Saturday, September 27, 2014

Getting Scalatra to work on Google App Engine - Part 2

In the previous post we made the necessary modifications to Scalatra to make it compatible with Servlets API 2.5 , which allow us to use it on App Engine. One compromise we had to make is to use web.xml to register all the servlets, instead of doing it programmatically.

There is indeed no way to make it possible to register the servlets programmatically in a 2.5 API environment we have no control of, but we can work around it by not really using servlets. This is actually a very old concept in Java web frameworks - register just one servlet in web.xml and have it dispatch the request to application non-servlet handlers. With a little work we can modify Scalatra to work this way, taking advantage of its API, and using an internal mapping mechanism for the top level resources, instead of the Servlet 3.0 registration.

Do note that as opposed to the changes we made in the previous post, the changes demonstrated here result in a different way the requests flow in your application, which may or may not affect your specific cases, especially when you have different frameworks/components in your web tier.

One of the objectives in the solution presented here is to stick to Scalatra's API as much as possible, so that it would be easier to port existing Scalatra code to App Engine, and to drop this solution when App Engine upgrades to Servlet API 3.0+ (if/whenever that happens).

The steps we need to take are:
  • Implement a resource class that will operate similar to ScalatraServlet but will ignore the URI prefix that corresponds to the top-level mapping
  • Implement a mapping structure to back the mounting operation
  • Override Scalatra's enriched ServletContext mount operation to delegate to the above mapping
  • Implement a dispatcher servlet that will use the above mapping to dispatch requests to the appropriate resource
First we implement our "resource" class (think JAX-RS resource or SpringMVC controller). Since it should be almost identical to ScalatraServlet, we inherit from it, and the only change we need to make is to trim the URI prefix from the top-level mapping, because Scalatra's original behavior is to use the part of the URI after the servlet mapping, and we need to use the part after the dispatcher mapping. Therefore we add a member to hold the length of the top-level mapping, and override requestPath() to return the path after that.

package org.scalatra.gae

import org.scalatra.ScalatraServlet
import javax.servlet.http.HttpServletRequest
import org.scalatra.Handler

/**
 * Instances of this class are meant to be mounted via 
 * [[org.scalatra.gae.AppMapper]] , as opposed to 
 * [[javax.servlet.ServletContext]], which cannot be done in API 2.5 .
 */
class ScalatraResource extends ScalatraServlet {

  /**
   * The length of the top-level URI mapping
   */
  protected[gae] var basePathLength: Int = _

  /*
   * Trim the top-level mapping from the URI before using Scalatra's mapping
   */
  override def requestPath(implicit request: HttpServletRequest): String = {
    super.requestPath(request).substring(basePathLength)
  }
}

To store the mappings, we implement an object that lets us add (mount) mapping of URI prefixes to resources, and get a resource that matches a URI prefix. We use a mutable map for storage. When we mount a resource, we set the top-level mapping length so it knows how to trim it.

package org.scalatra.gae

/**
 * This object holds the top-level resource mappings
 */
object AppMapper {

  val mapping = scala.collection.mutable.Map[String, ScalatraResource]()
  
  /*
   * Map a URI prefix to a resource
   */
  def mount(handler: ScalatraResource, path: String) {

    // Consistent handling of "/path", "/path/" and "/path/*"
    val fixedPath = if (path.endsWith("/*"))
      path.stripSuffix("*")
    else if (!path.endsWith("/"))
      path + "/"
    else
      path

    handler.basePathLength = fixedPath.length - 1
    mapping.put(fixedPath, handler)
  }
  
  /*
   * Get the resource that matches the prefix of a URI
   */
  def getHandler(uri: String) = mapping.find(m => uri.startsWith(m._1))
}

We could use AppMapper directly from our code, but to keep it close to the Scalatra way we will extend Scalatra's enriched ServletContext class and update ServletApiImplicits so our ScalatraBootstrap code will be decoupled from this solution.

package org.scalatra.gae

import org.scalatra.servlet.RichServletContext

import javax.servlet.ServletContext

/**
 * Enriched [[javax.servlet.ServletContext]] which delegates the mounting 
 * operation to application-level mapping, instead of 
 * [[javax.servlet.ServletContext#addServlet(String, javax.servlet.Servlet)]]
 * which is not available in API 2.5
 */
class RichAppServletContext(sc: ServletContext) extends RichServletContext(sc) {

  def mount(handler: ScalatraResource, urlPattern: String) = AppMapper.mount(handler, urlPattern)
}


package org.scalatra
package servlet

import javax.servlet.ServletContext
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse
import javax.servlet.http.HttpSession
import org.scalatra.gae.RichResponseLocalStatus
import org.scalatra.gae.RichAppServletContext

trait ServletApiImplicits {
  implicit def enrichRequest(request: HttpServletRequest): RichRequest =
    RichRequest(request)

  // Changed to overcome lack of HttpServletResponse.getStatus() in Servlet API 2.5
  implicit def enrichResponse(response: HttpServletResponse): RichResponse =
    new RichResponseLocalStatus(response)

  implicit def enrichSession(session: HttpSession): RichSession =
    RichSession(session)

  // Changed to overcome the lack of ServletContext.mount() in Servlet API 2.5
  implicit def enrichServletContext(servletContext: ServletContext): RichAppServletContext =
    new RichAppServletContext(servletContext)

}

object ServletApiImplicits extends ServletApiImplicits

Finally we implement the dispatcher servlet. It uses the request URI to get the matching resource instance from AppMapper and dispatches the request to it.

package org.scalatra.gae

import scala.collection.JavaConversions._
import javax.servlet.http.HttpServlet
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse

/**
 * This is the only servlet that needs to be registered in web.xml .
 * It dispatches the request to the resource that matches the URI prefix
 * as mounted to [[org.scalatra.gae.AppMapper]]
 */
class Dispatcher extends HttpServlet {

  override def service(request: HttpServletRequest, response: HttpServletResponse) {

    val uri = request.getRequestURI.substring(request.getServletPath().length())
    val o = AppMapper.getHandler(uri)
    if (o.isDefined) {
      val (basePath, handler) = o.get
      handler.handle(request, response)
    }

  }

}

Now all we need to do in our application code is:
  • Register and map the Dispatcher servlet in web.xml
  • Inherit from ScalatraResource instead of ScalatraServlet
Other than that, our code would be similar to any Scalatra application on a Servlet API 3.0+ environment, including calls to ServletContext.mount() in ScalatraBootstrap.

A complete code example is available here.

This is not the most generic code, for example - it doesn't handle filters, but it is a working starting point to get Scalatra to work on App Engine with minimum changes. If you have ideas for improvements - I'd love to know.

Wednesday, September 10, 2014

Getting Scalatra to work on Google App Engine - Part 1

Scalatra is a neat lightweight Scala web framework. Scalatra 2.3.0 is based on the Servlet API 3.1 , while App Engine Java runtime only supports Servlet API 2.5 . App Engine imposes additional restrictions as described here, but they are not as significant as lack of support for Servlet 3.0/3.1 .

After digging and experimenting, I succeeded in getting Scalatra to work on App Engine with a few workarounds. Basically we need to refrain from using Servlet 3.0/3.1 specific features in our code, and override Scalatra's parts that do use them with Servlet 2.5 compatible alternatives.

The first obstacle you'll encounter is that you can't mount your servlets and filters programmatically in your ScalatraBootstrap class like this:

import org.scalatra.LifeCycle

import javax.servlet.ServletContext

class ScalatraBootstrap extends LifeCycle {

   override def init(context: ServletContext) {

      // This will fail
      context mount (new MyCustomServlet, "/*")
   }
}

Instead, you leave the init() method empty, and add your servlets and filters to web.xml :

<servlet>
   <servlet-name>my-custom-servlet</servlet-name>
   <servlet-class>MyCustomServlet</servlet-class>
</servlet>
<servlet-mapping>
   <servlet-name>my-custom-servlet</servlet-name>
   <url-pattern>/*</url-pattern>
</servlet-mapping>

Then, at runtime when accessing a path which is mapped by your Scalatra endpoints, you will encounter a NoSuchMethodError because Scalatra attempts to call HttpServletResponse.getStatus(), which was added in Servlet API 3.0 . The problematic code is in RichResponse.status(). RichResponse is an implicit replacement for HttpServletResponse as defined in ServletApiImplicits.enrichResponse(). To overcome the reading of the status from the original response, we'll subclass RichResponse and change the setter and getter of the status to use a local variable:

import org.scalatra.servlet.RichResponse
import javax.servlet.http.HttpServletResponse
import org.scalatra.ResponseStatus

class RichResponseLocalStatus(response:HttpServletResponse) extends RichResponse(response) {

   var statusCode:Int = _

   override def status = ResponseStatus(statusCode)

   override def status_=(statusLine: ResponseStatus) {
      statusCode = statusLine.code
      super.status_=(statusLine)

   }
}

The Scalatra code made it impossible to override the implicit definition for our need, so we have to rewrite ServletApiImplicits to look like this:

package org.scalatra
package servlet

import javax.servlet.ServletContext
import javax.servlet.http.{HttpServletRequest, HttpServletResponse, HttpSession}
import test.RichResponseCustom

trait ServletApiImplicits {

   implicit def enrichRequest(request: HttpServletRequest): RichRequest =
      RichRequest(request)

   implicit def enrichResponse(response: HttpServletResponse): RichResponse =

      // This is changed
      new RichResponseLocalStatus(response)

   implicit def enrichSession(session: HttpSession): RichSession =
   RichSession(session)


   implicit def enrichServletContext(servletContext: ServletContext): RichServletContext =
      RichServletContext(servletContext)
}

object ServletApiImplicits extends ServletApiImplicits

Remember to place the changed ServletApiImplicits with your sources in the "org.scalatra.servlet" package. If you place it in another JAR, remember to make sure it takes precedence over Scalatra's JAR using App Engine's classloader JAR ordering.

That's it. It's not a perfect solution as it won't support Servlet 3.0/3.1 features such as asynchronous processing, and you have to define and map each servlet and filter in web.xml, but these are limitations you have to live with anyway on App Engine and this solution does let you use the nice and clean Scalatra code to write your web tier.

In the next post we will implement an alternative mapping mechanism, which will allow us to reduce the web.xml code and call mount operations in our ScalatraBootstrap code.

Sunday, May 25, 2014

Spring Social Google 1.0.0 Reaches General Availability

I am happy to announce the release of Spring Social Google 1.0.0.RELEASE ! I would like to thank everyone who contributed by testing and submitting issues and pull requests.

Spring Social Google contains bindings to Google Plus, Google Tasks and Google Drive APIs, and allows you to use any Java client library for any Google API while using Spring Social for authentication and connection management.

Project on GitHub: https://github.com/GabiAxel/spring-social-google

Distribution bundle: https://github.com/GabiAxel/spring-social-google/releases/tag/1.0.0.RELEASE

Reference manual: http://gabiaxel.github.io/spring-social-google-reference/

JavaDoc: http://gabiaxel.github.io/spring-social-google-javadoc/

Example application: https://github.com/GabiAxel/spring-social-google-example

To use with Maven add the following code to your POM:
<dependency>
   <groupId>org.springframework.social</groupId>
   <artifactId>spring-social-google</artifactId>
   <version>1.0.0.RELEASE</version>
</dependency>

Creating a Google Cloud Project and Using it with Spring Social Google


Navigate to https://console.developers.google.com/project

Click "Create Project" and enter a name and ID for the project.

Enter the project and in the left menu select "APIs & Auth". In "APIs" activate the APIs you want to use in the application. The example application uses Google+ API and Tasks API.

In the left menu go to "Credentials" and click "Create new client ID". For "Application Type" choose "Web Application" and for "Authorized Redirect URI" enter "http://localhost:8080/signin/google".

The new set of credentials will be added. Note the "Cliend ID" and "Client Secret" values - we will use them when running the example application.

Get the example application from GitHub:
git clone git://github.com/GabiAxel/spring-social-google-example.git

Enter the application directory and run it with Tomcat (replace CLIENT_ID and CLIENT_SECRET with the values you saw in Google Developers Console):
cd spring-social-google-example
mvn tomcat7:run \
   -Dgoogle.clientId=CLIENT_ID \
   -Dgoogle.clientSecret=CLIENT_SECRET

The application will run at http://localhost:8080/

The Road Ahead


There is no shortage of Google APIs. My current though is to proceed to YouTube and Calendar, but I really want to know what you want, so please submit your requests and issues in the GitHub project.

Enjoy using Spring Social Google!

Monday, April 9, 2012

Spring Social Google First Milestone Is Out



I'm happy to announce that the first milestone of Spring Social Google is out! Since the project started I received a lot of feedback from the community that helped shape this milestone and I would like to thank everyone who helped with suggestions, bug reports, patches and kind words.

This milestone includes integration with Google+, Portable Contacts and Google Tasks. It also lets you integrate with the GData Java Client Library, letting you use Spring Social Google for authentication and authorization, and applying the OAuth2 access token to an instance of a GData client. Please see the reference manual for usage instructions.

The example application demonstrates most of the available functionality. You can see it in action here.

To use Spring Social Google in a maven project, add the following repository and dependency to your pom.xml :

<repository>
    <id>spring.social.google</id>
    <name>Spring Social Google</name>
    <url>https://github.com/GabiAxel/maven/raw/master/</url>
</repository>

...

<dependency>
    <groupId>org.springframework.social</groupId>
    <artifactId>spring-social-google</artifactId>
    <version>1.0.0.M1</version>
</dependency>


or download the JAR here.

The project roadmap mostly depends on community feedback. Integration with more Google APIs will be added and you can help decide which will be first, so let me know what you think.

Enjoy using Spring Social Google!

Thursday, October 27, 2011

Replacing javac with eclipse compiler in Maven

I was working on a Java project with eclipse where I used cyclic dependencies. Specifically I implemented the Reverse MVP pattern with GWT Platform. Everything went well in as long as I was using eclipse to compile the project, but once I tried to use Maven to compile the project, I got compilation errors for every case where I had a cyclic dependency. I figured that if eclipse is good enough to compile the sources in development time, it might as well be used in build time instead of JDK's javac. Here is the maven-compiler-plugin configuration from the project POM I initially had:

   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-compiler-plugin</artifactId>
   2.3.2
   
      <source>1.6</source>
      1.6
   
In order to replace javac with eclipse compiler we need to do two things: we need to add a dependency for plexus-compiler-eclipse, and we need to tell the maven-compiler-plugin to use the eclipse compiler as described here. Here is the updated configuration:

   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-compiler-plugin</artifactId>
   2.3.2
   
      <compilerId>eclipse</compilerId>
      <source>1.6</source>
      1.6
   
   
      
         <groupId>org.codehaus.plexus</groupId>
         <artifactId>plexus-compiler-eclipse</artifactId>
         1.8.2
      
   
After that it was possible to build the project with Maven.

Thursday, September 22, 2011

Spring Social Meets Google APIs

For the past several weeks I have been working on the Spring Social Google project. The goal of Spring Social is to act as an abstraction layer between your application code and various social APIs, and removes the need for you to deal with authentication and HTTP-Java mapping, and now you can use it with a growing number of Google APIs, starting with the Contacts API and Google+ API.

Why do I need this? Google already provides Java libraries for its APIs.

Indeed you can use the Java libraries from Google, but there are cases where you may benefit from using Spring Social instead. For once, Spring Social comes with an authentication mechanism that removes the need for you to write Servlet code for the stages of the OAuth2 process. You can read about it here. Another goodie Spring Social provides is connection management and persistence, so you don't have to deal with session management and and writing your own database schema to store the users' access tokens. You can read about it here. You can see how to set up both mechanisms in the example application, specifically in SocialConfig.javaWebMvcConfig.java and spring-config.xml .

Spring Social also takes a different approach than Google's Java libraries in that Google's libraries provide a Java API that mirrors the underlying REST API (be it Atom, Portable Contacts or other), while Spring Social aims to provide you with data structure and operations you are likely to use.
Let's use both tools to write a program that fetches the user's contacts and prints their names and primary e-mail addresses.

Here is how it's done with Google Contact API Java library:

ContactsService service = getContactsService();
URL feedUrl = new URL("https://www.google.com/m8/feeds/contacts/default/full");
ContactFeed resultFeed = service.getFeed(feedUrl, ContactFeed.class);
for (ContactEntry entry : resultFeed.getEntries()) {
   String fullName = entry.getName().getFullName().getValue();
   String email = null;
   for(Email e : entry.getEmailAddresses() {
      if(email.getPrimary) {
         email = e;
         break;
      }
   }
   System.out.println(fullName + " " + email);
}

And here is the equivalent Spring Social Google code:

GoogleOperations google = getGoogleOperations();
List<Contact> contacts = google.contactOperations().getContactList();
for(Contact contact : contacts) {
   String fullName = contact.getFullName();
   String email = contact.getPrimaryEmail();
   System.out.println(fullName + " " + email);
}

It's up to you to decide if you prefer the simplicity of Spring Social or the flexibility of the Google API Java libraries.

Is the project stable? What is its roadmap?

Spring Social Google is still at an early stage, so there are no builds and the API may change. Aside of the currently implemented Contacts and Google+ APIs, the project is expected to include more and more of Google APIs over time. You are welcome to download the sources and play with the example application, which is also deployed on Cloud Foundry

Enjoy and send feedback!

Saturday, July 30, 2011

Dynamically Adding Styles to HTML Pages

I came into a situation where I needed to apply styles to the entire page after the page was loaded. Using techniques like targeting elements by their CSS selectors and applying styles to them with jQuery or similar tools would not be sufficient, because I would have to do it every time elements are added to the DOM and it would have required specifically adding a class (or some other attribute) to each element I wanted to apply the style to.

The solution I ended up implementing was to add styles to the page dynamically using JavaScript. Normal browsers let you manipulate the Head element as any other element, so all I needed to do was to append the Style element as a child of the Head element, and set the style's inner HTML with the CSS content:

function addCustomStyle() {
   var style = document.createElement("style");
   style.innerHTML = ".someclass {color: green;}";
   var head = document.getElementsByTagName("head")[0];
   head.appendChild(style);
}

Unsurprisingly like so many other cases, Internet Explorer has its own way of doing stuff. It doesn't allow manipulating the Head element as other elements, but it does provide an API for adding CSS rules to the document. Here is how it's done:

function addCustomStyle() {
   var style = document.createStyleSheet();
   style.addRule(".someclass", "color: green;");
}

You could wrap the two methods with IE conditional tags ([if !IE] and [if IE]) so only one of them will actually be used depending on the browser, or you can come up with your own mechanism. If you are using GWT, deferred binding is perfect for this.