Friday, January 11, 2013

Writing REST services in Java: Part 1 overview

This is the first in a series of posts walking through setting up a REST application in java. The code accompanying it can be found here https://github.com/iainporter/rest-java

The application contains all the necessary parts to start building new services and deploy them quickly and easily. After checking out the code and making a few configuration changes you can deploy a fully functioning API that includes:

* Signing up a user
* Logging in a user
* Facebook authentication
* Sending email verification
* Verifying email responses
* JSR 303 Validation
* Lost Password
* Role-based authorization
* Request signing
* Session Expiry

I will walk through the various parts in subsequent posts. This will include:


Technologies covered will include:
* Jax-RS (Jersey implementation)
* Spring Data
* Hibernate
* Spring Integration
* Java Mail
* Velocity
* Groovy (for Integration Tests only)
* Deploying to AWS


Building the project 


The project uses gradle to build and deploy. If you don't have gradle installed get it here http://www.gradle.org/

To debug when running in tomcat you can add the following to your bash profile:
export GRADLE_OPTS="-Xmx1024m -XX:MaxPermSize=128M -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8787"


If you have gradle installed then navigate to the root directory and execute

gradle clean build integrationTest

Once it has finished you can inspect the results in build/reports.


Running the Application


From the root directory execute

gradle tomcatRun

When the servlet container has loaded the war run the following curl request:


curl -v -H "Content-Type: text/html" -X GET  http://localhost:8080/java-rest/healthcheck


You should see the following response:


< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Date: Thu, 10 Jan 2013 19:21:02 GMT
< Running version 1.0.0

The application is now successfully running.


The Project Structure

At the root there are two files that control how the project is built:

build.gradle
gradle.properties


Build Tasks


gradle clean - cleans out the build directories
gradle build - compiles the source, runs the unit tests, writes test and coverage reports, builds the WAR file
gradle integrationTest - runs the groovy integration tests and appends to the test report
gradle tomcatRun - starts the tomcat container and loads the WAR file


Spring Profiles

The project makes use of Spring Profiles. When running from gradle commands the active profile is loaded from the gradle.properties file.

There are four profiles:

dev - runs in-memory database, Mock Mail Sender
local - runs against local instance of mySQL DB, Mock Mail Sender
staging - runs against a configured DB instance and configured staging Mail Sender
production - runs against a configured DB instance and configured production Mail Sender


Unit Tests and Code Coverage

Unit tests are in the src/test/java directory and are ran against the DEV profile. 
Integration tests are in the src/test/groovy directory. These tests are ran in a Jetty servlet container against the DEV profile. You can change the profile to local if you want to capture the results in your local DB store. The main principle of the integration tests is that they exercise the API in the guise of a client. This means that all requests and responses are json structures and not built against any server side API components.
Code coverage uses the emma plugin



The Architecture






A typical message flow is:

1. The client makes a RESTful request (either PUT, POST, GET or DELETE) using Json payloads.
2. The servlet filter routes all requests through the Authorization filter.
3. If there is a valid Authorization header the authorization service performs validation and identifies the User making the request
4. Control is passed to the Resource Controller that handles resources of that type. 
5. A Roles-based filter checks the user has access control for that resource request.
6. The Json payload (if present) is marshalled to an API object
7. The Resource Controller communicates with one or more Services by passing back and forth API objects
8. The Service tier is responsible for managing the lifecycle of entities and integration with any external services as well as transformation mappings between entities and API objects
9. The Repository tier manages persistence and retrieval of entities
10. Responses are returned by the Resource Controllers, unmarshalled to Json and returned to the client


In the next post I'll dig into the details of sign up and login.



56 comments:

  1. Good job, Iain! It's such an awesome serie. I'm going to follow through these tutorials. Thank you!

    ReplyDelete
  2. This is wonderful, thanks for your effort on putting this together.

    ReplyDelete
  3. Great Article.. very good example for beginners how to do professional project

    ReplyDelete
  4. Awesome job Iain !
    Is there open source license on this code? Can anyone use the code?

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Hi there, great series.

    How can I integrate the project within IntelliJ?

    ReplyDelete
  7. cd to the project root and run:

    gradle idea

    this will create the intellij project file

    for eclipse run:

    gradle eclips

    ReplyDelete
  8. I keep getting this output, and cannot continue to build:

    Loading > Resolving dependencies ':classpath'

    any idea what might be the problem?

    ReplyDelete
  9. What task are you executing?

    You should be able to run "gradle clean build"

    from the project root.

    ReplyDelete
  10. That's exactly what i'm doing..

    I'm getting this output on almost every task (idea, build, etc).

    ReplyDelete
  11. Can you post the relevant stack trace snippet

    ReplyDelete
  12. Hello, I am getting following error while trying to run any gradle commands on the project. I am not a gradle user, so I don't know what it means. I have gradle-1.8.

    * Where:
    Build file '../rest-java-master/build.gradle' line: 254

    * What went wrong:
    A problem occurred evaluating root project 'rest-java-master'.
    > Could not create task 'integrationTest': Unknown argument(s) in task definition: [depends]

    Would be great if you could help me in fixing this. Thanks!

    ReplyDelete
  13. This is a very interesting article for something I am doing currently (but I'm using Spring Security), I'm having a compilation error in the code from github that I don't understand. The error in Eclipse is
    "The entity has no primary key attribute defined SessionToken.java." SessionToken subclasses AbstractPersistable which does define a primary key. I'm using spring-data-jap 1.4.1.RELEASE. Any idea?

    ReplyDelete
  14. MAN,, this is BEAUUUUTIFUL... AMAZING JOB done.. you deserve tons of appreciation.... ;)

    ReplyDelete
  15. Thanks for a great tutorial! How we would go about using Cassandra or HBase instead of a regular SQL DB? Would we have to ditch Spring Data and Hibernate? Any info would be greatly appreciated.

    ReplyDelete
    Replies
    1. There is a Spring Data Cassandra project: https://github.com/spring-projects/spring-data-cassandra

      You would need to refactor the domain objects and the repository interfaces and then the data-context.xml for setting up connections to your Cassandra instance.

      Delete
  16. How can we make this app portable ? meaning build it on 1 machine but deploy it on another ?

    ReplyDelete
    Replies
    1. The properties that are loaded are based on spring profiles (dev, local, staging, production)
      You can set up your database, email config, etc in the various application context files that make use of profiles.

      See http://porterhead.blogspot.co.uk/2013/03/writing-rest-services-in-java-part-7.html for details.

      Delete
    2. What I'm asking is for example, I want to build the WAR locally on a dev machine, and deploy it to a different server. (without running gradle on the destination host)

      Delete
    3. on your dev machine the war file is placed in:

      build/libs

      If you just want to build the war run:

      > gradle war

      or

      > ./gradlew war

      Delete
  17. Thanks you very much. This is actually good.

    ReplyDelete
  18. This comment has been removed by a blog administrator.

    ReplyDelete
  19. I downloaded this project from github. But getting this while doing "gradle eclipse"


    * What went wrong:
    Could not compile build file 'D:\Work\Code\GitHub\rest-java\build.gradle'.
    > startup failed:
    build file 'D:\Work\Code\GitHub\rest-java\build.gradle': 54: unable to resolve class org.apache.ivy.plugins.resolver.URLResolver
    @ line 54, column 13.
    add(new org.apache.ivy.plugins.resolver.URLResolver()) {
    ^

    1 error


    I am using gradle2.0.
    Can't find any answer on google for this.

    Appreciate any help on this.

    Thanks.

    ReplyDelete
    Replies
    1. Pull the latest code.
      It has been tested against gradle 2.0

      You can also use the supplied gradle wrapper

      ./gradlew eclipse

      Delete
    2. Thank You very much for your help.
      It's working fine now.

      -Manan

      Delete
  20. Hi Iain,

    I'm new to Restfull API especially with securing it. I tried to compile your project but got a bunch of errors. Any hints on how to resolve this??

    Instrumenting the classes at C:\Development\rest-java-master\build\classes\main
    Creating C:\Development\rest-java-master\build\tmp\emma\instr to instrument from C:\Development\rest-java-master\build\classes\main

    com.porterhead.rest.authorization.RequestSigningAuthorizationServiceTest > initializationError FAILED
    java.lang.VerifyError

    com.porterhead.rest.authorization.SecurityContextTest > caseDoesNotMatter FAILED
    java.lang.VerifyError at SecurityContextTest.java:42

    com.porterhead.rest.authorization.SecurityContextTest > allowAnonymousRole FAILED
    java.lang.VerifyError at SecurityContextTest.java:36

    com.porterhead.rest.authorization.SecurityContextTest > authenticationFailure FAILED
    java.lang.Exception
    Caused by: java.lang.VerifyError at SecurityContextTest.java:48

    ReplyDelete
    Replies
    1. There was a problem with running emma with 1.8 JRE.
      It is disabled now. Try pulling and running again

      Delete
  21. the command didn't finish : Building 93% > :integrationTest > Starting Jetty

    ReplyDelete
    Replies
    1. What is the command you are executing?

      Can you run ./gradlew tomcatRun?

      Delete
  22. gradlew clean build integrationTest

    ReplyDelete
  23. the command : gradlew tomcatRun works fine, and i can to connect to the application, many thanks

    ReplyDelete
  24. First of all , congratulations on the blog. It is very helpful . I noticed that you used in previous post Spring Intregration for verification mail functionality , but in this no. Please, can you tell me why ?. Which approach is better ?. Spring Integration With or without Spring integration ?. Thank you very much ( and sorry for my English )

    ReplyDelete
    Replies
    1. I took the Spring Integration part out for this sample project, but I really should have left it in. What it gives you is a simple way to implement guaranteed delivery. If there is a communication failure with the smtp service then the message will be put back in the queue for a retry.

      Delete
    2. Thank you very much . I'll try to add that part to this project to implement guaranteed delivery. A greeting.

      Delete
  25. Ian, thanks very much.

    I'm trying to follow but I didn't get what is the request flow from the page until its last step.

    Could you please help me?

    Example: webpage -> porterhead/rest/authorization/impl -> next step -> next step -> next step -> .... -> last step (Repository I guess)

    And what do you suggest for using these rest services for webpages and mobile apps?

    Thanks,

    Eduardo.

    ReplyDelete
    Replies
    1. I am not sure what you are asking.
      In any case I would recommend that you work with this project: https://github.com/iainporter/oauth2-provider instead as it supersedes the work and ideas in this project.

      Delete
    2. What I meant is: From an user action in the webpage, let's say click on the sign up button, what are the next Java classes the request will pass through?

      Example: An user clicks on the sign up button, so it invokes a javascript function which makes a post. This request is received by XXX class and then call the XXX class and then call the class XXX.

      Did you get what I mean?

      I know that I could discover this flow debugging the whole project. But once I couldn't even compile the whole project (I'm a .NET developer and I'm having some basic environment troubles), I'd appreciate a lot if you could provide me.

      About the second and last project, do you have a blog post giving some guidance about it?

      Delete
    3. http://porterhead.blogspot.co.uk/2014/05/securing-rest-services-with-spring.html

      Delete
  26. Ian, thanks very much.

    Very good explanation for beginners.

    ReplyDelete
  27. After successful running the project when trying to sign up i will get "Please fix your email address/password."

    ReplyDelete
    Replies
    1. Is this via curl or the browser?
      Can you append a screenshot or stacktrace.

      Delete
    2. hi Lain Porter,

      I am getting error message because of password field length validation.

      {"errorCode":null,"consumerMessage":"Validation Error","applicationMessage":"The data passed in the request was invalid. Please check and resubmit"

      Now its working.

      Thank you.

      Delete
  28. Hi Iain Porter,

    I am getting this error while i am click on Sign Up via Facebook.


    The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
    java.lang.NoClassDefFoundError: org/springframework/http/converter/json/MappingJacksonHttpMessageConverter
    at org.springframework.social.oauth2.AbstractOAuth2ApiBinding.getJsonMessageConverter(AbstractOAuth2ApiBinding.java:151)
    at org.springframework.social.facebook.api.impl.FacebookTemplate.getJsonMessageConverter(FacebookTemplate.java:239)

    ReplyDelete
  29. Hi Iain, thanks for the work. It seems very helpful but I have a problem. You have a link at the top of the page to get the code. I am new to service and autentication and I am trying to write the code myself to get more information about these tasks but I can't understand which class you wrote first and which class is last. and this issue makes it imposibble to write code for me. Is there any possibility to get information about sorting classes and packages?

    ReplyDelete
    Replies
    1. If you want to take that approach I would suggest a test-driven design.
      Write a test first and then write the pieces to make the test work.

      See https://github.com/iainporter/rest-java/blob/master/src/test/java/com/porterhead/rest/user/UserServiceTest.java as a good starting point.

      Delete
  30. Hi Iain Porter,

    can u provides us the maven dependency file for this project(pom).or if possible build of this project as maven project instead of gradle.

    ReplyDelete
  31. Thanks for sharing your expertise.

    ReplyDelete
  32. Great site and brilliant code! I

    'm having a slight problem running it. I get an error about a missing dependency from the SLF4J jar. The error I get is classNotFound for org.slf4j.spi.LoggerFactoryBinder. I've checked and this jar file is in the rest-java.master.war file in the libs folder and the class is there in the war file.

    I'm running this via command line (so, no IDE involved) Any idea what could be causing the issue?

    thanks,

    Laura

    ReplyDelete
    Replies
    1. Can you share the stack trace and the command you executed

      Delete
    2. Can you try it using the gradle wrapper:

      ./gradlew tomactRun

      Delete
    3. hi Iain,

      Thanks very much for your help. It worked fine when using the gradle wrapper,

      Laura

      Delete
  33. It is a great tutorial. Thanks for sharing your experience. It really helps a lot.

    ReplyDelete