dropwizard with mongodb and google cloud

BILL APIONE DAY PROTOTYPE USING

,   ANDDROPWIZARD MONGODBGOOGLE CLOUD

Yun Zhi Lin +YunZhiLin @YunZhiLin

with Ray King+RayKing

ABOUT - YUN ZHI LINComputer Science @UNSW, Applied Finance @MacquarieJoined a startup in 2002, then moved into Banking ITMoonlights as a Web 2.0 (nosql, json, mobile) enthusiastLoves Java without bloated frameworks or appserversFather of one chubby "michelin" baby son

SPECEnd to end prototype of a Bill Aggregator that mimics

vendor API. See below for detail specifications.

DATA MODELProviders - bill provider

Premises - location being provisioned for

Subscriptions - subscribe to bill by account/reference #

Bill - matching subscriptions trigger notifications

Notifications - Sent to subscription.callbackUrl withUrlForBillRetrieval

FK: List(Subscriptions)

FK: List(Subscriptions)

FK: Provider.id, Premise.id, Bill.accountNumber, List(Bill.id, Status)

FK: Provider.id, Subscription.referenceNumber

FK: Provider, Premise, Subscription

ACTORSProducer - API hosted on Google Compute EngineAPI Operations - Insert, List, Get, Remove, Path on allentities, except Notification and Bill.

Consumer - Use Sample Data - JSON files provided

Postman HTTP client

REQUIREMENTS1. HTTP transport2. JSON payload3. Use compute engine4. Authentication5. API versioning6. Ops-friendly7. Data Validation8. Batch considerations

DESIGN

FRAMEWORKS CHOICENode.js - full stack JSON, but lacking expertisePlay! Framework - too complex for this exerciseDropwizard - simple, war-less and proven

DATABASE CHOICEGoogle Cloud Datastore - lacking expertiseMongoDB - perfect for JSON

DROPWIZARD + MONGODB = MEETREQUIREMENTS

1. HTTP - Jersey RESTful web services2. JSON - Jackson with Mongo-Jack, fastest Java parser3. Use compute engine - Both API jar and DB hosted on GCE4. Authentication - OAuth2 or custom Providers5. API versioning - annotation based url mappings6. Ops-friendly - healchecks and Codahale metrics7. Data Validation - Hibernate Validatior8. Batch considerations - use JSON lists

PLANDue to the time contraint, it's important to decide on the

approach taken to plan out a successfull prototype:

Start to End - build each component fully one by one, canonly partially demonstrate the storyMinimum Viable Product - selectively build only criticaloperations for all components. So that together they tell afull story

SCOPE ADJUSTMENTAuthentication - excludedAPI versioning - excludedBatch - simplified to single JSON parsingQueuing - another consideration for the real world,excludedNotification - manually triggered.

Only the following Operations demonstrate the coreprocesses of subscription, notification and bill retrieval:

FINAL USE CASE

1. create a subscriptoin

2. Manually trigger notification, where UrlForBillRetrieval =subscription.callbackURL + billID

3. retrieve a Bill

PUT /subscription/

GET /subscription/trigger/{subscriptoinID}/{billID}/

GET /bill/?billID={billID}

IMPLEMENTATION1. Build Tool2. Configuration3. MongoDB4. Healthcheck5. JSON POJOs6. Resources7. Application Glue

BUILD TOOLGradle combines the best of Ant and Maven

repositories { mavenLocal() mavenCentral()}project.ext { dropwizardVersion = '0.7.0-SNAPSHOT' mongojackVersion = '2.0.0-RC5' dropWizardConfig = './src/main/resources/billproto.yml'}dependencies { compile 'io.dropwizard:dropwizard-core:' + dropwizardVersion compile 'org.mongojack:mongojack:' + mongojackVersion testCompile group: 'junit', name: 'junit', version: '4.11'} run { args 'server', dropWizardConfig}

CONFIGURATIONYAML

POJO

mongo: host: ds039487.mongolab.com port: 39487 db: tycoon-mongo user: #### password: ####

public class MongoConfiguration { @NotNull public String host; @Min(1) @Max(65535) @NotNull public int port; @NotNull public String db; @NotNull public String user; @NotNull public String password;}

MONGODBHosting - GCE optionConnection

JacksonDBCollection

DB Import (must be list of single line json files)

MongLab

mongo = new Mongo(mongoConfig.host, mongoConfig.port);db = mongo.getDB(mongoConfig.db);db.authenticate(mongoConfig.user, mongoConfig.password.toCharArray());

bills = wrap(db.getCollection("bill"), Bill.class, String.class);dbCursor = bills.find().is("_id", billID);if (dbCursor.hasNext()) { Bill result = cursor.next();}

mongoimport --host --username --password --db --collection --file

HEALTHCHECKCode

Healthy

Not healthy:

@Overrideprotected Result check() throws Exception { mongo.getDB(dbname).getCollectionNames(); return Result.healthy();}

{"MongoHealthCheck":{"healthy":true},"deadlocks":{"healthy":true}}

{"MongoHealthCheck":{"healthy":false, "message":"not authorized for query on tycoon-mongo.system.namespaces", "error":{"message":"not authorized for query on tycoon-mongo.system.namespaces" "stack":[...] "deadlocks":{"healthy":true}}

JSON POJOSUse . Then apply simple

annotations for Jackson and Hibernate Validatior.http://www.jsonschema2pojo.org/

@JsonInclude(JsonInclude.Include.NON_NULL)public class Notification {

@Id public String notificationID;

@NotNull public String subscriptionID;

@NotNull public String notificationURLForBillDataRetrival;

@NotNull public String notificationURLForBillImageRetrival;

}

NOTE: Use @Id for UUID, not @ObjectId

RESOURCESAnnotate opertion, metrics, url path and output format.

Jackson automatically parse POJO into specified mediaType.

@GET@Timed@Path("trigger/{subscriptionID}/{billID}")@Produces(MediaType.APPLICATION_JSON)public Notification triggerNotification( @PathParam("subscriptionID") String subscriptionID, @PathParam("billID") String billID) { ... notification = new Notification(); notification.notificationID = UUID.randomUUID().toString(); notification.subscriptionID = subscriptionID; notification.notificationURLForBillDataRetrival = subscription.subscriptionCallBackURL + "?billID=" + billID; return notification;}

APPLICATION GLUEPutting all of the above together

@Overridepublic void run( BillprotoConfiguration configuration, Environment environment) throws Exception {

MongoManaged mongoManaged = new MongoManaged(configuration.mongo); environment.lifecycle().manage(mongoManaged); environment.healthChecks().register( "MongoHealthCheck", new MongoHealthCheck(mongoManaged));

environment.jersey().register(new SubscriptionResource(mongoManaged)); environment.jersey().register(new BillResource(mongoManaged));}

DEPLOY1. Install Java (defaults to OpenJDK)

2. Open firewall ports 8080 and 8081

3. Copy file to server

4. Run fatjar

sudo apt-get install java7-runtime-headless

gcutil addfirewall rest --description="http" --allowed="tcp:8080gcutil addfirewall admin --description="Iadmin" --allowed="tcp:8081"

gcutil --project={project-id} push {instance-name} {local-file} {remote-target-path}

java -jar billproto-0.2-fat.jar server billproto.yml

TESTINGREST URLS

Subscription: Trigger:

Bill: Healthcheck:

162.222.183.244:8080/subscription/

162.222.183.244:8080/subscription/{subscriptionID}/{billID}162.222.183.244:8080/bill/?billID={billID}

162.222.183.244:8081

CONCLUSIONBy building just the bare minimum operations in each

components that work well together, we were able to coverthe core user story and better assess our API feasibility.

We also managed to test out a new framework and built atemplate that can be fleshed out for future applications.

LINKS AND CREDITS

(Private Repo) built with

Managed by and hosted on .

dropwizard.iomongojack.orggradle.orgSource codeThis Slide reveal.js Export to PDF

bower GitHub Pages

QUESTIONS