(dev303) practical dynamodb programming in java
TRANSCRIPT
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Jason Fulghum, Development Manager –
AWS SDK
Karthi Thyagarajan, Enterprise Solutions
Architect
October 2015
DEV303
Practical DynamoDB with Java
What to Expect from the Session
REAL-TIME PROCESSING
• Lambda + DynamoDBStreams
• Traditional approaches
DEMO – Real-time processing w/ Lambda and DynamoDB
Streams
BATCH PROCESSING
• Lambda-based
• Traditional approaches
DEMO – Batch processing w/
Lambda
1 2 3 4
Scenario
- Game analytics service
- Simple scenario, but concepts carry over for other
applications
- IoT
- E-commerce
- Connect mobile games
NOTE: This session will be heavily focused on code
Lambda
No infrastructure headaches
nodeJS and Java
Trigger on events
1
DynamoDB Streams
Similar to DB triggers
Change records available up to 24 hours
Can process change records using λ
1
Near real-time processing
REAL-TIME PROCESSING
• Lambda + DynamoDBStreams
• Traditional approaches
DEMO – Real-time processing w/ Lambda and DynamoDB
Streams
BATCH PROCESSING
• Lambda-based
• Traditional approaches
DEMO – Batch processing w/
Lambda
1 2 3 4
Near real-time processing: Key challenges
Multiple views for multiple constituents
Minimize impact on performance
Near real-time availability of data in views
1
Proposed architecture
Main table – raw
scores
Player stats
“BI” table (for internal
use)
1
Satellite table –
player info
Multiple views for multiple
constituents
• Materialized views
Minimize impact on performance
• Separate tables so no
contention
Near real-time availability of data
in views
• Minimal delay
Proposed architecture (cont’d)
1
Comparison: SQL-based architecture
• Multiple views for multiple
constituents
• Joins, aggregations, group-
bys, etc.
• Minimize impact on performance
• Potential for deadlock
• Near real-time availability of
data in views
• Read replicas
1
Demo: Real-time processing
REAL-TIME PROCESSING
• Lambda + DynamoDBStreams
• Traditional approaches
DEMO – Real-time processing w/ Lambda and DynamoDB
Streams
BATCH PROCESSING
• Lambda-based
• Traditional approaches
DEMO – Batch processing w/
Lambda
1 2 3 4
Batch processing
REAL-TIME PROCESSING
• Lambda + DynamoDBStreams
• Traditional approaches
DEMO – Real-time processing w/ Lambda and DynamoDB
Streams
BATCH PROCESSING
• Lambda-based
• Traditional approaches
DEMO – Batch processing w/
Lambda
1 2 3 4
Batch processing
• Why batch processing?
• Backfill - may not have accounted for certain scenarios
• Not all views need to be real-time
3
Batch processing: Key challenges
Timely completion – even as data set grows
Must not interfere with real-time processing
3
Batch processing: Architecture
Raw scores
“BI” table
Player stats
Player stats by
date 3
Batch processing (cont’d)
Timely completion – even as data set grows
Must not interfere with real-time processing
3
Batch processing: Traditional approaches
DB: SQL
ETL
May not scale w/ data set growth
DB: NoSQL
Impala, Spark, Presto
Learning curve
3
Demo: Batch processing
REAL-TIME PROCESSING
• Lambda + DynamoDBStreams
• Traditional approaches
DEMO – Real-time processing w/ Lambda and DynamoDB
Streams
BATCH PROCESSING
• Lambda-based
• Traditional approaches
DEMO – Batch processing w/
Lambda
1 2 3 4
Summary
No infrastructure
Low barriers to entry
Force multiplier
DynamoDB
Streams λ
+
Next steps
- Links
- AWS SDK for Java: http://aws.amazon.com/sdk-for-java/
- AWS Toolkit for Eclipse: http://aws.amazon.com/eclipse/
- AWS Java Development blog: https://java.awsblog.com/
Related sessions
- ARC308 - The Serverless Company Using AWS
Lambda: Streamlining Architecture with AWS
- DAT401 - Amazon DynamoDB Deep Dive: Schema
Design, Indexing, JSON, Search, and More
Thank you!
Remember to complete
your evaluations!