the spring data mongodb project

MongoDB and Spring DataChris Harris

Twitter : cj_harris5Email : charris@10gen.com

© 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.

22

Solution Architect at 10gen (the company behind MongoDB)

- Previous Roles:

• EMEA Architect at SpringSource• EMEA Principal JBoss Consultant at RedHat

About Me

http://images.mirror.co.uk/upl/m4/mar2009/0/2/satnav-cliff-hanger-pic-ross-parry-image-3-213726463.jpg

http://www.simpleadhdexpert.com/wp-content/uploads/2009/08/frustration.jpg

http://daddytypes.com/archive/duct_tape_baby_mianro.jpg

Is there a better way?

Terminology

RDBMS MongoDBTable CollectionRow(s) JSON DocumentIndex IndexJoin Embedding & LinkingPartition ShardPartition Key Shard Key

As simple as possible, but no simpler

Depth of functionality

Sca

labi

lity

& P

erfo

rman

ceMemcached

Key / Value

RDBMS

MongoDB

Lets build a Spring Data MongoDB App!

Welcome to Dev News• User Story 1:

– As a user I should be able to create “News Articles”

What is a News Article?{ title: “What is new in 2.2 MongoDB”, date : new Date(), author : “Fred” text : “.......” }

Example 1 - Create

Welcome to MongoDB News Dev• User Story 1:

– As a user I should be able to create “News Articles”• User Story 2:

– As a user I should be able to find a “News Articles” by Name

Example 2 - Query

Example 3 - Repo

Welcome to MongoDB News Dev• User Story 1:

– As a user I should be able to create “News Articles”• User Story 2:

– As a user I should be able to find a “News Articles” by Name• User Story 3:

– As a user I should be able to comment on “News Articles”

What is a Comment?{ title: “What is new in 2.2 MongoDB”, date : new Date(), author : “Fred” text : “.......” tags : [“mongodb”, “nosql”] comments : [ {author : “chris”, text: “this is great” , date : new Date()}, {author : “tom”, text: “rock on”, date : new Date()} ] }

Example 4 - Comments

Welcome to MongoDB News Dev• User Story 1:

– As a user I should be able to create “News Articles”• User Story 2:

– As a user I should be able to find a “News Articles” by Name• User Story 3:

– As a user I should be able to comment on “News Articles”• User Story 4:

–As a user I want to know how many comments are on a “Articles”

What is a Comment?{ title: “What is new in 2.2 MongoDB”, .... comments : [ {author : “chris”, text: “this is great” , date : new Date()}, {author : “tom”, text: “rock on”, date : new Date()} ] , comments: count : 2}

Example 5 - Inc

“MongoDB lost my data!!”

Data: 125647383885969795743

Least durability - Don't use!

Driver MongoDB

apply in memory

write

Wait for Journal Sync - Same as RDBMSDriver MongoDB

apply in memory

write

j:trueWrite to journal

Replication

http://www.flickr.com/photos/10335017@N07/4570943043

Replica Set• Data Protection• Multiple copies of the data• Spread across Data Centers, AZs• High Availability• Automated Failover• Automated Recovery

Primary

Secondary

Secondary

Read

Write

Java App

Asynchronous Replication

Replica Sets

Replica Sets

Primary

Secondary

Secondary

Read

Write

Java App

Replica Sets

Primary

Primary

Secondary

Read

Write Automatic Election of new PrimaryJava App

Replica Sets

Recovering

Primary

Secondary

Read

Write New primary serves dataJava App

Replica Sets

Secondary

Primary

Secondary

Read

WriteJava App

Durability SummaryMemory Journal Secondary

Other Data Center

RDBMS

Default"Fire & Forget"

w=1

w=1j=true

w="majority"w=n

w="myTag"

Less More

Example 6 - Write Concern

Eventual Consistency

• http://www.flickr.com/photos/26095468@N04/3779692985

Understanding Eventual Consistency

Primary Secondary

v1

Thread #1

Insert

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Readv1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Readreads v1v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

v2Update

Readreads v1v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

v2

reads v1

v2

v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

Read

reads v1

v2

v2

v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

Readreads v2

reads v1

v2

v2

v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

Read

Thread #2

v2

v2

v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

Read

Thread #2

v2

v2

v1

v1

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

Read

Thread #2

v2

v2

v1

v1

v1 does not exist on

Secondary

Understanding Eventual Consistency

Primary SecondaryThread #1

Insert

Update

Read

Read

Thread #2

v2

v2

v1

v1

v1 does not exist on

Secondary

Reads v1 but Primary at v2

Read v2

Example 7 - Eventual Consistency

http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg

http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg

MongoDB Sharding• Automatic partitioning and management

• Range based

• Convert to sharded system with no downtime

• Fully consistent

How mongoDB Sharding works

• Range keys from -∞ to +∞ • Ranges are stored as "chunks"

-∞  +∞  

> db.runCommand({addshard: "shard1"});> db.runCommand({shardCollection: "mydb.users", key: {age: 1}})

How mongoDB Sharding works

• Data in inserted• Ranges are split into more "chunks"

-∞  +∞  

-∞  40 41 +∞  

> db.users.save({age: 40})

How mongoDB Sharding works

-∞  +∞  

-∞  40 41 +∞  

41 50 51 +∞  

61 +∞  51 60

> db.users.save({age: 40})> db.users.save({age: 50})> db.users.save({age: 60})

-∞  4041 50

61 +∞  

51 60

> db.runCommand({addshard: "shard2"});> db.runCommand({addshard: "shard3"});

How mongoDB Sharding works

-∞  4041 50

61 +∞  

51 60

shard1

> db.runCommand({addshard: "shard2"});> db.runCommand({addshard: "shard3"});

How mongoDB Sharding works

-∞  4041 50

61 +∞  

51 60

shard1 shard2 shard3

> db.runCommand({addshard: "shard2"});> db.runCommand({addshard: "shard3"});

How mongoDB Sharding works

Architecture

C1

C2

C3

Config Serversmongos mongos

app app

mongod

Shard 1

mongod

mongod

mongod

Shard 2

mongod

mongod

Shard 4

mongod

Shard 3

mongod

mongod

Replica Set

mongod

mongod

mongod

Balancing

config

config

config

Shard 1 Shard 2 Shard 4Shard 3

mongosbalancer

5

9

1

6

10

2

7

11

3

8

12

4

13 14 15 16 17 18 19 20 21 22 23 24

ImbalanceData inserted all in a range

mongos

Balancing

config

config

config

Shard 1 Shard 2 Shard 4Shard 3

mongosbalancer

5

9

1

6

10

2

7

11

3

8

12

4

13 14 15 16 17 18 19 20 21 22 23 24

Move chunk 1 to Shard 2

mongos

Balancing

config

config

config

Shard 1 Shard 2 Shard 4Shard 3

balancer

5

9

6

10

2

7

11

3

8

12

4

13 14 15 16 17 18 19 20 21 22 23 24

1

mongos

Balancing

config

config

config

Shard 1 Shard 2 Shard 4Shard 3

balancer

5

9

6

10

2

7

11

3

8

12

4

13 14 15 16 17 18 19 20 21 22 23 24

1

mongos

Balancing

config

config

config

Shard 1 Shard 2 Shard 4Shard 3

mongosbalancer

5

9

16

10

2

7

11

3

8

12

4

13 14 15 16 17 18 19 20 21 22 23 24

mongos

Example 8 - Sharding

• MongoDB 2.2– Improved concurrency–Tag-aware sharding–Aggregation framework–TTL collections

• Free online MongoDB training–Develop–Deploy–Classes start Oct. 2012

Questions