rapid development with schemaless data models

Rapid Development with Schema-less Data models Keith Chilek SVP, Engineering

The online professional network for students and early professionals

3/8/13 | 2

With our data & tools, our members can better

Manage their education

Tell their story

Get a job

3/8/13 | 3

Growth

Technology Stack What is the MyEdu Profile?

MongoDb vs MySQL Decision Version 1 Using MongoDB

Learning From Our Mistakes

3/8/13 | 4

3/8/13 | 5

6,000,000

5,000,000

4,000,000

3,000,000

2,000,000

1,000,000

0

7,000,000

8,000,000

9,000,000

Jul 2012

Aug 2012

Sep 2012

Oct 2012

Nov 2012

Dec 2012

Jan 2013

We’re Building for Growth

MyEdu’s Technology Stack

Javascript

Scripting

Middleware

Database

3/8/13 | 6

3/8/13 | 7

“The only constant is change, continuing change, inevitable change, that is the

dominant factor in society today.

No sensible decision can be made any longer without taking into account not only

the world as it is, but the world as it will be.”

Isaac Asimov

3/8/13 | 8

MongoDB vs MySQL

Move fast, iterate often Minimize the downtime required

to launch new features Make sure the new product will

really, really scale

3/8/13 | 9

“We can do all of that in MySQL.”

“Schema-less design is impossible

to maintain.”

MongoDB vs MySQL

3/8/13 | 10

“We can do all of that in

MySQL.”

MongoDB vs MySQL

3/8/13 | 11

“We can do all of that in

MySQL.”

MongoDB vs MySQL

3/8/13 | 12

SELECT * FROM User, School, Major!WHERE User.school_id = School.school_id!AND User.major_id = Major.major_Id!AND User.user_Id = 1021!

“We can do all of that in

MySQL.”

MongoDB vs MySQL

3/8/13 | 13

“We can do all of that in

MySQL.”

MongoDB vs MySQL

3/8/13 | 14

SELECT * FROM User, User_School, User_Major, ! ! School, Major!

WHERE User_School.school_id = School.school_id!AND User_Major.major_id = Major.major_Id!AND User.user_id = User_Major.user_id!AND User.user_id = User_School.user_id!AND User.user_Id = 1021!

“We can do all of that in

MySQL.”

MongoDB vs MySQL

3/8/13 | 15

“We can do all of that in

MySQL.”

MongoDB vs MySQL

3/8/13 | 16

SELECT * FROM User, User_School, User_Major, User_Work, User_Project, School, Major, Work, Project!WHERE User_School.school_id = School.school_id!AND User_Major.major_id = Major.major_Id!AND User_Project.project_id = Project.project_Id!AND User_Work.work_id = Work.work_Id!AND User.user_id = User_Major.user_id!AND User.user_id = User_School.user_id!AND User.user_id = User_Work.user_id!AND User.user_id = User_Project.user_id!AND User.user_Id = 1021!

Yikes!

MongoDB vs MySQL

3/8/13 | 17

MongoDB vs MySQL

Looks like a Mongo document to me!

3/8/13 | 18

MongoDB vs MySQL

3/8/13 | 19

db.user_profile.find({“user_id” : 1021});!

“Schema-less design

is impossible to maintain.”

MongoDB vs MySQL

Lucky for us, the web services team had already been employing a Service / Mapper /

Model pattern.

With this pattern we are able to control the structure of each mongo document with a

defined data model.

3/8/13 | 20

MongoDB FTW!

MongoDB allows us to…

Move fast, iterate often Minimize the downtime required

to launch new features Make sure the new product will

really, really scale

3/8/13 | 21

MyEdu Profile (iteration 1: October)

3/8/13 | 22

MyEdu Profile (iteration 2: November)

3/8/13 | 23

MyEdu Profile (iteration 3: January)

3/8/13 | 24

Make sure the new product will really, really scale

With proper indexing alone, we have over 600k mongo profile documents on just a 3

member replica set.

We lowered our overall site speed time by 0.3 seconds and lowered our profile product

speed 300% when we introduced MongoDB.

3/8/13 | 25

It’s not all peaches and crème…

Oops! Bad Index

3/8/13 | 26

Lessons Learned

A schema-less data model has many benefits.

We can rapidly change our data to meet the needs of new product features and

requirements.

The document object is a familiar structure for a web developer.

3/8/13 | 27

What’s next?

Event tracking with MongoDB

Map Reduce

Using Json Patch for doing updates

3/8/13 | 28

Thanks!

Keith Chilek SVP, Engineering keith.chilek@myedu.com

We’re Hiring!

Thanks!

Keith Chilek SVP, Engineering keith.chilek@myedu.com

We’re Hiring!