butter web browsing with margarine

margarine dissection

alex brandtsr developer

wayne wallscloud evangelist

Wednesday, August 21, 13

2

WHAT IS MARGARINE?

Wednesday, August 21, 13

3

3 PARTS

Wednesday, August 21, 13

4

BLEND

TINGE

SPREAD

Wednesday, August 21, 13

5

BLEND

TINGE

SPREAD

Wednesday, August 21, 13

6

BLEND

TINGE

SPREAD

Wednesday, August 21, 13

7

WHAT DO WE CARE ABOUT?

Wednesday, August 21, 13

8

SIMPLICITY!

Wednesday, August 21, 13

9

DESIGN DISCLAIMER

Wednesday, August 21, 13

10

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE

Wednesday, August 21, 13

11

BUT WHAT DOES IT ALL MEAN?!

Wednesday, August 21, 13

12

DATA TIER

Wednesday, August 21, 13

13

WHAT KIND OF DATA DO WE NEED TO STORE?

Wednesday, August 21, 13

14

Content of ArticlesUser Information / ProfilesSubscriptions by UsersTags by UsersNotations for Margin Notes

Wednesday, August 21, 13

15

Content of ArticlesUser Information / ProfilesSubscriptions by UsersTags by UsersNotations for Margin Notes

Wednesday, August 21, 13

16

Content of ArticlesUser Information / ProfilesSubscriptions by UsersTags by UsersNotations for Margin Notes

Wednesday, August 21, 13

17

Content of ArticlesUser Information / ProfilesSubscriptions by UsersTags by UsersNotations for Margin Notes

Wednesday, August 21, 13

18

Content of ArticlesUser Information / ProfilesSubscriptions by UsersTags by UsersNotations for Margin Notes

Wednesday, August 21, 13

19

DATA TIER DECISIONS

Wednesday, August 21, 13

20

Where to Store the DataHow to Organize the DataHow to Manage UsersAPI Strategy

Wednesday, August 21, 13

21

Where to Store the DataHow to Organize the DataHow to Manage UsersAPI Strategy

Wednesday, August 21, 13

Data Schema

•articles• url

• text

• tags-index

• notations

• location

• note

• votes

• created_at-index

• etag

• parsed_at

•users• username-unique index

• email

• name

• hash (md5)

22

•subscribers• uuid4-user

• uuid4-bookmark

• subscribed_at

Wednesday, August 21, 13

23

Where to Store the DataHow to Organize the DataHow to Manage UsersAPI Strategy

Wednesday, August 21, 13

24

Where to Store the DataHow to Organize the DataHow to Manage UsersAPI Strategy

Wednesday, August 21, 13

25

APPLICATION TIER

Wednesday, August 21, 13

26

APPLICATION TIER ACTION!

Wednesday, August 21, 13

27

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS & ARTICLES

Wednesday, August 21, 13

28

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS & ARTICLES

Wednesday, August 21, 13

29

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS & ARTICLES

Wednesday, August 21, 13

30

APPLICATION TIER DECISIONS

Wednesday, August 21, 13

31

HOW IS DATA PROCESSED?WHERE IS DATA PROCESSED?

Wednesday, August 21, 13

32

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS & ARTICLES

Wednesday, August 21, 13

33

A LOT OF WORK HERE...

Wednesday, August 21, 13

34

SPLIT THE APPLICATION TIER!

Wednesday, August 21, 13

34

SPLIT THE APPLICATION TIER!

BLEND SPREAD

Wednesday, August 21, 13

35

ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

BLEND

SPREAD

Wednesday, August 21, 13

36

HOW DOES ONE SPLIT THE APPLICATION TIER?

Wednesday, August 21, 13

37

MESSAGE QUEUE, BUT WHICH?

Wednesday, August 21, 13

38

NOW THAT WE HAVE THAT FIGURED OUT

Wednesday, August 21, 13

39

BLEND

SPREAD

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES

RabbitMQ

Wednesday, August 21, 13

40

BLEND

SEVEN CONSUMER/WORKER SERVICES

Create  Ar(cleUpdate  References  Sani(ze  HTMLCreate  UserUpdate  UserPassword  E-­‐mailPassword  Change

Wednesday, August 21, 13

41

BLEND

SPREAD

CRUD OPERATIONS FOR USERSCRUD OPERATIONS FOR ARTICLES

ADVANCED OPERATIONS FOR USERS ADVANCED OPERATIONS FOR ARTICLES

RabbitMQ

Wednesday, August 21, 13

42

WEB TIER

Wednesday, August 21, 13

43

WHAT ARE WE SERVING OUR CUSTOMERS?

Wednesday, August 21, 13

44

WEB PORTAL FOR...

Wednesday, August 21, 13

44

WEB PORTAL FOR...

Logins

Wednesday, August 21, 13

44

WEB PORTAL FOR...

LoginsPassword Resets

Wednesday, August 21, 13

44

WEB PORTAL FOR...

LoginsPassword ResetsViewing and Tagging of Articles 

Wednesday, August 21, 13

44

WEB PORTAL FOR...

LoginsPassword ResetsViewing and Tagging of Articles Submission of Articles

Wednesday, August 21, 13

45

WEB TIER DECISIONS

Wednesday, August 21, 13

46

OPERATING SYSTEM / STACKDATASTORE SOLUTION

LOAD BALANCINGDNS

CONTENT DELIVERY

Wednesday, August 21, 13

47

Ubuntu 12.04python; python-flask

nginx in front of uWSGI

Wednesday, August 21, 13

48

OPERATING SYSTEM / STACKDATASTORE SOLUTION

LOAD BALANCINGDNS

CONTENT DELIVERY

Wednesday, August 21, 13

49

CloudFiles as Key/Value StoreObjectRocket as Document Store

Exceptional.io Redis2Go as Key/Value Store

Wednesday, August 21, 13

50

CloudFiles as Key/Value StoreObjectRocket as Document Store

Exceptional.io Redis2Go as Key/Value Store

Wednesday, August 21, 13

51

CloudFiles as Key/Value StoreObjectRocket as DocumentStore

Exceptional.io Redis2Go as Key/Value Store

Wednesday, August 21, 13

52

OPERATING SYSTEM / STACKDATASTORE SOLUTION

LOAD BALANCINGDNS

CONTENT DELIVERY

Wednesday, August 21, 13

53

RACKSPACE

Wednesday, August 21, 13

54

OPERATING SYSTEM / STACKDATASTORE SOLUTION

LOAD BALANCINGDNS

CONTENT DELIVERY

Wednesday, August 21, 13

55

Wednesday, August 21, 13

56

OPERATING SYSTEM / STACKDATASTORE SOLUTION

LOAD BALANCINGDNS

CONTENT DELIVERY

Wednesday, August 21, 13

57

Tinge to CloudFilesSEO Management

Wednesday, August 21, 13

58

Tinge to CloudFilesSEO Management

Wednesday, August 21, 13

59

CANONICAL ARCHITECTUREWITH PILLARS

Wednesday, August 21, 13

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE

Wednesday, August 21, 13

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE Parallel

Modular

Horizontally scalable

Agile

Secure

Wednesday, August 21, 13

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE Parallel

Modular

Horizontally scalable

Agile

Secure

Wednesday, August 21, 13

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE Parallel

Modular

Horizontally scalable

Agile

Secure

Wednesday, August 21, 13

LB

FRONT END API

MQ

CONSUMER

LB

KEY/VALUESTORE

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

OBJECTSTORE Parallel

Modular

Horizontally scalable

Agile

Secure

Wednesday, August 21, 13

KEY/VALUESTORE

OBJECTSTORE

LB

FRONT END API

MQ

CONSUMER

LB

DOCUMENTSTORE

INTERNET

Routing

•Topic

•users

•articles

•Fanout

•article.sanitize

•article.references

Consumers

•create user

•password user

•article reference

•email user

•create article

•article sanitize

Parallel

Modular

Horizontally scalable

Agile

Secure

Wednesday, August 21, 13

66

https://github.com/raxsavvy/margarine

Contribute

Wednesday, August 21, 13