social apps & social scalability

124
Matias Paterlini Altodot Social Apps And Social Scalability We’re hiring! Tuesday, December 6, 11

Post on 12-Sep-2014

892 views

Category:

Technology


1 download

DESCRIPTION

Presentation I made in the Mar del Plata Developers Conference in 2011 about social apps, and social scalability.

TRANSCRIPT

Page 1: Social apps & social scalability

Matias PaterliniAltodot

Social AppsAnd Social Scalability

We’re hiring!

Tuesday, December 6, 11

Page 2: Social apps & social scalability

<?php echo “Hola Mundo!!!”; ?>

Tuesday, December 6, 11

Page 3: Social apps & social scalability

about me...

Tuesday, December 6, 11

Page 4: Social apps & social scalability

• CTO & Co-founder at Altodot (creators of The Fan Machine)

• Former CTO & Co-founder at Virtualinmobiliario.com

• Former CTO & Co-founder at Xristianos Producciones

• Jóven Empresario Bonaerense 2008

• Head at Emprendedores Cristianos

Tuesday, December 6, 11

Page 5: Social apps & social scalability

Tuesday, December 6, 11

Page 6: Social apps & social scalability

Tuesday, December 6, 11

Page 7: Social apps & social scalability

Tuesday, December 6, 11

Page 8: Social apps & social scalability

Tuesday, December 6, 11

Page 9: Social apps & social scalability

Tuesday, December 6, 11

Page 10: Social apps & social scalability

What we do

Tuesday, December 6, 11

Page 11: Social apps & social scalability

Social Apps!!!

Tuesday, December 6, 11

Page 12: Social apps & social scalability

What???

Tuesday, December 6, 11

Page 13: Social apps & social scalability

Web or desktop applications connected

to Social Networks

Tuesday, December 6, 11

Page 14: Social apps & social scalability

Which social Networks?

Tuesday, December 6, 11

Page 15: Social apps & social scalability

Foursquare

Twitter

Linkedin

Sonico

Orkut

Hi5Google

Facebook

Google +

Tuesday, December 6, 11

Page 16: Social apps & social scalability

Most Annoying API award?

Tuesday, December 6, 11

Page 17: Social apps & social scalability

Tuesday, December 6, 11

Page 18: Social apps & social scalability

Some Facts...

Tuesday, December 6, 11

Page 19: Social apps & social scalability

Twitter

Tuesday, December 6, 11

Page 20: Social apps & social scalability

• 500K users in Argentina

• 0,56 % of users produce 50% of the content.

• The majority uses SmartPhones

• Users with more than 10 friends: 60%

• Users with more than 130 friends (media at Facebook), 20%.

Tuesday, December 6, 11

Page 21: Social apps & social scalability

Open Social

Tuesday, December 6, 11

Page 22: Social apps & social scalability

• 1,6 Billion Users world wide (several times duplicated people)

• 100x “Social networks” implemented

• Orkut still leading at Brazil

• Just launched Open Social 2

Tuesday, December 6, 11

Page 23: Social apps & social scalability

Facebook

Tuesday, December 6, 11

Page 24: Social apps & social scalability

• 750+ MM Users

• +18 Million in Argentina (85% penetration)

• +1 M Active Users in Argentina in May

• 50% of active users login every day

• average user has 130 friends

• Billion dollars business

Tuesday, December 6, 11

Page 25: Social apps & social scalability

• devs from 190 countries

• 20 million installs every day

• 250 million uses FB Connect every day

• 10K websites integrated every day

• 900 million open graph objects

• 70 translations

Tuesday, December 6, 11

Page 26: Social apps & social scalability

What can we do?

Tuesday, December 6, 11

Page 27: Social apps & social scalability

Twitter

Tuesday, December 6, 11

Page 28: Social apps & social scalability

Real Stream API

Search API

REST API

Tuesday, December 6, 11

Page 29: Social apps & social scalability

Any limits?

Tuesday, December 6, 11

Page 30: Social apps & social scalability

350 Authorized Requests per hour

Tuesday, December 6, 11

Page 31: Social apps & social scalability

Deal with it...

Tuesday, December 6, 11

Page 32: Social apps & social scalability

Some Samples

Tuesday, December 6, 11

Page 33: Social apps & social scalability

Tuesday, December 6, 11

Page 34: Social apps & social scalability

Tuesday, December 6, 11

Page 35: Social apps & social scalability

Tuesday, December 6, 11

Page 36: Social apps & social scalability

Tuesday, December 6, 11

Page 37: Social apps & social scalability

Tuesday, December 6, 11

Page 38: Social apps & social scalability

THE Social Network

Tuesday, December 6, 11

Page 39: Social apps & social scalability

Facebook

Tuesday, December 6, 11

Page 40: Social apps & social scalability

Social Graph

Tuesday, December 6, 11

Page 41: Social apps & social scalability

Tuesday, December 6, 11

Page 42: Social apps & social scalability

THE Platform

Tuesday, December 6, 11

Page 43: Social apps & social scalability

REST API

Graph API explorer

Javascript SDK (FB Connect)

Social Plugins

ADS API

XFBML

Graph APIFacebook Credits

Places API

Tuesday, December 6, 11

Page 44: Social apps & social scalability

THE Protocol

Tuesday, December 6, 11

Page 45: Social apps & social scalability

Open Graph

Tuesday, December 6, 11

Page 46: Social apps & social scalability

Tuesday, December 6, 11

Page 47: Social apps & social scalability

Graph API

Tuesday, December 6, 11

Page 48: Social apps & social scalability

Objects Reference

Tuesday, December 6, 11

Page 49: Social apps & social scalability

Objects Handling:

POST, GET, DELETE

Tuesday, December 6, 11

Page 50: Social apps & social scalability

Public info, private info

Tuesday, December 6, 11

Page 51: Social apps & social scalability

Objects Connections

Tuesday, December 6, 11

Page 52: Social apps & social scalability

Tuesday, December 6, 11

Page 53: Social apps & social scalability

Tuesday, December 6, 11

Page 54: Social apps & social scalability

Tuesday, December 6, 11

Page 55: Social apps & social scalability

Tuesday, December 6, 11

Page 56: Social apps & social scalability

REST Api

Tuesday, December 6, 11

Page 57: Social apps & social scalability

Javascript SDK

Tuesday, December 6, 11

Page 58: Social apps & social scalability

Ads API

Tuesday, December 6, 11

Page 59: Social apps & social scalability

Social Channels

Tuesday, December 6, 11

Page 60: Social apps & social scalability

Canvas Page

Tuesday, December 6, 11

Page 61: Social apps & social scalability

Tuesday, December 6, 11

Page 62: Social apps & social scalability

Tuesday, December 6, 11

Page 63: Social apps & social scalability

Tuesday, December 6, 11

Page 64: Social apps & social scalability

Tuesday, December 6, 11

Page 65: Social apps & social scalability

FanPage Tabs (apps)

Tuesday, December 6, 11

Page 66: Social apps & social scalability

Tuesday, December 6, 11

Page 67: Social apps & social scalability

Tuesday, December 6, 11

Page 68: Social apps & social scalability

Tuesday, December 6, 11

Page 69: Social apps & social scalability

Tuesday, December 6, 11

Page 70: Social apps & social scalability

Feed Stories on Walls

Tuesday, December 6, 11

Page 71: Social apps & social scalability

Tuesday, December 6, 11

Page 72: Social apps & social scalability

Tuesday, December 6, 11

Page 73: Social apps & social scalability

Tuesday, December 6, 11

Page 74: Social apps & social scalability

Facebook Connect

Tuesday, December 6, 11

Page 75: Social apps & social scalability

Tuesday, December 6, 11

Page 76: Social apps & social scalability

Otras ideas...

Tuesday, December 6, 11

Page 77: Social apps & social scalability

Tuesday, December 6, 11

Page 78: Social apps & social scalability

Tuesday, December 6, 11

Page 79: Social apps & social scalability

Tuesday, December 6, 11

Page 80: Social apps & social scalability

Tuesday, December 6, 11

Page 81: Social apps & social scalability

Tuesday, December 6, 11

Page 82: Social apps & social scalability

Social Plugins

Tuesday, December 6, 11

Page 83: Social apps & social scalability

Tuesday, December 6, 11

Page 84: Social apps & social scalability

Tuesday, December 6, 11

Page 85: Social apps & social scalability

Like Box

Tuesday, December 6, 11

Page 86: Social apps & social scalability

Tuesday, December 6, 11

Page 87: Social apps & social scalability

Like and Send Button

Tuesday, December 6, 11

Page 88: Social apps & social scalability

Tuesday, December 6, 11

Page 89: Social apps & social scalability

Tuesday, December 6, 11

Page 90: Social apps & social scalability

Live Stream

Tuesday, December 6, 11

Page 91: Social apps & social scalability

Tuesday, December 6, 11

Page 92: Social apps & social scalability

Comments Box

Tuesday, December 6, 11

Page 93: Social apps & social scalability

Tuesday, December 6, 11

Page 94: Social apps & social scalability

Tuesday, December 6, 11

Page 95: Social apps & social scalability

Social Scalability

Tuesday, December 6, 11

Page 96: Social apps & social scalability

The Fan MachineScalability

Tuesday, December 6, 11

Page 97: Social apps & social scalability

The Friendfeed approach

Tuesday, December 6, 11

Page 98: Social apps & social scalability

Horizontal Scalability

Tuesday, December 6, 11

Page 99: Social apps & social scalability

Database 1

Web Server 1 Web Server 2 Web Server 3

Static File Caching

Database 1

(replicated DB)

Data Caching

Tuesday, December 6, 11

Page 100: Social apps & social scalability

What about the DB?

Tuesday, December 6, 11

Page 101: Social apps & social scalability

MasterDatabase

Web Server 1

Web Server 2

Web Server 3

Static File Caching

SlaveDatabase

Slave Database

(Transaccional, read-write)

(read) (read)

Data Caching

Tuesday, December 6, 11

Page 102: Social apps & social scalability

DB scalling issues...

Tuesday, December 6, 11

Page 103: Social apps & social scalability

Tuesday, December 6, 11

Page 104: Social apps & social scalability

• MAAAAANY Records

Tuesday, December 6, 11

Page 105: Social apps & social scalability

• MAAAAANY Records

• Caching not enough

Tuesday, December 6, 11

Page 106: Social apps & social scalability

• MAAAAANY Records

• Caching not enough

• Alters lock tables for hours!

Tuesday, December 6, 11

Page 107: Social apps & social scalability

• MAAAAANY Records

• Caching not enough

• Alters lock tables for hours!

• Maintenance Issues

Tuesday, December 6, 11

Page 108: Social apps & social scalability

• MAAAAANY Records

• Caching not enough

• Alters lock tables for hours!

• Maintenance Issues

• Que haces si se corrompe la base porque alguien se engancha el cable de la luz con el pié???

Tuesday, December 6, 11

Page 109: Social apps & social scalability

Possible solutions...

Tuesday, December 6, 11

Page 110: Social apps & social scalability

CassandraMongoDB

Hipertable

NoSQL SimpleDB

CouchDB

Membase

BigData

RaptorDB

Hbase

Tuesday, December 6, 11

Page 111: Social apps & social scalability

NoSQL = No Trust!

Tuesday, December 6, 11

Page 112: Social apps & social scalability

THE approach

Tuesday, December 6, 11

Page 113: Social apps & social scalability

MySQL

Tuesday, December 6, 11

Page 114: Social apps & social scalability

Sharded MySQL databases

Tuesday, December 6, 11

Page 115: Social apps & social scalability

MySQL emulating NoSQL over Sharded

Databases

Tuesday, December 6, 11

Page 116: Social apps & social scalability

If( MySQL == NoSQL) { die(‘WTF???’);}

Tuesday, December 6, 11

Page 117: Social apps & social scalability

Documents = Entities

Tuesday, December 6, 11

Page 118: Social apps & social scalability

One Table for entities, many tables for

indexes.

Tuesday, December 6, 11

Page 119: Social apps & social scalability

How does it look like?

Tuesday, December 6, 11

Page 120: Social apps & social scalability

Databases Server 1

Web Server 2

Shard 1Shard 2Shard 3

Shard 4Shard 5Shard 6

Shard 7Shard 8Shard 9

Shard 10Shard 11Shard 12

Web Server 1

Web Server 3

File Caching

Database Backups

Data Caching

} Databases

Tuesday, December 6, 11

Page 121: Social apps & social scalability

DB 1

Shard 1Shard 2Shard 3

Web Server 1

Web Server 2

Web Server 3

Web Server 3

Shard 3Shard 4Shard 5

DB 1

Shard 6Shard 7Shard 8

DB 1

Shard 9Shard 10Shard 11

DB 1

Static File Caching

Database Backups per Shard

Tuesday, December 6, 11

Page 122: Social apps & social scalability

THE Benefits...

Tuesday, December 6, 11

Page 123: Social apps & social scalability

• Server maintenance per shard

• Data Maintenance Per shard

• NO table alters

• Easy recovery

• Easy scalability

• Easy migration to other No-SQL data engines

Tuesday, December 6, 11

Page 124: Social apps & social scalability

Questions?

Matias [email protected]@paterlinimatias

We’re hiring!!!Tuesday, December 6, 11