social apps & social scalability
Post on 12-Sep-2014
892 views
DESCRIPTION
Presentation I made in the Mar del Plata Developers Conference in 2011 about social apps, and social scalability.TRANSCRIPT
Matias PaterliniAltodot
Social AppsAnd Social Scalability
We’re hiring!
Tuesday, December 6, 11
<?php echo “Hola Mundo!!!”; ?>
Tuesday, December 6, 11
about me...
Tuesday, December 6, 11
• CTO & Co-founder at Altodot (creators of The Fan Machine)
• Former CTO & Co-founder at Virtualinmobiliario.com
• Former CTO & Co-founder at Xristianos Producciones
• Jóven Empresario Bonaerense 2008
• Head at Emprendedores Cristianos
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
What we do
Tuesday, December 6, 11
Social Apps!!!
Tuesday, December 6, 11
What???
Tuesday, December 6, 11
Web or desktop applications connected
to Social Networks
Tuesday, December 6, 11
Which social Networks?
Tuesday, December 6, 11
Foursquare
Sonico
Orkut
Hi5Google
Google +
Tuesday, December 6, 11
Most Annoying API award?
Tuesday, December 6, 11
Tuesday, December 6, 11
Some Facts...
Tuesday, December 6, 11
Tuesday, December 6, 11
• 500K users in Argentina
• 0,56 % of users produce 50% of the content.
• The majority uses SmartPhones
• Users with more than 10 friends: 60%
• Users with more than 130 friends (media at Facebook), 20%.
Tuesday, December 6, 11
Open Social
Tuesday, December 6, 11
• 1,6 Billion Users world wide (several times duplicated people)
• 100x “Social networks” implemented
• Orkut still leading at Brazil
• Just launched Open Social 2
Tuesday, December 6, 11
Tuesday, December 6, 11
• 750+ MM Users
• +18 Million in Argentina (85% penetration)
• +1 M Active Users in Argentina in May
• 50% of active users login every day
• average user has 130 friends
• Billion dollars business
Tuesday, December 6, 11
• devs from 190 countries
• 20 million installs every day
• 250 million uses FB Connect every day
• 10K websites integrated every day
• 900 million open graph objects
• 70 translations
Tuesday, December 6, 11
What can we do?
Tuesday, December 6, 11
Tuesday, December 6, 11
Real Stream API
Search API
REST API
Tuesday, December 6, 11
Any limits?
Tuesday, December 6, 11
350 Authorized Requests per hour
Tuesday, December 6, 11
Deal with it...
Tuesday, December 6, 11
Some Samples
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
THE Social Network
Tuesday, December 6, 11
Tuesday, December 6, 11
Social Graph
Tuesday, December 6, 11
Tuesday, December 6, 11
THE Platform
Tuesday, December 6, 11
REST API
Graph API explorer
Javascript SDK (FB Connect)
Social Plugins
ADS API
XFBML
Graph APIFacebook Credits
Places API
Tuesday, December 6, 11
THE Protocol
Tuesday, December 6, 11
Open Graph
Tuesday, December 6, 11
Tuesday, December 6, 11
Graph API
Tuesday, December 6, 11
Objects Reference
Tuesday, December 6, 11
Objects Handling:
POST, GET, DELETE
Tuesday, December 6, 11
Public info, private info
Tuesday, December 6, 11
Objects Connections
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
REST Api
Tuesday, December 6, 11
Javascript SDK
Tuesday, December 6, 11
Ads API
Tuesday, December 6, 11
Social Channels
Tuesday, December 6, 11
Canvas Page
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
FanPage Tabs (apps)
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Feed Stories on Walls
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Facebook Connect
Tuesday, December 6, 11
Tuesday, December 6, 11
Otras ideas...
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Social Plugins
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Like Box
Tuesday, December 6, 11
Tuesday, December 6, 11
Like and Send Button
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Live Stream
Tuesday, December 6, 11
Tuesday, December 6, 11
Comments Box
Tuesday, December 6, 11
Tuesday, December 6, 11
Tuesday, December 6, 11
Social Scalability
Tuesday, December 6, 11
The Fan MachineScalability
Tuesday, December 6, 11
The Friendfeed approach
Tuesday, December 6, 11
Horizontal Scalability
Tuesday, December 6, 11
Database 1
Web Server 1 Web Server 2 Web Server 3
Static File Caching
Database 1
(replicated DB)
Data Caching
Tuesday, December 6, 11
What about the DB?
Tuesday, December 6, 11
MasterDatabase
Web Server 1
Web Server 2
Web Server 3
Static File Caching
SlaveDatabase
Slave Database
(Transaccional, read-write)
(read) (read)
Data Caching
Tuesday, December 6, 11
DB scalling issues...
Tuesday, December 6, 11
Tuesday, December 6, 11
• MAAAAANY Records
Tuesday, December 6, 11
• MAAAAANY Records
• Caching not enough
Tuesday, December 6, 11
• MAAAAANY Records
• Caching not enough
• Alters lock tables for hours!
Tuesday, December 6, 11
• MAAAAANY Records
• Caching not enough
• Alters lock tables for hours!
• Maintenance Issues
Tuesday, December 6, 11
• MAAAAANY Records
• Caching not enough
• Alters lock tables for hours!
• Maintenance Issues
• Que haces si se corrompe la base porque alguien se engancha el cable de la luz con el pié???
Tuesday, December 6, 11
Possible solutions...
Tuesday, December 6, 11
CassandraMongoDB
Hipertable
NoSQL SimpleDB
CouchDB
Membase
BigData
RaptorDB
Hbase
Tuesday, December 6, 11
NoSQL = No Trust!
Tuesday, December 6, 11
THE approach
Tuesday, December 6, 11
MySQL
Tuesday, December 6, 11
Sharded MySQL databases
Tuesday, December 6, 11
MySQL emulating NoSQL over Sharded
Databases
Tuesday, December 6, 11
If( MySQL == NoSQL) { die(‘WTF???’);}
Tuesday, December 6, 11
Documents = Entities
Tuesday, December 6, 11
One Table for entities, many tables for
indexes.
Tuesday, December 6, 11
How does it look like?
Tuesday, December 6, 11
Databases Server 1
Web Server 2
Shard 1Shard 2Shard 3
Shard 4Shard 5Shard 6
Shard 7Shard 8Shard 9
Shard 10Shard 11Shard 12
Web Server 1
Web Server 3
File Caching
Database Backups
Data Caching
} Databases
Tuesday, December 6, 11
DB 1
Shard 1Shard 2Shard 3
Web Server 1
Web Server 2
Web Server 3
Web Server 3
Shard 3Shard 4Shard 5
DB 1
Shard 6Shard 7Shard 8
DB 1
Shard 9Shard 10Shard 11
DB 1
Static File Caching
Database Backups per Shard
Tuesday, December 6, 11
THE Benefits...
Tuesday, December 6, 11
• Server maintenance per shard
• Data Maintenance Per shard
• NO table alters
• Easy recovery
• Easy scalability
• Easy migration to other No-SQL data engines
Tuesday, December 6, 11