facebook, robert johnson
DESCRIPTION
TRANSCRIPT
![Page 1: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/1.jpg)
![Page 2: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/2.jpg)
!"" Million Active Users
!M
"!M
#$!M
#%!M
$&!M
'!!M
#""$ #""! #""% #""& #""' #""(
!""M
![Page 3: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/3.jpg)
## property on the Internet (time on site)
)""s of billions of monthly page views
>$ trillion feed actions processed per day
)""s of millions of cache queries per second
Over # trillion objects cached
Over !"" million active users, half log in every day
)"" billion photo files stored
Over #" billion minutes spent every Day
Over # billion pieces of content uploaded every week
![Page 4: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/4.jpg)
Move FastServer ScalingReliability
![Page 5: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/5.jpg)
!"" Million Active Users
!M
"!M
#$!M
#%!M
$&!M
'!!M
#""$ #""! #""% #""& #""' #""(
!""M
![Page 6: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/6.jpg)
Over ) million active users per engineer
!
"#!,!!!
#$!,!!!
%"!,!!!
&'!,!!!
(,"!!,!!!
Facebook Google* Amazon* Microsoft*
&!,"""$%,"""#$","""
#,#"","""
* Conservative estimates based on publicly available data
![Page 7: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/7.jpg)
Few external deadlines... but
![Page 8: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/8.jpg)
The site can’t go down
![Page 9: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/9.jpg)
Frequent small changesNever a delay waiting for a pushEasier to isolate bugs
Major changes dark launched
![Page 10: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/10.jpg)
![Page 11: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/11.jpg)
![Page 12: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/12.jpg)
Traditional websites
Bob
Bob’s data
![Page 13: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/13.jpg)
Traditional websites
Bob
Bob’s data
Julie
Julie’s data
Dan
Dan’s data
Beth
Beth’s data
Sue
Sue’s data
Erin
Erin’s data
![Page 14: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/14.jpg)
Facebook the data is interconnectedBob ErinBeth
Servers
![Page 15: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/15.jpg)
![Page 16: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/16.jpg)
![Page 17: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/17.jpg)
Scale Horizontally
![Page 18: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/18.jpg)
Database
Memcache
Web Server
![Page 19: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/19.jpg)
Database
Memcache
Web Server
![Page 20: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/20.jpg)
Database
Memcache
Web Server
![Page 21: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/21.jpg)
Database
Memcache
Web Server
![Page 22: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/22.jpg)
Network Incast
Memcache Memcache Memcache Memcache
Switch
PHP Client
![Page 23: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/23.jpg)
Network Incast
Many SmallGet Requests
Memcache Memcache Memcache Memcache
Switch
PHP Client
![Page 24: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/24.jpg)
Memcache Memcache Memcache Memcache
Switch
PHP Client
Many bigdata packets
Network Incast
![Page 25: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/25.jpg)
Memcache Memcache Memcache Memcache
Switch
PHP Client
Network Incast
![Page 26: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/26.jpg)
Memcache Memcache Memcache Memcache
Switch
PHP Client
Network Incast
![Page 27: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/27.jpg)
Memcache Memcache Memcache Memcache
Switch
PHP Client
Network Incast
![Page 28: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/28.jpg)
Reliability
![Page 29: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/29.jpg)
Single Points of Failure
![Page 30: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/30.jpg)
Single Points of FailureSoftware can be a SPOF
![Page 31: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/31.jpg)
Don’t make small problems big
![Page 32: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/32.jpg)
Don’t make small problems bigDon’t push problems upstream
![Page 33: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/33.jpg)
Don’t make small problems bigDon’t push problems upstreamBe wary of “smart” failover
![Page 34: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/34.jpg)
Don’t make small problems bigDon’t push problems upstreamBe wary of “smart” failoverShed load when you’re in trouble
![Page 35: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/35.jpg)
If you lose half of your machinesyou’re doing well if you’re serving half of your traffic
![Page 36: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/36.jpg)
Measure Everything
![Page 37: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/37.jpg)
p(! vs p((
![Page 38: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/38.jpg)
![Page 39: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/39.jpg)
CultureAlways do a post-mortemRelease oftenControl and Responsibility
![Page 40: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/40.jpg)
Facebook Platform
![Page 41: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/41.jpg)
Lessons LearnedFederate everythingKeep failures containedMeasure distributionsUnderstand every problemMake a person responsible
![Page 42: Facebook, Robert Johnson](https://reader034.vdocuments.us/reader034/viewer/2022051514/54922e38ac79594e288b46d7/html5/thumbnails/42.jpg)
(c) $!!( Facebook, Inc. or its licensors. )"Facebook" is a registered trademark of Facebook, Inc.. All rights reserved. #.!