data @ nflx (tableau conference 2014 presentation)
DESCRIPTION
I presented this at a 2014 Tableau Conference session with Albert Wong. Netflix relies on data to make decisions ranging from buying and recommending content, to improving the streaming experience on devices. This presentation shares our Big Data analytics architecture and the tools used to make data accessible throughout our business, focusing on how Tableau fits into our organization and why it aligns well with our culture.TRANSCRIPT
![Page 1: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/1.jpg)
DATA @ NFLXBuilding a Culture of Analytics EverywhereTableau Customer Conference2014.09.09
Blake IrvineManager, Device AnalyticsData Science & [email protected]
Albert WongManager, Reporting PlatformsCloud & Platform [email protected]
![Page 2: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/2.jpg)
![Page 3: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/3.jpg)
![Page 4: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/4.jpg)
![Page 5: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/5.jpg)
Netflix and data in the news...“Giving Viewers What They Want” --New York Times
“The Science Behind the Netflix Algorithms That Decide What You’ll Watch Next” --Wired
Data-Mining Boosts Netflix's Subscriber Base, Showbiz Clout
--AdAge
![Page 6: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/6.jpg)
![Page 7: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/7.jpg)
BIGDATA
![Page 8: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/8.jpg)
Big Data at Netflix
Size● 50+ million members● 1000’s of devices● 100’s of systems● >300B data pipeline events daily● >10B row tables daily
Ubiquitous● Data is everywhere● Many complex systems● Many engineering teams
producing and consuming● Non-streaming teams produce
and consume data● Culturally data driven
![Page 9: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/9.jpg)
How do we innovate with Big Data?
![Page 10: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/10.jpg)
TOOLSCULTURE
![Page 11: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/11.jpg)
Tools
Sting
DATA STORAGE DATA PROCESSORS DATABASE REPORTING
![Page 12: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/12.jpg)
Team StructureData Science and Engineering
Marketing
analyst
reporting
data engineering
Finance Product Engineering ...
Business Functions
![Page 13: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/13.jpg)
Team StructureData Science and Engineering
Marketing
analyst
reporting
data engineering
Finance Product Engineering ...
NOT WHAT WE DO
![Page 14: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/14.jpg)
Netflix Team StructureData Science and Engineering
Marketing
analyst
reporting
data engineering
Finance Product Engineering ...
analyst
reporting
data engineering
analyst
reporting
data engineering
analyst
reporting
data engineering
Business Functions
![Page 15: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/15.jpg)
Highly Aligned, Loosely Coupled
Finance
analyst
reporting
data engineering
Marketing
analyst/reporting
data architect
data engineering
![Page 16: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/16.jpg)
Highly Aligned, Loosely Coupled
Finance
analyst
reporting
data engineering
Marketing
analyst/reporting
data architect
data engineering
![Page 17: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/17.jpg)
Freedom & Responsibility
FreedomDon’t limit accessDon’t limit choicesReduce constraints
ResponsibilityTrustDon’t allow chaosReduce accidents
![Page 18: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/18.jpg)
Protected access
CENTRAL DATA
![Page 19: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/19.jpg)
Protected access
CENTRAL DATA
NOT WHAT WE DO
![Page 20: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/20.jpg)
Unlocked access
CENTRAL DATA
![Page 21: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/21.jpg)
OPERATIONAL DATA
Don’t limit choice
CENTRAL DATA LOCAL DATA
![Page 22: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/22.jpg)
OPERATIONAL DATA
Don’t limit choice
CENTRAL DATA LOCAL DATA
Sting
![Page 23: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/23.jpg)
Why is Tableau a good choice?
QuickIntuitiveRich Visual AnalysisStorytellingEmailed ReportingReusability
![Page 24: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/24.jpg)
Who uses Tableau?
![Page 25: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/25.jpg)
How do we use Tableau?
![Page 26: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/26.jpg)
Examples
● Application build testing● Certification tracking● Operational Excellence
![Page 27: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/27.jpg)
Application Build Testing (1/4)
● Team: Product Engineering● Context
○ Application automatically tested at every code checkin
○ Several dozen performance tests run to measure change and avoid regression
● Problem○ Limited graphing tool built into test tool○ Difficult / no customization
![Page 28: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/28.jpg)
Application Build Testing (2/4){ "metadata" : {
"TestCaseName" : "Trunk.Rendering.Effects_Mask2","MarkerSetId" : 2472165,"ESN" : "DCQA01","UIBuild" : null,"Build" : "2689","JenkinsJob" : "http://builds.netflix.com/job/208/","Label" : "#2689 / 208","BuildTimestamp" : null,"Changelist" : "2177893"
}, "results" :
[{"MeanFps" : 60.284862537264004},{"MeanFps" : 60.264900662251655},{"MeanFps" : 60.234541577825162}]
}
Build / Test
![Page 29: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/29.jpg)
Application Build Testing (3/4)
![Page 30: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/30.jpg)
Application Build Testing (4/4)
![Page 31: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/31.jpg)
Certification Tracking (1/3)
Team: Certification OperationsContext● We certify the Netflix implementation on many new
consumer electronics devices
Problem● Time consuming to generate insights across multiple
disconnected systems
![Page 32: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/32.jpg)
Certification Tracking (2/3)
NTSCertification
Process
![Page 33: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/33.jpg)
Certification Tracking (3/3)
![Page 34: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/34.jpg)
Operational Excellence (1/3)
Team: Data Science & EngineeringContext● Ensure continuous development does not negatively
impact availability and resilience
Problem● Multiple programs and data sources● Need to link source data patterns to engineering tools
![Page 35: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/35.jpg)
Operational Excellence (2/3)
![Page 36: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/36.jpg)
Operational Excellence (3/3)
![Page 37: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/37.jpg)
Where are we with Tableau?
![Page 38: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/38.jpg)
● Netflix is known for being data driven● Big data is available everywhere● Our culture enables analysis everywhere
● Tableau complements our culture● We have organic growth throughout Netflix● Growing part of our reporting platform
DATA @ NFLX
![Page 39: DATA @ NFLX (Tableau Conference 2014 Presentation)](https://reader034.vdocuments.us/reader034/viewer/2022052316/558c0ea1d8b42aa8338b4695/html5/thumbnails/39.jpg)
Blake IrvineManager, Device AnalyticsData Science & [email protected]
Albert WongManager, Reporting PlatformsCloud & Platform [email protected]
What can we answer?