how “stranger things” can happen with visual analytics · how “stranger things” can happen...
TRANSCRIPT
![Page 1: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/1.jpg)
How “Stranger Things” can happen with Visual Analytics
Jason FlittnerSenior Analytics Engineer / ManagerNetflix - Content Data Engineering and Analytics
#NetflixData
![Page 2: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/2.jpg)
● About Netflix
● Tableau + Big Data
○ Lessons Learned
○ Where we are today
● Analytics and Iterating Quickly
![Page 3: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/3.jpg)
What is Netflix?
![Page 4: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/4.jpg)
![Page 5: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/5.jpg)
![Page 6: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/6.jpg)
● 93+ million members
● 190 countries
● 1,000+ devices
● 10B hours/qtr
We plan on spending ~$6B in 2017 on content for our members
Metrics
![Page 7: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/7.jpg)
● ~60 PB DW on S3
● ~1400 Tableau users
● Live & extract connections
● Analytics on billions of rows
![Page 8: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/8.jpg)
(Hadoop clusters)
Storage Compute Data Interface Data Access, Analytics and Visualization
AWS S3
![Page 9: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/9.jpg)
● About Netflix
● Tableau + Big Data
○ Lessons Learned
○ Where we are today
● Analytics and Iterating Quickly
![Page 10: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/10.jpg)
Choosing a source
● Hive
● Spark
● Presto
● Redshift
● Published Data Source
● etc...
![Page 11: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/11.jpg)
● Powerful and scalable backend
● “Slower” 1,000,000,000/hr
● Hive + Tableau
○ Thrift Servers
○ Custom SQL vs Tables
○ Metadata
○ ODBC Optimization
![Page 12: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/12.jpg)
● Scalable
● Faster than Hive in many cases
● Spark + Tableau
○ Thrift Servers
○ Long running job on Cluster
○ Query reliability
![Page 13: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/13.jpg)
● Fast query engine
● Great for experimenting and
“smaller” data sets
● Connecting to Tableau
○ Web data connector
○ ODBC
![Page 14: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/14.jpg)
● About Netflix
● Tableau + Big Data
○ Lessons Learned
○ Where we are today
● Analytics and Iterating Quickly
![Page 15: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/15.jpg)
Tableau Data Extract Publish to Server
Tableau Extract API
![Page 16: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/16.jpg)
Create Tableau Data ExtractProvision Container ResourceIssues Command Create Extract
Publish to Server
Distributed Tableau Extract API
![Page 17: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/17.jpg)
● Very fast loads from S3
● Native Tableau connector
● Quick Tableau Iteration
● Live or Extract
● Concurrency
Amazon Redshift
![Page 18: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/18.jpg)
BIG Data● Too big to extract?
● Optimized live connections
○ SQL
● Custom data viz with Druid
● Tableau + Hyper!?
![Page 19: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/19.jpg)
● About Netflix
● Tableau + Big Data
○ Lessons Learned
○ Where we are today
● Analytics and Iterating Quickly
![Page 20: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/20.jpg)
Business users
Analytics Engineer
Analytics:
● Binge Analysis
● Viewing Patterns
● Hours Viewed
● Customer Joy
● Content Quality
![Page 21: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/21.jpg)
Bringing it all together
● Content analytics
● Iterate quickly
● Move between backend sources
● Strong user adoption
![Page 22: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/22.jpg)
![Page 23: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/23.jpg)
![Page 24: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/24.jpg)
![Page 25: How “Stranger Things” can happen with Visual Analytics · How “Stranger Things” can happen with Visual Analytics Jason Flittner Senior Analytics Engineer / Manager Netflix](https://reader033.vdocuments.us/reader033/viewer/2022042302/5ecd80942fb9bf0104494a9a/html5/thumbnails/25.jpg)
Merci
Thank you
Jason Flittner -