![Page 1: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/1.jpg)
Choosing Content for Netflix:
How Data Leads The WayPaul Ellwood
VP, Data Engineering & Analytics
![Page 2: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/2.jpg)
Today’s TalkEvolution of NetflixData@NetflixChoosing Content for Netflix
![Page 3: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/3.jpg)
Evolution of NetflixA Brief History Lesson
![Page 4: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/4.jpg)
DVD Service1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 5: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/5.jpg)
Streaming Service1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 6: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/6.jpg)
1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 7: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/7.jpg)
Netflix Originals1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
2012 2013 2014 2015 2016
+ 3 more + 2 more + 20 more + 30 or so more
![Page 8: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/8.jpg)
Netflix Originals1999 DVD Service2007 Streaming Service2012 Netflix Originals2016 Netflix Global
![Page 9: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/9.jpg)
Netflix Studios1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 10: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/10.jpg)
Local Originals1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 11: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/11.jpg)
#netflixeverywhere1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 12: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/12.jpg)
Netflix in Hollywood1999 DVD Service2007 Streaming Service2012 Netflix Originals 2016 Netflix Global
![Page 13: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/13.jpg)
Data@Netflix
![Page 14: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/14.jpg)
Big Data
>40PB700
B
![Page 15: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/15.jpg)
Experimentation, Not Opinion
![Page 16: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/16.jpg)
2013
2015
User Experience
![Page 17: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/17.jpg)
BillboardEvidenceImageryRow / Content SelectionSearch Results… and more
Recommender Systems
The Netflix Recommender System: Algorithms, Business Value, and Innovation
![Page 18: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/18.jpg)
http://techblog.netflix.com/2016/05/selecting-best-artwork-for-videos.html
Cover Art Optimization
![Page 19: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/19.jpg)
https://media.netflix.com/en/company-blog/how-netflix-works-with-isps-around-the-globe-to-deliver-a-great-viewing-experience
Content Delivery
![Page 20: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/20.jpg)
http://techblog.netflix.com/2015/12/optimizing-content-quality-control-at-netflix-predictive-modeling.html
Quality Control
![Page 21: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/21.jpg)
But What About Choosing Content?
![Page 22: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/22.jpg)
Can a computer write a script that would win a competition? [benjamin.wtf]
Can Data Science Create Content?
![Page 23: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/23.jpg)
Not So Well (Yet)...
![Page 24: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/24.jpg)
Is Netflix Doing This?
No. It’s the opposite.
We give creative freedom to the creatives.
![Page 25: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/25.jpg)
But Can Data Help in Choosing Content?
Yes.All decisions are made by experienced
creatives, but analytic products can help.
![Page 26: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/26.jpg)
Value Attribution On a Subscription Service
![Page 27: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/27.jpg)
Valuing Content on Linear TV
![Page 28: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/28.jpg)
How Should Netflix Value Content?
Many possibilities…● Hours viewed?● Completed watching?● New subscribers?● Award nominations? Wins?● Buzz?Or All Of The Above?
![Page 29: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/29.jpg)
Content Efficiency = value / cost
Applying Netflix’s Notion of Value
![Page 30: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/30.jpg)
Value PredictionFor Licensed Content
![Page 31: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/31.jpg)
Predict Value For Licensed Content
Available Titles
DemandFeature
s
DemandPredictiv
eModel
Adjust If efficient, license
![Page 32: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/32.jpg)
Predicting Value For Licensed Content
E.g.● Past performance on Netflix (if previously
licensed)● Past performance of similar titles on Netflix● Broadcast ratings● Box Office● Talent involved● Critic & User Reviews● Awards
Available Titles
DemandFeature
s
DemandPredictiv
eModel
Adjust If efficient, license
![Page 33: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/33.jpg)
Predicting Value For Licensed Content
Available Titles
DemandFeature
s
DemandPredictiv
eModel
Adjust If efficient, license
![Page 34: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/34.jpg)
Predicting Value For Licensed Content
E.g.● Buyer judgements● Deal term adjustments
Available Titles
DemandFeature
s
DemandPredictiv
eModel
Adjust If efficient, license
![Page 35: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/35.jpg)
Predicting Value For Licensed Content
Available Titles
DemandFeature
s
DemandPredictiv
eModel
Adjust If efficient, license
![Page 36: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/36.jpg)
Value PredictionFor Original Content
![Page 37: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/37.jpg)
Predicting Value for Original Content
Originals is a more difficult problem
● Less data - no box office or reviews
● Moving target - ideas and scripts can evolve
● Fungible - execution varies with talent and budget
![Page 38: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/38.jpg)
‘House of Cards’ script = X
Those members also watch:
Finding Comparable Titles (that we know more about)
![Page 39: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/39.jpg)
Catalog Optimization
![Page 40: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/40.jpg)
Programming Linear TVDemographic Based
![Page 41: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/41.jpg)
Programming Netflix
Something for Everyone
![Page 42: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/42.jpg)
Global Complexity
![Page 43: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/43.jpg)
What’s Next?
![Page 44: Choosing Content for Netflix: How Data Leads the Way](https://reader035.vdocuments.us/reader035/viewer/2022062503/586e731f1a28ab99598b537d/html5/thumbnails/44.jpg)
Upcoming Challenges
•Choose the best mix of content globally•Identify valuable content earlier•Better analytic tools for buyers & creatives•Understand marketplace dynamics