netflix viewing data microservices - how we know where ......netflix viewing data microservices -...
TRANSCRIPT
![Page 1: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/1.jpg)
How We Know Where You Are in
House of Cards
Netflix’s Viewing Data Microservices
@zimmermatt
![Page 2: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/2.jpg)
![Page 3: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/3.jpg)
![Page 4: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/4.jpg)
Netflix Scale
@zimmermatt
![Page 5: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/5.jpg)
Netflix Scale
62,000,000
@zimmermatt
![Page 6: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/6.jpg)
Netflix Scale
62,000,000
50+
@zimmermatt
![Page 7: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/7.jpg)
Netflix Scale
62,000,000
50+
3,000,000,000+
@zimmermatt
![Page 8: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/8.jpg)
Netflix Scale
62,000,000
50+
3,000,000,000+
1000+
@zimmermatt
![Page 9: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/9.jpg)
Netflix Scale
62,000,000
50+
3,000,000,000+
1000+
37%
@zimmermatt
![Page 10: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/10.jpg)
Viewing Data
@zimmermatt
![Page 11: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/11.jpg)
Viewing Data: History
Who
@zimmermatt
![Page 12: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/12.jpg)
Viewing Data: History
Who, What
@zimmermatt
![Page 13: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/13.jpg)
Viewing Data: History
Who, What, When
6/10/2015 11:55
@zimmermatt
![Page 14: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/14.jpg)
Viewing Data: History
Who, What, When, Where
6/10/2015 11:55
@zimmermatt
![Page 15: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/15.jpg)
Viewing Data: History
Who, What, When, Where, How Long
6/10/2015 11:55
@zimmermatt
![Page 16: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/16.jpg)
Viewing Data: Telemetry
@zimmermatt
![Page 17: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/17.jpg)
What have I watched?
@zimmermatt
![Page 18: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/18.jpg)
What have I watched?
@zimmermatt
![Page 19: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/19.jpg)
Where was I at?
@zimmermatt
![Page 20: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/20.jpg)
What else am I watching?
@zimmermatt
![Page 21: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/21.jpg)
What Happened?
Video Bitrate Usage
@zimmermatt
![Page 22: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/22.jpg)
![Page 23: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/23.jpg)
Viewing Data Requests
@zimmermatt
![Page 24: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/24.jpg)
Viewing Data Requests
@zimmermatt
![Page 25: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/25.jpg)
Viewing Data Requests
@zimmermatt
![Page 26: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/26.jpg)
Viewing Data Requests
@zimmermatt
![Page 27: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/27.jpg)
Viewing Data Requests
@zimmermatt
![Page 28: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/28.jpg)
![Page 29: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/29.jpg)
Gen 3 (Existing) System
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 30: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/30.jpg)
Gen 3 System’s Architecture & Scale
Logs
@zimmermatt
![Page 31: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/31.jpg)
Gen 3 System’s Architecture & Scale
Viewing History
Logs
@zimmermatt
![Page 32: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/32.jpg)
Gen 3 System’s Architecture & Scale
Viewing History
His
tory
Cac
he
Logs
@zimmermatt
![Page 33: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/33.jpg)
Gen 3 System’s Architecture & Scale
Viewing History
His
tory
Cac
he
Logs
Summarized Sessions
@zimmermatt
![Page 34: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/34.jpg)
Gen 3 System’s Architecture & Scale
Viewing History
His
tory
Cac
he
Vie
win
g Se
rvic
e
Stateful Tier
…
Stateless Tier
(History, logs,
and fallback)
Logs
Summarized Sessions
@zimmermatt
![Page 35: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/35.jpg)
Gen 3 System’s Architecture & Scale
Viewing History
His
tory
Cac
he
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Logs
Summarized Sessions
@zimmermatt
![Page 36: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/36.jpg)
![Page 37: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/37.jpg)
Growth
2015
@zimmermatt
![Page 38: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/38.jpg)
Virtuous Cycle
Improved Personalization
Better Experience
Viewing
@zimmermatt
![Page 39: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/39.jpg)
Stateful Instance Count Over Time
@zimmermatt
![Page 40: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/40.jpg)
Guiding Lights
![Page 41: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/41.jpg)
![Page 42: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/42.jpg)
![Page 43: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/43.jpg)
![Page 44: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/44.jpg)
Externalize State…
Viewing History
His
tory
Cac
he
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Logs
Summarized Sessions
@zimmermatt
![Page 45: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/45.jpg)
Externalize State… DENIED!
Viewing History
His
tory
Cac
he
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Logs
Summarized Sessions
@zimmermatt
![Page 46: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/46.jpg)
Service Decomposition
Video Bitrate Usage
@zimmermatt
![Page 47: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/47.jpg)
Service Decomposition
Start Stop
@zimmermatt
![Page 48: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/48.jpg)
Active Sessions
Last Position
Viewing History
Data Feed
Service Decomposition
Start Stop
@zimmermatt
![Page 49: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/49.jpg)
Active Sessions
Last Position
Viewing History
Data Feed
Service Decomposition
Start Stop
Event Stream
Stream State
Session Summary
@zimmermatt
![Page 50: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/50.jpg)
Service Decomposition
Collect
Process
Provide
Events
Queries
@zimmermatt
![Page 51: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/51.jpg)
Gen 3 Review
Viewing History
His
tory
Cac
he
Logs
Summarized Sessions
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 52: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/52.jpg)
Gen 4 Generic Architecture
Processor Provider Collector
@zimmermatt
![Page 53: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/53.jpg)
Gen 4 Generic Architecture
Processor Provider Collector
Queue
@zimmermatt
![Page 54: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/54.jpg)
Gen 4 Generic Architecture
Processor Provider Collector
Materialized Views
Raw Data
Queue
@zimmermatt
![Page 55: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/55.jpg)
Gen 4 Generic Architecture
Processor Provider Collector
Materialized Views
Cache
Raw Data
Queue
@zimmermatt
![Page 56: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/56.jpg)
Gen 4 Architecture
Session Summary Processor
Summarized Sessions
Materialized View
Session Events Collector
Event Data Raw Data
Log Collector
Logs Raw Data
Viewing History
Positions
Active Sessions
Materialized View
Session Accounting Provider
Playback History Provider
History Raw Data
@zimmermatt
![Page 57: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/57.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing
Edge
Se
rvic
e
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Log Collector
Logs
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 58: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/58.jpg)
Shadow Testing: Start Play Event
Edge
Se
rvic
e
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 59: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/59.jpg)
Shadow Testing: Start Play Event
Edge
Se
rvic
e
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 60: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/60.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Start Play Event
Edge
Se
rvic
e
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
@zimmermatt
![Page 61: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/61.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Start Play Event
Edge
Se
rvic
e
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
@zimmermatt
![Page 62: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/62.jpg)
Shadow Testing: Start Play Event
Edge
Se
rvic
e
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 63: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/63.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Viewing History
Positions
Playback History Provider
History
Shadow Testing: Read Latest Position
Edge
Se
rvic
e
@zimmermatt
![Page 64: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/64.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Read Latest Position
Edge
Se
rvic
e
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 65: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/65.jpg)
Shadow Testing: Read Latest Position
Edge
Se
rvic
e
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 66: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/66.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 67: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/67.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 68: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/68.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 69: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/69.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 70: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/70.jpg)
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 71: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/71.jpg)
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 72: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/72.jpg)
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 73: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/73.jpg)
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 74: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/74.jpg)
Shadow Testing: Stop Event
Edge
Se
rvic
e
Summarized Sessions
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 75: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/75.jpg)
Shadow Testing: Stop Event
Summarized Sessions
Session Summary Processor
Summarized Sessions
@zimmermatt
![Page 76: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/76.jpg)
Shadow Testing
Edge
Se
rvic
e
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
Session Summary Processor
Summarized Sessions
Session Events Collector
Event Data
Log Collector
Logs
Active Sessions
Session Accounting Provider
Viewing History
Positions
Playback History Provider
History
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 77: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/77.jpg)
Traffic Dial Session Summary
Processor
Summarized Sessions
Session Events Collector
Event Data
Log Collector
Logs
Active Sessions
Session Accounting Provider
Edge
Se
rvic
e
Viewing History
Positions
Playback History Provider
History
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
@zimmermatt
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
![Page 78: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/78.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Traffic Dial Session Events
Collector
Event Data
Log Collector
Active Sessions
Session Accounting Provider
Edge
Se
rvic
e
Positions
Playback History Provider
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
Session Summary Processor
@zimmermatt
![Page 79: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/79.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Traffic Dial Session Events
Collector
Event Data
Log Collector
Active Sessions
Session Accounting Provider
Edge
Se
rvic
e
Positions
Playback History Provider
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
Session Summary Processor
@zimmermatt
![Page 80: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/80.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
…
Active Sessions
Latest Positions
Session Summary
Stateless Tier
(History, logs,
and fallback)
Events
Traffic Dial Session Events
Collector
Event Data Ed
ge S
erv
ice
Summarized Sessions
Viewing History
His
tory
Cac
he
Logs
Session Summary Processor
@zimmermatt
Active Sessions
Session Accounting Provider
Positions
Playback History Provider
![Page 81: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/81.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
… Latest Positions
Stateless Tier
(History, logs,
and fallback)
Traffic Dial
Edge
Se
rvic
e
Viewing History
His
tory
Cac
he
@zimmermatt
Positions
Playback History Provider
![Page 82: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/82.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
… Latest Positions
Stateless Tier
(History, logs,
and fallback)
Traffic Dial Log
Collector
Edge
Se
rvic
e
Logs
@zimmermatt
![Page 83: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/83.jpg)
Vie
win
g Se
rvic
e
Stateful Tier
0
1
n-2
n-1
… Latest Positions
Stateless Tier
(History, logs,
and fallback)
Traffic Dial
Edge
Se
rvic
e
Playback History Provider
Viewing History
His
tory
Cac
he
@zimmermatt
![Page 84: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/84.jpg)
15
![Page 85: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/85.jpg)
Migration Status Session Events
Collector
Event Data
Active Sessions
Session Accounting Provider
Edge
Se
rvic
e
Session Summary Processor
Summarized Sessions
Log Collector
Logs
Viewing History
Positions
Playback History Provider
History
@zimmermatt
![Page 86: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/86.jpg)
Remember this?
@zimmermatt
![Page 87: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/87.jpg)
This is a typical graph now…
@zimmermatt
![Page 88: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/88.jpg)
The Road Ahead
@zimmermatt
![Page 89: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/89.jpg)
Key Points
• Devour the whale a bite at a time.
• Use CQRS / Event Sourcing. Backup the raw event
data. Design for idempotency so it can be replayed.
• System architectures are throw-away artifacts, useful for
only a limited time.
@zimmermatt
![Page 90: Netflix Viewing Data Microservices - How We Know Where ......Netflix Viewing Data Microservices - How We Know Where You Are in House of Cards Author Matt Zimmer Subject Transitioning](https://reader034.vdocuments.us/reader034/viewer/2022052616/609ebf5dee17de72b551f49f/html5/thumbnails/90.jpg)
“If you don’t like how the table is set, turn over the table.”
Frank Underwood
@zimmermatt