Download - Rethinking Cloud Proxies
![Page 1: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/1.jpg)
A Cloud Gateway -A Large Scale Company’s First Line of Defense
Mikey CohenManager - Edge Gateway
Netflix
![Page 2: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/2.jpg)
Today, more than 36% of North America’s internet
traffic is controlled by systems in the Amazon
Cloud
![Page 3: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/3.jpg)
![Page 4: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/4.jpg)
Global Streaming of TV Shows and Movies
![Page 5: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/5.jpg)
Nearly 70 Million Subscribers
In over 80 Countries
![Page 6: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/6.jpg)
Netflix accounts for over 36% of Downstream Traffic in North America
![Page 7: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/7.jpg)
From the Internet to Services in the Cloud
GatewayGateway
?????
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
![Page 8: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/8.jpg)
Our Edge Gateway @ Netflix
Handles most netflix.com hostsOver 20 production Zuul clusters~ 50 elbs Gateway handles ~10 origin services
![Page 9: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/9.jpg)
Netflix Gateway Scale
Tens of billions of requests per day3 AWS regionsOver 1000 device types
Hundreds of permutations of protocols and device versions
![Page 10: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/10.jpg)
SuccessEvolutionScaleFailure
Our Journey
![Page 11: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/11.jpg)
So What!? - Change your perspective!!
![Page 12: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/12.jpg)
Traditional Cloud Proxy Mission
Simple static rule-based routingAPI portalRequest authenticationThrottling - request capsMonitoringCaching
![Page 13: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/13.jpg)
The Gateway - a grown-up proxy!●Dynamic routing●Deep Insights●Load balancing●Availability focused●Service protection●Quality assurance tool
![Page 14: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/14.jpg)
Evolving to a Gateway
![Page 15: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/15.jpg)
Netflix’s Public API
Late 2008MasheryDatacenter
![Page 16: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/16.jpg)
Streaming Devices using public API
Early Streaming Devices - 2009 Windows Media CenterXBoxPS3
![Page 17: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/17.jpg)
Migration to AWS2010Sonoa / Apigee proxy
Device traffic, not publicControlling DC -> cloud
migrationRunning in AWSUnder Netflix control
![Page 18: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/18.jpg)
Streaming Success2011ChaosComplexityFailureSuccessLeveraging
Cloud benefits
![Page 19: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/19.jpg)
Anti-patterns of most cloud proxiesStatic configurations
Service push needed to change behavior
Limited range of functionality
Limited to HTTP
![Page 20: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/20.jpg)
Zuul Created
2012Dynamically injected and compiled filters
Manipulate requests and responsesHeaders / Body / etc
Change routing Add metrics and other functions
Built on Netflix’s OSS stackOpen Sourced
![Page 21: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/21.jpg)
Zuul - A Victim of SuccessEasy and convenientInstant resultsHigh adoptionHappy customers
Business logic in proxyAffects system resiliency Zuul team in critical path
![Page 22: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/22.jpg)
Creating a Gateway Strategy
![Page 23: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/23.jpg)
Principles of Netflix’s Gateway Strategy
Creative RoutingDynamic RoutingDelivery FocusedTraffic ShapingReact Fast Insights
![Page 24: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/24.jpg)
Creative Routing - Subclusters with Purpose
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeeze
![Page 25: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/25.jpg)
Red / Green Deployments
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
InstrumentedInstrumented
squeezesqueeze
![Page 26: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/26.jpg)
Developer Test Branches
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
InstrumentedInstrumented
squeezesqueeze
![Page 27: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/27.jpg)
Instrumented Clusters
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
![Page 28: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/28.jpg)
Squeeze Testing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeeze
![Page 29: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/29.jpg)
Targeted Routing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeeze
![Page 30: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/30.jpg)
Service “Canarying”
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
![Page 31: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/31.jpg)
“Sticky” Canary
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
canary
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
![Page 32: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/32.jpg)
Failure Injection Testing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
![Page 33: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/33.jpg)
Degraded Experience Testing
GatewayGateway
Gateway
Origin (API)
v1
v2
test
debug
Instrumented
squeeze
“sticky” canarybaseline
“sticky” baseline
v1
v2
test
debug
baseline canary
“sticky” canary
“sticky” baselineFIT
Instrumented
squeezesqueeze
![Page 34: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/34.jpg)
Traffic Shaping
![Page 35: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/35.jpg)
A Global Cloud Deployment
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
![Page 36: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/36.jpg)
Global Cloud Routing
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
![Page 37: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/37.jpg)
A Failing region
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
![Page 38: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/38.jpg)
Gateway routing to other regions
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
Persistence Tier
Business services Tier
Presentation Tier
Network Tier
Websites API
Proxy
DB
![Page 39: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/39.jpg)
Attack prevention
GatewayGateway
Gateway
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
![Page 40: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/40.jpg)
Smart Load Balancing
GatewayGateway
Gateway
Origin (API)
![Page 41: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/41.jpg)
Smart Load Balancing - Bad Nodes
GatewayGateway
Gateway
Origin (API)
![Page 42: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/42.jpg)
Gateway Backoff and Blacklists Bad Nodes
GatewayGateway
Gateway
Origin (API)
![Page 43: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/43.jpg)
Zone Failure - Blacklist the Zone automatically
GatewayGateway
Gateway
Origin (API)
![Page 44: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/44.jpg)
React Quickly - Runtime Filter changes
GatewayGateway
Gateway
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
Runtime Policy Injection
![Page 45: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/45.jpg)
A Room with a View - Insights
GatewayGateway
Gateway
Origin (API)Origin (API)
API
Origin (API)Origin (API)
Website
Insights
![Page 46: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/46.jpg)
What’s Next for Netflix’s Gateway?
Gateway as a serviceSelf-service dynamic routing / route validationControl APIs for special routing functions
Netty Based Zuul (using RxNetty)Handling persistent connectionsnon-blocking, async
Transport protocol agnostic routingReactive Socket http://reactivesocket.io/
![Page 47: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/47.jpg)
Top Ten Lessons Learned
![Page 48: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/48.jpg)
Build for handling Failures
![Page 49: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/49.jpg)
Expect the Unexpected
![Page 50: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/50.jpg)
Using Routing Creatively
![Page 51: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/51.jpg)
Shard to Reduce Blast Radius
![Page 52: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/52.jpg)
Devices are WeirdProtocols are Weird
![Page 53: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/53.jpg)
Devices are ForeverProtocols are Forever
![Page 54: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/54.jpg)
It will be built “wrong”
![Page 55: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/55.jpg)
Keep Business Logic out of your Gateway
![Page 56: Rethinking Cloud Proxies](https://reader036.vdocuments.us/reader036/viewer/2022081521/586fdb221a28ab18428b5fcb/html5/thumbnails/56.jpg)
For More Info...
Zuul OSSNetflix Tech BlogRxNettyJobs