content delivery - in a devops world
DESCRIPTION
Content Delivery - in a DevOps World. Leif Hedstrom. Chief Architect, Cisco Cloud Services. July 8, 2012. One of the drivers to Open S ource Traffic Server Committer for Apache Traffic Server VP of Apache Traffic Server PMC ASF member Chief Architect, Cisco Cloud Services - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/1.jpg)
1
Leif Hedstrom
Content Delivery- in a DevOps World
Chief Architect, Cisco Cloud Services
July 8, 2012
![Page 2: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/2.jpg)
2
Who am I?• One of the drivers to Open Source Traffic Server• Committer for Apache Traffic Server• VP of Apache Traffic Server PMC• ASF member• Chief Architect, Cisco Cloud Services
@zwoop
+zwoop
![Page 3: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/3.jpg)
3
Why DevOps? A short story
![Page 4: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/4.jpg)
4
It’s a disaster!
http://www.flickr.com/photos/historyinanhour/4775644390/
![Page 5: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/5.jpg)
5
Smart people trying to figure it out…
![Page 6: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/6.jpg)
6
Still a disaster, oh the humanity!
http://www.flickr.com/photos/historyinanhour/4775644390/
![Page 7: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/7.jpg)
7
In the mean time …
http://people.apache.org/~weijin/weijin.jpg
![Page 8: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/8.jpg)
8
Small fix, big results
index 30ed932..b74defa 100644--- a/iocore/net/UnixNetVConnection.cc+++ b/iocore/net/UnixNetVConnection.cc@@ -574,6 +574,9 @@ UnixNetVConnection::do_io_close(int alerrno /* = -1 */ ) write.vio.nbytes = 0; write.vio.op = VIO::NONE; + EThread *t = this_ethread();+ bool close_inline = !recursion && nh->mutex->thread_holding == t;+ INK_WRITE_MEMORY_BARRIER; if (alerrno && alerrno != -1) this->lerrno = alerrno;@@ -582,11 +585,8 @@ UnixNetVConnection::do_io_close(int alerrno /* = -1 */ ) else closed = -1; - if (!recursion) {- EThread *t = this_ethread();- if (nh->mutex->thread_holding == t)- close_UnixNetVConnection(this, t);- }+ if (close_inline)+ close_UnixNetVConnection(this, t); }
![Page 9: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/9.jpg)
9
I’m happy!
http://www.flickr.com/photos/mhedstrom/6763754839/
![Page 10: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/10.jpg)
10
DevOps reward (well, maybe … )
![Page 11: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/11.jpg)
11
So what is DevOps?• To me, it’s a collaborative, community driven
way of developing software, services and IT solutions.
• Tear down the borders (and rifts!) between Development and Operations.
• Thrives on healthy Open Source communities.
![Page 12: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/12.jpg)
12
![Page 13: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/13.jpg)
13
![Page 14: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/14.jpg)
14
QPS is no longer a meaningful metric
![Page 15: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/15.jpg)
15
…Other things matters…
![Page 16: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/16.jpg)
16
… and don’t forget features• Handling thousands of concurrent users• Extensible (e.g. plugins, or scripts)• Correctness• HTTP/1.1
• Keep-Alive• Chunking• etc.
![Page 17: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/17.jpg)
17
When all you have is a hammer …
http://www.flickr.com/photos/aai/6936657289/
![Page 18: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/18.jpg)
18
Evaluate the features that matters to you!ATS HAproxy ngin
xSquid Varnish mod_proxy
Worker Threads Y N N N Y Y
Multi-Process N Y Y N Y YEvent-driven Y Y Y Y sometimes YPlugin APIs Y N Y part Y YForward Proxy Y N N Y N YReverse Proxy Y Y Y Y Y YTransp. Proxy Y Y N Y N NLoad Balancer weak Y Y Y Y YPersistent Cache Y N Y Y sorta YESI Y N N Y kinda NICP Y N N Y N NKeep-Alive Y N Y Y Y YSSL Y N Y Y N YPipeline Y N Y Y N Y
![Page 19: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/19.jpg)
19
Forward proxy
![Page 20: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/20.jpg)
20
Transparent (or intercepting) proxy
![Page 21: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/21.jpg)
21
Reverse proxy
![Page 22: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/22.jpg)
22
![Page 23: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/23.jpg)
23
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
![Page 24: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/24.jpg)
24
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
![Page 25: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/25.jpg)
25
TCP 3-way handshake
![Page 26: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/26.jpg)
26
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
![Page 27: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/27.jpg)
27
Congestion control
![Page 28: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/28.jpg)
28
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
![Page 29: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/29.jpg)
29
http://news.example.com/http://finance.example.com/http://groups.example.com/
![Page 30: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/30.jpg)
30
http://news.example.com/http://finance.example.com/http://groups.example.com/
vshttp://www.example.com/newshttp://www.example.com/financehttp://www.example.com/groups
![Page 31: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/31.jpg)
31
gzip matters
http://www.flickr.com/photos/marcovdz/4520986339/
![Page 32: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/32.jpg)
32
Reduce your round-trips!
plain gzipTheoretical limit on 20Mbps 78ms 15msLow latency (17ms RTT) 182ms 105msHigh latency (159ms RTT) 1,340ms 840ms
![Page 33: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/33.jpg)
33
Congestion control
![Page 34: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/34.jpg)
34
Real example (200KB uncompressed)
200KB (8 RTs) 40KB (5 RTs)17ms RTT 136ms 85ms159ms RTT 1,272ms 795ms
![Page 35: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/35.jpg)
35
Keep-alive• A primary tool to fight 3-way handshake
latency• Helps against congestion control too
• But not always… Slow starts begins again after some idle time
![Page 36: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/36.jpg)
36
Old Congestion Window Size Algorithm:
IW = min (4*SMSS, max (2*SMSS, 4380 bytes))
(RFC2581)
Typically, 2-4 packets
![Page 37: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/37.jpg)
37
New Linux Initial Congestion Window Size Algorithm:
10
![Page 38: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/38.jpg)
38
SPDY• Addresses many of the problems related to
latency• Not a standard (yet)• SSL requirements, and other quirks, can be a
hurdle
![Page 39: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/39.jpg)
39
![Page 40: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/40.jpg)
40
Some history
![Page 41: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/41.jpg)
41
The Concurrency problem
![Page 42: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/42.jpg)
42
Solution 1: Multi-threading
![Page 43: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/43.jpg)
43
Problems with Multi-threading
![Page 44: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/44.jpg)
44
Solution 2: Event Processing
![Page 45: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/45.jpg)
45
Problems with event processing
![Page 46: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/46.jpg)
46http://www.flickr.com/photos/stuartpilbrow/3345896050/
![Page 47: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/47.jpg)
47
Traffic Server model
![Page 48: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/48.jpg)
48
Four Horseman of Bad Performance• Data copies• Context Switches• Memory allocation• Lock contention
![Page 49: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/49.jpg)
49http://www.flickr.com/photos/simpologist/16734948/
![Page 50: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/50.jpg)
50
http://www.flickr.com/photos/airosan/2232394342/
![Page 51: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/51.jpg)
51
http://trafficserver.apache.org/
![Page 52: Content Delivery - in a DevOps World](https://reader035.vdocuments.us/reader035/viewer/2022062222/568165f0550346895dd91605/html5/thumbnails/52.jpg)
Thank you.