operating systems and networks where we are in the course ... · web (http) web (cdns) p2p...

17
1 Operating Systems and Networks Network Lecture 12: Application Layer Adrian Perrig Network Security Group ETH Zürich 2 Where we are in the Course Starting the Application Layer! Builds distributed “network services” (DNS, Web) on Transport services Physical Link Application Network Transport 3 Recall Application layer protocols are often part of an “app” But don’t need a GUI, e.g., DNS TCP IP 802.11 HTTP app OS User-level (NIC) 4 Recall (2) Application layer messages are often split over multiple packets Or may be aggregatedin a packet … 802.11 IP TCP HTTP 802.11 IP TCP HTTP 802.11 IP TCP HTTP HTTP Application Communication Needs Vary widely with app; must build on Transport services 5 UDP DNS TCP Series of variable length, reliable request/reply exchanges Web UDP Real-time (unreliable) stream delivery Skype See Book Short,reliable request/reply exchanges Message reliability! OSI Session/Presentation Layers Remember this? Two relevant concepts … 6 – Provides functions needed by users – Converts different data representations – Multiple sessions between same src-dst – Provides end-to-end delivery – Sends packets over multiple links – Sends frames of information – Sends bits as signals But consider part of the application, not strictly layered!

Upload: others

Post on 08-Aug-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

1

OperatingSystemsandNetworks

NetworkLecture12:ApplicationLayer

AdrianPerrigNetworkSecurityGroupETHZürich

2

WhereweareintheCourse• StartingtheApplicationLayer!

– Buildsdistributed“networkservices”(DNS,Web)onTransportservices

PhysicalLink

Application

NetworkTransport

3

Recall• Applicationlayerprotocolsareoftenpartofan“app”

– Butdon’tneedaGUI,e.g.,DNS

TCPIP

802.11

HTTPapp

OS

User-level

(NIC)

4

Recall(2)• Applicationlayermessagesareoftensplitovermultiplepackets– Ormaybeaggregatedinapacket…

802.11 IP TCP HTTP

802.11 IP TCP HTTP

802.11 IP TCP HTTP

HTTP

ApplicationCommunication Needs• Varywidelywithapp;mustbuildonTransportservices

5

UDP

DNS

TCP

Seriesofvariablelength, reliablerequest/replyexchanges

Web

UDP

Real-time(unreliable)

streamdelivery

Skype

SeeBook

Short,reliablerequest/replyexchanges

Messagereliability!

OSISession/Presentation Layers• Remember this?Tworelevantconcepts…

6

– Providesfunctionsneededbyusers– Convertsdifferentdatarepresentations– Multiplesessionsbetweensamesrc-dst– Providesend-to-enddelivery– Sendspacketsovermultiplelinks– Sendsframesofinformation– Sendsbitsassignals

Butconsiderpartoftheapplication,notstrictlylayered!

Page 2: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

2

7

Session Concept

• Asession isaseriesofrelated networkinteractions insupport ofanapplication task– Ofteninformal,notexplicit

• Examples:– Webpagefetchesmultipleimages– Skypecall involvesaudio,video,chat

8

PresentationConcept• Appsneedtoidentifythetypeofcontent,andencodeitfortransfer– ThesearePresentationfunctions

• Examples:– Media(MIME)types,e.g.,image/jpeg,identifythetypeofcontent

– Transferencodings,e.g.,gzip,identifycontentencoding– Applicationheadersareoftensimpleandreadableversuspackedforefficiency

9

Topics• EvolvingInternetapplications• DNS(DomainNameSystem)• HTTP(HyperText TransferProtocol)• Webproxiesandcaching• ContentDistributionNetworks• Peer-to-peer(BitTorrent)

• Real-timeapplications(VoIP)

Thislecture

SeeBook

EvolutionofInternetApplications• Alwayschanging,andgrowing…

10

20101970 19901980 2000

Traffic

FileTransfer(FTP)Email(SMTP)

News(NTTP)

SecureShell(ssh)Telnet

Email

Web(HTTP)Web(CDNs)

P2P(BitTorrent)Web(Video)

???

EvolutionofInternetApplications (2)• ForapeekatthestateoftheInternet:

– Akamai’sStateoftheInternetReport(quarterly)– Cisco’sVisualNetworkingIndex– Mary Meeker’s InternetReport

• RobustInternetgrowth,esp.video,wirelessandmobile– Mosttrafficisvideo,willbe90%ofInternetinafewyears– Wirelesstrafficwillsoonovertakewiredtraffic– Mobiletrafficisstillasmallportion(15%)ofoverall– Growingattack trafficfromChina,alsoU.S.andRussia

11

EvolutionoftheWeb

12

Source:h ttp ://www.evo lu tiono ftheweb .com, Vizzu al i ty, Google,and Hyperakt

Page 3: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

3

EvolutionoftheWeb (2)

13

Sou rce:h ttp ://www.evo lu tiono ftheweb .com, Vizzu al i ty, Google,and Hyperakt

14

DomainNameSystem(DNS)(§7.1.1-7.1.3)

• TheDNS(DomainNameSystem)– Human-readablehostnames,andmore– Part1:thedistributednamespace

www.uw.edu?

Network

128.94.155.135

NamesandAddresses• Names:higher-level(user-understandable)resourceidentifiers• Addresses: lower-levelresourcelocators

– Multiple levels, e.g., fullnameà emailà IPaddress à Ethernet address• Resolution (orlookup):mappinganametoanaddress

15

Directory

Name, e.g.“Andy Tanenbaum,”or“flits.cs.vu.nl”

Address, e.g.“Vrije Universiteit, Amsterdam”

or IPv4“130.30.27.38”Lookup

16

BeforetheDNS– HOSTS.TXT• DirectorywasafileHOSTS.TXTregularlyretrievedforallhostsfromacentralmachineattheNIC(NetworkInformationCenter)

• Nameswereinitiallyflat,becamehierarchical(e.g.,lcs.mit.edu)~1985

• NeithermanageablenorefficientastheARPANETgrew…

17

DNS• AnamingservicetomapbetweenhostnamesandtheirIP

addresses(andmore)– www.uwa.edu.auà 130.95.128.140

• Goals– Easytomanage(especiallywithmultipleparties)– Efficient(goodperformance,fewresources)

• Approach– Distributeddirectorybasedonahierarchicalnamespace– Automatedprotocoltotiepiecestogether

DNSNamespace• Hierarchical,startingfrom“.”(dot,typicallyomitted)

18

robot.cs.washington.edu

Page 4: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

4

TLDs(Top-LevelDomains)• RunbyICANN(InternetCorp.forAssignedNamesandNumbers)

– Startingin‘98;namingis financial,political, andinternational

• 22+genericTLDs– Initially.com,.edu ,.gov.,.mil,.org,.net– Added .aero,. info,.museum,etc.from’01through.xxxin’11– DifferentTLDshavedifferentusagepolicies

• ~250countrycodeTLDs– Twoletters,e.g.,“.au”,plusinternationalcharacterssince2010– Widelycommercialized,e.g.,.tv (Tuvalu)– Manydomainhacks,e.g.,instagr.am(Armenia),goo.gl(Greenland)

19

DNSZones• Azone isacontiguousportionofthenamespace

20

AzoneDelegation

21

DNSZones(2)• Zonesarethebasisfordistribution

– EDURegistraradministers .edu– UW administerswashington.edu– CS&E administers cs.washington.edu

• Eachzonehasanameserver tocontactforinformationaboutit– Zonemust include contacts fordelegations, e.g., .eduknows nameserver forwashington.edu

DNSResourceRecords• AzoneiscomprisedofDNSresourcerecordsthatprovideinformationaboutitsdomainnames

22

Type MeaningSOA Start of authority, has main zone parametersA IPv4 address of a hostAAAA (“quad A”) IPv6 address of a hostCNAME Canonical name for an aliasMX Mail exchanger for the domainNS Nameserver of domain or delegated subdomain

23

DNSResourceRecords(2)

IPaddressesofcomputers

Nameserver

Mailgateways

24

DNSResolution• DNSprotocolletsahostresolveanyhostname(domain)toIPaddress

• Ifunknown,canstartwiththerootnameserver andworkdownzones

• Let’s seeanexample first…

Page 5: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

5

DNSResolution (2)• flits.cs.vu.nlresolvesrobot.cs.washington.edu

25 26

Iterativevs.RecursiveQueries• Recursivequery

– Nameserver completesresolutionandreturnsthefinalanswer– E.g.,flitsà localnameserver

• Iterativequery– Nameserver returnstheanswerorwhotocontactnextfortheanswer

– E.g.,localnameserver à allothers

27

Iterativevs.RecursiveQueries(2)• Recursivequery

– Letsserveroffloadclientburden(simpleresolver)formanageability

– Letsservercacheoverapoolofclientsforbetterperformance

• Iterativequery– Letsserver“fileandforget”– Easytobuildhighloadservers

28

Caching• Resolutionlatencyshouldbelow

– Addsdelaytowebbrowsing• Cachequery/responsestoanswerfuturequeriesimmediately– Includingpartial(iterative)answers– ResponsescarryaTTLforcaching

Nameserver

query out

responseCache

Caching(2)• flits.cs.vu.nlnowresolveseng.washington.edu

– Andpreviousresolutionscutoutmostoftheprocess

29

1:query 2:query

UWnameserver(forwashington.edu)

3:eng.washington.edu4:eng.washington.edu

Localnameserver(forcs.vu.nl)

Iknowtheserverforwashington.edu!

Cache

30

LocalNameservers• Localnameservers typicallyrunbyIT(enterprise,ISP)

– ButmaybeyourhostorAP– Oralternativese.g.,GooglepublicDNS

• Clientsneedtobeabletocontacttheirlocalnameservers– TypicallyconfiguredviaDHCP

Page 6: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

6

31

RootNameservers• Root(dot) isservedby13servernames

– a.root-servers.net tom.root-servers.net– Allnameservers needrootIPaddresses– Handledviaconfigurationfile(named.ca)

• Thereare>250distributedserverinstances– Highlyreachable,reliableservice– Mostserversare reached byIPanycast (MultiplelocationsadvertisesameIP!Routestakeclienttotheclosestone.See§5.2.9)

– Servers areIPv4andIPv6reachable

RootServerDeployment

32

Source:http:/ /www.root-servers .org.Snapshoton27.02.12.Does notrepresentcurrentdeployment.

33

DNSProtocol• Queryandresponsemessages

– Built on UDPmessages, port 53– ARQ for reliability; server is stateless!– Messages linked bya16-bit IDfield

Query

Response

Time

Client ServerID=0x1234

ID=0x1234

34

DNSProtocol(2)• Service reliabilityviareplicas

– Runmultiplenameservers fordomain– Returnthelist;clientsuseoneanswer– Helpsdistributeloadtoo

NS foruw.edu?

ABC

UseA,BorC

35

DNSProtocol(3)• Securityisamajor issue

– Compromiseredirectstowrongsite!– Notpartofinitialprotocols..

• DNSSEC(DNS SecurityExtensions)– Longunderdevelopment,nowpartiallydeployed

Um,security??

36

HTTP,theHyperTextTransferProtocol(§7.3.1-7.3.4)

• HTTP,(HyperText TransferProtocol)– BasisforfetchingWebpages

requestNetwork

Page 7: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

7

37

SirTimBerners-Lee(1955–)• InventoroftheWeb

– DominantInternetappsincemid90s– HenowdirectstheW3C

• DevelopedWebatCERNin‘89– Browser,serverandfirstHTTP– PopularizedviaMosaic(‘93),Netscape– FirstWWWconferencein’94…

Source: ByP au l C larke, CC -BY-2 .0 ,viaWikimed ia Commons

WebContext

38

HTTPrequest

HTTPresponse

PageasasetofrelatedHTTPtransactions

39

WebProtocolContext• HTTPisarequest/responseprotocolforfetchingWebresources– RunsonTCP,typicallyport80– Partofbrowser/serverapp

TCPIP

802.11

browser

HTTPTCPIP

802.11

server

HTTPrequest

response

40

FetchingaWebpagewithHTTP• Start withthepageURL:

http://de.wikipedia.org/wiki/Chuc hichä schtl i

• Steps:– Resolve the server to IPaddress (DNS)– SetupTCPconnection to theserver– SendHTTPrequest for thepage– (Await HTTP response for thepage)– Execute/ fetchembedded resources / render– Clean upany idle TCPconnections

Protocol PageonserverServer

**

Staticvs DynamicWebpages• Staticwebpageiscontentofafile,e.g., image• Dynamicwebpage istheresultofprogramexecution

– Javascript onclient,PHPonserver,orboth

41

EvolutionofHTTP• Considersecurity(SSL/TLSforHTTPS)later

42

20101990 20001995 2005

1.0developed

1.1developed(persistentconnections)

0.9RFC1945

RFC2068,2109RFC2616

CookiesSSL2.0

SPDY(HTTP2.0)

Proliferationofcontenttypes

andbrowser/server

scriptingtechnologies

RFC2965

Page 8: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

8

43

HTTPProtocol• Originallyasimple protocol, withmany options added over

time– Text-based commands,headers

• Tryityourself:– Asa“browser”fetchingaURL– Run“telnetwww.scion-architecture.net 80”– Type“GET /HTTP/1.1”followedby“Host:www.scion-architecture.net” followedbyablankline

– Server willreturnHTTPresponsewiththepagecontents(orotherinfo)

44

HTTPGetResult$telnetwww.scion-architecture.net 80Trying129.132.85.42...Connectedtoscion-architecture.net.Escapecharacteris' ]̂'.GET/HTTP/1.1Host:www.scion-architecture.net

HTTP/1.1200OKDate:Wed,01Jun201621:04:24GMTServer:Apache/2.2.15(RedHat)X-Powered-By:PHP/5.3.3Transfer-Encoding:chunkedContent-Type:text/html;charset=UTF-8

3565

<!DOCTYPEHTML>

<html>...

45

HTTPProtocol(2)• Commandsusedintherequest

Method DescriptionGET ReadaWebpageHEAD ReadaWebpage's headerPOST Append toaWeb pagePUT StoreaWebpageDELETE Remove the WebpageTRACE Echo the incoming requestCONNECT Connect through aproxyOPTIONS Queryoptions forapage

FetchpageUploaddata

HTTPProtocol(3)• Codesreturnedwiththeresponse

46

Code Meaning Examples

1xx Information 100=serveragrees tohandle client's request

2xx Success 200= request succeeded; 204=nocontentpresent

3xx Redirection 301=pagemoved;304=cachedpagestill valid

4xx Client error 403= forbidden page;404 =pagenot found

5xx Servererror 500= internal server error;503 =tryagain later

Yes!

HTTPProtocol(4)• Manyheaderfieldsspecifycapabilitiesandcontent

– E.g.,Content-Type:text/html,Cookie:lect=12-1-http

47

Function ExampleHeadersBrowsercapabilities(clientà server)

User-Agent,Accept,Accept-Charset,Accept-Encoding,Accept-Language

Cachingrelated(mixeddirections)

If-Modified-Since,If-None-Match,Date,Last-Modified,Expires,Cache-Control,ETag

Browsercontext(clientà server) Cookie,Referer,Authorization,Host

Contentdelivery(serverà client)

Content-Encoding,Content-Length,Content-Type,Content-Language,Content-Range,Set-Cookie

48

HTTPPerformance (§7.3.4)• PerformanceofHTTP

– Parallelandpersistentconnections

requestNetwork

Page 9: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

9

49

PLT(PageLoadTime)• PLTisthekeymeasureofwebperformance

– From clickuntiluser seespage– Smallincreases inPLT decrease sales

• PLTdependsonmanyfactors– Structureof page/content– HTTP (and TCP!)protocol– NetworkRTT andbandwidth

50

EarlyPerformance(1)• HTTP/1.0usesoneTCPconnectiontofetcheachwebresource– MadeHTTPveryeasytobuild– ButgavefairlypoorPLT…

51

EarlyPerformance(2)• ManyreasonswhyPLTislargerthan

necessary– Sequential request/responses , evenwhen

todifferent servers– Multiple TCP connection setups to thesame

server– Multiple TCP slow-start phases

• Networkisnotusedeffectively– Worse with manysmall resources /page

52

WaystoDecreasePLT1. Reducecontentsizefortransfer

– Smallerimages,gzip2. ChangeHTTPtomakebetteruseofavailable

bandwidth(e.g.,avoidTCPslowstart)3. ChangeHTTPtoavoidrepeatedtransfersofthe

samecontent– Caching,andproxies

4. Movecontentclosertoclient– CDNs[later]

53

ParallelConnections• OnesimplewaytoreducePLT

– Browserrunsmultiple(8,say)HTTPinstancesinparallel– Serverisunchanged;alreadyhandlesconcurrentrequestsformanyclients

• Howdoesthishelp?– SingleHTTPwasn’tusingnetworkmuch…– Soparallelconnectionsaren’tslowedmuch– Pullsincompletiontimeoflastfetch

54

PersistentConnections• Parallelconnectionscompetewitheachotherfornetworkresources– 1parallelclient≈8sequential clients?– Exacerbates networkbursts, and loss

• Persistentconnectionalternative– Make1TCPconnection to1server– Use itformultiple HTTPrequests

Page 10: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

10

PersistentConnections (2)

55

Onerequestperconnection

Sequentialrequestsperconnection

Pipelined requestsperconnection

56

PersistentConnections (3)• WidelyusedaspartofHTTP/1.1

– Supportsoptionalpipelining– PLTbenefitsdependingonpagestructure,buteasyonnetwork

• Issueswithpersistentconnections– HowlongtokeepTCPconnection?– Canitbeslower?(Yes.Butwhy?)

57

HTTPCachingandProxies (§7.3.4,§7.5.2)

• HTTPcaching andproxies– Enablingcontentreuse

ServerClients

ProxyCache

58

WebCaching• Usersoftenrevisitwebpages

– Bigwinfromreusinglocalcopy!– Thisiscaching

• Keyquestion:– WhenisitOKtoreuselocalcopy?

NetworkCache

Local copies

Server

59

WebCaching(2)• Locallydetermine ifcopyisstillvalid

– Basedonexpiryinformationsuchas “Expires”headerfromserver

– Oruseaheuristictoguess(cacheable,freshlyvalid,notmodifiedrecently)

– Contentisthenavailablerightaway

NetworkCacheServer

60

WebCaching(3)• Revalidatecopywithremoteserver

– Basedontimestampofcopysuchas“Last-Modified”headerfromserver

– Orbasedoncontentsuchas“ETag”headerfromserver:EntityTag,computedbyserverasauniqueobjectidentifier

– Contentisavailableafter1RTT

NetworkCacheServer

Page 11: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

11

WebCaching(4)• Puttingthepiecestogether:

61 62

WebProxies• Placeintermediarybetweenpoolofclientsandexternalwebservers

• Proxycaching– Clients benefit from larger,sharedcache (other clientsmayhavealreadyaccessed content)

– Benefits limited bysecure /dynamic per-client content,aswellas“long tail”dataaccess pattern

– Enables application of organizationalaccess policies

WebProxies(2)• Clientscontactproxy;proxycontactsserver

63

Cache

Nearclient

Far fromclient

64

CDNs (ContentDeliveryNetworks)(§7.5.3)

• CDNs(ContentDeliveryNetworks)– Efficientdistributionofpopularcontent;fasterdeliveryforclients

ContentReplica

Consumers

65

Context• Asthewebtookoffinthe90s,trafficvolumesgrewandgrew.This:1. Concentratedloadonpopularservers2. Ledtocongestednetworksandneedtoprovisionmore

bandwidth3. Gaveapooruserexperience

• Idea:– Placepopularcontentnearclients– Helpswithallthreeissuesabove

66

BeforeCDNs• Sendingcontentfromthesourceto4userstakes4x3=12“networkhops”intheexample

Source

User

User

...

Page 12: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

12

67

AfterCDNs• Sendingcontentviareplicastakesonly4+2=6“networkhops”

Source

User

User

...Replica

68

AfterCDNs(2)• Benefitsassumingpopularcontent:

– Reducesserver,networkload– Improvesuserexperience(PLT)

Source

User

User

...Replica

69

Popularity ofContent• Zipf’s Law: fewpopularitems,manyunpopularones(“heavytail”ofprobabilitydistribution);bothmatter

Zipf popularity(kth item is 1/k)

RankSource:Wikipedia

GeorgeZipf (1902-1950)

70

Howtoplacecontentnearclients?• Usebrowserandproxycaches

– Helps,butlimitedtooneclientorclientsinoneorganization

• WanttoplacereplicasacrosstheInternetforusebyallnearbyclients– DonebycleveruseofDNS

ContentDeliveryNetwork

71

ContentDeliveryNetwork(2)• DNSresolutionofsitegivesanswerdependingonclient

– Directeachclienttothenearestreplica(usingIPgeolocation)

72

Page 13: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

13

Consumer

site

73

BusinessModel• ClevermodelpioneeredbyAkamai

– PlacingsitereplicaatanISPiswin-win– ImprovessiteexperienceandreducesbandwidthusageofISP

ISPUser

User

...Replica

74

TheFutureofHTTP• TheFutureofHTTP

– Howwillwemakethewebfaster?– Abrieflookatsomeapproaches

requestNetwork

ModernWebPages• Waterfalldiagramshowsprogressionofpageload

75

webpagetest toolforhttp://coursera.org(Firefox,5/1Mbps,fromVA,3/1/13)

76

ModernWebPages(2)

Yikes!-23requests-1Mbdata-2.6secs

webpagetest toolforhttp://coursera.org(Firefox,5/1Mbps,fromVA,3/1/13)

• WaterfallandPLTdependsonmanyfactors– Verydifferentfordifferentbrowsers– Verydifferentforrepeatpageviews– Dependsonlocalcomputationaswellasnetwork

ModernWebPages(3)

77

Yay!(Networkusedwell)

78

RecentworktoreducePLTPagesgrowevermorecomplex!

– Larger,moredynamic,andsecure– HowwillwereducePLT?

1. Betteruseofthenetwork– HTTP/2effortbasedonSPDY

2. Bettercontentstructures– mod_pagespeed serverextension

Page 14: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

14

79

SPDY(“speedy”)• AsetofHTTPimprovements

– Multiplexed(parallel)HTTPrequestsononeTCPconnection– Clientprioritiesforparallelrequests– CompressedHTTPheaders– Serverpushofresources

• Nowbeingtestedandimproved– DefaultinChrome,Firefox– BasisforHTTP/2

80

mod_pagespeed• Observation:

– Thewaypagesarewrittenaffectshowquicklytheyload– Manybooksonbestpracticesforpageauthorsanddevelopers

• Keyidea:– Haveserverre-write(compile)pagestohelpthemloadquickly!

– Apachemod_pagespeed isanexample

81

mod_pagespeed (2)• Apacheserverextension

– Softwareinstalledwithwebserver– Rewritespages“onthefly”withrulesbasedonbestpractices

• Examplerewriterules:– MinifyJavascript– Flattenmulti-levelCSSfiles– Resizeimagesforclient– …andmuchmore(100sofspecificrules)

82

Peer-to-PeerContentDelivery(BitTorrent)(§7.5.4)

• Peer-to-peercontentdelivery– Runswithoutdedicatedinfrastructure– BitTorrent asanexample

Peer

Peer

Peer

PeerPeer

83

Context• Deliverywithclient/serverCDNs:

– Efficient,scalesupforpopularcontent– Reliable,managedforgoodservice

• …butsomedisadvantagestoo:– Needfordedicatedinfrastructure– Centralizedcontrol/oversight

84

P2P(Peer-to-Peer)• Goalisdeliverywithout dedicatedinfrastructureorcentralizedcontrol– Stillefficient atscale,andreliable

• Keyideaistohaveparticipants(orpeers)helpeachother– InitiallyNapster‘99formusic (gone)– Now BitTorrent ‘01 onwards(popular!)

Page 15: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

15

85

P2PChallenges• Noserversonwhichtorelyon

– Communicationmustbepeer-to-peer andself-organizing,notclient-server

– Leadstoseveralissuesatscale…

Peer

Peer

Peer

PeerPeer

86

P2PChallenges (2)1. Limitedcapabilities

– Howcanonepeerdelivercontenttoallotherpeers?2. Participationincentives

– Whywouldpeershelpeachother?3. Decentralization

– Howwillpeersfindcontent?

87

OvercomingLimitedCapabilities• Peercansendcontenttoallotherpeersusingadistributiontree– Typicallydonewithreplicasovertime– Self-scalingcapacity(morenodesàmorecapacity)

Source

88

Providing ParticipationIncentives• Peersplaytworoles:

– Download()tohelpthemselves,andupload()tohelpothers

Source

89

Providing ParticipationIncentives(2)• Couplethetworoles:

– I’lluploadforyouifyouuploadforme– Encouragescooperation

Source

90

EnablingDecentralization• Peermust learnwheretogetcontent

– UseDHTs (DistributedHashTables)

• DHTs arefully-decentralized, efficient algorithms for adistributed index– Indexisspreadacrossallpeers– Indexlistspeerstocontactforcontent– Anypeercanlookuptheindex– Started asacademicworkin2001

Page 16: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

16

91

BitTorrent• MainP2Psysteminusetoday

– DevelopedbyCohenin‘01– Veryrapidgrowth,largetransfers– BigfractionofInternettraffic– Usedforlegalandcopyrightedcontent

• Deliversdatausing“torrents”:– Transfersfilesinpiecesforparallelism– Notablefortreatment ofincentives– Tracker ordecentralizedindex(DHT)

ByJacob Appelb aum,CC -BY-SA-2 .0 , fromWikimed iaCommons

BramCohen(1975—)

92

BitTorrent Protocol• Stepstodownloadatorrent:

1. Startwithtorrentdescription2. Contact trackertojoinand getlistofpeers(withatleast

seed peer)2. Or, useDHTindex forpeers3. Trade pieceswithdifferent peers4. Favorpeersthatupload toyourapidly;“choke” peers

thatdon’t byslowingyourupload tothem

BitTorrent Protocol (2)• Allpeers(exceptseed)retrieve torrentatthesametime

93

BitTorrent Protocol (3)• Dividingfileintopiecesgivesparallelismforspeed

94

BitTorrent Protocol (4)• Chokingunhelpfulpeersencouragesparticipation

95

STOPSTOP

STOP

XXX

BitTorrent Protocol (5)• DHTindex(spreadoverpeers)isfullydecentralized

96

DHT

DHT

DHTDHT

DHT

DHT

DHT

DHT

Page 17: Operating Systems and Networks Where we are in the Course ... · Web (HTTP) Web (CDNs) P2P (BitTorrent ) Web (Video)??? Evolution of Internet Applications (2) • For a peek at the

17

97

P2POutlook• Alternative toCDN-styleclient-servercontentdistribution– Withpotentialadvantages

• P2PandDHTtechnologiesfindingmorewidespreaduseovertime– E.g.,partofskype,Amazoncloudcomputing– Expecthybridsystemsinthefuture