Recap• TCPconnectionsetup/teardown• Slidingwindow,flowcontrol• Retransmissiontimeouts• Fairness,max-minfairness• AIMDachievesmax-minfairness

81

FeedbackSignals• Severalpossiblesignals,withdifferentpros/cons

– We’lllookatclassicTCPthatusespacketlossasasignal

82

Signal Example Protocol Pros / ConsPacket loss TCP NewReno

Cubic TCP (Linux)Hard to get wrong

Hear about congestion latePacket delay Compound TCP

(Windows)Hear about congestion early

Need to infer congestionRouter

indicationTCPs with Explicit

Congestion NotificationHear about congestion early

Require router support

TCPTahoe/Reno• Avoidcongestioncollapsewithout

changingrouters(orevenreceivers)

• Ideaistofixtimeoutsandintroduceacongestionwindow (cwnd)overtheslidingwindowtolimitqueues/loss

• TCPTahoe/RenoimplementsAIMDbyadaptingcwnd usingpacketlossasthenetworkfeedbacksignal

84

TCPTahoe/Reno(2)• TCPbehaviorswewillstudy:

– ACK clocking– Adaptivetimeout(meanandvariance)– Slow-start– FastRetransmission– FastRecovery

• Together,theyimplementAIMD

SlidingWindowACKClock• Eachin-orderACK advancestheslidingwindowandletsanewsegmententerthenetwork– ACKs “clock”datasegments

Ack 12345678910

20191817161514131211Data

BenefitofACKClocking• Considerwhathappenswhensenderinjectsaburstofsegmentsintothenetwork

86

Fastlink FastlinkSlow(bottleneck)link

Queue

BenefitofACKClocking(2)• Segmentsarebufferedandspreadoutonslowlink

87

Fastlink FastlinkSlow(bottleneck)link

Segments“spreadout”

BenefitofACKClocking(3)• ACKs maintainthespreadbacktotheoriginalsender

88

SlowlinkAcks maintainspread

BenefitofACKClocking(4)• Senderclocksnewsegmentswiththespread

– Nowsendingatthebottlenecklinkwithoutqueuing!

89

Slowlink

Segmentsspread Queuenolongerbuilds

BenefitofACKClocking(4)• Helpsthenetworkrunwithlow

levelsoflossanddelay!

• Thenetworkhassmoothedouttheburstofdatasegments

• ACK clocktransfersthissmoothtimingbacktothesender

• Subsequentdatasegmentsarenotsentinburstssodonotqueueupinthenetwork

91

TCPStartupProblem• Wewanttoquicklyneartherightrate,cwndIDEAL,butitvariesgreatly– Fixedslidingwindowdoesn’tadaptandisroughonthenetwork(loss!)

– AIwithsmallburstsadaptscwndgentlytothenetwork,butmighttakealongtimetobecomeefficient

92

Slow-StartSolution• Startbydoublingcwnd everyRTT

– Exponentialgrowth(1,2,4,8,16,…)– Startslow,quicklyreachlargevalues

AI

Fixed

TimeWindo

w(cwnd

)

Slow-start

93

Slow-StartSolution(2)• Eventuallypacketlosswilloccurwhenthenetworkiscongested– Losstimeouttellsuscwnd istoolarge

– Nexttime,switchtoAIbeforehand– Slowlyadaptcwnd nearrightvalue

Extentofsubtitles

• Question:whatisthecwnd atwhichpacketlosswillhappenduringslowstart?

94

95

Slow-StartSolution(3)• Combinedbehavior,afterfirsttime

– Mosttimespentnearrightvalue

AI

Fixed

Time

Window

ssthresh

cwndC

cwndIDEALAIphase

Slow-start

Slow-Start(Doubling)Timeline

96

Incrementcwndby1packetforeachACK

AdditiveIncreaseTimeline

97

Incrementcwnd by1packeteverycwndACKs(or1RTT)

98

TCPTahoe(Implementation)• Initialslow-start(doubling)phase

– Startwithcwnd =1(orsmallvalue)– cwnd +=1packetperACK

• LaterAdditiveIncreasephase– cwnd +=1/cwnd packetsperACK– Roughlyadds1packetperRTT

• Switchingthreshold(initiallyinfinity)– SwitchtoAIwhencwnd >ssthresh– Setssthresh =cwnd/2afterloss– Beginwithslow-startaftertimeout

Extentofsubtitles

• HowcanweimproveonTCPTahoe?

99

100

TimeoutMisfortunes• Whydoaslow-startaftertimeout?

– InsteadofMDcwnd (forAIMD)

• TimeoutsaresufficientlylongthattheACK clockwillhaverundown– Slow-startrampsuptheACK clock

• WeneedtodetectlossbeforeatimeouttogettofullAIMD– DoneinTCPReno

101

InferringLossfromACKs• TCPusesacumulativeACK

– Carrieshighestin-orderseq.number– Normallyasteadyadvance

• DuplicateACKsgiveushintsaboutwhatdatahasn’tarrived– Tellussomenewdatadidarrive,butitwasnotnextsegment

– Thusthenextsegmentmaybelost

102

FastRetransmit• TreatthreeduplicateACKsasaloss

– Retransmitnextexpectedsegment– Somerepetitionallowsforreordering,butstilldetectslossquickly

Ack 1234555555

FastRetransmit(2)

103

Ack 10Ack 11Ack 12Ack 13

...

Ack 13

Ack 13Ack 13

Data14...Ack 13

Ack 20......

Data20ThirdduplicateACK,sosend14 Retransmissionfills

intheholeat14ACKjumpsafterlossisrepaired

......

Data14waslostearlier,butgot15to20

104

FastRetransmit(3)• Itcanrepairsinglesegmentloss

quickly,typicallybeforeatimeout

• However,wehavequiettimeatthesender/receiverwhilewaitingfortheACKtojump

• AndwestillneedtoMDcwnd …

105

InferringNon-LossfromACKs• DuplicateACKsalsogiveushintsaboutwhatdatahasarrived– EachnewduplicateACKmeansthatsomenewsegmenthasarrived

– Itwillbethesegmentsaftertheloss– Thusadvancingtheslidingwindowwillnotincreasethenumberofsegmentsstoredinthenetwork

106

FastRecovery• Firstfastretransmit,andMDcwnd• ThenpretendfurtherduplicateACKsaretheexpectedACKs– LetsnewsegmentsbesentforACKs– ReconcileviewswhentheACKjumps

Ack 1234555555

FastRecovery(2)

107

Ack 12Ack 13Ack 13

Ack 13Ack 13

Data14Ack 13

Ack 20......

Data20ThirdduplicateACK,sosend14

Data14waslostearlier,butgot15to20

Retransmissionfillsintheholeat14

Setssthresh,cwnd =cwnd/2

Data21Data22

MoreACKsadvancewindow;maysend

segmentsbeforejump

Ack 13

ExitFastRecovery

108

FastRecovery(3)• Withfastretransmit,itrepairsasingle

segmentlossquicklyandkeepstheACKclockrunning

• ThisallowsustorealizeAIMD– Notimeoutsorslow-startafterloss,just

continuewithasmallercwnd

• TCPRenocombinesslow-start,fastretransmitandfastrecovery– MultiplicativeDecreaseis½

TCPReno

109

MDof½,noslow-start

ACKclockrunning

TCPsawtooth

110

TCPReno,NewReno,andSACK• RenocanrepaironelossperRTT

– Multiplelossescauseatimeout

• NewReno furtherrefinesACKheuristics– Repairsmultiplelosseswithouttimeout

• SACKisabetteridea– ReceiversendsACKrangessosender

canretransmitwithoutguesswork

Extentofsubtitles

• Checkoutsimulationat:– http://guido.appenzeller.net/anims/– Or:goo.gl/sqmGWp

111

ComputerNetworks

ExplicitCongestionNotification

113

CongestionAvoidancevs.Control• ClassicTCPdrivesthenetworkintocongestionandthenrecovers– Needstoseelosstoslowdown

• Wouldbebettertousethenetworkbutavoidcongestionaltogether!– Reduceslossanddelay

• Question:howcanwedothiswithroutersupport?

FeedbackSignals• Delayandroutersignalscanletusavoidcongestion

114

Signal Example Protocol Pros / ConsPacket loss Classic TCP

Cubic TCP (Linux)Hard to get wrong

Hear about congestion latePacket delay Compound TCP

(Windows)Hear about congestion early

Need to infer congestionRouter

indicationTCPs with Explicit

Congestion NotificationHear about congestion early

Require router support

ECN(ExplicitCongestionNotification)• Routerdetectstheonsetofcongestionviaitsqueue

– Whencongested,itmarks affectedpackets(IPheader)

115

ECN(2)• Markedpacketsarriveatreceiver;treatedasloss

– TCPreceiverreliablyinformsTCPsenderofthecongestion

116

117

ECN(3)• Advantages:

– Routersdeliverclearsignaltohosts– Congestionisdetectedearly,noloss– Noextrapacketsneedtobesent

• Disadvantages:– Routersandhostsmustbeupgraded

TCPVariants• TherearemanydifferentstrainsofTCPincluding:– Loss-basedcongestioncontrol:Reno,BIC,Cubic– Delay-basedcongestioncontrol:Vegas,Veno,Westwood

– High-speedcongestioncontrol:Scalable,HighSpeed,HTCP

DelayBasedCongestionControl

119

• Basicidea:– Before packetlossoccurs,detecttheearlystageofcongestion intheroutersbetweensourceanddestination

– Additivelydecreasethesendingratewhenincipientcongestionisdetected

TCPVegas

120

• Expected =cwnd/BaseRTT• Actual = cwnd/RTT• DIFF = (Expected-Actual)

BaseRTT:theminimumofallmeasuredRTTif( DIFF*BaseRTT < α )

cwnd = cwnd + 1elseif( DIFF*BaseRTT > β )

cwnd = cwnd – 1elsecwnd = cwnd

RTT:theactualround-triptimeofataggedpacket

αandβareconstantvaluesthataresetbyexperimentation

TCPVegas

121

• ModifiedSlowStart– Trytofindthecorrectwindowsizewithoutincurringaloss

– exponentiallyincreasingitswindoweveryother RTTandusetheotherRTTtocalculateDIFF

– AssoonasVegasdetectsqueuebuildupduringslowstart,ittransitionstocongestionavoidance

Cubic• Twokeymodifications:

– Cubicwindowgrowthwithinflectionpointatcongestionwindowatpreviousloss

– Safeexitforslowstart(i.e.,transitionfromexponentialgrowthtolineargrowth)

Multipath• Mobileuser

– WiFi andcellularatthesametime• High-endservers

– MultipleEthernetcards• Datacenters

– Richtopologieswithmanypaths

• Question:whatarethebenefitsofmultipath?

MultipathTCPProtocol

124

WorkingWithUnmodifiedApps• PresentthesamesocketAPIandexpectations

– Identifiedbythe“fivetuple”(IPaddress,port#,protocol)

From http://queue.acm.org/detail.cfm?id=2591369

WorkingWithUnmodifiedHosts• EstablishtheTCPconnectioninthenormalway

– CreateasockettoasingleremoteIPaddress/port

• Andthenaddmoresubflows,ifpossible

A B

Each host tells its Initial Sequence Number (ISN)

to the other host.

NegotiatingMPTCPCapability• HowdohostsknowtheybothspeakMPTCP?

– Duringthe3-waySYN/SYN-ACK/ACKhandshake

• IfSYN-ACKdoesn’tcontainMP_CAPABLE– Don’ttrytoaddanysubflows!

AddingSubflows,Idealized• Howtoassociateanewsubflow withtheconnection?

– Useatokengeneratedfromoriginalsubflow set-up• Howtostartusingthenewsubflow?

– SimplystartsendingpacketswithnewIP/portpairs– …andassociatethemwiththeexistingconnection

• Howcouldtwoend-pointslearnaboutextraIPaddressesforestablishingnewsubflows?– Implicitly:oneend-pointestablishesanewsubflow,toalready-knownaddress(es)attheotherend-point

SequenceNumbers• Challengesacrosssubflows

– Out-of-orderpacketsduetoRTTdifferences– Accessnetworksthatrewritesequencenumbers– Middleboxes upsetbydiscontinuousTCPbytestream– Needtoretransmitlostpacketsonadifferentsubflow

• Twolevelsofsequencenumbers– Sequencenumberspersubflow– Sequencenumbersfortheentireconnection

• Enables– Efficientdetectionoflossoneachsubflow– Retransmissionoflostpacketonadifferentsubflow

ReceiveBufferSpace• EachTCPconnectionhasareceivebuffer

– Bufferspacetostoreincomingdata– …untilitisreadbytheapplication

• TCPflowcontrol– Receiveradvertisestheavailablebufferspace– …usingthe“receivewindow”

• Shouldeachsubflow haveitsownreceivewindow?– Starvationofsomesubflows inaconnection?– FairnessrelativetootherTCPconnections?– Fragmentationoftheavailablebufferspace?

• Instead,useacommonreceivewindow

FairnessandEfficiencyinMultipathCongestionControl

SlidesfromDamonWischik

Goal#1:FairnessatSharedBottlenecks

Tobefair,MultipathTCPshouldtakeasmuchcapacityasTCPatabottlenecklink,nomatterhowmanypathsitisusing.

A multipath TCP flow with two subflows

RegularTCP

Goal#2:UseEfficientPaths

Eachflowhasachoiceofa1-hopanda2-hoppath.Howshouldsplititstraffic?

12Mb/s

12Mb/s12Mb/s

UseEfficientPaths

Ifeachflowsplititstraffic1:1...

8Mb/s

8Mb/s

8Mb/s

12Mb/s

12Mb/s 12Mb/

s

UseEfficientPaths

Ifeachflowsplititstraffic2:1...

9Mb/s

9Mb/s

9Mb/s

12Mb/s

12Mb/s

12Mb/s

UseEfficientPaths

Better:Eachconnectiononaone-hoppathEachconnectionshouldsendalltrafficontheleast-congestedpaths

12Mb/s

12Mb/s

12Mb/s

12Mb/s

12Mb/s12Mb/s

UseEfficientPaths

Better:Eachconnectiononaone-hoppathEachconnectionshouldsendalltrafficontheleast-congestedpathsButkeepsometrafficonthealternatepathsasaprobe

12Mb/s

12Mb/s

12Mb/s

12Mb/s

12Mb/s12Mb/s

Goal#3:BeFairComparedtoTCP• Least-congestedpathsmaynotbebest!

– Duetodifferencesinround-triptime

• Twopaths– WiFi:highloss,lowRTT– Cellular:lowloss,highRTT

• Usingtheleast-congestedpath– Choosethecellularpath,duetolowloss– But,theRTTishigh– Sothroughputislow!

BeFairComparedtoTCP• Tobefair,MultipathTCPshouldgiveaconnectionatleastasmuchthroughput

asitwouldgetwithasingle-pathTCPonthebestofitspaths.– EnsureincentivefordeployingMPTCP

• AMultipathTCPshouldtakenomorecapacityonanypath(orcollectionofpaths)thanifitwasasingle-pathTCPflowusingthebestofthosepaths.– Donoharm!

AchievingTheseGoals• RegularTCP

– Maintainacongestionwindoww– OnanACK,increaseby1/w(increase1perwindow)– Onaloss,decreasebyw/2

• MPTCP– Maintainacongestionwindowperpathwr– OnanACKonpathr,increasewr– Onalossonpathr,decreasebywr/2

• Howmuchtoincreasewr onanACK??– Ifristheonlypathatthatbottleneck,increaseby1/wr

IfMultiplePathsShareBottleneck?• Don’ttakeanymorebandwidthonalinkthanthebestoftheTCPpathswould

– But,wheremightthebottlenecksbe?– Multiplepathsmightsharethesame bottleneck

• So,considerallpossiblesubsets ofthepaths– SetRofpaths– SubsetSofRthatincludespathr

• E.g.,considerpath3– Supposepaths1,3,and4shareabottleneck– …but,path2doesnot– Then,wecareaboutS={1,3,4}

AchievingTheseGoals• Whatisthebest ofthesesubflowsachieving?

– Pathsisachievingthroughputofws/RTTs– Sobestpathisgettingmaxs(ws/RTTs)

• Whattotal bandwidtharethesesubflowsgetting?– Acrossall subflowssharingthatbottleneck– SumoversinSofws/RTTs

• Considertheratio ofthetwo– Increasebylessifmanysubflowsaresharing

• AndpicktheresultsforthesetSwithminratio– Toaccountforthemost pathssharingabottleneck