table of contents€¦ · http2 explained this is a detailed document describing http/2 (rfc 7540),...
TRANSCRIPT
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
1.10
1.11
1.12
1.13
1.14
1.15
TableofContentsIntroduction
Background
HTTPToday
Thingsdonetoovercomelatencypains
UpdatingHTTP
http2concepts
Thehttp2protocol
Extensions
Anhttp2world
http2inFirefox
http2inChromium
http2incurl
Afterhttp2
Furtherreading
Thanks
2
http2explainedThisisadetaileddocumentdescribingHTTP/2(RFC7540),thebackground,concepts,protocolandsomethingaboutexistingimplementationsandwhatthefuturemighthold.
Seehttps://daniel.haxx.se/http2/forthecanonicalhomeforthisproject.
Seehttps://github.com/bagder/http2-explainedforthesourcecodeofallbookcontents.
CONTRIBUTINGIencourageandwelcomehelpandcontributionsfromanyonewhomayhaveimprovementstooffer.Weacceptpullrequests,butyoucanalsojustfileissuesorsendemailtodaniel-http2@haxx.sewithyoursuggestions!
/DanielStenberg
Introduction
3
1.BackgroundThisdocumentdescribeshttp2fromatechnicalandprotocollevel.ItstartedoutasapresentationDanieldidinStockholminApril2014thatwassubsequentlyconvertedandextendedintoafull-blowndocumentwithalldetailsandproperexplanations.
RFC7540istheofficialnameofthefinalhttp2specificationanditwaspublishedonMay15th2015:https://www.rfc-editor.org/rfc/rfc7540.txt
Allandanyerrorsinthisdocumentaremyownandtheresultsofmyshortcomings.Pleasepointthemoutandtheywillbefixedinupdatedversions.
InthisdocumentI'vetriedtoconsistentlyusetheword"http2"todescribethenewprotocolwhileinpuretechnicalterms,thepropernameisHTTP/2.Imadethischoiceforthesakeofreadabilityandtogetabetterflowinthelanguage.
1.1AuthorMynameisDanielStenberg.I'vebeenworkingwithopensourceandnetworkingforovertwentyyearsinnumerousprojects.PossiblyI'mbestknownforbeingtheleaddeveloperofcurlandlibcurl.I'vebeeninvolvedintheIETFHTTPbisworkinggroupforseveralyearsandthereI'vekeptup-to-datewiththerefreshedHTTP1.1workaswellasbeinginvolvedinthehttp2standardizationwork.
Email:[email protected]
Twitter:@bagder
Web:daniel.haxx.se
Blog:daniel.haxx.se/blog
1.2Help!Ifyoufindmistakes,omissions,errorsorblatantliesinthisdocument,pleasesendmearefreshedversionoftheaffectedparagraphandI'llmakeamendedversions.Iwillgivepropercreditstoeveryonewhohelpsout!Ihopetomakethisdocumentbetterovertime.
Thisdocumentisavailableathttps://daniel.haxx.se/http2
1.3License
ThisdocumentislicensedundertheCreativeCommonsAttribution4.0license:https://creativecommons.org/licenses/by/4.0/
1.4Documenthistory
Background
4
ThefirstversionofthisdocumentwaspublishedonApril25th2014.Herefollowsthelargestchangesinthemostrecentdocumentversions.
Version1.13
ConvertedthemasterversionofthisdocumenttoMarkdownsyntax13:Mentionmoreresources,updatedlinksanddescriptions12:UpdatedtheQUICdescriptionwithreferencetodraft8.5:Refreshedwithcurrentnumbers3.4:Theaverageisnow40TCPconnections6.4:Updatedtoreflectwhatthespecsays
Version1.12
1.1:HTTP/2isnowinanofficialRFC6.5.1:LinktotheHPACKRFC9.1:MentiontheFirefox36+configswitchforhttp212.1:AddedsectionaboutQUIC
Version1.11
Lotsoflanguageimprovementsmostlypointedoutbyfriendlycontributors8.3.1:MentionnginxandApachehttpdspecificacitivities
Version1.10
1:Theprotocolhasbeen“okayed”4.1:Refreshedthewordingsince2014islastyearFront:Addedimageandcallit“http2explained”there,fixedlink1.4:AddeddocumenthistorysectionManyspellingandgrammarmistakescorrected14:Addedthankstobugreporters2.4:BetterlabelsfortheHTTPgrowthgraph6.3:Correctedthewagonorderinthemultiplexedtrain6.5.1:HPACKdraft-12
Version1.9
UpdatedtoHTTP/2draft-17andHPACKdraft-11Addedsection"10.http2inChromium"(==onepagelongernow)LotsofspellfixesAt30implementationsnow8.5:Addedsomecurrentusagenumbers8.3:Mentioninternetexplorertoo8.3.1Added"missingimplementations"8.4.3:MentionthatTLSalsoincreasessuccessrate
Background
5
2.HTTPtodayHTTP1.1hasturnedintoaprotocolusedforvirtuallyeverythingontheInternet.Hugeinvestmentshavebeenmadeinprotocolsandinfrastructurethattakeadvantageofthis,totheextentthatitisofteneasiertodaytomakethingsrunontopofHTTPratherthanbuildingsomethingnewonitsown.
2.1HTTP1.1ishugeWhenHTTPwascreatedandthrownoutintotheworld,itwasprobablyperceivedasarathersimpleandstraightforwardprotocol,buttimehasprovedthattobefalse.HTTP1.0inRFC1945isa60-pagespecificationreleasedin1996.RFC2616thatdescribesHTTP1.1wasreleasedonlythreeyearslaterin1999andhadgrownsignificantlyto176pages.YetwhenwewithinIETFworkedontheupdatetothatspec,itwassplitupandconvertedintosixdocumentswithamuchlargerpagecountintotal(resultinginRFC7230andfamily).Byanycount,HTTP1.1isbigandincludesamyriadofdetails,subtletiesand,nottheleast,alotofoptionalparts.
2.2AworldofoptionsHTTP1.1'snatureofhavinglotsoftinydetailsandoptionsavailableforlaterextensionshasgrownasoftwareecosystemwherealmostnoimplementationeverimplementseverything–anditisn'tevenreallypossibletoexactlytellwhat“everything”is.Thishasledtoasituationwherefeaturesthatwereinitiallylittle-usedsawveryfewimplementations,andthosethatdidimplementthefeaturesthensawverylittleuseofthem.
Lateron,thiscausedaninteroperabilityproblemwhenclientsandserversstartedtoincreasetheuseofsuchfeatures.HTTPpipeliningisaprimaryexampleofsuchafeature.
2.3InadequateuseofTCPHTTP1.1hasahardtimereallytakingfulladvantageofallthepowerandperformancethatTCPoffers.HTTPclientsandbrowsershavetobeverycreativetofindsolutionsthatdecreasepageloadtimes.
OtherattemptsthathavebeengoingoninparallelovertheyearshavealsoconfirmedthatTCPisnotthateasytoreplace,andthuswekeepworkingonimprovingbothTCPandtheprotocolsontopofit.
Simplyput,TCPcanbeutilizedbettertoavoidpausesorwastedintervalsthatcouldhavebeenusedtosendorreceivemoredata.Thefollowingsectionswillhighlightsomeoftheseshortcomings.
2.4TransfersizesandnumberofobjectsWhenlookingatthetrendforsomeofthemostpopularsitesonthewebtodayandwhatittakestodownloadtheirfrontpages,aclearpatternemerges.Overtheyears,theamountofdatathatneedstoberetrievedhasgraduallyrisenuptoandabove1.9MB.Whatismoreimportantinthiscontextisthat,onaverage,over100individualresourcesarerequiredtodisplayeachpage.
Asthegraphbelowshows,thetrendhasbeengoingonforawhile,andthereislittletonoindicationthatitwillchangeanytimesoon.Itshowsthegrowthofthetotaltransfersize(ingreen)andthetotalnumberofrequestsusedonaverage(inred)toservethemostpopularwebsitesintheworld,andhowtheyhavechangedoverthelastfouryears.
HTTPToday
6
2.5Latencykills
HTTP1.1isverylatencysensitive,partlybecauseHTTPpipeliningisstillriddledwithenoughproblemstoremainswitchedofftoalargepercentageofusers.
Whilewe'veseenagreatincreaseinavailablebandwidthtopeopleoverthelastfewyears,wehavenotseenthesamelevelofimprovementsinreducinglatency.High-latencylinks,likemanyofthecurrentmobiletechnologies,makeithardtogetagoodandfastwebexperienceevenifyouhaveareallyhighbandwidthconnection.
Anotherusecaserequiringlowlatencyiscertainkindsofvideo,likevideoconferencing,gamingandsimilarwherethere'snotjustapre-generatedstreamtosendout.
2.6.Head-of-lineblockingHTTPpipeliningisawaytosendanotherrequestwhilewaitingfortheresponsetoapreviousrequest.Itisverysimilartoqueuingatacounteratthebankorinasupermarket:youjustdon'tknowifthepersoninfrontofyouisaquickcustomerorthatannoyingonethatwilltakeforeverbeforehe/sheisdone.Thisisknownashead-of-lineblocking.
HTTPToday
7
Sure,youcanattempttopickthelineyoubelieveisthecorrectone,andattimesyoucanevenstartanewlineofyourown.Butintheend,youcan'tavoidmakingadecision.Andonceitismade,youcannotswitchlines.
Creatinganewlineisalsoassociatedwithaperformanceandresourcepenalty,sothat'snotscalablebeyondasmallernumberoflines.There'sjustnoperfectsolutiontothis.
Eventoday,mostdesktopwebbrowsersshipwithHTTPpipeliningdisabledbydefault.
AdditionalreadingonthissubjectcanbefoundintheFirefoxbugzillaentry264354.
HTTPToday
8
3.ThingsdonetoovercomelatencypainsWhenfacedwithproblems,peopletendtogathertofindworkarounds.Someoftheworkaroundsarecleveranduseful,butothersarejustawfulkludges.
3.1Spriting
Spritingisthetermoftenusedtodescribecombiningmultiplesmallimagestoformasinglelargerimage.Then,usingJavaScriptorCSS,you“cutout”piecesofthatbigimagetoshowsmallerindividualones.
Asitewouldusethistrickforspeed.GettingasinglebigimageinHTTP1.1ismuchfasterthangetting100smallerindividualones.
Ofcourse,thishasitsdownsidesforthepagesofthesitethatonlywanttoshowoneortwoofthesmallpictures.Spritingalsocausesallimagestoberemovedsimultaneouslywhenthecacheisclearedinsteadofpossiblylettingthemostcommonlyusedonesremain.
3.2InliningInliningisanothertrickusedtoavoidsendingindividualimages,andthisisdonebyusingdataURLsembeddedintheCSSfile.Thishassimilarbenefitsanddrawbacksasthespritingcase.
.icon1{
background:url(data:image/png;base64,<data>)no-repeat;
}
.icon2{
background:url(data:image/png;base64,<data>)no-repeat;
}
3.3ConcatenationAbigsitecanendupwithalotofdifferentJavaScriptfiles.Developerscanusefront-endtoolstoconcatenate,orcombine,multiplescriptssothatthebrowserwillgetasinglebigfileinsteadofdozensofsmallerones.Toomuchdataissentwhenonlylittleisneededand,likewise,toomuchdataneedstobereloadedwhenachangeismade.
Thispracticeis,ofcourse,mostlyaninconveniencetothedevelopersinvolved.
Thingsdonetoovercomelatencypains
9
3.4ShardingThefinalperformancetrickI'llmentionisoftenreferredtoas“sharding.”Itbasicallymeansservingaspectsofyourserviceonasmanydifferenthostsaspossible.Atfirstglancethisseemsstrange,butthereissoundreasoningbehindit.
Initially,theHTTP1.1specificationstatedthataclientwasallowedtouseamaximumoftwoTCPconnectionsforeachhost.So,inordertonotviolatethespec,cleversitessimplyinventednewhostnamesand–voilà–youcouldgetmoreconnectionstoyoursiteanddecreasedpageloadtimes.
Overtimethatlimitationwasremoved,andtodayclientseasilyusesixtoeightconnectionsperhostname.Buttheystillhavealimit,sositescontinuetousethistechniquetobumpupthenumberofconnections.AsthenumberofobjectsrequestedoverHTTPisever-increasing–asIshowedbefore–thelargenumberofconnectionsisthenusedtomakesureHTTPperformswellandallowyoursitetoloadquickly.Itisnotunusualforsitestousewellover50orevenupto100ormoreconnectionsnowforasinglesiteusingthistechnique.Recentstatsfromhttparchive.orgshowthatthetop300KURLsintheworldneed,onaverage,40(!)TCPconnectionstodisplaythesite,andthetrendsaysthisisstillincreasingslowlyovertime.
Anotherreasonforshardingistoputimagesorsimilarresourcesonaseparatehostnamethatdoesn'tuseanycookies,asthesizeofcookiesthesedayscanbequitesignificant.Byusingcookie-freeimagehosts,youcansometimesincreaseperformancesimplybyallowingmuchsmallerHTTPrequests!
TheimagebelowshowswhatapackettracelookslikewhenbrowsingoneofSweden'stopwebsitesandhowrequestsaredistributedoverseveralhostnames.
Thingsdonetoovercomelatencypains
10
Thingsdonetoovercomelatencypains
11
4.UpdatingHTTPWouldn'titbenicetomakeanimprovedprotocol?Itwould...
1. Belesslatencysensitive2. Fixpipeliningandtheheadoflineblockingproblem3. Eliminatetheneedtokeepincreasingthenumberofconnectionstoeachhost4. Keepallexistinginterfaces,allcontent,theURIformatsandschemes5. BemadewithintheIETF'sHTTPbisworkinggroup
4.1.IETFandtheHTTPbisworkinggroupTheInternetEngineeringTaskForce(IETF)isanorganizationthatdevelopsandpromotesinternetstandards,mostlyontheprotocollevel.They'rewidelyknownfortheRFCseriesofmemosdocumentingeverythingfromTCP,DNS,andFTP,tobestpractices,HTTP,andnumerousprotocolvariantsthatneverwentanywhere.
WithinIETF,dedicated“workinggroups”areformedwithalimitedscopetoworktowardagoal.Theyestablisha“charter”withsomesetguidelinesandlimitationsforwhattheyshouldproduce.Everyoneandanyoneisallowedtojoininthediscussionsanddevelopment.Everyonewhoattendsandsayssomethinghasthesameweightandchancetoaffecttheoutcomeandeveryoneiscountedasanindividual,withlittleregardtowhichcompanytheyworkfor.
TheHTTPbisworkinggroup(seelaterforanexplanationofthename)wasformedduringthesummerof2007andtaskedwithcreatinganupdateoftheHTTP1.1specification.Withinthisgroupthediscussionsaboutanext-versionHTTPreallystartedduringlate2012.TheHTTP1.1updatingworkwascompletedearly2014andresultedintheRFC7230series.
Thefinalinter-opmeetingfortheHTTPbisWGwasheldinNewYorkCityinthebeginningofJune2014.TheremainingdiscussionsandtheIETFproceduresperformedtoactuallygettheofficialRFCoutcontinuedintothefollowingyear.
SomeofthebiggerplayersintheHTTPfieldhavebeenmissingfromtheworkinggroupdiscussionsandmeetings.Idon'twanttomentionanyparticularcompanyorproductnameshere,butclearlysomeactorsontheInternettodayseemtobeconfidentthatIETFwilldogoodwithoutthesecompaniesbeinginvolved...
4.1.1.The"bis"partofthename
ThegroupisnamedHTTPbiswherethe"bis"partcomesfromtheLatinadverbfortwo.BisiscommonlyusedasasuffixorpartofthenamewithintheIETFforanupdateorthesecondtakeonaspec;inthiscase,theupdatetoHTTP1.1.
4.2.http2startedfromSPDYSPDYisaprotocolthatwasdevelopedandspearheadedbyGoogle.Theycertainlydevelopeditintheopenandinvitedeveryonetoparticipatebutitwasobviousthattheybenefitedbybeingincontroloverbothapopularbrowserimplementationandasignificantserverpopulationwithwell-usedservices.
WhentheHTTPbisgroupdecideditwastimetostartworkingonhttp2,SPDYhadalreadyproventhatitwasaworkingconcept.IthadshownitwaspossibletodeployontheInternetandtherewerepublishednumbersthatprovedhowwellitperformed.Thehttp2workbeganwiththeSPDY/3draftthatwasbasicallymadeintothehttp2draft-00withalittlesearchandreplace.
UpdatingHTTP
12
5.http2conceptsSowhatdoeshttp2accomplish?WherearetheboundariesforwhattheHTTPbisgroupsetouttodo?
Theboundarieswereactuallyquitestrictandputmanyrestraintsontheteam'sabilitytoinnovate:
http2hastomaintainHTTPparadigms.ItisstillaprotocolwheretheclientsendsrequeststotheserveroverTCP.
http://andhttps://URLscannotbechanged.Therecanbenonewschemeforthis.TheamountofcontentusingsuchURLsistoobigtoexpectthemtochange.
HTTP1serversandclientswillbearoundfordecades,weneedtobeabletoproxythemtohttp2servers.
Subsequently,proxiesmustbeabletomaphttp2featurestoHTTP1.1clientsone-to-one.
Removeorreduceoptionalpartsfromtheprotocol.Thiswasn'treallyarequirementbutmoreamantracomingfromSPDYandtheGoogleteam.Bymakingsureeverythingismandatorythere'snowayyoucannotimplementanythingnowandfallintoatraplateron.
Nomoreminorversion.Itwasdecidedthatclientsandserversareeithercompatiblewithhttp2ortheyarenot.Ifaneedarisestoextendtheprotocolormodifythings,thenhttp3willbeborn.Therearenomoreminorversionsinhttp2.
5.1.http2forexistingURIschemesAsmentionedalready,theexistingURIschemescannotbemodified,sohttp2mustusetheexistingones.SincetheyareusedforHTTP1.xtoday,weobviouslyneedawaytoupgradetheprotocoltohttp2,orotherwiseasktheservertousehttp2insteadofolderprotocols.
HTTP1.1hasadefinedwaytodothis,namelytheUpgrade:header,whichallowstheservertosendbackaresponseusingthenewprotocolwhengettingsucharequestovertheoldprotocol,atthecostofanadditionalround-trip.
Thatround-trippenaltywasnotsomethingtheSPDYteamwouldaccept,andsincetheyonlyimplementedSPDYoverTLS,theydevelopedanewTLSextensionwhichshortcutsthenegotiationsignificantly.Usingthisextension,calledNPNforNextProtocolNegotiation,theservertellstheclientwhichprotocolsitknowsandtheclientcanthenusetheprotocolitprefers.
5.2.http2forhttps://Alotoffocusofhttp2hasbeentomakeitbehaveproperlyoverTLS.SPDYrequiresTLSandthere'sbeenastrongpushformakingTLSmandatoryforhttp2,butitdidn'tgetconsensussohttp2shippedwithTLSasoptional.However,twoprominentimplementershavestatedclearlythattheywillonlyimplementhttp2overTLS:theMozillaFirefoxleadandtheGoogleChromelead,twooftoday'sleadingwebbrowsers.
ReasonsforchoosingTLS-onlyincluderespectforuser'sprivacyandearlymeasurementsshowingthatthenewprotocolshaveahighersuccessratewhendonewithTLS.Thisisbecauseofthewidespreadassumptionthatanythingthatgoesoverport80isHTTP1.1,whichmakessomemiddle-boxesinterferewithordestroytrafficwhenanyotherprotocolsareusedonthatport.
ThesubjectofmandatoryTLShascausedmuchhand-wringingandagitatedvoicesinmailinglistsandmeetings–isitgoodorisitevil?Itisahighlycontroversialtopic–beawareofthiswhenyouthrowthisquestioninthefaceofanHTTPbisparticipant!
Similarly,there'sbeenafierceandlong-runningdebateaboutwhetherhttp2shoulddictatealistofciphersthatshouldbemandatorywhenusingTLS,orifitshouldperhapsblacklistaset,orifitshouldn'trequireanythingatallfromtheTLS“layer”butleavethattotheTLSworkinggroup.ThespecendedupspecifyingthatTLSshouldbeatleastversion1.2and
http2concepts
13
thereareciphersuiterestrictions.
5.3.http2negotiationoverTLSNextProtocolNegotiation(NPN)istheprotocolusedtonegotiateSPDYwithTLSservers.Asitwasn'taproperstandard,itwastakenthroughtheIETFandtheresultwasALPN:ApplicationLayerProtocolNegotiation.ALPNisbeingpromotedforusebyhttp2,whileSPDYclientsandserversstilluseNPN.
ThefactthatNPNexistedfirstandALPNhastakenawhiletogothroughstandardizationhasledtomanyearlyhttp2clientsandhttp2serversimplementingandusingboththeseextensionswhennegotiatinghttp2.Also,NPNiswhat'susedforSPDYandmanyserversofferbothSPDYandhttp2,sosupportingbothNPNandALPNonthoseserversmakesperfectsense.
ALPNdiffersfromNPNprimarilyinwhodecideswhatprotocoltospeak.WithALPN,theclientgivestheserveralistofprotocolsinitsorderofpreferenceandtheserverpickstheoneitwants,whilewithNPNtheclientmakesthefinalchoice.
5.4.http2forhttp://Aspreviouslymentioned,forplain-textHTTP1.1thewaytonegotiatehttp2isbypresentingtheserverwithanUpgrade:header.Iftheserverspeakshttp2itrespondswitha“101Switching”statusandfromthenonitspeakshttp2onthatconnection.Ofcoursethisupgradeprocedurecostsafullnetworkround-trip,buttheupsideisthatit'sgenerallypossibletokeepanhttp2connectionalivemuchlongerandre-useitmorethanatypicalHTTP1connection.
Whilesomebrowsers'spokespersonsstatedtheywillnotimplementthismeansofspeakinghttp2,theInternetExplorerteamonceexpressedthattheywould-althoughtheyhaveneverdeliveredonthat.curlandafewothernon-browserclientssupportclear-texthttp2.
Today,nomajorbrowsersupportshttp2withoutTLS.
http2concepts
14
6.Thehttp2protocolEnoughaboutthebackground,thehistoryandpoliticsbehindwhatgotushere.Let'sdiveintothespecificsoftheprotocol:thebitsandtheconceptsthatmakeuphttp2.
6.1.Binaryhttp2isabinaryprotocol.
Justletthatsinkinforaminute.Ifyou'vebeeninvolvedininternetprotocolsbefore,chancesarethatyouwillnowbeinstinctivelyreactingagainstthischoice,marshalingyourargumentsthatspellouthowprotocolsbasedontext/asciiaresuperiorbecausehumanscanhandcraftrequestsovertelnetandsoon...
http2isbinarytomaketheframingmucheasier.FiguringoutthestartandtheendofframesisoneofthereallycomplicatedthingsinHTTP1.1and,actually,intext-basedprotocolsingeneral.Bymovingawayfromoptionalwhitespaceanddifferentwaystowritethesamething,implementationbecomessimpler.
Also,itmakesitmucheasiertoseparatetheactualprotocolpartsfromtheframing-whichinHTTP1isconfusinglyintermixed.
ThefactthattheprotocolfeaturescompressionandwilloftenrunoverTLSalsodiminishesthevalueoftext,sinceyouwon'tseetextoverthewireanyway.WesimplyhavetogetusedtotheideaofusingsomethinglikeaWiresharkinspectortofigureoutexactlywhat'sgoingonattheprotocollevelinhttp2.
Debuggingthisprotocolwillprobablyhavetobedonewithtoolslikecurl,orbyanalyzingthenetworkstreamwithWireshark'shttp2dissectorandsimilar.
6.2.Thebinaryformat
http2sendsbinaryframes.Therearedifferentframetypesthatcanbesentandtheyallhavethesamesetup:Length,Type,Flags,StreamIdentifier,andframepayload.
Therearetendifferentframetypesdefinedinthehttp2specandperhapsthetwomostfundamentalonesthatmaptoHTTP1.1featuresareDATAandHEADERS.I'lldescribesomeoftheframesinmoredetailfurtheron.
6.3.MultiplexedstreamsTheStreamIdentifiermentionedintheprevioussectionassociateseachframesentoverhttp2witha“stream”.Astreamisanindependent,bi-directionalsequenceofframesexchangedbetweentheclientandserverwithinanhttp2connection.
Thehttp2protocol
15
Asinglehttp2connectioncancontainmultipleconcurrently-openstreams,witheitherendpointinterleavingframesfrommultiplestreams.Streamscanbeestablishedandusedunilaterallyorsharedbyeithertheclientorserverandtheycanbeclosedbyeitherendpoint.Theorderinwhichframesaresentwithinastreamissignificant.Recipientsprocessframesintheordertheyarereceived.
Multiplexingthestreamsmeansthatpackagesfrommanystreamsaremixedoverthesameconnection.Two(ormore)individualtrainsofdataaremadeintoasingleoneandthensplitupagainontheotherside.Herearetwotrains:
Thetwotrainsmultiplexedoverthesameconnection:
6.4.PrioritiesanddependenciesEachstreamalsohasapriority(alsoknownas“weight”),whichisusedtotellthepeerwhichstreamstoconsidermostimportant,incasethereareresourcerestraintsthatforcetheservertoselectwhichstreamstosendfirst.
UsingthePRIORITYframe,aclientcanalsotelltheserverwhichotherstreamthisstreamdependson.Itallowsaclienttobuildapriority“tree”whereseveral“childstreams”maydependonthecompletionof“parentstreams”.
Thehttp2protocol
16
Thepriorityweightsanddependenciescanbechangeddynamicallyatrun-time,whichshouldenablebrowserstomakesurethatwhenusersscrolldownapagefullofimages,thebrowsercanspecifywhichimagesaremostimportant,orifyouswitchtabsitcanprioritizeanewsetofstreamsthatsuddenlycomeintofocus.
6.5.HeadercompressionHTTPisastatelessprotocol.Inshort,thismeansthateveryrequestneedstobringwithitasmuchdetailastheserverneedstoservethatrequest,withouttheserverhavingtostorealotofinfoandmeta-datafrompreviousrequests.Sincehttp2doesn'tchangethisparadigm,ithastoworkthesameway.
ThismakesHTTPrepetitive.Whenaclientasksformanyresourcesfromthesameserver,likeimagesfromawebpage,therewillbealargeseriesofrequeststhatalllookalmostidentical.Aseriesofalmostidenticalsomethingsbegsforcompression.
Whilethenumberofobjectsperwebpagehasincreased(asmentionedearlier),theuseofcookiesandthesizeoftherequestshavealsokeptgrowingovertime.Cookiesalsoneedtobeincludedinallrequests,oftenthesameonesinmultiplerequests.
TheHTTP1.1requestsizeshaveactuallygottensolargethattheysometimesenduplargerthantheinitialTCPwindow,whichmakesthemveryslowtosendastheyneedafullround-triptogetanACKbackfromtheserverbeforethefullrequesthasbeensent.Thisisanotherargumentforcompression.
6.5.1.Compressionisatrickysubject
HTTPSandSPDYcompressionwerefoundtobevulnerabletotheBREACHandCRIMEattacks.Byinsertingknowntextintothestreamandfiguringouthowthatchangestheoutput,anattackercanfigureoutwhat'sbeingsentinanencryptedpayload.
Doingcompressionondynamiccontentforaprotocol-withoutbecomingvulnerabletooneoftheseattacks-requiressomethoughtandcarefulconsideration.ThisiswhattheHTTPbisteamtriedtodo.
EnterHPACK,HeaderCompressionforHTTP/2,which–asthenamesuggests-isacompressionformatespeciallycraftedforhttp2headers,anditisbeingspecifiedinaseparateinternetdraft.Thenewformat,togetherwithothercounter-measures(suchasabitthatasksintermediariestonotcompressaspecificheaderandoptionalpaddingofframes),shouldmakeithardertoexploitcompression.
InthewordsofRobertoPeon(oneofthecreatorsofHPACK):
“HPACKwasdesignedtomakeitdifficultforaconformingimplementationtoleakinformation,tomakeencodinganddecodingveryfast/cheap,toprovideforreceivercontrolovercompressioncontextsize,toallowforproxyre-indexing(i.e.,sharedstatebetweenfrontendandbackendwithinaproxy),andforquickcomparisonsofHuffman-encodedstrings”.
6.6.Reset-changeyourmindOneofthedrawbackswithHTTP1.1isthatwhenanHTTPmessagehasbeensentoffwithaContent-Lengthofacertainsize,youcan'teasilyjuststopit.Sure,youcanoften(butnotalways)disconnecttheTCPconnection,butthatcomesatthecostofhavingtonegotiateanewTCPhandshakeagain.
Abettersolutionwouldbetojuststopthemessageandstartanew.Thiscanbedonewithhttp2'sRST_STREAMframewhichwillhelppreventwastedbandwidthandtheneedtoteardownconnections.
6.7.Serverpush
Thehttp2protocol
17
Thisisthefeaturealsoknownas“cachepush”.TheideaisthatiftheclientasksforresourceX,theservermayknowthattheclientwillprobablywantresourceZaswell,andsendsittotheclientwithoutbeingasked.IthelpstheclientbyputtingZintoitscachesothatitwillbetherewhenitwantsit.
Serverpushissomethingaclientmustexplicitlyallowtheservertodo.Eventhen,theclientcanswiftlyterminateapushedstreamatanytimewithRST_STREAMshoulditnotwantaparticularresource.
6.8.FlowControlEachindividualhttp2streamhasitsownadvertisedflowwindowthattheotherendisallowedtosenddatafor.IfyouhappentoknowhowSSHworks,thisisverysimilarinstyleandspirit.
Foreverystream,bothendshavetotellthepeerthatithasenoughroomtohandleincomingdata,andtheotherendisonlyallowedtosendthatmuchdatauntilthewindowisextended.OnlyDATAframesareflowcontrolled.
Thehttp2protocol
18
7.ExtensionsThehttp2protocolmandatesthatareceivermustreadandignoreallunknownframes(thosewithanunknownframetype).Twopartiescannegotiatetheuseofnewframetypesonahop-by-hopbasis,butthoseframesaren'tallowedtochangestateandtheywillnotbeflowcontrolled.
Thesubjectofwhetherhttp2shouldallowextensionsatallwasdebatedatlengthduringtheprotocol'sdevelopmentwithopinionsswingingforandagainst.Afterdraft-12thependulumswungbackonelasttimeandextensionswereultimatelyallowed.
Extensionsarenotpartoftheactualprotocolbutwillbedocumentedoutsideofthecoreprotocolspec.Therearealreadytwoframetypesthathavebeendiscussedforinclusionintheprotocolthatwillprobablybethefirstframessentasextensions.I'lldescribethemherebecauseoftheirpopularityandpreviousstateas“native”frames:
7.1.AlternativeServicesWiththeadoptionofhttp2,therearereasonstosuspectthatTCPconnectionswillbemuchlengthierandbekeptalivemuchlongerthanHTTP1.xconnectionshavebeen.Aclientshouldbeabletodoalotofwhatitwantswithasingleconnectiontoeachhost/site,andthatconnectioncouldpotentiallybeopenforquitesometime.
ThiswillaffecthowHTTPloadbalancersworkandtheremayarisesituationswhenasitewantstosuggestthattheclientconnecttoanotherhost.Itcouldbeforperformancereasons,orifasiteisbeingtakendownformaintenance,etc.
TheserverwillsendtheAlt-Svc:header(orALTSVCframewithhttp2)tellingtheclientaboutanalternativeservice:anotherroutetothesamecontent,usinganotherservice,host,andportnumber.
Aclientshouldthenattempttoconnecttothatserviceasynchronouslyandonlyusethealternativeifthenewconnectionsucceeds.
7.1.1.OpportunisticTLS
TheAlt-Svcheaderallowsaserverthatprovidescontentoverhttp://toinformtheclientthatthesamecontentisalsoavailableoveraTLSconnection.
Thisisasomewhatdebatablefeature.SuchaconnectionwoulddounauthenticatedTLSandwouldn'tbeadvertizedas“secure”anywhere,wouldn'tuseanypadlockintheUI,andinfactthereisnowaytotelltheuserthatitisn'tplainoldHTTP,butthisisstillopportunisticTLSandsomepeopleareveryfirmlyagainstthisconcept.
7.2.BlockedAframeofthistypeismeanttobesentexactlyoncebyanhttp2partywhenithasdatatosendoffbutflowcontrolforbidsittosendanydata.Theideaisthatifyourimplementationreceivesthisframeyouknowyouhavemessedupsomethingand/oryou'regettinglessthanperfecttransferspeeds.
Aquotefromdraft-12,beforethisframewasmovedouttobecomeanextension:
“TheBLOCKEDframeisincludedinthisdraftversiontofacilitateexperimentation.Iftheresultsoftheexperimentdonotprovidepositivefeedback,itcouldberemoved”
Extensions
19
8.Anhttp2worldSowhatwillthingslooklikewhenhttp2getsadopted?Willitgetadopted?
8.1.Howwillhttp2affectordinaryhumans?http2isnotyetwidelydeployednorused.Wecan'ttellforsureexactlyhowthingswillturnout.WehaveseenhowSPDYhasbeenusedandwecanmakesomeguessesandcalculationsbasedonthatandotherpastandcurrentexperiments.
http2reducesthenumberofnecessarynetworkround-tripsanditavoidstheheadoflineblockingdilemmacompletelywithmultiplexingandfastdiscardingofunwantedstreams.
Itallowsalargeamountofparallelstreamsthatgowayovereventhemostshardedsitesoftoday.
Withprioritiesusedproperlyonthestreams,chancesaremuchbetterthatclientswillactuallygettheimportantdatabeforethelessimportantdata.Allthistakentogether,I'dsaythatthechancesareverygoodthatthiswillleadtofasterpageloadsandtomoreresponsivewebsites.Shortlyput:abetterwebexperience.
Howmuchfasterandhowmuchimprovementwewillsee,Idon'tthinkwecansayyet.First,thetechnologyisstillveryearlyandthenwehaven'tevenstartedtoseeclientsandserverstrimimplementationstoreallytakeadvantageofallthepowersthisnewprotocoloffers.
8.2.Howwillhttp2affectwebdevelopment?OvertheyearswebdevelopersandwebdevelopmentenvironmentshavegatheredafulltoolboxoftricksandtoolstoworkaroundproblemswithHTTP1.1,recallthatIoutlinedsomeoftheminthebeginningofthisdocumentasajustificationforhttp2.
Lotsofthoseworkaroundsthattoolsanddevelopersnowusebydefaultandwithoutthinking,willprobablyhurthttp2performanceoratleastnotreallytakeadvantageofhttp2'snewsuperpowers.Spritingandinliningshouldmostlikelynotbedonewithhttp2.Shardingwillprobablybedetrimentaltohttp2asitwillprobablybenefitfromusingfewerconnections.
Aproblemhereisofcoursethatwebsitesandwebdevelopersneedtodevelopanddeployforaworldthatintheshorttermatleast,willhavebothHTTP1.1andhttp2clientsasusersandtogetmaximumperformanceforalluserscanbechallengingwithouthavingtooffertwodifferentfront-ends.
Forthesereasonsalone,Isuspecttherewillbesometimebeforewewillseethefullpotentialofhttp2beingreached.
8.3.http2implementationsTryingtodocumentspecificimplementationsinadocumentsuchasthisisofcoursecompletelyfutileanddoomedtofailandonlyfeeloutdatedwithinareallyshortperiodoftime.InsteadI'llexplainthesituationinbroadertermsandreferreaderstothelistofimplementationsonthehttp2website.
Therewerealargenumberofimplementationsearlyon,andtheamounthasincreasedovertimeduringthehttp2work.Atthetimeofwritingthisthereareover40implementationslisted,andmostofthemimplementthefinalversion.
8.3.1Browsers
Firefoxhasbeenthebrowserthat'sbeenontopofthebleedingedgedrafts,Twitterhaskeptupandoffereditsservicesoverhttp2.GooglestartedduringApril2014toofferhttp2supportonafewtestserversrunningtheirservicesandsinceMay2014theyofferhttp2supportintheirdevelopmentversionsofChrome.Microsofthasshownatechpreviewwithhttp2
Anhttp2world
20
supportfortheirnextInternetExplorerversion.Safari(withiOS9andMacOSXElCapitan)andOperahavebothsaidtheywillsupporthttp2.
8.3.2Servers
Therearealreadymanyserverimplementationsofhttp2.
ThepopularNginxserveroffershttp2supportwithsince1.9.5releasedonSeptember22,2015(whereitreplacestheSPDYmodule,sotheycannotbothruninthesameserverinstance).
Apache'shttpdserverhasahttp2modulemod_http2since2.4.17whichwasreleasedonOctober9,2015.
H2O,ApacheTrafficServer,nghttp2,CaddyandLiteSpeedhaveallreleasedhttp2capableservers.
8.3.3Others
curlandlibcurlsupportinsecurehttp2aswellastheTLSbasedoneusingoneoutofseveraldifferentTLSlibraries.
Wiresharksupportshttp2.Theperfecttoolforanalyzinghttp2networktraffic.
8.4.Commoncritiquesofhttp2Duringthedevelopmentofthisprotocolthedebatehasbeengoingbackandforthandofcoursethereisacertainamountofpeoplewhobelievethisprotocolendedupcompletelywrong.Iwantedtomentionafewofthemorecommoncomplaintsandmentiontheargumentsagainstthem:
8.4.1.“TheprotocolisdesignedormadebyGoogle”ItalsohasvariationsimplyingthattheworldgetsevenfurtherdependentorcontrolledbyGooglebythis.Thisisn'ttrue.TheprotocolwasdevelopedwithintheIETFinthesamemannerthatprotocolshavebeendevelopedforover30years.However,weallrecognizeandacknowledgeGoogle'simpressiveworkwithSPDYthatnotonlyprovedthatitispossibletodeployanewprotocolthiswaybutalsoprovidednumbersillustratingwhatgainscouldbemade.
GooglepubliclyannouncedthattheywouldremovesupportforSPDYandNPNfromChromein2016andurgedserverstomigratetoHTTP/2instead.InFeburaryof2016theyannouncedthatSPDYandNPNwouldfinallyberemovedinChrome51.SinceChrome51,ithasshippedwithoutSPDYandNPNsupport.
8.4.2.“Theprotocolisonlyusefulforbrowsers”
Thisissortoftrue.Oneoftheprimarydriversbehindthehttp2developmentisthefixingofHTTPpipelining.Ifyourusecaseoriginallydidn'thaveanyneedforpipeliningthenchancesarehttp2won'tdoalotofgoodforyou.Itcertainlyisn'ttheonlyimprovementintheprotocolbutabigone.
Assoonasservicesstartrealizingthefullpowerandabilitiesthemultiplexedstreamsoverasingleconnectionbrings,Isuspectwewillseemoreapplicationuseofhttp2.
SmallRESTAPIsandsimplerprogrammaticusesofHTTP1.xmaynotfindthesteptohttp2toofferverybigbenefits.Butalso,thereshouldbeveryfewdownsideswithhttp2formostusers.
8.4.3.“Theprotocolisonlyusefulforbigsites”
Notatall.Themultiplexingcapabilitieswillgreatlyhelptoimprovetheexperienceforhighlatencyconnectionsthatsmallersiteswithoutwidegeographicaldistributionsoftenoffer.Largesitesarealreadyveryoftenfasterandmoredistributedwithshorterround-triptimestousers.
8.4.4.“ItsuseofTLSmakesitslower”
Anhttp2world
21
Thiscanbetruetosomeextent.TheTLShandshakedoesaddalittleextra,butthereareexistingandongoingeffortsonreducingthenecessaryround-tripsevenmoreforTLS.TheoverheadfordoingTLSoverthewireinsteadofplain-textisnotinsignificantandclearlynotablesomoreCPUandpowerwillbespentonthesametrafficpatternasanon-secureprotocol.Howmuchandwhatimpactitwillhaveisasubjectofopinionsandmeasurements.Seeforexampleistlsfastyet.comforonesourceofinfo.
Telecomandothernetworkoperators,forexampleintheATISOpenWebAlliance,saythattheyneedunencryptedtraffictooffercaching,compressionandothertechniquesnecessarytoprovideafastwebexperienceoversatellite,inairplanesandsimilar.http2doesnotmakeTLSusemandatorysoweshouldn'tconflatetheterms.
ManyInternetusershaveexpressedapreferenceforTLStobeusedmorewidelyandweshouldhelptoprotectusers'privacy.
ExperimentshavealsoshownthatbyusingTLS,thereisahigherdegreeofsuccessthanwhenimplementingnewplain-textprotocolsoverport80astherearejusttoomanymiddleboxesoutintheworldthatinterferewithwhattheywouldthinkisHTTP1.1ifitgoesoverport80andmightlooklikeHTTPattimes.
Finally,thankstohttp2'smultiplexedstreamsoverasingleconnection,normalbrowserusecasesstillcouldendupdoingsubstantiallyfewerTLShandshakesandthusperformfasterthanHTTPSwouldwhenstillusingHTTP1.1.
8.4.5.“NotbeingASCIIisadeal-breaker”Yes,welikebeingabletoseeprotocolsintheclearsinceitmakesdebuggingandtracingeasier.Buttextbasedprotocolsarealsomoreerrorproneandopenupformuchmoreparsingandparsingproblems.
Ifyoureallycan'ttakeabinaryprotocol,thenyoucouldn'thandleTLSandcompressioninHTTP1.xeitheranditsbeenthereandusedforaverylongtime.
8.4.6.“Itisn'tanyfasterthanHTTP/1.1”
Thisisofcoursesubjecttodebateanddiscussionsonhowtomeasurewhatfastermeans,butalreadyintheSPDYdaysmanytestswereperformedthatprovedbrowserpageloadswerefaster(like"HowSpeedyisSPDY?"bypeopleatUniversityofWashingtonand"EvaluatingthePerformanceofSPDY-enabledWebServers"byHervéServy)andsuchexperimentshavebeenrepeatedwithhttp2aswell.I'mlookingforwardtoseeingmoresuchtestsandexperimentsgettingpublished.Abasicfirsttestmadebyhttpwatch.commightimplythatHTTP/2holdsitspromises.
8.4.7.“Ithaslayeringviolations”
Seriously,that'syourargument?Layersarenotholyuntouchablepillarsofaglobalreligionandifwe'vecrossedintoafewgrayareaswhenmakinghttp2ithasbeenintheinterestofmakingagoodandeffectiveprotocolwithinthegivenconstraints.
8.4.8.“Itdoesn'tfixseveralHTTP/1.1shortcomings”
That'strue.WiththespecificgoalofmaintainingHTTP/1.1paradigmstherewereseveraloldHTTPfeaturesthathadtoremain,suchasthecommonheadersthatalsoincludetheoftendreadedcookies,authorizationheadersandmore.Buttheupsideofmaintainingtheseparadigmsisthatwegotaprotocolthatispossibletodeploywithoutaninconceivableamountofupgradeworkthatrequiresfundamentalpartstobecompletelyreplacedorrewritten.Http2isbasicallyjustanewframinglayer.
8.5.Willhttp2becomewidelydeployed?(Thissectionwaswrittenin2015andshowsthestateofaffairsbackthen.Thingshavemovedanddevelopedsignificantlysince.)
Itistooearlytotellforsure,butIcanstillguessandestimateandthat'swhatI'lldohere.
Anhttp2world
22
Thenaysayerswillsay“lookathowgoodIPv6hasdone”asanexampleofanewprotocolthat'stakendecadestojuststarttogetwidelydeployed.http2isnotanIPv6though.ThisisaprotocolontopofTCPusingtheordinaryHTTPupdatemechanismsandportnumbersandTLSetc.Itwillnotrequiremostroutersorfirewallstochangeatall.
GoogleprovedtotheworldwiththeirSPDYworkthatanewprotocollikethiscanbedeployedandusedbybrowsersandserviceswithmultipleimplementationsinafairlyshortamountoftime.WhiletheamountofserversontheInternetthatofferSPDYtodayisinthe1%range,theamountofdatathoseserversdealwithismuchlarger.SomeoftheabsolutelymostpopularwebsitestodayofferSPDY.
http2,basedonthesamebasicparadigmsasSPDY,IwouldsayislikelytobedeployedevenmoresinceitisanIETFprotocol.SPDYdeploymentwasalwaysheldbackabitbythe“itisaGoogleprotocol”stigma.
Thereareseveralbigbrowsersbehindtheroll-out.RepresentativesfromFirefox,Chrome,Safari,InternetExplorerandOperahaveexpressedtheywillshiphttp2capablebrowsersandtheyhaveshownworkingimplementations.
Thereareseveralbigserveroperatorsthatarelikelytoofferhttp2soon,includingGoogle,TwitterandFacebookandwehopetoseehttp2supportsoongetaddedtopopularserverimplementationssuchastheApacheHTTPServerandnginx.H2oisanewblazinglyfastHTTPserverwithhttp2supportthatshowspotential.
Someofthebiggestproxyvendors,includingHAProxy,SquidandVarnishhaveexpressedtheirintentionstosupporthttp2.
Allthroughout2015,theamountofhttp2traffichasbeenincreasing.InearlySeptember,Firefox40usagewasat13%outofallHTTPtrafficand27%outofallHTTPStraffic,whileGoogleseesroughly18%ofincomingrequestasHTTP/2.ItshouldbenotedthatGooglerunsothernewprotocolexperimentsaswell(seeQUICin12.1)whichmakesthehttp2usagelevelslowerthanitcouldotherwisebe.
Anhttp2world
23
9.http2inFirefoxFirefoxhasbeentrackingthedraftsverycloselyandhasprovidedhttp2testimplementationsformanymonths.Duringthedevelopmentofthehttp2protocol,clientsandservershavetoagreeonwhatdraftversionoftheprotocoltheyimplementwhichmakesitslightlyannoyingtoruntests.Justbeawaresothatyourclientandserveragreeonwhatprotocoldrafttheyimplement.
9.1.First,makesureitisenabledInallFirefoxversionssinceversion35,releasedJanuary13th2015,http2supportisenabledbydefault.
Enter'about:config'intheaddressbarandsearchfortheoptionnamed“network.http.spdy.enabled.http2draft”.Makesureitissettotrue.Firefox36addedanotherconfigswitchnamed“network.http.spdy.enabled.http2”whichissettruebydefault.Thelatteronecontrolsthe“plain”http2versionwhilethefirstoneenablesanddisablesnegotiationofhttp2-draftversions.BotharetruebydefaultsinceFirefox36.
9.2.TLS-onlyRememberthatFirefoxonlyimplementshttp2overTLS.Youwillonlyeverseehttp2inactionwithFirefoxwhengoingtohttps://sitesthatofferhttp2support.
9.3.Transparent!
ThereisnoUIelementanywherethattellsthatyou'retalkinghttp2.Youjustcan'ttellthateasily.Onewaytofigureitout,istoenable“Webdeveloper->Network”andchecktheresponseheadersandseewhatyougotbackfromtheserver.Theresponseisthen“HTTP/2.0”somethingandFirefoxinsertsitsownheadercalled“X-Firefox-Spdy:”asshowninthescreenshotabove.
TheheadersyouseeintheNetworktoolwhentalkinghttp2havebeenconvertedfromhttp2'sbinaryformatintotheold-styleHTTP1.xlook-alikeheaders.
http2inFirefox
24
9.4.Visualizehttp2useThereareFirefoxpluginsavailablethathelpvisualizeifasiteisusinghttp2.Oneofthemis“HTTP/2andSPDYIndicator”.
http2inFirefox
25
10.http2inChromiumTheChromiumteamhasimplementedhttp2andprovidedsupportforitinthedevandbetachannelforalongtime.StartingwithChrome40,releasedonJanuary27th2015,http2isenabledbydefaultforacertainamountofusers.Theamountstartedoffreallysmallandthenincreasedgraduallyovertime.
SPDYsupportwasremovedinChrome51infavorofhttp2.Inablogpost,theprojectannouncedinFebruary2016:
“Over25%ofresourcesinChromearecurrentlyservedoverHTTP/2,comparedtolessthan5%overSPDY.Basedonsuchstrongadoption,startingonMay15th—theanniversaryoftheHTTP/2RFC—ChromewillnolongersupportSPDY.”
10.1.First,makesureitisenabledIfyouuseaveryoldChromeversionyoumaywanttocheckifthesupportisthere.
Enter“chrome://flags/#enable-spdy4"inyourbrowser'saddressbarandclick“enable”ifitisn'talreadyshowingitasenabled.Thisflaghasbeenremovedinrecentversionandthesupportisnowalwaysimplied.
10.2.TLS-onlyRememberthatChromeonlyimplementshttp2overTLS.Youwillonlyeverseehttp2inactionwithChromewhengoingtohttps://sitesthatofferhttp2support.
10.3.VisualizeHTTP/2useThereareChromepluginsavailablethathelpsvisualizeifasiteisusingHTTP/2.Oneofthemis“HTTP/2andSPDYIndicator”.
10.4.QUICChrome'scurrentexperimentswithQUIC(seesection12.1)dilutetheHTTP/2numberssomewhat.
http2inChromium
26
11.http2incurlThecurlprojecthasbeenprovidingexperimentalhttp2supportsinceSeptember2013.
Inthespiritofcurl,weintendtosupportjustabouteveryaspectofhttp2thatwepossiblycan.curlisoftenusedasatesttoolandtinkerer'swaytopokeonwebsitesandweintendtokeepthatupforhttp2aswell.
curlusestheseparatelibrarynghttp2forthehttp2framelayerfunctionality.curlrequiresnghttp21.0orlater.
NotethatcurrentlyonlinuxcurlandlibcurlarenotalwaysdeliveredwithHTTP/2protocolsupportenabled.
11.1.HTTP1.xlook-alikeInternally,curlwillconvertincominghttp2headerstoHTTP1.xstyleheadersandprovidethemtotheuser,sothattheywillappearverysimilartoexistingHTTP.ThisallowsforaneasiertransitionforwhateverisusingcurlandHTTPtoday.Similarlycurlwillconvertoutgoingheadersinthesamestyle.GivethemtocurlinHTTP1.xstyleanditwillconvertthemontheflywhentalkingtohttp2servers.ThisalsoallowsuserstonothavetobotherorcareverymuchwithwhichparticularHTTPversionthatisactuallyusedonthewire.
11.2.Plaintext,insecurecurlsupportshttp2overstandardTCPviatheUpgrade:header.IfyoudoanHTTPrequestandaskforHTTP2,curlwillasktheservertoupdatetheconnectiontohttp2ifpossible.
11.3.TLSandwhatlibrariescurlsupportsawiderangeofdifferentTLSlibrariesforitsTLSback-end,andthatisstillvalidforhttp2support.ThechallengewithTLSforhttp2'ssakeistheALPNsupportandtosomeextentNPNsupport.
BuildcurlagainstmodernversionsofOpenSSLorNSStogetbothALPNandNPNsupport.UsingGnuTLSorPolarSSLyouwillgetALPNsupportbutnotNPN.
11.4.CommandlineuseTotellcurltousehttp2,eitherplaintextoroverTLS,youusethe--http2option(thatis“dashdashhttp2”).curldefaultstoHTTP/1.1forHTTP:URLssotheextraoptionisnecessarywhenyouwanthttp2forthat.ForHTTPSURLs,curlwillattempttousehttp2.
11.5.libcurloptions
11.5.1EnableHTTP/2
Yourapplicationwouldusehttps://orhttp://URLslikenormal,butyousetcurl_easy_setopt'sCURLOPT_HTTP_VERSIONoptiontoCURL_HTTP_VERSION_2tomakelibcurlattempttousehttp2.Itwillthendoabesteffortanddohttp2ifitcan,butotherwisecontinuetooperatewithHTTP1.1.
11.5.2Multiplexing
Aslibcurltriestomaintainexistingbehaviorstoafarextent,youneedtoenableHTTP/2multiplexingforyourapplicationwiththeCURLMOPT_PIPELININGoption.Otherwiseitwillcontinueusingonerequestatatimeperconnection.
http2incurl
27
Anotherlittledetailtokeepinmindisthatifyouaskforseveraltransfersatoncewithlibcurl,usingitsmultiinterface,anapplicationcanverywellstartanynumberoftransfersatonceandifyouthenratherhavelibcurlwaitalittletoaddthemalloverthesameconnectionratherthanopeningnewconnectionsforallofthematonce,youusetheCURLOPT_PIPEWAIToptionforeachindividualtransferyouratherwait.
11.5.3Serverpush
libcurl7.44.0andlatersupportsHTTP/2serverpush.YoucantakeadvantageofthisfeaturebysettingupapushcallbackwiththeCURLMOPT_PUSHFUNCTIONoption.Ifthepushisacceptedbytheapplication,it'llcreateanewtransferasanCURLeasyhandleanddelivercontentonit,justlikeanyothertransfer.
http2incurl
28
12.Afterhttp2Alotoftoughdecisionsandcompromiseshavebeenmadeforhttp2.Withhttp2gettingdeployedthereisanestablishedwaytoupgradeintootherprotocolversionsthatworkwhichlaysthefoundationfordoingmoreprotocolrevisionsahead.Italsobringsanotionandaninfrastructurethatcanhandlemultipledifferentversionsinparallel.Maybewedon'tneedtophaseouttheoldentirelywhenweintroducenew?
http2stillhasalotofHTTP1“legacy”broughtwithitintothefuturebecauseofthedesiretokeepitpossibletoproxytrafficbackandforthbetweenHTTP1andhttp2.Someofthatlegacyhampersfurtherdevelopmentandinventions.Perhapshttp3candropsomeofthem?
Whatdoyouthinkisstilllackinginhttp?
12.1.QUICGoogle'sQUIC(QuickUDPInternetConnections)protocolisaninterestingexperiment,performedmuchinthesamestyleandspiritastheydidwithSPDY.QUICisaTCP+TLS+HTTP/2replacementimplementedusingUDP.
QUICallowsthecreationofconnectionswithmuchlesslatency,itsolvespacketlosstoonlyblockindividualstreamsinsteadofallofthemlikeitdoesforHTTP/2anditmakesconnectionspossibletobedoneoverdifferentnetworkinterfaceseasily-thusalsocoveringareasMPTCPismeanttosolve.
QUICissofaronlyimplementedbyGoogleinChromeandtheirserverendsandthatcodeisnoteasilyre-usedelsewhere,evenifthere'salibquicefforttryingexactlythat.TheprotocolhasbeenbroughtasadrafttotheIETFtransportworkinggroup.
Afterhttp2
29
13.FurtherreadingIfyouthinkthisdocumentwasabitlightoncontentortechnicaldetails,hereareadditionalresourcestohelpyousatisfyyourcuriosity:
TheHTTPbismailinglistanditsarchives:https://lists.w3.org/Archives/Public/ietf-http-wg/
Theactualhttp2specificationinaHTMLifiedversion:https://httpwg.github.io/specs/rfc7540.html
Firefoxhttp2networkingdetails:https://wiki.mozilla.org/Networking/http2
curlhttp2implementationdetails:https://curl.haxx.se/docs/http2.html
Thehttp2website:https://http2.github.io/andperhapsinparticulartheFAQ:https://http2.github.io/faq/
IlyaGrigorik'sHTTP/2chapterinhisbook“HighPerformanceBrowserNetworking”:https://hpbn.co/http2/
Furtherreading
30
14.ThanksInspirationandthepackageformatLegoimagefromMarkNottingham.
HTTPtrenddatacomesfromhttps://httparchive.org/.
TheRTTgraphcomesfrompresentationsdonebyMikeBelshe.
MykidsAgnesandRexforlettingmeborrowtheirLegofiguresfortheheadoflinepicture.
Thankstothefollowingfriendsforreviewsandfeedback:KjellEricson,BjornReese,LinusSwälasandAnthonyBryan.Yourhelpisgreatlyappreciatedandhasreallyimprovedthedocument!
Duringthevariousiterations,thefollowingfriendlypeoplehaveprovidedbugreportsandimprovementstothedocument:MikaelOlsson,RemiGacogne,BenjaminKircher,saivlis,florin-andrei-tp,BrettAnthoine,NickParlante,MatthewKing,NicolasPeels,JonForrest,sbrickey,MarcinOlak,GaryRowe,BenFrain,MatsLinander,RaulSiles,AlexLee,RichardMoore
Thanks
31