neutron dscp - policing your network
TRANSCRIPT
NateJohnston - PrincipalEngineer - ComcastDavidShaughnessy - NetworkSoftwareEngineer - Intel
PolicingYourNetworkNeutronDSCP
l WhatisDSCP?
l DSCPUseCases
l ImplementingDSCPinNeutronQoS
l What’sNext?
l Conclusion
PresentationOutline
WhatisDSCP?
DSCPstandsfor“DifferentiatedServicesCodePoint”.
DSCPisaprotocolforspecifyingandcontrollingnetworktrafficbyclasssothatcertaintypesoftrafficgetprecedence- forexample,voicetraffic,whichrequiresarelativelyuninterruptedflowofdata,mightgetprecedenceoverotherkindsoftraffic.
l DSCPisdefinedinRFC2474"DefinitionoftheDifferentiatedServicesField(DSfield)intheIPv4andIPv6Headers”
l DSCPforTunnelsisgovernedbyRFC2983"DifferentiatedServicesandTunnels"
What is DSCP?
DSCPisasix-bitfieldintheIPheader– itcomprisesthehighsixbitsoftheeightbitDS(“DiffServ”)fieldinanIPv4header.TheDSfieldwasformerlyreferredtoastheToS (“TypeofService”)field.
InIPv6,theDSfieldhasbeenrenamedtheTrafficClassfield.ItfunctionsidenticallytotheDSfieldinIPv4.
InIPv4,DSisthethirdfieldintheIPheader,andinIPv6TrafficClassisthesecond,whichindicatestheimportanceofDSCP.
TheDSCPBitsintheIPHeader
DSCPintheIPv4Packet
DSCPintheIPv6Packet
The8bitsoftheDSbytearedividedinto2sections:DSCPhasthehigh6bits,andECN(“ExplicitCongestionNotification”)hasthelast2bits.Forthepurposesofthispresentation,weareignoringtheECNbitsentirely.
The6DSCPbitsarearrangedinto4sections:l Thehighest3bitsareusedasthe“Precedence”setting,whichdefinestheClassSelector.
l Thenexttwobitsdesignate“Delay”and“Throughput”,andcollectivelydefine the“AssuredForwarding”(AF)setting.
l Thelowestbitdesignates“Reliability”andisunused.
ThecontentsoftheDSCPbitsarecollectivelyreferredtoasa“mark”or“codepoint”.
StructureoftheDSByte
l Per-HopBehavior(PHB)describeshowtrafficishandledateachhopbasedontheDSCPvaluethatisset.
l Incaseswherepacketswillbedroppedbecauseofcongestion,trafficwithalowerDSCPmark,ornoneatall,willbedroppedbeforetrafficwithahigherDSCPmark.
l Thereare4kindsofPHBsetting:none(thedefault),ClassSelector1-7,threeAssuredForwardingsubclassesforClassSelectors1-4,andExpeditedForwarding.
l BeforeyouimplementDSCP,youneedtoreallyknowhowthesethingsworkoryoumaycauseunintendedeffects.ContactanetworkengineertoverifyyourDSCPimplementationdetails.
Per-HopBehavior
AllDSCPMarks
DSCPUseCases
Typically,networksoperateonabest-effortdeliverybasis,whichmeansthatalltraffichasequalpriorityandanequalchanceofbeingdeliveredinatimelymanner.Whencongestionoccurs,alltraffichasanequalchanceofbeingdropped.
DSCPallowsyoutoselectspecificnetworktrafficforprioritizationaccordingtoitsrelativeimportanceandusecongestion-managementandcongestion-avoidancetechniquestoprovidepreferentialtreatment.
ThisiswhatDSCPwasdesignedfor.
UseCase1:PreferentialTreatmentUnderCongestion
l DSCPmarks,aspartoftheTCPheader,canbeusedascriteriainfirewallrulesandnetworkdeviceACLs.HereisaCiscoexample:
access-list 101 permit ip any any dscp cs1
l Thereforeyoucouldcomeupwithaconvention,forexample:l CS4isproduction guestsl CS3isQAguestsl Andsoforth…
l ThenensurethattheACLspermittrafficwiththegivenmarkstothenetworkscorrespondingtotheirfunction.
UseCase2:DSCPMarksasSecurityPolicy
ImplementingDSCPinNeutronQoS
TheNeutronDSCPcodedidnotmakeitintotheMitakareleaseofNeutronbecauseoflastminuteissues.
Thoseissueshavebeenresolved,andtheDSCPchangeshavebeenmergedintoNeutronmaster.
DSCPfunctionalitywillbeavailableintheNewtonrelease.
Newton
QoSObjectRelationships
QosDscpMarkingRule
QosRule
QosBandwidthLimitRule
QosPolicy
Port
extendsextends
l CreateaQoS policy
l CreateaDSCPruleforaQoS policy
l AssignaQoS policy toaport
AttachingaQoS PolicytoaPort
# openstack network qos-policy-create ‘urgent’ \--description ‘Deliver now’
# openstack network qos-dscp-marking-rule-create urgent \--dscp-mark 26
# openstack network port-update \48c6256f-9123-4e39-a321-108782807cfc --qos-policy urgent
QoS PolicyAddsandUpdates
Controller "DSCP mark 26, please"
"Update a port, please."
"What are the port details?"
Compute
ML2 Plugin and OVS Mechanism
DriverL2 Agent and
QoS Agent Extension
"Here, including QoS policy <uuid>."
"What are policy <uuid>’s rules?"
"Here they are."
"I’m subscribing to policy <uuid>."
"Hey, policy <uuid> changed!"
QoS ExtensionArchitecture
Controller
Core API
Compute
L2 Agent
1
QoS API Extension
2
ML2 Plugin and OVS Mechanism Driver
3
4
QoS Agent Extension
1 User assigns QoS policy containinga DSCP mark rule, to port
2 OVS driver sends RPC message
3 QoS agent extension receives RPC message
4 QoS agent extension notifies OVS agent
VM1
VM2
management network
"DSCP mark 26, please"
5Open
vSwitchOVS Agent
5 OVS agent sets DSCP mark on port
neutron-openvswitch-agent
ProviderNetworkwithOVS
Legend
Provider network Generic network (vlans) Management network External network
(Parenthetical numbers indicate OpenFlow ofport.)
Compute
Instance 1
VLAN Tagging
eth010.251.2.156
p1p110.1251.1.37
Linux Bridgeqbr
qvb
OVS Integration Bridgebr-int
p1p2 (1)
phy-br-ex (2)
tap
Instance 2
eth010.251.2.157
tap
qvo (2)
DSCP Marking
int-br-ex (1)
br-int (65534)Linux Bridge
qbr
qvbtap qvo (3)
DSCP Marking
Security Groups
Security Groups
tap
VLANs
OVS Provider Bridgebr-ex
br-ex (65534)
phy-br-ex (1)
Physical Network
p1p2
IntroductiontoOpenFlow
OpenFlow Switch
Packet In Packet OutTABLE 0
1 Find highest-priority matching flow entry
2 Execute instructions: - apply action list - update (clear or write) action set
3 Apply action set
Match Fields - ingress port - packet headers - metadata
Actions - modify packet headers - update metadata - send packet to another table or out of the pipeline
2 2 31 1
TABLE N
DSCPinOVS
cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24) cookie=1234, table=0, priority=1,in_port=6 actions=mod_nw_tos:104,NORMAL cookie=1234, table=0, priority=0 actions=NORMAL
cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL
cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24) cookie=1234, table=0, priority=0 actions=NORMAL
cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL
OVS Flow Table Before DSCP Marking Added
OVS Flow Table After DSCP Marking Added
DSCPMarkWitnessed
cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24) cookie=1234, table=0, priority=1,in_port=6 actions=mod_nw_tos:104,NORMAL cookie=1234, table=0, priority=0 actions=NORMAL
cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL
03:36:15.516084 fa:16:3e:41:90:36 > fa:16:3e:41:90:37, ethertype IPv4 (0x0800), length 73: (tos 0x68, ttl 10.251.2.136.58321 > 10.251.2.132.8999: Flags [P.], cksum 0x7a0e (correct), seq 78:85, ack 1, win 229, 03:36:15.516156 fa:16:3e:41:90:37 > fa:16:3e:82:32:e0, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 10.251.2.132.8999 > 10.251.2.136.58321: Flags [.], cksum 0x1b28 (incorrect -> 0x6528), seq 1, ack 85,
OVS Flow Table After DSCP Marking Added
tcpdump Output Reflecting DSCP Mark
DSCPMarkWitnessed:Wireshark
l Thesession IDof theL2agentisusedasthecookievalueinOVSflowentriesl WhenanL2agentreboots,modifiesaportorupdatesafirewallitremovesanyflowswhichflow_cookie_value !=my_session_ID
l BecauseL2agentextensions,suchastheQoS extension,managetheirownOVSflowentries,thechallengeistoensurethatanagentdoesnotremoveextensions’flowentries
L2AgentUpdates:Challenges
l Agentassignseachextensionitsowncookievalue
l Uponagentreboot orportupdateextension-owned flowentriesarepreserved
L2AgentUpdates:Solution
QoS Agent Extension
DVR Agent Extension
SG Agent Extension
OVS Agent Extension
L2 Agent
0x1234 0xefgh0x56780xabcd
L2-Agent-Extensions-API
QoS AgentExtension OVSCookieBridge OVSAgentBridge
Cookie=0x1234In_port=6,actions=mod_nw_tos:104,Normal
In_port=6,actions=mod_nw_tos:104,Normal
In_port=6,actions=mod_nw_tos:104,Normal
Cookie=0x1234,In_port=6,actions=mod_nw_tos:104,Normal
l WeneedtoallowNeutrontouseOVStoapplymultiplefeatures(e.g.,DSCPmarkingandVLANtagging)toasinglepacket
l Weneedtoensurethatafeaturedoesn’thijackotherOVSprocessingbyremovingapacketfromthepipeline
l Weneedtoensurethatfeatureflowsdon’taffect,andaren’taffectedby,anyotherpipelineprocessing,includinganyexistingorfuturefeatures
FeatureIsolation:Challenges
l Usemetadatatocorralpacketsforfeatureapplication
l Apacketenteringthepipelinehasallmetadatafieldssetto0
l Afeature’stable0flowentrywillmatch(inpart)onaparticularmetadatafieldandresubmitmatchingpacketstoa“featuretable”
l Thefeature-tableflowlogicwillapplythefeaturetothepacket,setanon-0valueinthepacket’sfeature-specificmetadatafield,andresubmitthepacketbacktotable0forfurtherprocessing
FeatureIsolation:Solution
OpenStack’s DSCPinOVS
cookie=abcd, table=0, priority=65535,reg2=0x0,in_port=6 actions=resubmit(,10)cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24)cookie=1234, table=0, priority=0 actions=NORMAL
cookie=abcd, table=10, priority=0 actions=load:0x37->NXM_NX_REG2[0..5],mod_nw_tos:104,resubmit(,0)
cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL
cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24)cookie=1234, table=0, priority=0 actions=NORMAL
cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL
OVS Flow Table Before DSCP Marking Added
OVS Flow Table After Feature’s DSCP Marking Added
l Additional ruletypesmaycomeavailableduring aserverupgradel DifferentagentsmayusedifferentAPIobjects(e.g.,oneagentmaynotknowaboutDSCPruleswhileanotherdoes)
l Differentagentsmayusedifferentlyversionedobjects(e.g.,oneagentknowsaboutQoSpolicyversion1.0whileanotheragentknowsaboutQoSpolicy1.1)
l Anagentneeds toknowaboutagivenpolicyinstanceid
Server-AgentCommunications:Challenges
Server-AgentCommunications:Solution
ReportVersionedObjects
CreateVersionedFanoutQueue
CreateVersionedFanoutQueue
ReportVersionedObjects
What’sNext?
l IngressDSCPfiltering:WediscussedachangetoSecurityGroupsthatwouldallowingressDSCPfiltering,butSecurityGroupchangesarefraughtwithperil.ThiscouldbecomearoadmapitemforFWaaS atsomepointhowever.
l MarkingencapsulatingpacketswiththeDSCPmarkoftheencapsulatedtraffic:WebelieveitveryunlikelythatDSCP-basedfilteringwilloccurbetweenpartsofaregion,becausesomuchofthatwillbetunneled;theassumptionisthatanyDSCP-relatedbehaviorwillbecomerelevantonlyafterthetrafficexitsthespines.
OmittedfromReferenceImplementation…
l NeutronsupportforExplicitCongestionNotification(ECN)
l Neutrontrafficclassification
l Minbandwidthguarantees
l Ingressbandwidthlimiting
FutureRoadmap
Conclusion
lNeutronserver(neutron.conf)service_plugins = neutron.services.qos.qos_plugin.QoSPlugin
notification_drivers = message_queue
lML2pluginandL2agent(ml2_conf.ini)
[ml2]
extension_drivers = qos
[agent]
extensions = qos
ConfiguringQoS
lDevStack (local.conf)
enable_plugin neutron git://git.openstack.org/openstack.neutron
enable_service q-qos
[[post-config|$NEUTRON_CONF]]
[DEFAULT]
service_plugins=neutron.services.qos.qos_plugin.QoSPlugin
[[post-config|/$Q_PLUGIN_CONF_FILE]]
[ml2]
extension_drivers=qos
[agent]
extensions=qos
ConfiguringQoS,cont.
GeneralDocumentationl NetworkingGuide:UsingOpenStackNetworkingwithQoSl Tokyopresentation“QoS- ANeutronn00bie”
PrerequisitesfortheDSCPChangel L2agentextensionsimplementation: agentAPIl RPCcallbacksrollingupgrades implementationl RPCcallbacksrollingupgrades implementation: reportingandintegration
ChangesAssociatedwiththeDSCPChangel OriginalQoSAPIextensionspecificationl QoSAPIextensionwithDSCPspecificationl ServerandagentDSCPQoSruleimplementationl DSCPimplementation inneutronclientl DSCPinHeat:specificationl DSCPinHeat:implementationoftheQosDscpMarkingRule resource
OtherOpenStack Resources
DiffServ RFCsl RFC2474— DefinitionoftheDifferentiatedServicesField(DSField)intheIPv4andIPv6Headers
l RFC2475— AnArchitectureforDifferentiatedServices
l RFC2597— AssuredForwardingPHBGroupl RFC2983— DifferentiatedServicesandTunnels
l RFC3086— DefinitionofDifferentiatedServicesperDomainBehaviorsandRulesfortheirSpecification
l RFC3140— PerHopBehaviorIdentificationCodes(obsoletes RFC2836)
l RFC3246— AnexpeditedforwardingPHB(obsoletes RFC2598)
l RFC3247— SupplementalInformationfortheNewDefinitionoftheEFPHB(ExpeditedForwarding Per-HopBehavior)
l RFC3260— NewTerminologyandClarifications forDiffserv (updatesRFCs2474,2475,and2597)
l RFC4594— ConfigurationGuidelinesforDiffServ ServiceClasses
l RFC5865— ADifferentiatedServicesCodePoint(DSCP)forCapacity-AdmittedTraffic (updates RFCs4542and4594)
DiffServ ManagementRFCsl RFC3289—ManagementInformationBasefortheDifferentiatedServicesArchitecture
l RFC3290— AnInformalManagementModelforDiffserv Routers
l RFC3317— DifferentiatedServicesQuality ofServicePolicyInformationBase
IfYouWanttoReadMoreaboutDSCP…
AllpatentslistedareU.S.patents.l US20070199064 — Methodandsystemforqualityofservicebasedwebfilteringl US20080089324 — Indicatingorremarkingofa dscp forrtp ofaflow(call) toandfromaserver
l US20080144502 — In-band quality-of-service signaling toendpoints that enforcetraffic policies attraffic sources using policy messages piggybacked onto DiffServbits
l US8767569B2 — Dynamic DSCP availability requestmethodl US20130283379 — System,methodandapparatus thatemployvirtualprivatenetworkstoresistip qos denialofserviceattacks
DSCPUseCasesinPatents
Itwasacollectiveeffort…soourthanksgoesto:
Margaret Frances
Miguel Angel Ajo
Ihar Hrachyshka
Victor Howard
James Reeves
Gary Kotton
John Schwarz
Acknowledgements
l Inteltechnologies’ featuresandbenefitsdependonsystemconfigurationandmayrequireenabledhardware,softwareorserviceactivation.Learnmoreatintel.com,orfromtheOEMorretailer.
l Nocomputersystemcanbeabsolutely secure.
l Testsdocument performanceofcomponents onaparticulartest,inspecific systems.Differencesinhardware,software,orconfigurationwillaffectactualperformance.Consult othersourcesofinformationtoevaluateperformanceasyouconsider yourpurchase. Formorecompleteinformationaboutperformanceandbenchmarkresults, visithttp://www.intel.com/performance.
l Intel,theIntellogoandothersaretrademarksofIntelCorporation intheU.S.and/orothercountries.*Othernamesandbrandsmaybeclaimedasthepropertyofothers.
l ©2016IntelCorporation.
l ©2016ComcastCorporation.
LegalNoticesandDisclaimers
Thank You
Wanttheseslides?