about eantc test report: huawei sd-wan solution – page 3 of 13 the endpoint and link resiliency...

About EANTCEANTC (European AdvancedNetworking Test Center) isinternationally recognized asone of the world's leadingindependent test centers fortelecommunication technologies.Based in Berlin, the companyoffers vendor-neutral consultancy

and realistic, reproducible high-quality testing servicessince 1991. Customers include leading networkequipment manufacturers, tier 1 service providers,large enter-prises and governments worldwide.EANTC's Proof of Concept, acceptance tests andnetwork audits cover established and next-generationfixed and mobile network technologies.

Table of ContentsIntroduction ........................................................2

Test Highlights .................................................2Hardware and Software.......................................3Test Bed .............................................................3

Test Equipment.................................................3Test Results..........................................................4Scalability ..........................................................4

Number of VPNs per Site .................................4Throughput per Number of VPNs per Site (CPE Performance)...................................................5

Endpoint and Link Resiliency .................................6CPE Failure .....................................................6Primary Link Failure..........................................7

Performance Monitoring.......................................8Performance Monitoring Accuracy .....................8Application-based Traffic Steering....................10

Conclusion .......................................................12

IntroductionThe market for Software Defined Wide Area Network(SD-WAN) solutions has grown rapidly over the lasttwo years. In an area where cloud computing enablescustomers to deploy compute resources on-demandand in a matter of seconds, and where SDN is beingactively deployed in data centers networks, enterprisesrequest a flexible and affordable connectivity solutionfor the Wide Area Networks (WAN).

By aggregating different types of WAN links andmaking intelligent forwarding decisions across theselinks based on policies and network conditions, SD-WAN has become an important transformation tech-nology for the telecommunications industry. Thebeauty (and somehow also the challenge) of SD-WANis its nature as an overlay technology: Adaptable toany connectivity solution while unable to control theunderlying transport network, SD-WAN implementa-tions need a different approach to ensure networkquality and reliability.EANTC has been testing IP/MPLS independently formore than 15 years and SDN-WAN solutions since2011. Naturally, we are reviewing SD-WAN solutionsas well. Huawei is the first vendor to pass SD-WANtesting at EANTC. It is our pleasure to publish their SD-WAN solution review now.EANTC has started an SD-WAN testing initiative opento any manufacturer of SD-WAN solutions worldwide.Participating vendors have the opportunity to choosetheir bespoke test package from EANTC’s SD-WANtesting portfolio based on the primary technical focusareas. For its participation, Huawei chose thefollowing test areas: Scalability, Endpoint and LinkResiliency, and Performance Monitoring.In the scalability area, we successfully verified thenumber of VPNs per site supported by Huawei’s SD-WAN solution and the throughput per VPN per site.Two test cases out of five defined in this area werecompleted. This was due to the amount of time andresources required to complete some of the scalabilitytests.

Test Highlights

2,000+ full-mesh VPNs per CPE

Protection switching within milliseconds in the event of a CPE failure or a primary link failure

Application-aware traffic re-routing based on live network performance

Accurate performance monitoring and GIS-based multi-dimensional visualization

EANTC Test Report: Huawei SD-WAN Solution – of 13

The Endpoint and Link Resiliency tests includedCustomer Premise Equipment (CPE) failure and thefailure of one of the primary links, which enableservice providers to measure the readiness of the SD-WAN to be deployed for the transport of voice, videoand data.

Finally, with their SD-WAN solution, Huawei seeks tooptimize application performance and reliability bydynamically steering critical traffic across the bestlinks, as well as potentially mitigating circuit perfor-mance issues. We were excited to test performancemonitoring and application-based traffic steering,which are key enablers of the SD-WAN.

After two weeks of testing at Huawei’s office inNanjing, we are happy to present the results of ourjoint work with Huawei’s team in this report.

Hardware and Software

Test BedHuawei provided a comprehensive end-to-end labenvironment for the test. The test network reflected areal production network and had a hierarchical WideArea Network (WAN) design, consisting of threebranches and a headquarter site. As depicted inFigure 1 and Figure , two branch sites with a singleCPE each were associated in the management systemwith emulated locations of Nanjing and Shenzhenrespectively; the third branch designated as Shanghaiwas comprised of two redundant Customer PremiseEquipment (CPEs).

All branch sites were connected to the hub at theheadquarter site at the emulated location of Beijingthrough two underlying networks, an MPLS networkand an emulated Internet connection. CPEs and hubdevices in turn redundantly peered with the HuaweiAgile Controller cluster, acting as the SD-WAN

controller via both the MPLS network and the emulatedInternet. The Agile Controller cluster’s deploymentconsisted of several Virtual Machines (VMs) carryingvarious components and spread across three computenodes to provide hardware redundancy. VMware ESXiwas deployed in each compute node as the virtualiza-tion technology.

The WAN constructed by Huawei included Huawei’sAR161EW, AR169W, and AR1220EVW at thebranch sites. A Huawei AR3670 was provided at thehub site. The underlying MPLS core network was builtusing Huawei AR1220C and AR1220EVW ProviderEdge (PE) routers. A Huawei SRG1320E was used toemulate the Internet in the lab.

For the purpose of one single test case, we used aWindows server connected to a CPE in a branch siterunning a Huawei proprietary Dynamic Smart VirtualPrivate Network (DSVPN) application to emulate otherCPE nodes, which in turn were used to simulate alarge number of overlay tunnels to mimic a typicallarge WAN deployment scenario.

Figure 1: Test Setup - SD-WAN Controller’s View

Test Equipment

Executing the test required a traffic generator, ananalyzer and an impairment tool. We used SpirentTest Center equipped with EDM-20038 24-ports Giga-bitEthernet cards for traffic generation and analysis.For impairment generation, we used Wide AreaNetwork Emulator (WANem) from TATA consultancyservices.

We complemented the test equipment (traffic gener-ator/emulator) with an x86 Windows server. Basedon the test bed design, a single CPE would have toestablish a large number of spoke to spoke DSVPNtunnels. Based on this requirement, we used a DSVPNapplication, running on an x86 Windows server toemulate a large number of DSVPNs towards the CPE.

Hardware Type Software Version

Huawei AR161EWV200R009C00Patch Version:V200R009C00CP0711

Huawei AR169W

V200R009C00Huawei AR1220EVW

Huawei AR3670

Huawei AR1220EVW V200R008C50

Huawei AR1220C V200R008C50

Huawei SRG1320E V200R008C20

Huawei Agile Controller V300R002C00


Test ResultsIn the following section we describe the test results forthe test areas selected by Huawei.

ScalabilityAs SD-WAN gains popularity as an alternative to thetraditional wide-area VPN technologies, it is onlylogical that enterprises and service providers areconcerned with the scalability of the solution that theyare buying.Through the following series of tests, EANTC verifiedthe various scalability and performance claims ofHuawei’s SD-WAN solution, specifically:• Number of VPNs per Site• Throughput per number of VPNs per site

(CPE Performance)

Number of VPNs per Site

In a large-scale multi-tenant SD-WAN deployment with alarge number of branch sites per customer tenant, asingle CPE device is often required to support a high

number of overlay tunnels to enable connection tomultiple hub and spoke sites.

Since the test setup would have required a largenumber of physical CPEs and hub devices to verify theability of a CPE to support a large number of DSVPNs,we deployed a windows server at a branch site andconnected it to the CPE to complement our test bed.The intention was to run a Huawei proprietary DSVPNapplication on the server to simulate a large numberof spokes connected to the CPE, acting as the hubdevice. This allowed us to simulate a scenario where aCPE creates a high number of dynamic spoke-spokeDSVPNs.

In our test scenario we installed two SD-WAN edgedevices — Huawei AR3670, acting as the hub nodeand Huawei AR1220EVW, served as the CPE — asdepicted in Figure 4. The devices were connected tothe SD-WAN controller via both an MPLS network andthe emulated Internet. Both SD-WAN edge nodes werein turn interconnected via two WANs: the path overthe MPLS network comprised of a single link betweenthe two sites. The path over the Internet was made upof three links between the two sites.

Figure 2: Physical Test Setup

SD-WAN Enabled Network

x3

x3

HuaweiAR3670

HuaweiSRG1320E

Shenzhen

Nanjing

Shanghai

Beijing

AR169WHuawei

AR169WHuawei

HuaweiAR1220EVW

SD-WAN Controller

HuaweiS5321

AR161EWHuawei

Impairment Generator

Traffic Generator/Emulator

Physical Link

MPLS Network

InternetHub Site Branch Site

. . .Emulated CPE

(Windows Server)

em_CPE1

em_CPE2

em_CPE2k

..

.

HuaweiAR1220C

AR1220EVWHuawei


Figure 3: Number of DSVPNs per Site - Huawei SD-WAN Controller View

Initially, we configured the SD-WAN controller toorchestrate four DSVPNs between CPE and hubnodes: three DSVPNs over the emulated Internet andone through the MPLS network. We expected all fourDSVPNs to be correctly deployed onto CPE and hubdevices.

We started sending bidirectional UDP traffic througheach of the DSVPN tunnels using Spirent Test Center tomake sure that they were correctly setup. There was nopacket loss observed during this reference test. Whiletraffic was running through the four provisionedDSVPN tunnels, Huawei engineers used the DSVPNapplication installed on the server to request thecreation of 3,000 DSVPNs on the CPE. The CPE’sCommand Line Interface (CLI) displayed that a total of2,004 DSVPNs were created. During the test, wemonitored the CPE’s CPU utilization, which wasapproximately 11.4% as well as memory consump-tion, which was 60%. No impact was observed on theUDP traffic traversing through the four DSVPN tunnels.Our test revealed that Huawei AR1220EVW whenused as an SD-WAN edge device can setup and main-tain four hub-spoke DSVPNs and up to 2,000 spoke-spoke DSVPNs.

Throughput per Number of VPNs per Site (CPE Performance)

As the number of DSVPNs supported by a single CPEscales to the maximum, one can imagine that as moredevices are added to the Branch/hub sites, moretraffic will be generated resulting in more bandwidth.The CPE with its provisioned DSVPN is expected todeal with an increase in throughput per VPN require-ments.

Figure 4: Number of VPNs per Site

To assess the CPE performance of HuaweiAR1220EVW, acting as CPE loaded with four hub-spoke and 2,000 emulated spoke-spoke DSVPNs, ourtest used the same test bed as in the “Number of VPNsper Sites” test (see Figure 4). This time we intentionallyoffered traffic at varying loads and determined thehighest rate at which the SD-WAN CPE could forwardall packets with zero loss.

We used the same methodology described by the IETFin RFC 2544 for a selected range of packet sizes —64, 100, 373, 570, 1300, 1518 bytes. Additionallywe tested with Internet mix (IMIX)1 traffic, allowing arealistic overview relevant for an SD-WAN realdeployment scenario. IMIX is a packet mix thatattempts to replicate real Internet traffic.

We performed the test using Spirent Test Centerconnected to both hub and CPE nodes and generatedUDP traffic through the four hub-spoke DSVPN tunnels.

1. IMIX consisted of 5% 64-byte packets, 42% 100 byte packets, 9.5% 373-byte packets, 8% 570 byte packets, 26% 1518-byte packets and 9.5% 1300-byte packets

SD-WAN ControllerTraffic

HuaweiAR3670

x3

x3

. . .em_CPE1 em_CPE2 em_CPE2000

HuaweiAR1220C

HuaweiAR1220EVW

Generator

Ethernet Link

Management

IBGP

Branch Site

Hub SiteMPLS Network

Internet

DSVPN

Windows Server

x3

Link

Huawei AR1220EVW

HuaweiSRG1320E


The aggregated bidirectional throughput test result isdepicted in Figure 5.

Figure 5: CPE Performance for 4 DSVPNs and Loaded with 2004 Emulated DSVPNs

Endpoint and Link ResiliencyNetwork reliability has been always a major concernin any enterprise and service provider network trans-porting voice, video and business-critical data.Whether a link fails due to cable pull or a CPE failsdue to power outage, a reliable network should reactto sudden failures without any manual intervention.

Huawei SD-WAN solution provides a range of solu-tions to address such network failures.

CPE Failure

To ensure the highest level of availability for gold andsilver level service, SD-WAN requires a configurationthat has redundant CPEs at a branch location to peerwith the remote hub site using multiple circuits. Sinceboth circuits can be grouped into single link groups,they can be used to simultaneously carry differentapplication types in an active-active fashion. Whileone circuit is active for a specific application, it is usedas backup for other applications running on a differentcircuit. Upon failure of one CPE, the SD-WAN solutioncauses the switchover of applications from the failedCPE to the other CPE.

This test was carried out with the network configura-tion shown in Figure 6. Our test setup consisted of abranch site with two redundant SD-WAN CPEs (twoHuawei AR169W’s) each of which was connected tothe hub device (Huawei AR3670) over an emulatedInternet and an MPLS network. CPEs and hub wereconnected to the SD-WAN controller by SSL sessions.Both CPEs were configured to operate in master-backup redundancy mode using the Virtual RoutingRedundancy Protocol (VRRP).

Figure 6: CPE Failure

Huawei

Huawei

Tester Ports

AR169W

S5321

Gigabit Ethernet

Management Link

IBGP

OSPF

MPLS Network

Internet

HuaweiAR1220EVW

HuaweiAR1220C

Huawei

Huawei

Tester Ports

AR169W

S5321

HuaweiAR3670

HuaweiAR1220EVW

HuaweiAR1220C

DSVPN

Failure

SD-WAN Controller

SD-WAN Controller

Branch Site

Hub Site

HuaweiAR3670

AR169WHuawei

HuaweiAR169W

HuaweiSRG1320E

HuaweiSRG1320E


For testing purposes, we used Huawei’s SD-WANcontroller to define two DSVPNs, each starting atdifferent CPEs, terminating on the same hub andtraversing different links.

Both tunnels were configured to be in an activestate.At that point, we validated that the CPEconnected to the MPLS network was the VRRP masterand the one connected to the Internet was the VRRPslave.

The SD-WAN solution was asked to identify thedifferent traffic types and steer them towards thecorrect link using pre-defined policies:

• Steer general UDP traffic towards the Internet linkas the primary link and use the MPLS link on failureof the Internet link

• Steer FTP traffic towards the MPLS link as itsprimary link and make use of the Internet link asbackup

Once the Huawei Agile controller displayed estab-lished DSVPN tunnels as shown in Figure 7, wesupplied bidirectional FTP and UDP traffic between SD-WAN edge devices using Spirent Test Center and theFTP application server respectively.

Figure 7: DSVPNs between on Two CPEs and a Hub

Huawei configured Bidirectional Forwarding Detection(BFD) on the VRRP interface on the link between bothCPEs. Additionally the OSPF session, running betweenboth CPEs, was monitored by BFD. Same for theoverlay IBGP session, configured between CPEs andthe hub. All BFD sessions were used to monitor the live-ness of the connections.

Given that Huawei configured the BFD hello interval at50 milliseconds and the multiplier at three, weexpected that the detection time would be theoreticallybetween 100 and 150 milliseconds. After a failure isdetected, time for convergence is needed and this maytake additional time

First we verified that FTP and general UDP traffic wereindeed steered towards the MPLS and the Internet linkrespectively and that traffic was forwarded to thedestination without any packet loss.

Once this step was completed, we emulated the failureof the CPE connected to the Internet link and acting as

the VRRP backup by issuing an administrative “RebootFast” command, causing the SD-WAN solution to steerthe UDP traffic towards the MPLS link. We recordedout of service time of maximum 135 milliseconds inthe direction from the hub to the CPE and of maximum295 milliseconds in the opposite direction. We testedthe recovery scenario during the restart of the CPE.After the Wait to Restore (WTR) timer expired, trafficloss and reversion time was measured when thegeneral UDP traffic was steered again towards theInternet link. We measured recovery time of maximum167 milliseconds in the direction from the CPE to thehub and no packet loss at all in the opposite direction.

Afterwards, we emulated the failure of the second CPE- the VRRP master - connected to the MPLS link, bypowering it down using “Reboot Fast” command.Once successful, we followed the recovery testwherein the device was powered up. We measuredfailover time of maximum 132 milliseconds in thedirection from the hub to the CPE and of maximum139 milliseconds in the other direction. We measuredrecovery time of maximum 424 milliseconds in thedirection from the hub to the CPE and no packet loss atall in the opposite direction. During this step, weobserved that as soon as the previously failed masterrecovered from failure, it took over the VRRP master-ship.

In this test, Huawei R3670 participated as the hubnode at the headquarter site, and Huawei AR169Wserved as the CPE access device at the branch site.Huawei Agile Controller successfully participated asSD-WAN controller.

Primary Link Failure

One of the key advantages of SD-WAN technology isthe ability to aggregate different WAN links (such asMPLS, xDSL, LTE/4G). The ability of the SD-WAN solu-tion to mitigate the failure of the primary link is critical.

Our setup consisted of one CPE (Huawei AR161EW)and one hub device (Huawei R3670), each of whichwere connected to the Huawei Agile Controller usingthe MPLS and Internet network, over which the SSLcontroller channel was established. CPE and hubnodes were in turn connected together via two WANs:an MPLS network and a network emulating the publicInternet.

The SD-WAN controller was configured to setup twoDSVPN tunnels between the CPE and the hub device:one DSVPN across the MPLS network and another oneover the emulated Internet. Likewise the traffic policieswere provisioned on the SD-WAN controller to steergeneral UDP traffic towards the Internet and FTPtowards the MPLS network. To perform this test,Huawei engineers setup two Internal Border Gateway


Protocol (IBGP) sessions — one over each DSVPNtunnel — between CPE and hub to distribute all localprefixes available on the SD-WAN sites.

Figure 8: Primary Link Failure

Once CPE and hub were successfully registered withthe SD-WAN controller, we validated that all DSVPNand traffic policy configurations were correctlydeployed to CPE and hub nodes. We verified that allcontrol plane sessions had been established and IPprefixes had been exchanged between SD-WAN edgenodes.

Figure 9: SD-WAN edge devices registration to controller

When we sent UDP and FTP traffic using Spirent TestCenter connected to both Huawei AR161EW andHuawei AR3670, we validated that the UDP trafficwas indeed steered towards the emulated Internet andthe FTP traffic was steered towards the MPLS network.We observed no packet drops.

On failure of the Internet link, which was induced byphysical link disconnection between the CPE and the

Internet node, we successfully verified that UDP trafficwas redirected to the backup MPLS link as expected.The simulated Internet connectivity failure generatedfailover times ranging from 127 to 155 milliseconds inthe direction from the hub to the CPE and from 170 to200 milliseconds in the opposite direction.

We also tested the recovery scenario by reconnectingthe previously disconnected link. After the Wait toRestore (WTR) timer expired, we expected the UDPtraffic in this case to be steered towards the Internetlink. We measured the recovery time of 0 seconds. Alltest runs showed no traffic loss at all.

During this test, a Huawei engineer configured BFD onthe overlay BGP sessions between CPE and hubdevices to monitor the liveness of the connection.

Performance Monitoring

One of the main drivers for SD-WAN is the ability toaggregate WAN links and to make intelligentforwarding decisions across these links based on poli-cies and network conditions. To allow real-time visi-bility into these WAN links, which is key for SD-WANintelligent application-based forwarding, accurateperformance monitoring is required.

Performance Monitoring Accuracy

SD-WAN relies on the performance monitoring tool toprovide visibility into application performance bymonitoring performance parameters such as packetloss, packet delay, and packet delay variation. Accu-rate measurement of these performance metrics iscrucial to the correct operation of an SD-WAN-enabled network.

To mimic a typical network deployment scenario, webuilt a test bed consisting of an SD-WAN CPE (HuaweiAR161EW), an SD-WAN hub (Huawei AR3670) andan SD-WAN controller (Huawei Agile Controller). TheSD-WAN CPE and hub were connected to the SD-WAN controller via the MPLS network and emulatedInternet. The SD-WAN CPE was connected to the SD-WAN hub via two WANs: an emulated Internet andan MPLS network. Huawei engineers established twoInternal Border Gateway Protocol (IBGP) sessionsbetween the SD-WAN CPE and hub over each of thelinks to distribute SD-WAN site prefixes.

In our test, we requested the Huawei SD-WANcontroller to provision two Dynamic Smart VirtualPrivate Networks (DSVPNs) between Branch and hubsites — one was provisioned over the emulatedInternet and the other over the MPLS network. TheHuawei SD-WAN controller reported both tunnelsoperational as depicted in Figure 10.

Tester Ports

HuaweiAR1220C

HuaweiAR1220EVW

ManagementIBGP

MPLS Network

Internet

DSVPN

Failure

SD-WAN Controller

Branch Site

Hub Site

Link

HuaweiAR161EW

HuaweiSRG1320E

HuaweiAR3670

Gigabit Ethernet


Once this procedure was successfully completed, weused the SD-WAN controller to configure traffic poli-cies to steer UDP traffic towards the Internet link andFTP towards the MPLS link.

Figure 10: DSVPNs Across Internet and MPLS Network

Figure 11: Performance Monitoring Accuracy Test Setup

In our test we used Wide Area Network Emulator(WANem), inserted on the link between the SD-WANCPE and the node emulating the Internet, to introducecontrolled impairment such as packet loss, packetdelay and jitter to emulate network congestion. For aspecific type of impairment, we verified the measure-ment results provided by the SD-WAN solution againstthe emulator configuration.

We initially ran a baseline reference test without inser-tion of any impairment to validate performance moni-toring implementation of the Huawei SD-WAN solu-tion. During this procedure we verified that theHuawei SD-WAN solution used a Network QualityAnalysis (NQA) tool to monitor the performance andhealth of the network links, which reported on packetloss, packet delay and jitter. As part of the baseline,we generated UDP traffic using Spirent Test Center,connected to the SD-WAN edge nodes to validate thatthe test network was running correctly and experi-enced zero packet loss.

For loss measurement, we sent bidirectional UDPtraffic through the provisioned DSVPN via the Internetlink and introduced 10% loss in one direction usingthe WAN emulator. The Huawei SD-WAN solutionshowed the same loss value as shown in Figure 12.

Figure 12: Packet Loss Ratio Reported by SD-WAN Controller

Figure 13: Packet Loss Measurement as Displayed by CPE

Once we removed the impairment profile, the lossvalue reverted to the normal condition.

In order to test packet delay measurement, we addedan unidirectional constant delay of 20 milliseconds to

Tester Ports

HuaweiAR1220C

HuaweiAR1220EVW

IBGP

MPLS Network

Internet

DSVPN

PM Session

SD-WAN Controller

Management

Branch Site

Hub Site

Link

HuaweiAR161EW

HuaweiSRG1320E

HuaweiAR3670

Gigabit Ethernet

ImpairmentGenerator


all packets on the Internet link. One-way Packet delayon all UDP packets on the Internet link as report by theSD-WAN solution increased by 20 milliseconds.

For the average packet delay variation or Jittermeasurement, we introduced 25 milliseconds delayand 15 milliseconds jitter on all packets on the Internetlink. Packet delay and jitter as reported by the SD-WAN solution was increased by 25 milliseconds and15 milliseconds respectively.

Figure 14: Packet Delay Measurementas Shown by the SD-WAN

Figure 15: Packet Delay Variation Measurement as Shown by the SD-WAN

Application-based Traffic Steering

The quality, performance and reliability of the networkare essential to user experience and customer satisfac-tion. With their SD-WAN solution, Huawei seeks tooptimize application performance and reliability bydynamically steering critical traffic across the bestlinks, as well as potentially mitigating circuit perfor-mance issues. The key element to address theserequirements is the application-based traffic steeringfunctionality of the SD-WAN solution.

The idea behind this feature is to monitor the quality ofeach link used by application traffic with continuousperformance monitors. If a link experiences qualitydegradation for example due to excessive packet lossor jitter, which severely impact applications running onthat link, the sending SD-WAN edge devices instantlyand dynamically switch the application traffic toanother link with the best quality available at thatmoment.

To perform this test, we used the same test bed as inthe “Performance Monitoring Accuracy” test (seeFigure 11). On this occasion we intentionallyrequested that the SD-WAN controller configure appli-cation policies as depicted in Table . Figure 16 depictssmart routing policy configuration for the UDP applica-tion on the SD-WAN controller.

To make sure that the defined policies were not boundto the link but to the application itself, we purposelysteered two sets of application traffic towards thesame link with different traffic policies applied on eachof them.

Figure 16: Smart Routing Policy for UDP Application

We started our test by validating that DSVPNs andapplication policies configured on the SD-WANcontroller were correctly deployed on both CPE andhub nodes.

During this step, we observed a smart-policy-routeinstalled on the SD-WAN CPE and hub nodes,reflecting the traffic policies defined on the SD-WANcontroller. We initially ran baseline tests by generating


UDP, HTTP and FTP application traffic using eitherSpirent Test Center or a real application server,connected to the SD-WAN edge nodes to establishthat the test bed was running correctly and experi-enced no loss. Indeed, UDP and HTTP traffic weresteered towards the Internet and FTP towards the MPLSlink and were received at the destination with no lossas expected.

Table 1: SD-WAN Pre-defined Application Policies

We completed this test in three phases:

Phase 1. Degradation condition due to packet loss

As soon as this baseline procedure was successfullycompleted, we verified that despite the 2% constantpacket loss introduced in the direction from branch tohub site using the impairment tool, UDP and HTTPwere delivered over the Internet link as expected.

When we increased the value of the packet loss to 5%on the same link and in the same direction, weobserved results that match our expectations: due tothe excessive degradation condition on the Internetlink, UDP traffic was rerouted to the MPLS link. HTTPtraffic was steered towards the Internet and FTPtowards the MPLS. We increased the packet loss onthe Internet link to 10%. We expected UDP, HTTP, andFTP traffics to be all steered towards the MPLS asdepicted in Figure 17.

Figure 17: Intelligent Traffic Steering - Packet Loss Degradation Conditions

Once we disabled the impairment generation, wesuccessfully validated that UDP and HTTP traffic werereverted back to the Internet link.

Phase 2. Degradation condition due to packet delay

As soon as phase 1 was successfully completed, weadded a unidirectional constant delay of 100 millisec-onds on all packets traversing the Internet link usingthe impairment generator. We expected no traffic tobe rerouted by the SD-WAN solution. We then config-ured WANem to increase the delay on all traffic onthe internet link to 160 milliseconds.

As expected, the SD-WAN solution moved the UDPtraffic from the Internet link to the MPLS link. When wefurther increased the value of the packet delay on theInternet link to 350 milliseconds - excessive degrada-tion condition on the HTTP traffic - the HTTP wassuccessfully steered towards the MPLS by the SD-WANsolution as illustrated in Figure 18.

Figure 18: Intelligent Traffic Steering - Packet Delay Degradation Conditions

UDP and HTTP were steered again to the Internet linkonce the impairment generation was disabled.

Phase 3. Degradation condition due to jitter

To verify the degradation condition due to jitter, wefirst introduced a constant delay of 50 millisecondsand a jitter of 10 milliseconds on all packets on theInternet link. None of the application traffic wasrerouted. When we increased the delay value to 100milliseconds and the jitter to 50 milliseconds, the SD-WAN solution redirected HTTP traffic to the MPLS link,while the UDP remained on the Internet link asdepicted in Figure 19.

Figure 19: Intelligent Traffic Steering - Jitter Degradation Conditions

Applications

UDP HTTP FTP

Packet loss [%] 3 8 5

Packet delay [ms] 150 300 200

Packet delay variation [ms] 150 20 100

Working link Internet Internet MPLS

Backup link MPLS MPLS Internet


ConclusionThis section highlights the key results achieved duringour test.With the test areas selected by Huawei, we verifiedand measured the capability of their SD-WAN solutionfrom a number of perspectives.Based on the test results, we can confirm that the SD-WAN solution is meeting Huawei’s claims. The solu-tion is scalable in terms of number of DSVPNs that canbe established on a single CPE node. A CPE success-fully established up to 2,004 DSVPNs and remainedmanageable. No signs of performance degradationwere observed during this test. To ensure the reliabilityof the solution, we ran a throughput test per CPE andper number of DSVPNs for a selected range of packetsizes, which showed acceptable results.We also verified that Huawei’s SD-WAN solution isequipped with failover mechanism to help serviceproviders maintain operations during link and CPE fail-ures.The test results demonstrated that performance moni-toring tools were functioning correctly and can beused by the SD-WAN edge nodes to provide visibilityinto application performance.The application-based traffic steering scenario estab-lished that the Huawei’s SD-WAN solution would makeintelligent forwarding decisions based on policies andnetwork conditions.The observed results concludes that Huawei’s SD-WAN solution offers a combination of CPE scalabilityfor supporting a large number of DSVPNs, failoverand application performance monitoring capabilities.


about eantc test report: huawei sd-wan solution – page 3 of 13 the endpoint and link resiliency...

Documents