carrier ethernet world congress 2011 public multi-vendor … · 2011. 10. 7. · 4 carrier ethernet...

Carrier Ethernet WorldCongress 2011

Public Multi-VendorInteroperability Event

White Paper

2

Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test

EDITOR’S NOTE2011 marks the seventh yearof the Carrier Ethernet WorldCongress and its EANTCmulti-vendor Carrier Ethernetinteroperability tests. In prepa-ration of our showcase, Iwondered how muchinnovation we would be ableto witness year. Typically,vendors who were early

adopters aim to capitalize on their investment whenthe market matures, as the absence of some largebackbone equipment vendors shows this time.Substantial innovation happens in other areas thesedays, closing some gaps for Carrier Ethernet services:

• The active integration of packet-basedmicrowave solutions into Carrier Ethernetnetworks is driven by fierce competition. Wetested full-blown switching functionality withclock synchronization, Ethernet Ring ProtectionSwitching and more.

• End-to-end service activation quality assuranceprocedures is progressing to reduce the carriers‘huge provisioning cost and helping to ensureservice levels.

• Protection in provider bridging- and MPLS-TP-based access and aggregation networks,bringing them up to speed with MPLS in theaccess.

We have tested MPLS-TP since 2009. The industry isstill undecided about two alternative, incompatibleways to implement protection.As usual we asked all vendors tobring innovative solutions withnew aspects, as opposed torepeat previous showcases. Thisreduced the field to twoequipment vendors committed toBFD-based solutions. Sometimesthe difference between politicalstatements and technologyreality can be surprising.Packet-based clock synchroni-zation continues to be an activetest area. Our event provides aunique opportunity to test multi-vendor solutions in a realisticend-to-end network scenario. This time we focuses onboundary clocks, but had to deal with a number offunctional multi-vendor interoperability points.Although progress has been made, IEEE 1588:2008is still clearly a non-trivial technology.Overall the Carrier Ethernet paradigm has gainedoverwhelming support in the industry. Basic E-Lineservices are a staple worldwide. The moreadvanced concepts which enable service providersto build value-added services, permit competitivedifferentiation and reduce operational cost,however, still require care in multi-vendor environ-ments.

INTRODUCTIONSometimes diplomatic skills are required when weorganize interoperability events at EANTC. We pollvendors’ technical interests, author documentsdescribing what will be tested, provide a platform toexecute tests and when things do not work asplanned, bring the participants to the table, mitigateand help solve problems.One of the major cornerstones of our interoperabilityshowcases is that they focus on new technologysolutions and/or new products. This is why thereader will find most test cases updated and refinedcompared to previous years‘ events, or new productsbeing put to test.In 2011, our focus was on pure Ethernet transporttests. 12 out of the 14 test cases were concernedwith Ethernet testing from clock synchronization toservice activation.After months of preparation and two intense weeksof hot staging in our new offices in Berlin in whichwe created 106 different test permutations, weconcluded a successful event. As in each interopera-bility event we had long debugging sessions andmeetings to discuss test expectations and results, butthe incremental increase in implementations andadvanced features does show that Carrier Ethernet ismoving forward.

INTEROPERABILITY TEST RESULTSThe following sections of the white paper describethe test areas and results of the interoperability

event. The document generallyfollows the structure of the test plan— a document edited by EANTCand reviewed together withvendors in preparation for theevent.

Terminology. We use the term“tested” when reporting on multi-vendor interoperability tests. Theterm “demonstrated” refers toscenarios where a service orprotocol was terminated byequipment from a single vendoron both ends.

Test Equipment. In order to perform our tests wehad to generate, measure, impair, and analyzeEthernet traffic and perform synchronizationanalysis. We thank Calnex Solutions, Ixia, SpirentCommunications, Sunrise Telecom, Symmetricomand Veryx for their test equipment and supportthroughout the hot staging.

Carsten RossenhövelManaging Director

NewPhoto

TABLE OF CONTENTSParticipants and Devices ............. 3Service Activation Test................. 3Resiliency and Redundancy ......... 6Topology ................................... 8Clock Synchronisation............... 10Ethernet Microwave Tests........... 12Demonstration Network............. 14References............................... 15

3

Participants and Devices

PARTICIPANTS AND DEVICES

SERVICE ACTIVATION TESTA brand new test area this year was Ethernet serviceactivation. The test followed the ITU-T standardY.1564 “Ethernet Service Activation Test Method-ology,” a standard approved in January 2011. Thestandard provides methodologies and specific teststo allow service providers to validate that newCarrier Ethernet services coming online meet ServiceLevel Agreements (SLA) sold to the customer.An informative appendix of the standard defines anumber of tests that evaluate Frame Loss Ratio,Frame Transfer Delay, Frame Delay Variation, andAvailability service performance parameters. Theseparameters are commonly known as objectiveservice performance indicators and are also oftendescribed in SLAs. In order to meet the performanceparameters defined for a service, the provider has toconfigure certain Quality of Service parameters onthe devices used to construct the service — this isnormally where the difficulties lie.In order to support multiple service classes, a serviceprovider has to configure parameters such asCommitted Information Rate (CIR), Excess Infor-mation Rate (EIR), Committed Burst Size (CBS),Excess Burst Size (EBS) as well as color mode (color-aware or color-blind). As our testing showed, config-uring all these parameters is a challenging task sinceMEF standard nomenclature does not necessarilyalign with different vendors‘ configuration language.The test was conducted over two Ethernet VirtualPrivate Line (EVPL) services that were defined as“Services Under Tests”. In essence these were thenew services that were being activated. The Firstservice, EVPL1, was defined to operate in color-aware mode. The service was provisioned with threeBandwidth Profiles (BWP) per EVC per CoS asspecified in the MEF 10.2 standard and the MEF23.1 draft standard. The profiles were applied onthe UNIs of EVPL1, with no policing on subsequentports in the network:BWP1 – CIR 5 Mbps, CBS 16,000 bytes, EIR 0Mbps, EBS 0 bytes; The service was defined as veryimportant (we called this service Metro High) andhad one color: Green.The second bandwidth profile was used for mediumpriority traffic. Its characteristics were: CIR 10 Mbps,CBS 16,000 bytes, EIR 10 Mbps, EBS 16,000 Bytes.Here, two colors were defined: Green and Yellow.The third bandwidth profile, BWP3, was defined for

Vendor Devices

Albis Technologies ACCEED 1416ACCEED 2202

Aviat Networks Eclipse Packet Node

Brocade MLXe-4NetIron CER

Calnex Paragon-X

ECI SR9705SR9608BG9310

Ericsson MINI-LINK TNMINI-LINK LHMINI-LINK PT2010MINI-LINK CN510MINI-LINK PT6010MINI-LINK SP110MINI-LINK SP210MINI-LINK SP310SPO1460SPO1410

Extreme Networks E4G-200E4G-400

Hitachi AMN1710

Ixia IxNetworkImpairNet

Linkra WIDHOP

NEC iPASOLINK 400

Nokia Siemens Networks (NSN)

FlexiPacket MultiRadioFlexiPacket FirstMile 200FlexiPacket Hub 800

Rohde & Schwarz SITLine ETH 1G

Siklu EtherHaul-1200

SpirentCommunications

Spirent TestCenterSpirent GEMSpirent XGEMSpirent Anue 3500

Sunrise Telecom RxTSL GigE RSPDR

Symmetricom Cesium Reference CsIIITimeProvider 1500TimeProvider 5000TimeProvider 5000 EvolutionSSU 2000e

Veryx XC1000

Vendor Devices

4


low priority traffic and included CIR 0 Mbps, CBS 0bytes, EIR 15 Mbps, EBS 16,000 Bytes. Here onlyYellow color was defined.In order to make sure that the tests are thorough,each bandwidth profile at each end of the networkwas first separately tested to verify its configurationand performance prior to connecting the service andtesting the entire service. This procedure separatelyconfirmed the proper operation of each bandwidthprofiler. In EVPL1, the Albis ACCEED 2202 wasdefined as the customer premises equipment and theECI 9608 was defined as the Aggregation orProvider Edge device. Once the configuration of each bandwidth profilewas confirmed, a loopback test confirmed the perfor-mance of the EVC to Y.1564.

In the second, color-blind scenario, we defined asecond EVPL service with slightly different per EVCper CoS bandwidth profiles:BWP1 – CIR 15 Mbps, CBS 16,000 bytes, EIR 0Mbps, EBS 0 bytes and BWP2 – CIR 20 Mbps, CBS16,000 bytes, EIR 30 Mbps, EBS 16,000 Bytes.

We repeated the same procedure, first verifying thatthe participating vendors devices’, which includedAlbis ACCEED 1416, ECI 9608 and Ericsson MINI-LINK TN, had the correct configuration applied andthen verified that the service parameters such asFrame Loss Ratio, Frame Transfer Delay, FrameDelay Variation, and Availability reported theexpected values within the Service AcceptanceCriteria defined by Y.1564 and the Metro Class ofService Performance Tier of MEF 23.1 (under devel-opment).One of the biggest challenges of the test was toachieve a common understanding of QoS configu-ration between vendors. Since QoS architecture isnot standardized well, every vendor has its owninterpretation of QoS. For example, the committedinformation rate parameter on one device wasconfigured with an Information Rate value whileanother device configured the same parameter usingUtilized Line Rate (ULR). We also found that someimplementations did not allow the operator toconfigure burst size at the granularity we had origi-nally specified in the test plan. We actually asked for12,176 bytes of burst size, but due to some limita-tions of some of the devices we converged on avalue of 16,000 bytes. In the same configurationarea (burst size) we also found that some of theimplementations allowed a very granular configu-ration, however, the tests showed that the configu-ration was actually not enforced by the hardware.For service providers we see these initial results as apositive sign. The instrumentation and standards toverify service activation now exist and both solutionsand services can be tested before an Ethernetservice is handed over to a customer. Especially nowthat Ethernet services are becoming more and moremission critical and are replacing legacy TDMsolutions, having the standards to verify that theservices are really working as advertised.

Performance Monitoring

Once an Ethernet service has been activated,monitoring its performance in-service is one of themost desirable requirements for service providers.After all service providers sign contracts with theirbusiness customers to deliver a certain quality ofservice (defined in Service Level Agreements, SLAs).These SLAs often stipulate financial implications inthe case that the service is “out of SLA”.The metrics to perform monitoring of services isdefined in the ITU-T specification Y.1731. The speci-fication introduces techniques to measure serviceperformance objectives such as frame loss, framedelay, and frame delay variation on point-to pointEthernet services. We started interoperability testing of Y.1731 imple-mentations when the standard was young in 2008.But just around now, three years later, serviceproviders really begin to deploy performancemonitoring in production-grade Carrier Ethernetservices. Vendors brought new or updated productsto our test this time. The level of interest was so high

Color blind Test

ECI 9705

ECI 9608

AlbisACCEED 2202

AlbisACCEED 1416

AlbisACCEED 1416

EVC1

EVC2

Stream Direction

Customer Domain

Color aware and Color-Blind Test

Figure 1: Service Activation Tests

Analyzer/Traffic Generator

Loopback device

EricssonMINI-LINK TN

EricssonMINI-LINK TN

ECI 9705

ECI 9608

AlbisACCEED 2202

Carrier EthernetNetwork

SunriseSL GigE RSPDR

SunriseRxT

SunriseRxT

SunriseRxT

SunriseRxT


SunriseSL GigERSPDR

SunriseSL GigERSPDR

5

Service Activation Test

that we were able to create a list of 25 vendor pairsfor the test.We distinguished between two measurement typefor the tests: Two-way frame delay and two-wayframe delay variation.We used a procedure that allowed us first to validatethat the protocol exchange between two implemen-tations worked as expected focusing on Y.1731Delay Measurement Messages and Replies (DMMsand DMRs). Once this test procedure was completedand a baseline was reached, we added animpairment generator for all further tests. We usedCalnex Paragon-X, Ixia ImpairNet and Spirent GEMimpairment generators to introduce controlled delayand delay variations so we could verify that not onlythe implementations worked with each other, butalso worked correctly. For the frame delay measurement tests we used theimpairment generators to add constant delay of20 milliseconds (ms) to all DMM packets. Weexpected that the implementations will report anaverage frame delay value of 20 ms. It was not anunrealistic expectation given that the propagationdelay in our lab approached zero. In order to verifythat the two implementations measured frame delayvariation correctly we asked the impairmentgenerator vendors to introduce packet delay of 15ms to every second DMM packet and 25 ms to theremaining packets. With this procedure weexpected to observe average frame delay of 20 ms,and frame delay variation of 10ms.The following devices successfully participated in theY.1731 delay/delay variation tests (see Figure 2):Albis ACCEED 2202, Albis ACCEED 1416,Brocade MLXe-4, ECI 9608, ECI 9705, EricssonSPO1460, Ericsson MINI-LINK SP310, IxiaIxNetwork, Spirent TestCenter.Each pair in the figure represents a vendor pairwhich constructed a Carrier Ethernet service usingVLANs (IEEE 802.1Q standard) and that havepassed the test.After we completed performance measurement testsbetween Brocade MLXe-4 and Ericsson SPO1460,as well as between Albis ACCEED 2202 and ECI9608, we inserted the Rohde & Schwarz SITLineETH 1G encryption devices and repeated the testwith the same measurements. Our goal was to verifyif the measurements could still be executed betweentwo devices over an encrypted EVPL service.We then verified that the performance measurementswere still working over the encrypted service andalso, using the delay measurements themselves, thatthe service‘s delay and delay variation was onlymarginally impacted.

Emulated PSN

Y.1731 frame delay

Y.1731 frame delay variation

EricssonSPO1460

SpirentGEMBrocade MLXe-4

Rohde & SchwarzSITLine ETH 1G


AlbisACCEED 2202

IxiaImpairNetECI 9608



AlbisACCEED 2202

SpirentTestCenter

SpirentGEM

AlbisACCEED 2202

SpirentGEM ECI 9608

AlbisACCEED 1416

SpirentGEM

EricssonSPO1460

SpirentTestCenter

SpirentGEMECI 9608

EricssonMINI-LINK SP310

CalnexParagon-X ECI 9608

IxiaIxNetwork

SpirentGEMBrocade MLXe-4

EricssonSPO1460

CalnexParagon-X Brocade MLXe-4

Spirent TestCenterBrocade MLXe-4

Spirent GEMECI 9705Brocade MLXe-4

Provider/Provider Edge

Impairment tool

Aggregation

Emulator

Layer2encryption devicedevice

Figure 2: Performance Monitoring

6


Brocade MLXe-4 and Ixia IxNetwork also performedthe baseline performance monitoring test over100 Gigabit Ethernet (GbE) interface. As we couldnot instrument the event with a 100GbE impairmentgenerator, we had to skip the accuracy verification. Another performance measurements test that wasexecuted using the Brocade MLXe-4 over 100GbEinterface used a Virtual Private LAN Service (VPLS)service with Spirent TestCenter. Again, only the basicprotocol interoperability was verified in this caseand no delay and delay variation accuracy wasmeasured.During the performance monitoring test, weobserved three implementations for delaymeasurement. One vendor supported only on-demand delay measurement. This implementation isuseful where the delay measurement function istriggered by an operator or by an externalmanagement system. Two vendors supported scheduled measurement. Inthis case, the values of delay and delay variation arecalculated over a specific time interval. The countersused for the calculation were cleared at the end ofthe time interval, and the measurement continuedover the next interval continuously. There was also one implementation where the meanvalue of delay and delay variation could beobserved only once the DMM session was stoppedmanually. In order to follow the procedure in the testplan, a restart was required at each step. It waspossible to read the live run delay values however.We were pleased with the results we collected forthis test and believe that some of the implementationsbenefited from the opportunity to test against somany other implementations. We see that theprotocol gets used in a growing number of imple-mentations. This is a positive signal to the industrythat services could be delivered with objective andongoing quality measurements.

Hierarchical Service OAMThe intention of Hierarchical Service OAM was toverify the interoperability of Service OAM functionsbetween different vendor Maintenance AssociationEnd Points (MEPs) at multiple maintenance levels.Since no other vendor participated in this test case,Albis created a demo to show the support of EthernetAlarm Indication (ETH-AIS), and Ethernet LockedSignal (ETH-LCK) functionality in an OAM hierarchy. We verified that the ETH-AIS messages werereceived successfully at the subscriber level in casethat the transport path and the configured servicefailed. We also verified that the ETH-LCK informationwas received periodically at the subscriber level withno impact on the traffic, until the administrative lockcondition was removed. The demo was presentedsuccessfully with no issues observed.

RESILIENCY AND REDUNDANCYSome assumptions could be made these days withrespect to the way carrier packet networks aredesigned: The core of the network is likely to bebased on MPLS (with or without transport profile)and the access will be built based on availablephysical layer infrastructure. The test scenariosplanned for this area were meant to provide anoverview on the available tools for service providerto offer reliable Ethernet services to customers. Resil-ience is after all one of the five attributes thatdescribe Carrier Ethernet so we set out to investigatethe options available in the market.

Ethernet Ring Protection SwitchingEthernet has come a long way since the days inwhich resiliency was performed using spanning treeprotocols. These days the target set in the ITU-TG.8032 standard for resiliency is 50 ms — a targetas aggressive as the gold standard set by SDH.G.8032 or Ethernet Ring Protection Switching (ERPS)is a standard for fast recovery in Ethernet based ringtopologies using automatic protection switching(APS) protocol.ERPS-enabled Ethernet nodes send ring automaticprotection switching signal failure (R-APS(SF))messages as soon as they detect a failure on a ring.The failure signal causes the switch on that ring thathas one of its ports in blocking state (RPL owner) tounblock its RPL port therefore forcing a change to thering in order to bypass the failed ring segment. Thenodes adjacent to the failed link isolated the failed

Subscriber network

Service Provider network

Operator network

Site 1-Site 2 E-Line

DUTEmulator

Up-MEP

Down-MEPMaintenance Association

Subscriber ME Level

EVC ME Level

Operator ME Level

Figure 3: Hierarchical Service OAM

7

Resiliency and Redundancy

network segment by blocking their ports and trafficis then forwarded through the open RPL port.In revertive operation mode the nodes detect that thefailure is resolved and are expected to send ringautomatic protection switching no request (R-APS(NR)) message causing the RPL owner to start theWait to Restore (WTR) timer. Upon expiration of theWTR timer, the RPL owner blocks its RPL port. Nodesadjacent to the former failed link unblock their portsand the ring then resumes its original operation. Atthis point all nodes flush their Forwarding DataBases(FDBs). This whole procedure should theoreticallyoccur within 50 milliseconds. The standard allowsfor non-revertive operations, however, we did nottest this mode.

Our goal was to verify that a ring, constructed by a

number of vendor solutions, will behave asdescribed above. The assumption we made was thatthe access is exactly where carriers would require afunctioning multi-vendor rings. After all, access isoften the network area that includes switches thatwere positioned in the core of the network or newsolutions that were added at a second networkrollout phase.The following vendors participated in this test area(see Figure 4): Brocade MLXe-4, ECI BG9310,Ericsson MINI-LINK PT2010/ MINI-LINK CN510/SPO1410, Ericsson MINI-LINK SP310/MINI-LINK CN510/MINI-LINK PT2010, Extreme E4G-400, NEC iPASOLINK 400, Siklu EtherHaul-1200,and Veryx XC1000.In all test scenarios two rings were constructed: amajor ring and a sub-ring. Both were interconnectedvia a shared link. In each ring one dedicated nodewas provisioned as RPL owner. In most of the tests aRPL neighbor was configured in both major and sub-ring to block the other end of the RPL link.In order to perform the test we defined two profilesto accommodate different implementations: The firstprofile (Profile 1) was designed for vendorsallocating different VLAN ID in both ring for R-APScommunication. The second profile was intended forvendors that use the last octet of the MAC addressas RingID for the R-APS communication to distinguishthe R-APS channel in between the rings.Initially, Connectivity Fault Management (CFM) wasrequested to be used to monitor the liveliness of thering. Since vendors take different approaches torealize CFM - Some vendors used a separate VLANID for CFM and R-APS communication whereas othervendors used the same VLAN ID for both CFM andR-APS, it was difficult to find a common linklivelihood. After a long debugging session we couldnot converge on a single method and in order tomove on with the ERPS testing we decided to run thetest without CFM. Instead we triggered the failoverby pulling the cable from one of the devices.In all test scenarios, traffic was sent at 1,000frames/seconds on the service configured in the ringusing either Ixia IxNetwork, Spirent TestCenter orVeryx. We emulated link failure between the inter-connection nodes and verified the proper reaction ofthe ring.In the first test scenario, ECI BG9310 participated inthe sub-ring as ERPSv1 and all other devices partici-pated in this test scenario and in all subsequent testscenarios with their ERPSv2 implementation.The Veryx solution acted as a ring node and wasalso able to generate traffic in the ring. When wewanted to verify that the Veryx XC 1000 had the RPLport block, we realized, together with the Veryxengineers that there are no commands available onthe solution to show such port state. None the lesswe could verify, using traffic flow directionality in thering that the Veryx solution was indeed blocking theport as expected.

ExtremeE4G-400

Primary path

Port blocking

Link failure

Backup path

Figure 4: Ethernet Ring Protection Switching

Major ring

Sub-ring

ExtremeE4G-400

ECIBG9310

Ericsson MINI-LINKSP310/CN510/PT2010

SikluEtherHaul-1200

ExtremeE4G-400

VeryxXC1000

NECiPASOLINK 400

ExtremeE4G-400

ExtremeE4G-400

VeryxXC1000

NECiPASOLINK 400

BrocadeMLXe-4

Ericsson MINI-LINKPT2010/CN510

/SPO1410NEC

iPASOLINK 400

SikluEtherHaul-1200

8


gation

INI-LINK10

n MINI-LINK10/PT2010 M

MI

son410

CalnexParagon-X

IxiaIxNetwork

G9310

Ericsson MINI-LSP310

ECI 9608

IxiaIxNetwork

MPLS

-TP

n

E

Access device

Emulator

TOPOLOGY

Ethernet Aggre

Aviat NetworksEclipse Packet Node

Video Client

VideoSource

CalnexParagon-X

CalnexParagon-X


SymmetricomSSU 2000e

SymmetricomTimeProvider 1500

PackeTime ptp

12:50:00Symmetricom

TimeProvider 5000

Ericsson MSP1

EricssoCN5

HitachiAMN1710

EricsSPO1

Evolution

AlbisACCEED 2202

IxiaImpairNet

AlbisACCEED 2202


Traffic Generator

Ericsson MINI-LINKTN

Traffic Generator

IxiaIxNetwork

AlbisACCEED 1416

AlbisACCEED 1416

SikluEtherHaul-1200

Linkra WIDHOP

NSN FlexiPacketMicrowave

ECI B

EricssonMINI-LINK SP310

ECI 9705

12:50:00Symmetricom

TimeProvider 5000

IP/

MPLS

Ethernet Aggregatio

Connection Types

Analyzer/Traffic Generator

Device Types

Provider/Provider Edge deviceClock link

1Gbit Ethernet link

100Gbit Ethernet link

10Gbit Ethernet link

TDM linkAggregation device

Bonded SHDSL link

Topology

9

SpiGE

EricssonI-LINK SP210

Ericsson-LINK SP310

SunriseRxT

R

SpirenAnue 35

BrocadeCER

iPAS

Extreme4G-400

Eth

K

SpirenTestCent

BrocadeMLXe-4

hernet Ac

12:50:00 IEEE

Lay

Devi

TOPOLOGY

SymmetricomCesium Cslll

EricssonSPO1460

rentM

Ericsson MINI-LINKPT6010

Traffic GeneratorNECiPASOLINK 400

VeryxXC1000

SpirentAnue 3500

Traffic Generator

NECiPASOLINK 400

xpic

Ericsson MINI-LINKLH


Traffic Generator

Aviat NetworksEclipse Packet Node

SpirentAnue 3500

ExtremeE4G-200

CalnexParagon-X

ohde & SchwarzSITLine ETH 1G

t00

NECOLINK 400

SikluerHaul-1200

ter

cess

Network Areas

Reference Clock

Impairment toolMicrowave radiosystem

Ethernet Aggregation network

MPLS-TP network

IP/MPLS network

Access network

ERPS Rings

Loopback device

1588v2 Grandmaster

er2 encryption device

ce Types

10


In all test scenario after pulling the cable betweeninterconnection ring nodes, we successfully verifiedthat the protection switching was triggered - RPLnode in the major ring unblocked the RPL port andthe port connected to the failed link was blocked.The RPL port in the sub-ring remained blocked. Thefailover and restoration time was asymmetric andranged from 51 to 290 milliseconds and 7 to 96milliseconds respectively.We also successfully tested the following adminis-trative commands on all major ring RPL ownerdevices: “Manual Switch” and “Force Switch” bothof which move the blocked port as desired aroundthe ring. We also tested the “Clear” command,which removes both “Manual Switch” and “ForceSwitch” commands. The failover time during theseset of tests ranged from 0 to 103 milliseconds andthe restoration ranged from 0 to 91 milliseconds.During the test we encountered several issuesranging from configuration to interoperability.Specially we were unable to perform the test usingprofile 2. Using said profile, and while emulating thelink failure on the shared link, nodes connected tothe failed link (interconnection nodes) started to sendSignal Fail (SF). After receiving this message, thesub-ring node interprets the messages and removesthe RPL block, which causes the traffic to flow on adifferent direction than expected.

MPLS-TP OAM 1:1 ProtectionOne of the areas of interest for MPLS-TP vendors hasbeen protection scenarios. For this event threevendors were ready to support MPLS-TP 1:1protection tests. Two impairment generators —Calnex Paragon-X and Spirent Anue 3500 —offered to provide impairment functions. In prepa-ration for the tests we asked vendors to bring newimplementations as usual. As a result of the poll, wefocused on BFD implementations only this time sinceall available Y.1731 implementations have beentested in our interoperability events frequently in thepast.The test scenario was straightforward from a testingperspective: Build an MPLS-TP based network using1:1 protection, fail the active link and verify that thenetwork converges in under 50 ms. This test wasperformed by statically configuring two bidirectionalLabel Switched Paths (LSP) - primary and secondary -using a single physical connection between EricssonSPO1410 and Ixia IxNetwork. BFD ContinuityCheck (CC) sessions were running on both primaryand secondary LSPs to monitor the liveliness of theLSP and the pseudowire service configured on top ofthe LSPs. BFD-CC was transmitted over the GenericAssociated Channel (G-ACh) and GenericAssociated Label (GAL). Both BFD sessions wererunning in a slow start mode of operation. Once theBFD sessions moved into the UP state the trans-mission and receive intervals changed to 3.33ms.Once both primary and secondary LSPs were estab-lished we offered load to verify that the traffic wasindeed using the primary path. We then emulated an

unidirectional link failure on the primary path bydropping all traffic and BFD packets. We verifiedthat traffic moved to the secondary path. Therecorded failover when only the Worker BFD-CCflow was impaired was 0 milliseconds. The recordedfailover when both the Worker BFD-CC and WorkerPseudowire flows were impaired was 12.4 milli-seconds. After disabling the link failure emulation thetraffic reverted back to the primary path without anyloss. The restoration time was then zero milliseconds.

Besides protection switching due to the link failure,Administrative commands “Lockout of Protection”,“force Switch normal traffic signal-to-protection”,“manual switch normal traffic signal-to-protection”were all successfully tested based on draft-ietf-mpls-tp-linear-protection-03.

CLOCK SYNCHRONISATIONEANTC started testing clock synchronization interop-erability already in early 2008. Over time we haveseen a constant increase in the number of implemen-tations and more and more positive results.Participating vendors were still interested to continueclock synchronisation tests so we obliged. We alsohear from our service provider customers that thelevel of trust in the multi-vendor interoperability of theassociated protocols (specifically IEEE 1588v2) isstill not as high as it should be. With this mandate,we set two areas of testing for the IEEE PrecisionTime Protocol (PTP or IEEE 1588v2): OrdinaryClocks and boundary clocks.

IEEE 1588 Master/Slave – Phase and Frequency SynchronizationIn certain conditions transporting clock signal over apacket based solution is the only option available forservice providers. For example, SynchronousEthernet can not be used over Ethernet servicesleased from other providers as the majority do notoffer such services. At this point a service providerrequiring synchronization services, for mobilebackhaul or other services, could use the IEEE1588v2 Precision Time Protocol (PTP). The protocolis also attractive when Time of Day and phasesynchronization is needed.

ImpairmentTool

Traffic GeneratorWorkingMPLS-TP LSP

ProtectionMPLS-TP LSP

Figure 5: MPLS-TP 1:1 Protection

CalnexParagon-X

IxiaIxNetwork

IxiaIxNetwork

EricssonSPO1410

Ethernet access

MPLS-TP networkPseudowire

Emulator

11

Clock Synchronisation

Our methodology for verifying 1588v2 focuses onprotocol interoperability as well as on clockaccuracy. The latter is only used in order to verifythat the interoperable implementations are alsodelivering useful clock information to the clients ofthe service.The measurements were performed by connectingeach slave device to a wander analyzer forfrequency, via either E1 or 2048 KHz interfaces,and a frequency counter via 1 PPS (pulse persecond) for phase (time of day, or TOD) deviationmeasurement. All test scenarios were required topass the 15ppb mask for MTIE, and a maximumtime error of 3 microseconds for the time of daydeviation.Impairment was introduced into the networkaccording to ITU-T G.8261 Test Case 12 using aCalnex Paragon-X or Spirent Anue 3500 impairmentgenerators. Frequency wander measurement wasperformed using a Spirent Anue 3500 and anothermeasurement device over a period of 4 hours ormore. Phase measurements were planned, but at theend were not tested since the slave clocks partici-pating in the test only supported frequency output.Configuration of all slave clocks was done overunicast IP/UDP on a specified VLAN, with 64 Syncmessages per second in Two-Step mode.We successfully tested two PTP slaves (see Figure 6)under impairment, the Ericsson SPO1460 and NSNFlexiPacket Hub 800, and one slave withoutimpairment emulated by Ixia IxNetwork.

In addition to successful tests, we found severalinterop issues among some vendors. These weremostly due to differing features (such as supportedsync message rates or One-Step only vs. Two-Steponly). An additional reason for the lower than expectedresults for this test was the fact that the test durationwas so long. This meant that synchronization was

lost during the measurement period, we did not haveenough time to repeat the tests.

IEEE 1588 Boundary Clocks There were two Boundary Clock scenarios run, asonly one vendor attending this event brought aBoundary Clock to be tested.Like in the Master/Slave synchronization test cases,the Symmetricom TimeProvider 5000 served as thePTP Grandmaster. Impairment was once againbased on G.8261 Test Case 12, and provided byeither an Calnex Paragon-X and Spirent Anue 3500.The Ericsson MINI-LINK SP310 served as theBoundary Clock in both scenarios tested. The SlaveClocks tested were in a second PTP domain, as aslave to the Boundary Clock. This role wassupported by an Ericsson MINI-LINK SP110 and anExtreme E4G-200 (see Figure 7). In the case of theEricsson-Extreme interop, Ericsson used the MINI-LINK PT6010 microwave as a fiber replacement inbetween the Master and Slave Clock devices.Both Boundary Clock and Slave Clock wereconfigured with 64 sync packets per second in Two-Step mode, but the VLAN and PTP Domain weredifferent on either side of the Boundary Clock torepresent a realistic real-world scenario.Frequency measurement was done with the CalnexParagon-X or Spirent Anue 3500. 1PPS (Time ofDay) measurement was done with a CalnexParagon-X or a frequency counter. Like in theMaster/Slave test case, all tests had to pass theG.823 15ppb mask or better for frequency, andhave a maximum Time of Day deviation of 3 micro-seconds or less.

Impairmenttool

Figure 6: PTP Master/Slave Clock

CalnexParagon-X

Spirent12:50:00

SymmetricomTimeProvider

EricssonSPO1460

Frequency

5000

E1/2048KHz

Ethernet link

12:50:00

PTP domain

PTP grandmaster

PTP slave

NSNFPH800Anue 3500

Analyzer

Emulator

IxiaIxNetwork

SymmetricomCsIII

Referenceclock

Impairmenttool

CalnexParagon-X

SymmetricomCsIII

Ericsson MINI-LINK SP310

FrequencyAnalyzer

PhaseAnalyzer

1PPS link

Figure 7: PTP Boundary Clock

Ericsson MINI-LINK SP110

ExtremeE4G-200

Referenceclock

12:50:00PTP Grandmaster

E1/2048KHz

Ethernet link

PTP domain 1

PTP domain 2

12:50:00SymmetricomTimeProvider

5000

Anue 3500Spirent

12


Ethernet Synchronization Messaging Channel (ESMC)To test ESMC, three devices were configured in alinear fashion to have varying reference clockqualities. Supplied externally to the devices arePrimary Reference Clock (PRC) and SynchronizationSupply Unit (SSU) signals. A device that does nothave a clock source connected is expected to sendDNU (Do Not Use) messages or SEC (SDHEquipment Clock), since the internal clock on mostdevices are expected to comply to SEC specifica-tions at best. Due to the clock sources attached todevices in specific locations, we will refer to thosenodes as PRC, SEC, and SSU. Clock sources were connected and disconnectedfrom each node, allowing them to adapt while wemonitored the links between them to be sure that theSSMs (Synchronization Status Messages) being sentwere changed accordingly.

Six ESMC scenarios were tested. The followingvendors all took on various roles (see Figure 8) ineach given configuration: Albis ACCEED 1416,Aviat Networks Eclipse Packet Node, Ericsson MINI-LINK LH, Ericsson MINK-LINK PT2010, Extreme

E4G-400, Ixia IxNetwork, NSN FlexiPacketMicrowave, Siklu EtherHaul-1200, and Symmet-ricom TimeProvider 5000 Evolution.The Spirent Anue 3500, Calnex Paragon-X, andSpirent TestCenter were used to evaluate theSynchronous Ethernet clock Quality Levels on theline, which were sourced from the SymmetricomCsIII.One vendor found a bug in which they were notsending messages when the device reported theywere, and one had an issue with switching to alower-priority clock source. Both vendors testedsuccessfully later with other products or bug fixes.

ETHERNET MICROWAVE TESTSOur test ideas in this area focused on Ethernetfunctionality and not on microwave physical layerfunctionality. The microwave vendors were morethan ready to demonstrate interoperability with otherswitches and routers.We understand the reason behind the inclusion ofmore and more advanced features in microwavesolutions: The microwave solutions are an integralpart of many mobile backhaul solutions and as suchhave to satisfy an extensive set of requirements.These requirements on microwave solutions couldclearly be seen in the tests in which microwavevendors exclusively participated as well as in thetests that were opened to all.We requested microwave vendors to participate asactive components in all the tests we describe belowand, as a policy, we did not focus on air interfacetesting (including capacity).

Microwave QoS SupportAs microwave systems are evolving from puretransport to a more intelligent and complex solutions,one transport capability required is support forquality of service (QoS). The nature of microwaveslink is its capacity variation that depends on weatherconditions. It is therefore often desirable, that thebandwidth is preserved for more critical data, if thelink capacity decreases.In this test we configured two classes for which wegenerated test traffic: Priority and Best Effort. Withthe BestEffort test traffic we emulated Internet traffic,with the Priority test traffic we emulated legacy TDMvoice traffic encapsulated into packets by means ofCircuit Emulation Service (CES). For each test we used two microwave systems of twodifferent vendors that were connected via anEthernet wire. Both systems were required toconfigure QoS and differentiate traffic based onEthernet Priority Code Points (PCP) - a part of IEEE802.1Q VLAN tag. We used PCP 0 for BestEffortand PCP 6 for priority class. During the test wegenerated 60 Mbit/s bidirectional IMIX (Internetmix) BestEffort traffic and 2 Mbit/s bidirectionalPriority traffic with packet size equal to 256 Bytes.During the test we changed modulation of

Figure 8: ESMC

SyncE link

Ethernet linkSyncE Node

PRC SEC SSU

ECIBG9310

Aviat EclipsePacket Node

SpirentTestCenter

ECISR9608

ExtremeE4G-400

SymmetricomCsIII

ExtremeE4G-400

IxiaIxNetwork

ExtremeE4G-400

CalnexParagon-X

E1/2048KHz

Impairment tool

SymmetricomTP 5000 Evolution

NSN FPMultiRadio

SymmetricomCsIII

SikluEtherHaul-1200

Ericsson MINI-LINK PT2010

AlbisACCEED 1416

SikluEtherHaul-1200

SikluEtherHaul-1200

Ericsson MINI-LINK LH

ReferenceClock

Symmetricom TP5000 Evolution

ExtremeE4G-400


EricssonMINI-LINK LH

13

Ethernet Microwave Tests

microwave links on both participating systems firstmanually then via attenuating of the microwave link -emulating bad weather condition and triggeringadaptive modulation on the systems. Our expec-tation was that we do not loose any packets from thePriority class and do loose packets of BestEffort classwhen link capacity was reduced.11 Microwave System pair combinations success-fully passed this test (see Figure 9): Aviat NetworksEclipse Packet Node and Ericsson MINI-LINK TN,Aviat Networks Eclipse Packet Node and SikluEtherHaul-1200, Aviat Networks Eclipse PacketNode and Linkra WIDHOP, Ericsson MINI-LINK TNand Siklu EtherHaul-1200, Ericsson MINI-LINK TNand Linkra WIDHOP, Ericsson MINI-LINK TN andNSN FlexiPacket Microwave, Ericsson MINI-LINKTN and NEC iPASOLINK 400, Linkra WIDHOP andSiklu EtherHaul-1200, Linkra WIDHOP and NSNFlexiPacket Microwave, Linkra WIDHOP and NECiPASOLINK 400, NEC iPASOLINK 400 and SikluEtherHaul-1200.

On the microwave links we used differentmodulation per vendor. When we emulated normalweather condition we used the following modula-tions: on the Aviat Eclipse Packet Node 256QAMwith 28MHz channel bandwidth; on the EricssonMINI-LINK TN 1024QAM with 28Mhz channelbandwidth; on the Linkra WIDHOP 256QAM with28MHz channel bandwidth; on the NEC iPASOLINK400 512QAM with 28MHz channel bandwidth; onthe NSN FlexiPacket Microwave 256QAM with28MHz channel bandwidth; on the Siklu EtherHaul-1200 64QAM with 500MHz channel bandwidth.When we emulated bad weather conditions, allvendors were stepping back to a most robustmodulation which was 4QAM with 256 MHzchannel capacity for Siklu and 4QAM with 28 MHzchannel capacity for all other vendors.On the Ericsson MINI-LINK TN and NSN FlexiPacket

Microwave combination we configured SAToPCESoIP according to RFC4553, and on the EricssonMINI-LINK TN and NEC iPASOLINK 400 combi-nation we configured CESoETH (Circuit EmulationService over Ethernet) according to MEF8. Themicrowave systems successfully converted the E1signal from and to Ethernet packets and transportedit over microwave in the priority class.

Synchronous Ethernet over MicrowaveOne of the other capabilities that we looked at forMicrowave vendors was transporting SynchronousEthernet, a physical layer technology, overmicrowave links. In that respect, the challenge for amicrowave vendor is to transform the SynchronousEthernet clock received on an Ethernet interface to aformat that could be transported across the airinterface and then recover the clock signal on theother microwave base station. At this point thesolution will convert the signal back to SynchronousEthernet and provide it further in the clock chain.As is seen in Figure 10 we defined one microwaveas a clock master, starting the clock chain and thentransporting the clock over its own microwave link,attaching to a second vendor solution using anEthernet link and then repeating the story again. Thisprovided a way to both verify the ability of thesolution to act as a clock master, slave and transportthe clock signal over two different air interfaces.Five Vendors were tested in a Synchronous EthernetMaster/Slave scenario.A PRC clock source was provided via a Symmet-ricom SSU 2000e, which also provided a referencefor measurements on a Calnex Paragon-X andSpirent Anue 3500.We waited for the master and slave clocks tosynchronize monitoring the duration the clocks tookin moving from free-running to synchronize state andthen ran measurements for a minimum of two hours.All test measurements were required to pass theG.8262 SSU Option 1 or Option 2 Mask and havea PPB (parts per billion) of under 16 nanoseconds ata sample rate of 1 second.Eight vendor pairings were successfully tested (seeFigure 10) consisting of: Aviat EclipsePacket Node,Ericsson MINI-LINK LH, Ericsson MINI-LINKPT2010/SPO1460, Ericsson MINI-LINK PT2010,NEC iPASOLINK 400, NSN FlexiPacket Microwave,and Siklu EtherHaul-1200.

Bidirectional traffic

Figure 9: Microwave QoS Support

(Priority and Best Effort classes)

Microwave radiosystem

PSN


NECiPASOLINK 400

Ericsson MINI-LINKTN

SikluEtherHaul-1200

NSN FlexiPacketMicrowave

Linkra WIDHOP

14


DEMONSTRATION NETWORKThe demonstration network was built based onselected successful results and vendor demonstra-tions. Since we could not show all results in the samenetwork topology the detailed test results aredescribed in the test sections. In the demonstration network we reflected thedifferent network technologies tested in the eventaccording to different by network areas. We tried tominic a real provider network as possible. Wecreated Ethernet Access, Aggregation Ethernet,MPLS-TP aggregation and IP/MPLS core networkareas.In the IP/MPLS area, Brocade MLXe-4, IxiaIxNetwork, and Spirent TestCenter configured IP/MPLS over 100Gbit/s Ethernet interfaces. Ixia andSpirent emulated multiple PE devices connectingthem to a VPLS service that was build with theBrocade MLXe-4 router.In the MPLS-TP area, Ericsson SPO1410, HitachiAMN1710 and Ixia IxNetwork all established MPLS-TP LSPs and pseudowire services as well as success-fully established BFD-CC and PSC sessions. TheMPLS-TP services were transporting test trafficgenerated from the emulated devices.In the aggregation area we showed two ERPS rings,synchronous Ethernet over microwave, transport ofPTP sessions from a PTP Grandmaster towards a PTPBoundary Clock over the IP/MPLS core and Ethernetaggregation network areas. We also demonstratedredistribution of the Precision Time Protocol signalfrom a boundary clock towards ordinary clocks.Another demonstration included service activationfor a service configured between ECI and Albisdevices.Based on QoS over microwave test results wecreated a chain of microwave systems that transportCircuit Emulation Services (CES), video signal, andInternet-like data in the aggregation area over 5microwave hops.We also demonstrating an encrypted EVPL trans-ported over IP/MPLS network and performancemonitoring for this service. The performancemonitoring results of Albis and ECI devices demon-strate that the encryption of the EVPL did notnegatively influence the service.

AcknowledgementsWe would like to thank Stephen Murphy from theUniversity of New Hampshire IOL for his supportduring the testing.Editors. This report has been written and edited byJambi Ganbar, Sergej Kaelberer, Stephen Murphy,Ronsard Pene, Carsten Rossenhoevel and ShimaSajjad.

Master Slave

SyncE link

E1/2048KHz link


FrequencyAnalyzer

ReferenceClock

Microwave systemsupporting SyncE

Figure 10: SyncE Test Results

NECiPASOLINK 400

NECiPASOLINK 400



SpirentAnue 3500


NSN FlexiPacket

CalnexParagon-X

AviatEclipse Packet Node

AviatEclipse Packet Node

SikluEtherHaul-1200

SikluEtherHaul-1200

NECiPASOLINK 400

Ericsson MINI-LINKPT2010/SPO1460

NSN FlexiPacket



Microwave

Microwave

NECiPASOLINK 400


15

References

Acronyms

REFERENCES“Ethernet ring protection switching”, ITU-T G.8032version 2“MPLS Transport Profile Data Plane Architecture”,RFC 5960“A Framework for MPLS in Transport Networks”,RFC 5921“Proactive Connection Verification, Continuity Checkand Remote Defect indication for MPLS TransportProfile”, draft-ietf-mpls-tp-cc-cv-rdi“MPLS-TP Linear Protection”, draft-ietf-mpls-tp-linear-protection, work in progress“Precision Time Protocol (PTP)”, IEEE 1588-2008“Mobile Backhaul Implementation Agreement Phase2”, MEF technical specification, work in progress“Timing and Synchronization Aspects in PacketNetworks”, ITU-T G.8261/Y.1361“Precision Time Protocol Telecom Profile forfrequency synchronization”, ITU-T G.8265.1“Distribution of timing information through packetnetworks”, ITU-T G.8264“Synchronization layer functions”, ITU-T G.781“OAM Functions and Mechanisms for EthernetBased Networks”, ITU-T Y.1731“Transport Performance Metrics MIB”, RFC 4150“Service OAM Fault Management ImplementationAgreement”, MEF 30 technical specification“Ethernet ring protection switching”, ITU-T G.8032version 2“Ethernet service activation test methodology”, ITU-TY.1564“Timing characteristics of synchronous Ethernetequipment slave clock (EEC)”, ITU-T G.8262/Y.1362“Distribution of timing through packet networks”,ITU-T G.8264/Y.1364

Term Definition

APS Automatic Protection Switching

BC Boundary Clock

BFD Bidirectional Forwarding Detection

BFD-CC Bidirectional Forwarding Detection - Continuity Check

BWP Bandwidth Profiles

CBS Committed Burst Size

CC Continuity Check

CCM Continuity Check Message

CFM Connectivity Fault Management

CIR Committed Burst Size

CoS Class of Service

CPE Customer Premise Equipment

DMM Delay Measurement Message

DMR Delay Measurement Reply

DNU Do Not Use

EBS Excess Burst Size

EIR Excess Information Rate

ERPS Ethernet Ring Protection Switching

ESMC Ethernet Synchronization Messaging Channel

ETH-AIS Ethernet Alarm Indication

ETH-LCK Ethernet Locked Signal

EVC Ethernet Virtual Connection

EVPL Ethernet Virtual Private Line

FDB Forwarding DataBase

G-ACh Generic Associated Channel

GAL G-ACh Label

IEEE Institute of Electrical and Electronics Engineers

IETF Internet Engineering Task Force

ITU-T International Telecommunication Union Telecommunication Standard-ization Sector

LAN Local Area Network

LSP Label Switched Path

MA Maintenance Association

ME Maintenance Entity

MEP Maintenance Entity Point

MIP Maintenance Association Interme-diate Point

MTIE Maximum Time Interval Error

OAM Operations, Administration and Management

OC Ordinary Clock

PRC Primary Reference Clock

PTP Precision Time Protocol

R-APS Ring Automatic Protection Switching

R-APS(NR) ring automatic protection switching no request

R-APS(SF) ring automatic protection switching signal failure

RPL Ring Protection Link

SEC SONET/SDH Equipment Clock

SLA Service Level Agreement

SSM Synchronization Status Messages

SSU Synchronization Supply Unit

SyncE Synchronous Ethernet

TDEV Time DEViation

TDM Time Division Multiplexing

TIE Time Interval Error

TOD Time of Day

VPLS Virtual Private LAN Service

WTR Wait to Restore timer

Term Definition


EANTC AGEuropean Advanced Networking Test Center

Hosted by:IIR Telecoms

Salzufer 1410587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 [email protected]://www.eantc.com

29 Bressenden PlaceLondon SW1E 5DR, UKTel: +44 20 7017 7483Fax: +44 20 7017 [email protected]

This report is copyright © 2011 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information containedherein.All brand names and logos mentioned here are registered trademarks of their respective companies in the UnitedStates and other countries.20111004 v1

IntroductionInteroperability Test ResultsParticipants and DevicesService Activation TestPerformance MonitoringHierarchical Service OAM

Resiliency and RedundancyEthernet Ring Protection Switching

TopologyTopologyMPLS-TP OAM 1:1 Protection

Clock SynchronisationIEEE 1588 Master/Slave – Phase and Frequency SynchronizationIEEE 1588 Boundary Clocks

Ethernet Synchronization Messaging Channel (ESMC)Ethernet Microwave TestsMicrowave QoS SupportSynchronous Ethernet over Microwave

Demonstration NetworkAcknowledgementsAcronyms

References

carrier ethernet world congress 2011 public multi-vendor … · 2011. 10. 7. · 4 carrier ethernet...

Documents