![Page 1: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/1.jpg)
“Real Life” System Testing of Networking
EquipmentBill Eklow, Cisco Systems Inc.
Josh Goldstein, Ixia
![Page 2: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/2.jpg)
The Pervasiveness of the Internet• Home access
– High speed connections– Home banking/stocks– Travel– Shopping– Google– IM
• Business– B2B– Web meetings/VOD– Application sharing– VoIP
![Page 3: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/3.jpg)
The New, Critical Nature of Data
• 911 VoIP call
• Nasdaq Stock Transactions
• On-line banking
![Page 4: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/4.jpg)
BranchOffice
LeasedLine
Main Office
BranchOffice
ExtranetPartner
AccessNetwork
Customers
Service ProviderNetwork
EdgeRouter
Core Router
CPERouter
LAN Router
The Function of Routers
![Page 5: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/5.jpg)
Common Router Architecture
1. Routing protocol and data traffic is received via the physical ports (i.e. Ethernet, ATM)
PortPort
PortPort
SwitchingFabric
Typical RouterArchitecture
ControlPlane
Applications
1
2
3
4
1
2. Routing traffic is sent to the control plane and stored in the Routing Information Base (RIB)
3. Based on routing algorithms, the best route is populated in the Forwarding Information Base (FIB)
4. Applications are router specific – QoS, LSPs, multicast, filtering, load balancing
• Hardware vs Software implementation• Efficiency of routing algorithms• Switch fabric packet performance
![Page 6: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/6.jpg)
Plethora of Routing Protocols
MP-BGP (RFC 3107)
LDP
IS-ISv6
RSVP-TEMPLS Signaling
BGP+
OSPFv3
IPv6
IPv4
Category
RIPng
BGP
IS-IS
OSPF
RIP
Protocol
Layer 2 VPNMPLS VPN
BGP Layer 3 VPN (RFC2547bis)
Inter-AS VPN
VPLS
PIM-SMv6
MLDv1
MLDv2
PIM-SM
IGMPv3
IGMPv2
IGMPv1Multicast
Category Protocol
![Page 7: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/7.jpg)
4 Elements of a Router
![Page 8: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/8.jpg)
Key Metrics for Router Performance
• Throughput/latency/consistency
• Data integrity
• Routing capabilities/protocols
• Services (Security, QoS)
![Page 9: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/9.jpg)
Challenging Areas In Router Testing
• Keeping up with new technologies and developments
• Scaling the test environment
• Emulating realistic traffic scenarios
• Simplifying the test process
![Page 10: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/10.jpg)
Some State of the Art Test Areas
• Test requirements induced by real-time IP traffic– Videos should arrive smoothly, not choppy– Voice services need to work all the time, with quality
comparable to public telephone network
• New applications are based on router services– High availability, routing convergence– Multicast transport, IPv6 support
![Page 11: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/11.jpg)
Trend: Testing Routed Services
• Verify end-to-end VPN mapping over a routed core backbone network
• Both Layer 2 (ATM, Ethernet, Frame Relay) and Layer 3 VPNs increasingly becoming integrated into routers
• Complexity: Many different hierarchies of protocols, plus different IP address realms in a single test
![Page 12: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/12.jpg)
Trend: Testing Special Devices
• Routers are no longer just IP forwarding boxes
• Firewalls, content and general application switching, soft (telephony) switching and other functions become more and more integrated in routers
• Application layer functions within the router need to be tested with tailored emulators
![Page 13: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/13.jpg)
Single vs. Multiple Device Tests
• Single device configurationtests the routing engineand the switching fabric
• Multiple device configuration adds ports and protocols to stress the device, and to do convergence testing
TestPort 1
TestPort 2Device
Under Test
SystemUnder Test
TestPort 1
Test Port 3
TestPort 2
![Page 14: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/14.jpg)
Data Plane Testing Objectives
• Validate the router forwards traffic to the correct interface
• Measure data plane performance– Latency– Throughput– Data Integrity– Jitter– Sequence of packets
• Test advanced features such as non-stop forwarding
![Page 15: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/15.jpg)
Data Plane Testing Setup
Emulated Routers Send Traffic to Advertised Routes
Measure Throughput
Measure Latency
![Page 16: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/16.jpg)
Data Plane Test – Sequence Checking
Real-time StatisticsSmall Errors
3
BigErrors
Reverse Errors
Total Errors
1 2
Transmit Sequence
1 2 3 4 5 6 7 8 9 10
1 3 2 4 5 8 6 7 9 10
Reverse Error Big Error Small Error
Error Threshold = 2
6
Track sequence validity for up to 128,000 flowsPerform sequence checking while getting first/last latency values per flow
![Page 17: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/17.jpg)
Data Plane Test – Data IntegrityInto Router:
TTL = 64Checksum = 0x829F
Out of Router:
TTL = 63Checksum = 839F
![Page 18: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/18.jpg)
Data Plane Test – Data Integrity
Problem: What if a router bug causes a packet payload change?
• Since the checksum is recalculated, the packet will look valid, but the user’s data is corrupt.
Solution: Test equipment adds second checksum, only calculated on packet payload
• Now, corrupt user data can be identified.
![Page 19: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/19.jpg)
Control Plane Testing Objectives
• Validate the router correctly populates its RIB and FIB
• Find the maximum number of routes the router can handle
• Find the maximum number of routers the router can talk to
• Measure response time to network topology changes
![Page 20: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/20.jpg)
Route Capacity Test Setup
NetworkA
I know how to reach Network A.
I know how to reach Network B.
Please send this traffic to Network XYZ.
RouterUnderTest
I know how to reach Network XYZ.
NetworkB
NetworkXYZ
?
![Page 21: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/21.jpg)
Routing Convergence Test Setup
NetworkA
I know how to reach Network A.My cost to reach Network A is 10
I know how to reach Network A.
My cost to reach Network A is 50
Please send this traffic to Network A.
RouterUnderTest
![Page 22: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/22.jpg)
Routing Convergence Test Setup
NetworkA
I know how to reach Network A.
My cost to reach Network A is 50
Please send this traffic to Network A.
RouterUnderTest
I know how to reach Network A.My cost to reach Network A is 75
![Page 23: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/23.jpg)
Test Equipment Hardware Architecture
1. High port density2. Layer 2-3 wire-speed data
plane testing features3. High scalability and
performance for routing protocol emulations
4. Supports Layer 4-7 statefultransactions; “Real World Traffic”
5. Enables porting of 3rd party applications to Ixia platform
6. System scalability to several hundred ports
1
3 45
CPU per port
2 Dedicated FPGA logic per port
![Page 24: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/24.jpg)
Advanced Router Testing – IPSEC
IxChariot:Layer 4 Traffic
IPSEC tunnels
• Test router services such as IPSEC VPNs• Performance, scalability, real world traffic
![Page 25: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/25.jpg)
Routing System Test• Connect multiple devices and benchmark
performance as a system or network• Routing Protocol Performance & Conformance• Validate Wire-Speed Traffic, QoS, Sequence
Checking, Throughput, Latency, Packet Loss• Use Real World Traffic to emulate deployed
network
• Connect multiple devices and benchmark performance as a system or network
• Routing Protocol Performance & Conformance• Validate Wire-Speed Traffic, QoS, Sequence
Checking, Throughput, Latency, Packet Loss• Use Real World Traffic to emulate deployed
network
![Page 26: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/26.jpg)
©2004 Ixia. All rights reserved.
IxChariot – Customer Market Acceptance
Market Facts
IxChariot and its predecessor, Chariot have been successfully used by thousands of enterprises, network service providers and equipment manufacturers for more than 6 years:
Enterprise Customers:
Carriers/MSOs:
Network Equipment Manufacturers:
Silicon Chip Developers:
![Page 27: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/27.jpg)
©2004 Ixia. All rights reserved.
IxChariot – How it works
IxChariot reproduces this conversation in rhythm using the same:
Transport layer protocol
Packet sizes and quantities
Packet headers
IxChariot can reproduce scalable amounts of enterprise application transactions –maintaining the same rhythm.
IxChariot does not need to talk to real application servers.
IxChariot does not deviate from its scripts.
![Page 28: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/28.jpg)
©2004 Ixia. All rights reserved.
Application Testing – Stateful Transport Layer Sessions
Emulate traffic patterns of multiple applications
Use full TCP stack
Measure “goodput” available to the application layer
![Page 29: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/29.jpg)
©2004 Ixia. All rights reserved.
IxChariot – Principal Application AreasTest the performance of a device to handle mixed mode, multi-protocol application traffic.
Determine the ability of WAN links to cope with additional
traffic introduced by new applications.
Test the ability of a system to handle network congestion caused by application traffic. Measure the impact of traffic prioritization.
![Page 30: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/30.jpg)
©2004 Ixia. All rights reserved.
IxChariot Hardware Platforms – ALM1000T8
PowerPC 750FX per port
Up to 1GB of RAM per port
1. Eight port 10/100/1000 Ethernet Load Module
2. RJ-45 Copper interfaces
3. Layer 4-7 capability
Use whenOnly Layer 4-7 testing is neededPerformance needs are medium to high ALM1000T8 – IxChariot,
WebLOAD, IxLoad, IxVPN
![Page 31: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/31.jpg)
©2004 Ixia. All rights reserved.
IxChariot Hardware Platforms – PCs, etc.
Combine Ixia Hardware with other platforms whenExisting infrastructure is entrenched (experience, ROI, relationships)Test plans rely on specific devices (e.g., WLAN PDA testing)Remote labs cannot cost justify Ixia hardware
![Page 32: “Real Life” System Testing of Networking Equipmentbtw.tttc-events.org/material/BTW04/BTW2004-Presentations/... · MP-BGP (RFC 3107) LDP IS-ISv6 MPLS RSVP-TE Signaling BGP+ OSPFv3](https://reader030.vdocuments.us/reader030/viewer/2022020215/5b2fc8377f8b9adc6e8d9ea4/html5/thumbnails/32.jpg)
Conclusions
• Testing a modern router is a complex endeavor
• Test equipment technology has matured in both hardware and software
• Careful test planning at multiple product development stages reduces time-to-market and increases quality