1© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

John Berg – Lead Engineer September 16, 2014

Comparing IPv6 and IPv4 Performance

2© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• CableLabs has been conducting IPv6 interops since 2009– Observed subtle but persistent IPv6 performance benefit

• Collected real performance data from one of our member operators

• Conducted lab testing to measure IPv4/IPv6 performance– Native IPv6– IPv4 with one layer of NAT– IPv4 with two layers of NAT

Background

2

3© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• Service Provider produced >1.5 million IPv4/IPv6 data records– Analysis performed jointly by CableLabs and MSO

• Metrics collected on data records:– Average Round Trip Time (AVGRTT) for a variety of popular IPv4 and IPv6 web sites– Statistics Include: Minimum, Maximum, Mean, Median, Standard Deviation, etc.

• Measurement agents configured on backbone POPs in ten hubs– Randomized latency checks executed each hour– RTT interval = completion of TCP handshake between client and server

How Service Provider Data Was Collected

3

TCP SYN

SYN + ACK

TCP ACK

4© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Where the Data Was Collected

4

New York

Washington, DC

Charlotte

Atlanta

Houston

Dallas/Ft. Worth

Los Angeles

San Jose

Chicago

Seattle

5© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Is IPv6 Really Faster?

5

These visits to apple.com make it appear that is so…..

6© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Is IPv6 Really Faster?

6

RTT for wikipedia.org is still measurably faster for IPv6 on average…..

7© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Is IPv6 Really Faster?

7

But on Facebook RTT for IPv6 and IPv4 are nearly identical…..

8© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Is IPv6 Really Faster?

8

Again, nearly identical results for Netflix…..

9© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Is IPv6 Really Faster?

9

And this time IPv4 is significantly faster, by nearly 25 ms…..

10© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Is IPv6 Really Faster?

10

IPv4 wins again. What are we to make of these results?

11© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

IPv4 vs IPv6 Aggregate View

11

12© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

Lab Testing

12

13© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• Conducted matched IPv4-IPv6 ping tests over the course of an IPv6 interop– First sets: 2 NATS enabled– Second sets: 1 NAT enabled

• Identify effect of NAT without IPv4-IPv6 path differences

• Did NOT measure effects of traffic engineering or differentiated peering

Lab Testing Methodology

13

14© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• Could NAT be responsible for slower IPv4 performance?

• IPv6 performed measurably better than IPv4– Lower mean, median, standard deviation, and

minimum– Statistically significant differences

Results – One Layer of NAT

14

Common Home Network Scenario

15© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• What happened?

• One NAT showed differences, two NATs show nearly identical results– 81% chance that the means are really the same*– IPv4 in both cases still shows higher variability– IPv6 performed better half the time across 6 test

runs

• Could NAT performance vary per device?

Results – Two Layers of NAT

15

Possible CGN Effect

*Based on Students t-test analysis

16© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• How do we explain the disparity in test results?– Analysis shows a subtle IPv6 performance advantage – However, there are more questions than answers….

• Some possible explanations:– Does NAT performance vary across some IPv4 platforms?– Is hop count a factor for IPv4 vs. IPv6?– How do network effects, such as tunnels, traffic engineering and congestion factor in?– What about Content Provider IPv6 transition strategies?

• Better tools and methodologies for gathering end-to-end performance metrics are needed

• Bottom line – further research is needed

Interpreting The Results Data

16

17© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.

• CableLabs is developing performance measurement tools and methodologies as part of our IP Performance Evaluation & Reporting (CLIPPER) project

• Project will standardize testing methodologies for MSOs– Aligns with IETF IPPM & LMAP and BBF WT-143 working groups– Introduce common test platform– Gather data on jitter, delay, packet loss, throughput and DNS response time for both

IPv4 and IPv6– Better correlation of speed tests between home router and wireless devices

• Recommendation: Systematic performance metric collection should consider differences between IPv6 and IPv4

CableLabs Initiative

17

© Cable Television Laboratories, Inc. 2014.  Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.