Improving end-to-end performance of TCP/IP

via satelliteGorry Fairhurst

University of Aberdeen

IET Satcom Security and Efficiency 26 November 2014 | IET Birmingham: Austin Court, UK

Satellite Internet

ClientTerminal

Gateway

Ground Network/Content Server

Goals: Present the state of the art with respect to web and Internet technologiesRelate this to the present and future performance of satellite systems

Getting a web page via satellite

System round trip delay(satellite, crypto, modem, etc)

directly impacts web page load time

TCP Setup

GET request

Initial data

More data

etc.Page/object sizedirectly impacts

web page load time

Getting a page with multiple objects

Time

Parallel connections reduce page load time

Time

A web page has multiple objects, each need to be received.

But, this can still be painfully slow with satellite delay

Satellite PEPs

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ipsec ipsec

TCP TCP

TLSHTPP

TLSHTPP

No transport or application acceleration

TCP

HTPP

A-PEP

TCP

HTPP T-PEP

A-PEP

TCP

HTPP T-PEP

TCP

HTPP

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

T-PEPT-PEP

With a good PEP satellite Internet performance is often acceptable

T-PEP

A-PEPT-PEP

No PEP

The Internet has Changed

HTTP/1.1 specified in RFC 2616 (June 1999)

Internet changes in last 10 years:

▶ Web pages from ~10s objects -> many 10s of objects, longer downloads

▶ Sharding and caching common to improve performance

▶ Increase in dynamic content and apps that use HTTP

▶ Support for multiple platforms (e.g. mobile devices, desktop PC, TV, etc)

Over same time, TCP transport protocols changed little:

▶ Compare RFC2581(Apr1999) with RFC5681(Sept2009)

Warning: HTTP/2.0 is coming soon!Coordinated effort to replace HTTP/1.1 by:

▶ Internet Engineering Task Force (IETF) & Worldwide Web Consortium (W3C)

▶ Reduce volume of data

▶ Reduce number of Round Trip Times

▶ Fix security model

Features:

▶ Multiplexing request and response messages on a single connection

▶ Binary messages (compressed) and encrypted.

▶ Mandatory server authentication

▶ Smarter server with bidirectional communication (e.g. Server Push)

HTTP/2.0 Multiplexing

Only one setup delay to each server

▶ Once setup, objects are multiplexed on same connection

▶ Reduced “Head of Line” blocking

▶ Reduces incentives for “Domain Sharding”

▶ Adds flexible prioritisation mechanisms

BUT! Loss of a packet has much more impact on performance!

Time

HTTP/2.0 How much difference?

243 small flag icons (811 bytes/flag on average)

► HTTP/1.1 consumes one RTT per flag per connection

► HTTP/2.0 uses one connection (in some case two)

Satellite Performance for “flags” III

Multiplexing over a single connection is key: The flag webpage test

Multiple HTTP transactions can be interleaved over one TCP connection Implemented using Session and Message IDs Mitigates “Head of Line” blocking Mitigates “Domain Sharding” Allows flexible prioritisation mechanisms

Page load time (sec)

© 2

010

-Sat

NEx

-All

right

s re

serv

ed

12th January 2014 – ESA ESTECSatNEx III – COO 3 Task 3

The benefits of multiplexing are illustrated by experiments with Flags.html Flags.html consists of 243 small flag icons (811

bytes/flag on average) HTTP/1.1 consumes one RTT per flag per

connection HTTP/2.0 uses one connection (in some case

two)

Number of TCP connections

III

Multiplexing over a single connection is key: The flag webpage test

Multiple HTTP transactions can be interleaved over one TCP connection Implemented using Session and Message IDs Mitigates “Head of Line” blocking Mitigates “Domain Sharding” Allows flexible prioritisation mechanisms

Page load time (sec)

© 2

010

-Sat

NEx

-All

right

s re

serv

ed

12th January 2014 – ESA ESTECSatNEx III – COO 3 Task 3

The benefits of multiplexing are illustrated by experiments with Flags.html Flags.html consists of 243 small flag icons (811

bytes/flag on average) HTTP/1.1 consumes one RTT per flag per

connection HTTP/2.0 uses one connection (in some case

two)

Number of TCP connections

Page Load Time (secs)

Number of TCP sessions

LAN LAN Satellite Satellite +250mS +VPN

LAN LAN Satellite Satellite +250mS +VPN

PEP +RR

noPEP

???

Satellite Web PerformanceIII

PLT comparisons between HTTP/1.1 and SDPY in Haylas network (UoS/UoA testbed)

© 2

010

-Sat

NEx

-All

right

s re

serv

ed

12th January 2014 – ESA ESTECSatNEx III – COO 3 Task 3

SDPY reduces latency across several sizes of a webpage TCP accelerator interacts with SDPY server SDPY improves performance when PEPs are enabled (multiplexing)

Kernel 3.8 TCP modifications (large IW) do not significantly impact IW could be already large when connection starts after TLS exchange

SDPY improves satellite performance by reducing latency for many sizes of a webpage

▶ Compressed header/contents and streamlined protocol reduces RTTs per page

▶ SDPY performance improved when using T-PEPs

T-PEP

Satellite Web Performance with IPsecIII

PLT comparisons between HTTP/1.1 and SDPY in Hylas using VPN

© 2

010

-Sat

NEx

-All

right

s re

serv

ed

12th January 2014 – ESA ESTECSatNEx III – COO 3 Task 3

When TCP PEPs are disabled (VPN hiding TCP headers), using SDPY still shows improvements However performance without T-PEPs are generally degraded

Default IW=10 in Linux Kernel 3.8, Default IW=3 in Linux Kernel 2.6IW does not change performance

SDPY improves satellite performance with T-PEPs disabled (VPN hides TCP headers),▶ HTTP/2.0 resembles some key functions in A-PEPs (compression, pipeling/multiplexing, push) ▶ Overall gain depends on web page characteristics

Fewer SPDY TCP connections ▶ Eases workload of NAPT▶ causes traffic to be more vulnerable to loss

So, HTTP/2.0 works over satellite?

Mostly performance is as good as an A-PEP

Actual performance depends a lot on the server policies

Server Push may in future be used to send web resources without explicit request

▶ Servers can notify real-time applications of events

▶ Can push web resources correlated to previous requests

▶ Assumes knowledge about network and user characteristics

▶ Some mobile operators fear a rise in undesirable data

Latest work may provide an opportunity to implement a satellite proxy as an A-PEP?

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

Sender changes to TCP

Larger IW – Experimental RFC 6928 (Apr 2013)▶ Enhances web browsing allowing larger TCP window at start

TCP Fast Open (TFO) – TCPM work in progress▶ Allows data on SYN

New-CWV – TCPM work in progress▶ Better congestion control for rate-limited applications

Sender pacing▶ Reduces impact of application bursts on network

Tail Loss Probe & RTO Restart▶ Reduces the negative impact of packets drops at the end of a traffic burst

Optimistic conclusion: enhancements to standard TCP can accommodate some fundamental performance issues posed by satellite networks

Ah, but what happens at a PEP?

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

ip4ip6!!!!

Link

TCP

Applications

TLSHTTP

A Taller, Thinner Hourglass• HTTP (+TLS) is a universal

session layer • Driven by endpoints

(browsers as front-end) as well as the network (HTTP-/proxy-only connectivity)

• HTTP implies TCP, which is not always what we want. • Transport stagnates, or

innovation happens beyond the stack.

T-PEPT-PEP

Updated server

Updated client

satellite system without updatesstops new feature being used

As we enter a time where TCP and HTTP change:

PEPs need to follow transport changes

There is also new work on opportunistic TCP encryption (TCPinc)

▶ If this becomes common: PEPs need to reflect the new security model

Other considerations...

The evolving protocol stack will lead to more “bursty” network traffic

Rapid increase in traffic other than web (including “HAS/DASH”, Web video)

Renewed interest in using diffserv QoS within the Internet

▶ Satellite QoS - prioritising streams/flows and requesting radio resources

▶ Managing graceful degradation of the service under load/threat

ip4ip6!!!

Link

HTTPTLSTCP

Applications

UDP

new transport(udp35)

An Initial Proposal• UDP gives us a partial defense

against middleboxes, provides port multiplexing, and works from userspace.

• Building recommendations / “mix-ins” for transport atop UDP will make this work better. • Congestion control in particular

is hard to do well. • Provide hooks for policy decision

(e.g. enterprise firewall traversal) • Allow evolution beyond and

coexistence with “Internet over HTTP”.

There is much more to come....Some see TCP and UDP as the only “usable” transport protocols

▶ NATs, Firewalls, proxies are a major obstacle to other transports

▶ TCP and UDP are used because other transports could not be deployed

▶ In many apps they only use HTTP/TCP

▶ AND updating middlebox TCP is hard

New proposals to use UDP as a substrate for new transports (IPSE)

▶ QUIC (Quick UDP Internet Connections) HTTP/UDP

▶ UDP 3.5 (a new family of transport protocols)

▶ These likely will prevent T-PEP

Pessimistic conclusion: new UDP protocols replace TCP with an end-to-end security model, improving terrestrial performance, but degrading satellite performance

ConclusionsBe ready!

After many years, things are finally changing!!

▶ HTTP/2.0 expected to be widely implemented and replace HTTP/1.1 in a few years

▶ Satellite people need to understand the new protocol

Transports are also evolving now (There is still some time to influence design!)

▶ “ossification” of technologies needs to be avoided to enable the new protocols

▶ It is imperative to identify issues and find solutions before these become obstacles

to performance of satellite network services.

The changes can improve performance using existing T-PEP

▶ Actual performance depends on many things (including server policy)

▶ Performance without a T-PEP is impacted by satellite packet loss

▶ A-PEP functionality may not be needed ? (but server policy hard to know)

▶ Role of intermediaries in HTTP/2.0 is still to be clarified and may help make an A-PEP

References to new standards

HTTP:

M. Belshe, Twist, R. Peon, M. Thomson, A. Melnikov, Hypertext Transfer Protocol version 2 draft-ietf-httpbis-http2-15 Work-in-progress, October 2014

PEPs:

J. Border, M. Kojo, J. Griner, G. Montenegro, Z. Shelby, “Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations”, RFC 3135, Informational, June 2001.

TurboPage with Active Compression, Hughes (R), http://www.hughes.com/technologies/satellite-systems/hughes-technology/turbopage-with-activecompression, Sept 2012.

UDP Transport:

J. Roskind, “QUIC (Quick UDP Internet Connections), Multiplexed Stream Transport over UDP”, Google technical report, [online], Apr 2012.

IP Stack Evolution Program, Internet Architecture Board, Nov 2014.

References to new standardsChanges to TCP

J. Chu, N. Dukkipati, Y. Cheng, M. Mathis, “Increasing TCP’s Initial Window”, RFC 6928, Experimental, April 2013.

Y. Cheng, Jerry Chu, S. Radhakrishnan, A. Jain, “TCP Fast Open”, IETF Work-in-Progress draft-ietf-tcpm-fastopen-15.txt Experimental, October, 2014

J. Roskind, “QUIC (Quick UDP Internet Connections), Multiplexed Stream Transport over UDP”, Google technical report, [online], Apr 2012.

G. Fairhurst, A. Sathiseelan, R. Secchi, “Updating TCP to Support Rate-Limited Traffic”, Internet draft draft-ietf-tcpm-newcwv-07.txt, Sept 2014.

N. Dukkipati, N. Cardwell, Y. Cheng, M. Mathis, “Tail Loss Probe (TLP): An Algorithm for Fast Recovery of Tail Losses”, Internet draft draft-dukkipati-tcpm-tcp-loss-probe-01.txt, Feb 2013.

P. Hurtig, A. Brunstrom, A. Petlund, and Micheal Welzl, “TCP and SCTP RTO Restart”, Internet draft draft-ietf-tcpm-rtorestart-04, Oct 2014.

J. Iyengar, S. Cheshire, and J. Greassley, “Minion – Service Model and Conceptual API”, Internet draft draft-iyengar-minion-concept-02, Oct 2013, draft-iyengar-minion-protocol-02.

The Flags page

Abstract

This talk explores how changes in Internet technology could impact the design and operation of a secure SATCOM service. SATCOM services are increasingly IP-based, utilising COTS networking equipment, but often using a form of Performance Enhancing Proxy (PEP) to both optimise network performance and to promote efficient interaction with the dynamic resource management of satellite system layers.

For SATCOM links, a PEP must also address the challenge of supporting IP encryption, the need for multiple levels of QoS, and graceful service degradation when capacity is limited. The talk reviews the demands and benefit expected from PEP mechanisms against changes in Internet traffic patterns, highlighting important changes being introduced by the emergent HTTP2 web protocols.

ACKNOWLEDGMENTS

Work on web performance via Satellite has been co-funded by ESA and the University of Aberdeen in the framework of the ESA Support to the SatNEx III Network of Experts, CoO3 “State-of-the-art Web Technologies and Protocols”, ESA Contract 23089/10/NL/CLP.

Work on understanding the root causes of Internet Latency was funded by the European Community under its Seventh Framework Programme through the Reducing Internet Transport Latency (RITE) project (ICT- 317700).

Work on understanding deployment opportunities for rural satellite broadband was supported by the award made by the RCUK Digital Economy programme to the dot.rural Digital Economy Hub at University of Aberdeen; award reference: EP/G066051/1.