Connection Migration: Why & How

Hari BalakrishnanNetworks and Mobile Systems Group

MIT Lab for Computer Science

http://nms.lcs.mit.edu/

Joint work with Alex Snoeren & Dave Andersen

Anatomy of a connection

• Connection defined by IPA:PortA IPB:PortB

• An IP address does not identify a host; it only identifies a network interface

• Is this a good definition of a connection?

Some socket onbreeze.lcs.mit.edu

www.cnn.com’sHTTP service (port)

18.31.0.83:5678 207.25.71.23:80

Problem #1: Host mobility

• Cerf’s comment from DoD Internet paper:“If a host were to move, its network (and host) addresses would change and this would affect the connection identifiers used by the TCP. This is rather like a problem called "dynamic reconnection" which has plagued network designers since the inception of the ARPANET project in 1968.”

• Two options today for connections: Terminate and retry Somehow preserve IP address and continue

• “Horizontal” mobility isn’t quite enough...

Vertical mobility: Seamless inter-provider movement

In-building & In-room

Campus-Area Packet Radio

Metro-Area

Regional-Area/”wireless cable”

Problem #2: Unreliable components

• What happens to a bound connection on failure or unresponsiveness?

cnn.com

• Individual components rather unreliable

• Replicate for improved reliability and availability

Possible solutions

• 1. Force constant IP address for end-point Mobile IP Layer-N switches with “Virtual IPs”

• 2. Make names routable All packets identify destination by name, which

serves as routing identifier Intentional naming (late binding), TRIAD

• 3. In-band migration Don’t confound end-point and routing identifiers!

Home AgentIntercepts

pkts

Address constancy: Mobile IP

Foreign Agent (FA)

Mobile Host

Correspondent Hosts

Temporary address dtmp

changes with mobility

D

“Tunnel” to FA

D

dFA

“Detunnel” to Don addr dtmp

D

D

Why Mobile IP isn’t right

• Requires additional network support and infrastructure (HA, FA, authentication,…)

• Triangle routing even for “local” interactions

• Many types of mobile applications Connections that don’t care for seamlessness Connection initiators Both initiators and responders

• Ingress filtering reverse tunneling too!

• Vertical mobility can’t be properly handled

• Applications can’t be made aware of mobility

Address constancy: Layer-N switching

Client

But we want: Global distribution

With work can solve: Local distribution

Name-based routing (example)

[service = camera][building = NE43

[room = 510]]

[service = camera][building = NE43

[room = 510]]

Intentional name

Late binding: integrate resolution and message routingLate binding: integrate resolution and message routing

image

Lookup

Intentional name resolvers

form an overlay network

What should a connection be?

• Between communicating applications, not network interfaces

• Should be possible for an application to easily change network interface of connection: While preserving good unicast routes Securely

• Should not require a priori knowledge of valid network interfaces

• Dynamism should not affect semantics or correctness, nor worsen reliability

If done right, can solve both problems at once!

Migrate overview

Name server

Migrating Hostfoo.bar.edu

Location Query(DNS Lookup)

Connection Initiation

Location Update(Dynamic DNS Update)

Connection Migration

xxx.xxx.xxx.xxxyyy.yyy.yyy.yyy

FixedHost

Problems

• Consistency of name mapping

• Correctness Handling packet losses around time of movement What if someone else gets your old address?

• Security Connection hijacking Denial-of-service protection

• Semantics How to maintain semantics of connection sequence

across different nodes?

Dynamic name mappings

• Zero TTL on A-records for migratable names

• Use Dynamic DNS (RFC 2136, 2137) for Internet names

• Potential problems: Race condition between name update and movement

• Simply retry! This isn’t a new failure mode What about old BIND implementations?

• Pray that they’ll vanish off the face of the earth What about extra DNS load?

• What load? Ask Akamai!

Migrating a connection

• Initiate migration from new network address Identify previous connection with token, on SYN Secure token to protect against hijacking Requires some state machine changes to guarantee

correctness

• Preserves service model to application

• Handles “middle boxes” Works with most NATs, PEPs, stateful firewalls

• Requires changes to transport protocol Kernel TCP, SCTP, RTP (linked library)

TCP ConnectionMigration

1. Initial SYN

2. SYN/ACK

3. ACK (with data)

4. Normal data transfer

5. Migrate SYN

6. Migrate SYN/ACK

7. ACK (with data)

TCP ConnectionMigration

1. Initial SYN

2. SYN/ACK

3. ACK (with data)

4. Normal data transfer

5. Migrate SYN

6. Migrate SYN/ACK

7. ACK (with data)

TCP ConnectionMigration

1. Initial SYN

2. SYN/ACK

3. ACK (with data)

4. Normal data transfer

5. Migrate SYN

6. Migrate SYN/ACK

7. ACK (with data)

Two correctness issues

• SYN uses 1 byte of sequence space; what should SYN ACK value be set to? Needed to correctly handle lost segments

• What if someone else gets your previous address? Peer TCP will reset connection

Correctness: SYN ACK corresponds to data

1. Initial SYN

2. SYN/ACK

3. ACK (with data)

4. Normal data transfer

5. Migrate SYN

6. Migrate SYN/ACK

7. ACK (with data)

Modified TCP State Machine

MIGRATE_WAIT2MSL timeout

recv

: SY

N (

mig

rate

T, R

)se

nd: S

YN

, AC

K

recv:

RST

appl:

migrate

send:

SYN (migr

ate T, R

)recv: SYN (migrate T, R)

send: SYN, ACK

• 2 new transitions between existing states

- and -• 1 new state

handles potential race condition due to rapid readdressing

Securing the Migration

• Problem: Increased vulnerability to hijacking Ingress filtering (RFC 2827) doesn’t help Attacker only needs token and sequence space

• Solution: Keep the token secret Negotiate it using Diffie-Hellman exchange (Elliptic-Curve

DH) Use sequence numbers to prevent replay Complete crypto exchange in SYN handshake

• Result: Connections are as secure as standard TCP Use IPsec or SSH for real security

Semantics of multi-machine migration

• Sequence spaces across different machines may not have same application-layer semantics

7801-9000ACK 9000

One solution: Soft-state synchronzation

• Technique for static content (e.g., file)

• Information about mapping between filename and TCP initial sequence periodically disseminated

7801-9000ACK 9000

HealthMonitor

foo.mp3 ISN = I Token = T caddr = Ca

cport = Cp

preload-tcb

Implementation

• Use application-specific stream mapper to map between sequence space and app (e.g., HTTP range requests)

HTTP GET parser/creator

HTTP header parser/stripper

Data relay

Responsehandler

BackendHTTPserver

HTTP rangerequest

ResponseTo client

• Stream mapper involved in initial connection processing and in re-establishment

Clientrequest

Experiment #1: Mobility

FixedBasestation

FixedServer

100Mbps Ethernet

MobileLocation 1

19.2KbpsModem

MobileLocation 2

19.2KbpsModem

…then moves to a new location

Mobile client initiates a transfer…

Migration Trace

SYN/ACK

BufferedPackets

(old address)

Migrate SYN

A Lossy Trace with SACK

SYN/ACK

Migrate SYN

BufferedPackets

(old address)

ACKw/SACK

Experiment #2: Failover works!

0.000 cl.1065 > sA.8080: . ack 0505 win 31856

----> (Erroneous) sA death pronouncement issued

0.080 sA.8080 > cl.1065: P 0505:1953(1448) ack 1 win 31856

----> Successful connection migration to sB

0.095 sB.1033 > cl.1065: S 0:0(0) win 0 <migrate PRELOAD 1>

0.096 cl.1065 > sB.1033: S 0:0(0) ack 1953 win 32120

0.142 sB.1033 > cl.1065: . ack 1 win 32120

----> Continued data transmission from sA (Reset by client)

0.174 sA.8080 > cl.1065: P 0505:1953(1448) ack 1 win 31856

0.174 cl.1065 > sA.8080: R 1:1(0) win 0

----> Resumed data transmission from sB

0.241 sB.1033 > cl.1065: P 1953:3413(1460) ack 1 win 32120...

Oscillations aren’t a problem

Summary

• Host mobility and service failover are examples of the same fundamental problem: Connections must be between applications

• The Migrate architecture enables connections to be separated from, and move between, IP addresses

• Mobility & service failover are both really end-to-end issues!

• Got code?

Migrate code for Linux 2.2 available from:

http://nms.lcs.mit.edu/software/

Migrate project Web page:

http://nms.lcs.mit.edu/projects/migrate/

Networks and Mobile Systems