centaur: realizing the full potential of centralized wlans through a hybrid data path vivek...

CENTAUR: Realizing the Full Potential of Centralized WLANs Through a Hybrid

Data Path

Vivek Shrivastava* , Shravan Rayanchu, Suman BanerjeeUniversity of Wisconsin-Madison

Nabeel Ahmed, Srinivasan KeshavUniversity of Waterloo, Ontario

Konstantina PapagiannakiIntel Labs, Pittsburgh

Arunesh MishraGoogle Inc.

* viveks@cs.wisc.edu

2

Centralization of Enterprise WLANs

2Vivek Shrivastava

Wireless controller

Access Point

Clients

Internet

3

Centralization of Enterprise WLANs

3Vivek Shrivastava

Power controlChannel assignment

Common control plane functions

4

Centralization of Enterprise WLANs

4Vivek Shrivastava

What about data plane functions?

Power controlChannel assignmentData scheduling ?

5 5Vivek Shrivastava

Can centralized scheduling help?

6 6Vivek Shrivastava

Can centralized scheduling help?

Hidden terminals

7 7Vivek Shrivastava

Hidden terminals

Can centralized scheduling help?

8 8Vivek Shrivastava

Hidden terminals

Can centralized scheduling help?

9 9Vivek Shrivastava

1. Carrier sense

Hidden terminals

Can centralized scheduling help?

10 10Vivek Shrivastava

2. Channel free, transmit

Hidden terminals

Can centralized scheduling help?

11 11Vivek Shrivastava

• Collision!• Backoffs• Low throughputs

3. Collision !

Can centralized scheduling help?

12

How bad is it ?

12Vivek Shrivastava

Experiments on production building-wide WLANs •W1: 5 floors

9 APs, 45 clients

• W2: 1 floor 21 APs, 51 clients

10% links suffer more than 70% throughput

reduction

13 13Vivek Shrivastava

A lost opportunity ?

Suppose infrastructure can gather conflict data

14 14Vivek Shrivastava

A lost opportunity ?

And when packets arrive …

15 15Vivek Shrivastava

A lost opportunity ?

… realize interference will happen …

16 16Vivek Shrivastava

A lost opportunity ?

1. Transmit first packet

17 17Vivek Shrivastava

A lost opportunity ?

1. Transmit first packet2. Transmit second packet

with delay

18 18Vivek Shrivastava

Use an in-band scheduler

Simple FIFO schedule with interference

avoidance

Scheduling functionality

19

What about exposed terminals ?

19Vivek Shrivastava

• In our experiments, about 41%of link pairs suffer from exposed terminal interference

• Disabling carrier sense to solve the problem can be dangerous for uplink, non-enterprise traffic

• We will show how centralization can help even here

20

Challenge

•Centralization has obvious overheads

•How to make this feasible and useful under

• Real applications

• Common large-scale wireless environments

• Presence of uplink and non-enterprise traffic

• Requirements of no client modifications

20Vivek Shrivastava

21

Contributions

1.Design CENTAUR, a hybrid (partly centralized, partly distributed) scheduling approach

• Resolves hidden and exposed terminals

• Requires no client modifications

• Requires no carrier sense disabling

2.Evaluate CENTAUR on two WLAN testbeds with real-world traffic traces

• 1.48x greater throughput for bulk data traffic

• 1.38x reduction in web-transaction times

21Vivek Shrivastava

22

Outline

• A naïve attempt at centralized scheduling (DET)

•Our hybrid centralized scheduler (CENTAUR)

•Evaluation

•Related Work

•Summary

22Vivek Shrivastava

23

Outline

• A naïve attempt at centralized scheduling (DET)

•Our hybrid centralized scheduler (CENTAUR)

•Evaluation

•Related Work

•Summary

23Vivek Shrivastava

24 24Vivek Shrivastava

DET: A Simple Deterministic Scheduler

25 25Vivek Shrivastava

DET: A Simple Deterministic Scheduler

Input: Conflict graph

26 26Vivek Shrivastava

DET: A Simple Deterministic Scheduler

Goal: Schedule each incoming downlink packet

Input: Conflict graph

27 27Vivek Shrivastava

DET: A Simple Deterministic Scheduler

1 2 3 4Transmission

slots

Goal: Schedule each incoming downlink packet

Input: Conflict graph

28 28Vivek Shrivastava

DET: A Simple Deterministic Scheduler

1 2 3 4Transmission

slots

1. New packet arrives

29 29Vivek Shrivastava

DET: A Simple Deterministic Scheduler

1 2 3 4Transmission

slots

1. New packet arrives2. Find the earliest conflict

free slot

30 30Vivek Shrivastava

DET: A Simple Deterministic Scheduler

1 2 3 4Transmission

slots

1. New packet arrives2. Find the earliest conflict

free slot

31 31Vivek Shrivastava

DET: A Simple Deterministic Scheduler

1 2 3 4Transmission

slots

1. New packet arrives2. Find the earliest conflict

free slot3. Schedule the packet in that

slot

5

Performance of DET

4xNo gains for exposed terminals; Non-

conflicting links perform worse under load

32Vivek Shrivastava

33

Outline

• A naïve attempt at centralized scheduling (DET)

•Our hybrid centralized scheduler (CENTAUR)

•Evaluation

•Related Work

•Summary

33Vivek Shrivastava

34

Overview of CENTAUR

• Incorporate basic DET scheduler

• Tackle DET’s shortcomings:

• Amortize scheduling overhead

• Improve performance for exposed links

• Avoid degrading normal links

• Coexist with non-enterprise and uplink traffic

34Vivek Shrivastava

35

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

35Vivek Shrivastava

36

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

36Vivek Shrivastava

37

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

37Vivek Shrivastava

Wired ack

38

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

38Vivek Shrivastava

39

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

39Vivek Shrivastava

40

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

40Vivek Shrivastava

41

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

41Vivek Shrivastava

42

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

42Vivek Shrivastava

Wired ack

43

(1) Avoid Scheduling Overheads

• Problem: Per-packet scheduling performs poorly under high network loads

• Solution:

• Schedule packets in batches (or epochs)

43Vivek Shrivastava

44

(2) Improve Exposed Terminals

44Vivek Shrivastava

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

45 45Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

46 46Vivek Shrivastava

Variable wired delay

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

47 47Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

48 48Vivek Shrivastava

Carrier sense, deferral

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

49 49Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

50 50Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

51 51Vivek Shrivastava

Schedule packets in batches

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

52 52Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

53 53Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

54 54Vivek Shrivastava

First packets can be out of sync

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

55 55Vivek Shrivastava

Waiting packets synchronized by carrier sense !

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

56 56Vivek Shrivastava

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

57 57Vivek Shrivastava

After first packet, both APs transmit

simultaneously.

(2) Improve Exposed Terminals

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets, fix backoff periods

58

(3) Avoid Degrading Normal Links

58Vivek Shrivastava

A B

C D

Hidden terminals

Non-hidden/Non-exposed

59

(3) Avoid Degrading Normal Links

59Vivek Shrivastava

A B

C D

Scheduler

60

(3) Avoid Degrading Normal Links

60Vivek Shrivastava

A B

C D

Scheduler

61

(3) Avoid Degrading Normal Links

61Vivek Shrivastava

A B

C D

Scheduler

62

(3) Avoid Degrading Normal Links

62Vivek Shrivastava

A B

C D

Scheduler

Hybrid scheduling

63

Outline

• A naïve attempt at centralized scheduling (DET)

•Our hybrid centralized scheduler (CENTAUR)

•Evaluation

•Related Work

•Summary

63Vivek Shrivastava

64

Large-Scale Experiments

•Platform:

• Two WLAN testbeds in separate buildings

•Topology

• Representative: 7 APs, 12 clients

•Traffic and metrics

• UDP, TCP, VoIP, HTTP (real traces)

• Throughput, delay, MOS, web transaction delay

64Vivek Shrivastava

65

Result 1: UDP/TCP Performance

65Vivek Shrivastava

66

Result 1: UDP/TCP Performance

66Vivek Shrivastava

DCF

67

Result 1: UDP/TCP Performance

67Vivek Shrivastava

DCF

68

Result 1: UDP/TCP Performance

68Vivek Shrivastava

DCF Per Packet

69

Result 1: UDP/TCP Performance

69Vivek Shrivastava

DCF Per Packet

70

Result 1: UDP/TCP Performance

70Vivek Shrivastava

Epoch BasedDCF Per Packet

71

Result 1: UDP/TCP Performance

71Vivek Shrivastava

Epoch BasedDCF Per Packet

Hidden terminal starves some clients

72

Result 1: UDP/TCP Performance

72Vivek Shrivastava

Epoch BasedDCF Per Packet

Better fairness

than DCF

73

Result 1: UDP/TCP Performance

73Vivek Shrivastava

Epoch BasedDCF Per Packet

Exploits exposed

terminals, higher system

throughput

74

Result 1: UDP/TCP Performance

74Vivek Shrivastava

Epoch BasedDCF Per Packet

Avg. delay is smallest for epoch

based scheduling

75

Result 1: UDP/TCP Performance

75Vivek Shrivastava

Epoch BasedDCF Per Packet

90th percentile delays are higher for

epoch based scheduling

76

Result 1: UDP/TCP Performance

76Vivek Shrivastava

Epoch BasedDCF Per Packet

CENTAUR yields up to 60% higher total throughput and 50% lower per-packet delay

compared to DCF

77

More Results in Paper

•Centaur micro-benchmarks: performance for exposed and hidden terminals under centaur

•Uplink traffic: coexistence and persistent gains with different fractions of uplink traffic

•Data rate: robust to changes in data rate and ARF

•Realistic HTTP traces: significant reduction in web transaction delay

•VoIP traffic: better performance (MOS) for voice traffic with small epoch duration

77Vivek Shrivastava

78

Related Work

•Commercial WLAN offerings (Aruba, Meru)

•Theoretical formulations (Vaidya ‘00, Kanodia ‘01)

•Epoch based scheduling (Kompella ‘05, 802.11n/e)

• Interference mitigation (CMAP, SIC, Shuffle)

78Vivek Shrivastava

79

Summary

• Interference a growing problem in enterprises

•Careful design of a centralized data plane provides substantial performance gains

•CENTAUR implements a hybrid data path to improve aggregate performance without client modifications

•CENTAUR does not disable carrier sense and co-exists with non-enterprise and uplink traffic

79Vivek Shrivastava

Vivek Shrivastava 80

Future work

•Even more efficient conflict graph generation

•What if we were allowed client modifications ?

Questions ?

Characterizing System Latencies

System delays are high and variable, leading to inaccuracies for per-packet

scheduling

81

82

Evaluation of Micro-Probing

82Vivek Shrivastava

Topologies 20 node 30 nodeBandwidth

Tests16.2 mins 1hr 11 mins

Micro-Probing

~4 secs ~11 secs

Can be computed in stages with each instance taking ~

2.5ms

83

Result 2: Impact of Uplink Traffic

83Vivek Shrivastava

•Vary proportion of downlink/uplink traffic

•6 different configurations

•80/20 ➔ 40/60 (downlink/uplink)

•Results:

•Downlink: 1.6x ➔ 6.8x gain in throughput

•Uplink: 1x ➔ 1.18x gain in throughput

CENTAUR provides persistent gains for different proportions of uplink and downlink

traffic load

84

• Three topologies

• Hidden Heavy topology ➔ 10 links

• Exposed Heavy topology ➔ 6 links

• Mixed Topology ➔ 12 links

• Results: Up to 50% gain in overall system throughput

• Up to 6x gain for HT; Up to 1.7x gain for ET

Result 3: Impact of Topology

84Vivek Shrivastava

Improvements from using CENTAUR can be seen across many different network

topologies

85

Result 2: Impact of Uplink Traffic

85Vivek Shrivastava

CENTAUR provides persistent gains for different proportions of uplink and downlink

traffic load

86

Result 3: Impact of Topology

86Vivek Shrivastava

Improvements from using CENTAUR can be seen across many different network

topologies

87

CENTAUR Micro-Benchmarks (1)

87Vivek Shrivastava

Exposed Terminals

88

CENTAUR Micro-Benchmarks (II)

88Vivek Shrivastava

Hidden Terminals

89

Result 1: UDP/TCP Performance

89Vivek Shrivastava

Client Index (1-12)

Client Index (1-12)

CENTAUR yields up to 60% higher total throughput and 50% lower per-packet delay

compared to DCF

Other approaches to Hidden/Exposed Terminals

Mechanism

Target Proble

m

Approach

Client Changes Evaluation

CMAP [NSDI ‘07]

ZigZag [Sigcomm ’08] SIC [Mobicom ’08]

Centaur

Exposed

Hidden

ExposedHidde

n

Disable CS

Signal manipulatio

n

Centralized Scheduling

Yes

Yes

No

802.11

GNU Radio

802.11

Adaptive RTS/CTS [VTC ’03]

HiddenExtra

SignalingYes 802.1

1

March,30 2009Prelim 2009

Other approaches to Hidden/Exposed Terminals

Mechanism

Target Proble

m

Approach

Client Changes Evaluation

CMAP [NSDI ‘07]

ZigZag [Sigcomm ’08] SIC [Mobicom ’08]

Centaur

Exposed

Hidden

ExposedHidde

n

Disable CS

Signal manipulatio

n

Centralized Scheduling

Yes

Yes

No

802.11

GNU Radio

802.11

Adaptive RTS/CTS [VTC ’03]

HiddenExtra

SignalingYes 802.1

1

Solve both hidden/expos

ed

March,30 2009Prelim 2009

Other approaches to Hidden/Exposed Terminals

Mechanism

Target Proble

m

Approach

Client Changes Evaluation

CMAP [NSDI ‘07]

ZigZag [Sigcomm ’08] SIC [Mobicom ’08]

Centaur

Exposed

Hidden

ExposedHidde

n

Disable CS

Signal manipulatio

n

Centralized Scheduling

Yes

Yes

No

802.11

GNU Radio

802.11

Adaptive RTS/CTS [VTC ’03]

HiddenExtra

SignalingYes 802.1

1

No client side changes for

Centaur

CENTAUR: Realizing the Full Potential of Centralized WLANs

Through a Hybrid Data Path

• Vivek Shrivastava* , Shravan Rayanchu, Suman Banerjee

• University of Wisconsin-Madison

Nabeel Ahmed, Srinivasan KeshavUniversity of Waterloo, Ontario

Konstantina PapagiannakiIntel Labs, Pittsburgh

Arunesh MishraGoogle Inc.

94

WLANs HP LabsSeoul

National University

Our Testbed

Exposed Terminals

39% 9% 50%

Hidden Terminals

43% 70% 39%

Growth of Interference in Enterprise Wireless LANs

94

Interference an increasing problem according to leading enterprise WLAN

vendor

Vivek ShrivastavaVivek Shrivastava

95

Centralization of Enterprise WLANs

95

Centralized control for better network security and manageability

Vivek Shrivastava

96

Can Centralized Data Plane Scheduling Help?

96Vivek Shrivastava

97

(2) Improve Exposed Terminals

97Vivek Shrivastava

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets to each exposed AP

• Fix back-off periods and use carrier-sensing to align transmissions -- double throughput!

98

Related Work

•Commercial WLAN offerings (Aruba, Meru)

•Research proposals (MiFi, DenseAP, Smarta)

•TXOP in 802.11e/802.11n packet aggregation

• Interference Mitigation (CMAPs, ZigZag, SIC)

98Vivek Shrivastava

99

Types of Interference

Hidden Terminals

Collis

ions

CarrierSense

Exposed Terminals

Focus on downlink conflicts

Vivek Shrivastava 11

100

Can Centralized Data Plane Scheduling Help?

100Vivek Shrivastava

X Y

101

Can Centralized Data Plane Scheduling Help?

101Vivek Shrivastava

X Y

102

Can Centralized Data Plane Scheduling Help?

102Vivek Shrivastava

X Y

103 103Vivek Shrivastava

Can Centralized Data Plane Scheduling Help?Hidden

terminals

104

Quantifying Downlink Hidden Terminals

104Vivek Shrivastava

10% links suffer severe hidden terminal interference

105

Quantifying Downlink Exposed Terminals

105

41% links can obtain double the throughput with CS disabled, indicating

exposed terminal interference

Vivek Shrivastava

106

Result 1: UDP/TCP Performance

106Vivek Shrivastava

107

Result 1: UDP/TCP Performance

107Vivek Shrivastava

108

Result 1: UDP/TCP Performance

108Vivek Shrivastava

109

Centralization of Enterprise WLANs

109Vivek Shrivastava

110

(2) Improve Exposed Terminals

110Vivek Shrivastava

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets to each exposed AP

• Fix back-off periods and use carrier sensing to align transmissions -- double throughput!

111

(2) Improve Exposed Terminals

111Vivek Shrivastava

• Problem: Exposed links can operate in parallel but don’t due to carrier-sensing

• Solution:

• Schedule batch of packets to each exposed AP

• Fix back-off periods and use carrier sensing to align transmissions -- double throughput!

112

(3) Avoid Degrading Normal Links

112Vivek Shrivastava

113

1. Can it solve any interference problems prevalent in enterprise WLANs

• Hidden and exposed terminals

• 88% of links suffer some losses due to co-channel interference (Jigsaw, Sigcomm 2007)

2. If so, how can we implement it efficiently for practical WLAN deployments

• No client modifications, support legacy clients

• Coexistence with non-enterprise, uplink traffic

113Vivek Shrivastava

Can Centralized Data Plane Help?

114

Quantifying Downlink Interference

•Prior Work: Jigsaw [Sigcomm2006] analysis reveals 56% of all interference traffic is downlink in nature.

•Our Work: Two production WLANs

• W1: 5 floors, 9 APs, 45 clients

• W2: 1 floor, 21 APs, 51 clients

•Download ‘bulk’ traffic from the Internet

114Vivek Shrivastava

115

WLANsUW-

MadisonUW-

Ontario

Exposed Terminals

39% 9%

Hidden Terminals

43% 70%

Growth of Interference in Enterprise Wireless LANs

115

Interference an increasing problem according to leading enterprise WLAN

vendor

Vivek ShrivastavaVivek Shrivastava

116

Centralization of Enterprise WLANs

116Vivek Shrivastava

Can centralized data plane be useful for improving performance in

WLANs ?• About 70-80% of enterprise traffic

is downlink in nature

117

(3) Avoid Degrading Normal Links

117Vivek Shrivastava

A B

C D

Scheduler

118

(3) Avoid Degrading Normal Links

118Vivek Shrivastava

A B

C D

Scheduler

119

(3) Avoid Degrading Normal Links

119Vivek Shrivastava

A B

C D

Scheduler

120

(3) Avoid Degrading Normal Links

120Vivek Shrivastava

A B

C D

Scheduler

Hybrid scheduling

121

Outline

•DET - A deterministic scheduler

•CENTAUR - A hybrid centralized scheduler

•Evaluation

•Related Work

•Summary

121Vivek Shrivastava

122

Outline

•DET - A deterministic scheduler

•CENTAUR - A hybrid centralized scheduler

•Evaluation

•Related Work

•Summary

122Vivek Shrivastava

Performance of DET

4xNo gains for exposed terminals; Non-

conflicting links perform worse under load

123Vivek Shrivastava

124

Outline

•DET - A deterministic scheduler

•CENTAUR - A hybrid centralized scheduler

•Evaluation

•Related Work

•Summary

124Vivek Shrivastava

125

DET: A Simple Deterministic Scheduler

• Key Idea: Perform per-packet scheduling

• Given

• Conflict graph G = (L,E)

• Scheduled packets {P1, P2, ..., Pr}; Unscheduled Pr+1

• Objective

• Minimize t(Pr+1)

• Constraint: No two packets on interfering links are scheduled together

• Schedule downlink packets only

125Vivek Shrivastava

126

1. What are the problems it can solve ?

• Hidden and exposed terminals

2. Can we implement it efficiently ?

• No client modifications

• Coexistence with non-enterprise & uplink traffic

• Don’t disable carrier sensing

126Vivek Shrivastava

Can centralized scheduling help?

127 127Vivek Shrivastava

Can centralized scheduling help?

128 128Vivek Shrivastava

DET: A Simple Deterministic Scheduler

Packets arriving

• Schedule one packet at a time• Consider packets in order of arrival

1 2 3 4Transmission

slots

129

What about exposed terminals ?

129

41% links can obtain double the throughput with CS disabled, indicating exposed terminal

interference

Vivek Shrivastava

130 130Vivek Shrivastava

Use an in-band scheduler

Simple FIFO schedule with interference

avoidance

scheduler