Sharing Cloud Networks

Lucian Popa, Gautam Kumar, Mosharaf ChowdhuryArvind Krishnamurthy, Sylvia Ratnasamy, Ion Stoica

UC Berkeley

State of the Cloud

Network???

2

Guess the Share

3

B2

A1

B1

A2

A3

Link L

Alice : Bob = ? : ?

3 : 1 1 : 1 2 : 1 3 : 2TCP Per flow Per Source Per Destination

Per-VMProportional

ChallengesNetwork share of a virtual machine (VM) V depends on

»Collocated VMs,»Placement of destination VMs, and »Cross-traffic on each link used by V

Network differs from CPU or RAM»Distributed resource»Usage attribution (source, destination, or both?)

Traditional link sharing concepts needs rethinking

4

Requirements

5

High Utilization

Min Bandwidth Guarantee

Aggregate Proportionali

ty

Network shares

proportional to the

number of VMs

Introduce performan

ce predictabili

ty

Do not leave bandwidth unused if there is demand

Requirement 1:Guaranteed Minimum B/WProvides a minimum b/w guarantee for each VM

Captures the desire of tenants to get performance isolation for their applications

6

A B X

BminA

BminB

BminX

…

All VMs

Requirement 2:Aggregate ProportionalityShares network resources across tenants in proportion to the number of their VMs

Captures payment-proportionality»Similar to other resources like CPU, RAM etc.

Desirable properties»Strategy-proofness: Allocations cannot be

gamed»Symmetry: Reversing directions of flows does

not change allocation

7

Design Space

8

High Utilization

Min Bandwidth Guarantee

Aggregate Proportionali

ty

Strategy-Proofness

Symmetry

Requirement 3:High Utilization

Provides incentives such that throughput is only constrained by the network capacity

»Not by the inefficiency of the allocation or by disincentivizing users to send traffic

Desirable properties»Work Conservation: Full utilization of

bottleneck links» Independence: Independent allocation of one

VM’s traffic across independent paths

9

Tradeoff 1

Design Space

10

High Utilization

Min Bandwidth Guarantee

Aggregate Proportionali

ty

Work Conservatio

n

Strategy-Proofness

Independence

Symmetry

Tradeoff 1: Min B/W vs. Proportionality

11

Share of Tenant A can decrease arbitrarily!

B2 A1 B1

A2

Link L with Capacity C

C/2

C/2

B3

C/3

2C/3

Tradeoff 1 Tradeoff 2

Design Space

12

High Utilization

Min Bandwidth Guarantee

Aggregate Proportionali

ty

Work Conservatio

n

Strategy-Proofness

Independence

Symmetry

Tradeoff 2:Proportionality vs. Utilization

13

A1

A3

B1

B3

A2

A4

B2

B4

Link Lwith

Capacity C

C/4

C/4

C/4

C/4

To maintain proportionality, equal amount of traffic must be moved from A1-A2 to A1-A3 =>

Underutilization of A1-A3

Per-link Proportionality

Restrict to congested links only

Share of a tenant on a congested link is proportional to the number of its VMs sending traffic on that link

14

Per Endpoint Sharing (PES)

Five identical VMs (with unit weights) sharing a Link L

15

B2

A1

B1

A2

A3

Link L

Per Endpoint Sharing (PES)

Resulting weights of the three flows:

16

Link L

3/23/22

To generalize, weight of a flow A-B on link L is . .

NA = 2

B2

A1

B1

A2

A3

Per Endpoint Sharing (PES)

Symmetric

Proportional»sum of weights of flows of a tenant on a link L =

sum of weights of its VMs communicating on that link

Work Conserving

Independent

Strategy-proof on congested links17

WAWA-B = NA NB

WB+ = WB-A

Generalized PES

18

A B

α > β if L is more important to A than to B (e.g., A’s access link)

W

AWA-B = WB-

A= α NAN

B

WB+ β

L

Scale weight of A by α Scale weight of B

by β

One-Sided PES (OSPES)

19

A B

α = 1, β = 0 if A is closer to L

α = 0, β = 1 if B is closer to L

W

AWA-B = WB-

A= α NAN

B

WB+ β

L

Scale weight of A by α Scale weight of B

by β

Per Source closer to sourcePer Destination closer to

destination

Highest B/W Guarantee

*In the Hose Model

Comparison

20

Per Flow

Per Sourc

e

Static Reservatio

n

Link PES

OSPES

Link Proportionality Symmetry ✓ ✗ ✓ ✓ ✓

Strategy-Proofness

✗ ✓ ✓ ✓ ✓

Utilization Independence ✓ ✓ ✓ ✓ ✓

Work Conservation

✓ ✓ ✗ ✓ ✓

B/W Guarantee

Full-bisection B/W Network

21

Tenant 1 has one-to-one communication pattern

Tenant 2 has all-to-all communication pattern

OSPES

MapReduce Workload

22W1:W2:W3:W4:W5 = 1:2:3:4:5

OSPES

Summary

Sharing cloud networks is all about making tradeoffs

»Min b/w guarantee VS Proportionality»Proportionality VS Utilization

Desired solution is not obvious»Depends on several conflicting requirements and

properties » Influenced by the end goal

23