can we emulate local circuit switching in cloud storage?

.

Setting the Mood

• "It's time to get rid of TCP/UDP protocols in DCs"

• DCs are closed worlds, new technologies are welcome

• with bulk transfers (BigData, ...), the business value of TCP/UDPalternative is high

• circuits are an alternative to packets

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 2/24...

2/24

.

Ethernet is the Best

.Ethernet.....

.

... is the cheapest and most available technology with e2esupport

• Fiber Channel (FC), SATA, etc. require expensive hardware, lowcompatibility, no e2e support

• FCoE = Ethernet, same problems, expensive hardware, no e2e support

• network virtualization is best fit for Ethernet

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 3/24...

3/24

.

Ethernet is the Worst

.Ethernet.....

.... is the worst technology in terms of throughput• CSMA/CD is the biggest throughput limitation

◦ not in modern switches, but still major problem in wireless

• apart from CSMA/CD, contention cannot be easily resolved

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 4/24...

4/24

.

Ethernet Contention

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 5/24...

5/24

.

Ethernet and Contention

• whaterver you do, Ethernet L2 domains cannot avoid contention

Switch Switch

Qualitatively Identical

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 6/24...

6/24

.

Some Contention Control• you can turn off your NIC, no TX, but some RX

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 7/24...

7/24

.

Contention in Practice (2 flows)

20 24 28 32 36 40Transfer time in contention (s)

20

24

28

32

36

40Tr

ansf

er ti

me

by e

xclu

sive

circ

uits

(s)

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 8/24...

8/24

.

Switches

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 9/24...

9/24

.

Switches : Basic Facts

• cut-through versus store-and-forward• cut-through is 10 15x better

• Cisco has advanced cut-through : +bytes versus routing decision tradeoff

• store-and-forward is subjected to QoS classes◦ L3 DSCP versus L2 CoS, AF, EF, BE, SBE models

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 10/24...

10/24

.

Switches : Modeling

C: Cut Through

Check, etc. Q: Queue

D: Drop QoS classes

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 11/24...

11/24

.

Switches : Modeling (2)

.Model Reading..

.

Shorter paths have lower e2e delayand higher throughput

C: Cut Through

Check, etc. Q: Queue

D: Drop QoS classes

• Ta means throughput of path a

• TC > TSF

• TSF > TSQF

• TSQF > TSQD

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 12/24...

12/24

.

Proposal

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 13/24...

13/24

.

Proposal : Circuits

.Circuits..

.

... are emulations which allow for exclusive access to L2 domain byindividual parties

• circuits-over-packets emulation

• cut-through mode for each circuit is guaranteed

• highest possible throughput

• NOTE: will work with cheepest switches

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 14/24...

14/24

.

Basic Math

• circuits

R = C

• packets

R = f(wCkV

)• C capacity, R throughput/rate, wweight, k parallel count, V bulk in bytes,d e2e delay, pL loss rate

• packets: w = V case is theupper bound for throughput

• packets: in reality, upper bound isnever achieved

• circuits: R = C is achievable locally(DC intranet)

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 15/24...

15/24

.

Trivial Optimization

• special case: traffic engineering + scheduling• time shifts are optimized, bulk is known in advance

• implementation: GA, works well in small (DC-level) networks

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 16/24...

16/24

.

Non-Trivial Optimization

Bulk size per transmission

Goodput

2 potential distributions in practice

Response curve(s)

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 17/24...

17/24

.

Implementation

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 18/24...

18/24

.

Implementation : 2 cases• left: book-then-send, right: separate control layer

SWITCH

NOC

Storage Node A

Storage Node B

Step 1: Book

session

Step 2: Transfer bulk

SWITCH

Storage Node A

Storage Node B

SWITCH

Bookingsegment

BulkSegment

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 19/24...

19/24

.

Impl.: Centralized Case

SWITCH

NOC

Storage Node A

Storage Node B

Step 1: Book

session

Step 2: Transfer bulk

• same network for booking andcircuits

• inefficient but still valid/practical

• legacy-compatible,partial implementation, etc.

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 20/24...

20/24

.

Impl.: Distributed Case

SWITCH

Storage Node A

Storage Node B

SWITCH

Bookingsegment

BulkSegment

• book on one network, send on another

• legacy-incompatible• contention-sensing possible →fully distributed models

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 21/24...

21/24

.

Implementation : Details

• working prototype of legacy-compatible system is ready

• using for backup, transfers of VM images, etc.

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 22/24...

22/24

.

Future FollowUps

• Goal 1: non-legacy, sensing implementation• Goal 2: optimizations in DC-size 6-10 hop networks

• Goal 3: SDN package with PICO8 switches

• Goal 4: Advanced/Curve Tradeoff Optimization models

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 23/24...

23/24

.

That’s all, thank you ...

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...

24/24

.

[01] myself (2014)High Availability Cloud Storage...NS研

[02] Cisco (2014)LAN Switching and Wireless, CCNA Exploration Companion GuideCisco Press

[03] Cisco (2014)Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency....Cisco Press

[04] NetOptics (2014)Cut-Through Ethernet Switching: A Versatile Resource for Low Latency...White Paper

[05] Cisco (2006)QoS: DSCP Classification GuidelinesRFC4594

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...

24/24

.

[06] Cisco (2010)A Differentiated Services Code Point (DSCP)...RFC5865

[07] open source (current)PICA8 Project for Low Latency Virtual Networkinghttp://www.pica8.com/

M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...

24/24