.
Setting the Mood
• "It's time to get rid of TCP/UDP protocols in DCs"
• DCs are closed worlds, new technologies are welcome
• with bulk transfers (BigData, ...), the business value of TCP/UDPalternative is high
• circuits are an alternative to packets
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 2/24...
2/24
.
Ethernet is the Best
.Ethernet.....
.
... is the cheapest and most available technology with e2esupport
• Fiber Channel (FC), SATA, etc. require expensive hardware, lowcompatibility, no e2e support
• FCoE = Ethernet, same problems, expensive hardware, no e2e support
• network virtualization is best fit for Ethernet
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 3/24...
3/24
.
Ethernet is the Worst
.Ethernet.....
.... is the worst technology in terms of throughput• CSMA/CD is the biggest throughput limitation
◦ not in modern switches, but still major problem in wireless
• apart from CSMA/CD, contention cannot be easily resolved
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 4/24...
4/24
.
Ethernet Contention
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 5/24...
5/24
.
Ethernet and Contention
• whaterver you do, Ethernet L2 domains cannot avoid contention
Switch Switch
Qualitatively Identical
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 6/24...
6/24
.
Some Contention Control• you can turn off your NIC, no TX, but some RX
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 7/24...
7/24
.
Contention in Practice (2 flows)
20 24 28 32 36 40Transfer time in contention (s)
20
24
28
32
36
40Tr
ansf
er ti
me
by e
xclu
sive
circ
uits
(s)
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 8/24...
8/24
.
Switches
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 9/24...
9/24
.
Switches : Basic Facts
• cut-through versus store-and-forward• cut-through is 10 15x better
• Cisco has advanced cut-through : +bytes versus routing decision tradeoff
• store-and-forward is subjected to QoS classes◦ L3 DSCP versus L2 CoS, AF, EF, BE, SBE models
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 10/24...
10/24
.
Switches : Modeling
C: Cut Through
Check, etc. Q: Queue
D: Drop QoS classes
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 11/24...
11/24
.
Switches : Modeling (2)
.Model Reading..
.
Shorter paths have lower e2e delayand higher throughput
C: Cut Through
Check, etc. Q: Queue
D: Drop QoS classes
• Ta means throughput of path a
• TC > TSF
• TSF > TSQF
• TSQF > TSQD
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 12/24...
12/24
.
Proposal
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 13/24...
13/24
.
Proposal : Circuits
.Circuits..
.
... are emulations which allow for exclusive access to L2 domain byindividual parties
• circuits-over-packets emulation
• cut-through mode for each circuit is guaranteed
• highest possible throughput
• NOTE: will work with cheepest switches
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 14/24...
14/24
.
Basic Math
• circuits
R = C
• packets
R = f(wCkV
)• C capacity, R throughput/rate, wweight, k parallel count, V bulk in bytes,d e2e delay, pL loss rate
• packets: w = V case is theupper bound for throughput
• packets: in reality, upper bound isnever achieved
• circuits: R = C is achievable locally(DC intranet)
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 15/24...
15/24
.
Trivial Optimization
• special case: traffic engineering + scheduling• time shifts are optimized, bulk is known in advance
• implementation: GA, works well in small (DC-level) networks
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 16/24...
16/24
.
Non-Trivial Optimization
Bulk size per transmission
Goodput
2 potential distributions in practice
Response curve(s)
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 17/24...
17/24
.
Implementation
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 18/24...
18/24
.
Implementation : 2 cases• left: book-then-send, right: separate control layer
SWITCH
NOC
Storage Node A
Storage Node B
Step 1: Book
session
Step 2: Transfer bulk
SWITCH
Storage Node A
Storage Node B
SWITCH
Bookingsegment
BulkSegment
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 19/24...
19/24
.
Impl.: Centralized Case
SWITCH
NOC
Storage Node A
Storage Node B
Step 1: Book
session
Step 2: Transfer bulk
• same network for booking andcircuits
• inefficient but still valid/practical
• legacy-compatible,partial implementation, etc.
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 20/24...
20/24
.
Impl.: Distributed Case
SWITCH
Storage Node A
Storage Node B
SWITCH
Bookingsegment
BulkSegment
• book on one network, send on another
• legacy-incompatible• contention-sensing possible →fully distributed models
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 21/24...
21/24
.
Implementation : Details
• working prototype of legacy-compatible system is ready
• using for backup, transfers of VM images, etc.
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 22/24...
22/24
.
Future FollowUps
• Goal 1: non-legacy, sensing implementation• Goal 2: optimizations in DC-size 6-10 hop networks
• Goal 3: SDN package with PICO8 switches
• Goal 4: Advanced/Curve Tradeoff Optimization models
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 23/24...
23/24
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...
24/24
.
[01] myself (2014)High Availability Cloud Storage...NS研
[02] Cisco (2014)LAN Switching and Wireless, CCNA Exploration Companion GuideCisco Press
[03] Cisco (2014)Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency....Cisco Press
[04] NetOptics (2014)Cut-Through Ethernet Switching: A Versatile Resource for Low Latency...White Paper
[05] Cisco (2006)QoS: DSCP Classification GuidelinesRFC4594
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...
24/24
.
[06] Cisco (2010)A Differentiated Services Code Point (DSCP)...RFC5865
[07] open source (current)PICA8 Project for Low Latency Virtual Networkinghttp://www.pica8.com/
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...
24/24