can we emulate local circuit switching in cloud storage?
DESCRIPTION
Hardware in clouds in commonly connected by Ethernet LAN. There are alternatives but all are much more expensive by comparison. The problem in Ethernet used to be its CSMA/CD protocol which helps detect and resolve collisions. Even though modern switches are 100\% collision-free, high-rate multiparty chatter over the shared medium creates congestion and reduced overall utilization efficiency. One way to drastically improve efficiency is to revert to the old technology of circuit switching. This paper looks into possibilities of implementing a distributed scheduler which would emulate a circuit switching environment for multiparty communications. The immediate practical application in mind is access to local shared storage in clouds, or specifically, data centers.TRANSCRIPT
.
Setting the Mood
• "It's time to get rid of TCP/UDP protocols in DCs"
• DCs are closed worlds, new technologies are welcome
• with bulk transfers (BigData, ...), the business value of TCP/UDPalternative is high
• circuits are an alternative to packets
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 2/24...
2/24
.
Ethernet is the Best
.Ethernet.....
.
... is the cheapest and most available technology with e2esupport
• Fiber Channel (FC), SATA, etc. require expensive hardware, lowcompatibility, no e2e support
• FCoE = Ethernet, same problems, expensive hardware, no e2e support
• network virtualization is best fit for Ethernet
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 3/24...
3/24
.
Ethernet is the Worst
.Ethernet.....
.... is the worst technology in terms of throughput• CSMA/CD is the biggest throughput limitation
◦ not in modern switches, but still major problem in wireless
• apart from CSMA/CD, contention cannot be easily resolved
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 4/24...
4/24
.
Ethernet Contention
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 5/24...
5/24
.
Ethernet and Contention
• whaterver you do, Ethernet L2 domains cannot avoid contention
Switch Switch
Qualitatively Identical
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 6/24...
6/24
.
Some Contention Control• you can turn off your NIC, no TX, but some RX
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 7/24...
7/24
.
Contention in Practice (2 flows)
20 24 28 32 36 40Transfer time in contention (s)
20
24
28
32
36
40Tr
ansf
er ti
me
by e
xclu
sive
circ
uits
(s)
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 8/24...
8/24
.
Switches
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 9/24...
9/24
.
Switches : Basic Facts
• cut-through versus store-and-forward• cut-through is 10 15x better
• Cisco has advanced cut-through : +bytes versus routing decision tradeoff
• store-and-forward is subjected to QoS classes◦ L3 DSCP versus L2 CoS, AF, EF, BE, SBE models
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 10/24...
10/24
.
Switches : Modeling
C: Cut Through
Check, etc. Q: Queue
D: Drop QoS classes
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 11/24...
11/24
.
Switches : Modeling (2)
.Model Reading..
.
Shorter paths have lower e2e delayand higher throughput
C: Cut Through
Check, etc. Q: Queue
D: Drop QoS classes
• Ta means throughput of path a
• TC > TSF
• TSF > TSQF
• TSQF > TSQD
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 12/24...
12/24
.
Proposal
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 13/24...
13/24
.
Proposal : Circuits
.Circuits..
.
... are emulations which allow for exclusive access to L2 domain byindividual parties
• circuits-over-packets emulation
• cut-through mode for each circuit is guaranteed
• highest possible throughput
• NOTE: will work with cheepest switches
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 14/24...
14/24
.
Basic Math
• circuits
R = C
• packets
R = f(wCkV
)• C capacity, R throughput/rate, wweight, k parallel count, V bulk in bytes,d e2e delay, pL loss rate
• packets: w = V case is theupper bound for throughput
• packets: in reality, upper bound isnever achieved
• circuits: R = C is achievable locally(DC intranet)
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 15/24...
15/24
.
Trivial Optimization
• special case: traffic engineering + scheduling• time shifts are optimized, bulk is known in advance
• implementation: GA, works well in small (DC-level) networks
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 16/24...
16/24
.
Non-Trivial Optimization
Bulk size per transmission
Goodput
2 potential distributions in practice
Response curve(s)
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 17/24...
17/24
.
Implementation
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 18/24...
18/24
.
Implementation : 2 cases• left: book-then-send, right: separate control layer
SWITCH
NOC
Storage Node A
Storage Node B
Step 1: Book
session
Step 2: Transfer bulk
SWITCH
Storage Node A
Storage Node B
SWITCH
Bookingsegment
BulkSegment
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 19/24...
19/24
.
Impl.: Centralized Case
SWITCH
NOC
Storage Node A
Storage Node B
Step 1: Book
session
Step 2: Transfer bulk
• same network for booking andcircuits
• inefficient but still valid/practical
• legacy-compatible,partial implementation, etc.
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 20/24...
20/24
.
Impl.: Distributed Case
SWITCH
Storage Node A
Storage Node B
SWITCH
Bookingsegment
BulkSegment
• book on one network, send on another
• legacy-incompatible• contention-sensing possible →fully distributed models
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 21/24...
21/24
.
Implementation : Details
• working prototype of legacy-compatible system is ready
• using for backup, transfers of VM images, etc.
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 22/24...
22/24
.
Future FollowUps
• Goal 1: non-legacy, sensing implementation• Goal 2: optimizations in DC-size 6-10 hop networks
• Goal 3: SDN package with PICO8 switches
• Goal 4: Advanced/Curve Tradeoff Optimization models
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 23/24...
23/24
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...
24/24
.
[01] myself (2014)High Availability Cloud Storage...NS研
[02] Cisco (2014)LAN Switching and Wireless, CCNA Exploration Companion GuideCisco Press
[03] Cisco (2014)Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency....Cisco Press
[04] NetOptics (2014)Cut-Through Ethernet Switching: A Versatile Resource for Low Latency...White Paper
[05] Cisco (2006)QoS: DSCP Classification GuidelinesRFC4594
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...
24/24
.
[06] Cisco (2010)A Differentiated Services Code Point (DSCP)...RFC5865
[07] open source (current)PICA8 Project for Low Latency Virtual Networkinghttp://www.pica8.com/
M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24...
24/24