1 lbe tests between infn & ucl. 2 contents purpose/scenario equipment/topology udp tests –per...
TRANSCRIPT
1
LBE Tests between INFN & UCL
2
Contents
• Purpose/Scenario • Equipment/Topology• UDP TESTS
– Per frame-size 1:1 proportion (2 flows)– Per frame-size 5:5, 7:3, 9:1 proportion– Visual summary of results
• TCP TESTS (ongoing…..)• Conclusions• Future work
3
2 Possible test Scenarios
• Bottleneck(s) in the core network [Activity 2 in Project GN1 (Geant);D9.9].
• Bottleneck(s) in the edge-networks (NRENs) only: This is supported by the idea that a network can deploy
LBE incrementally, in points where congestion is more likely to occur (currently in the NRENs only)
4
Objectives…..
• Assess– BE protection under congestion due to the
LBE attack in terms of:• Throughput (IP and TCP layer)• Packet Loss• Jitter (OWD Standard Deviation)
– OWD Distribution buckets were engineered to achieve a compromise between lost of accuracy (buckets too large) and the network-noise/clocks non-synchronization vulnerability (buckets too small)
• Reordering (mainly for BE)…ongoing
5
Edges Equipment
NRENs + GEANT (“Overprovisioned”…some noise to filter out)
• 1 x 7200 Cisco Router. (1x GigE + 3 x feth + 1 x eth )
• 1 x SmartBits 6000B chassis (with 12 x 10/100 BaseT and 2 x GigE) in each site
• Italian site: 2 Linux boxes dual-processor 2.4.9 (red hat 7.3)
• UK site: 1 Supermicro super server 6002P6, Dual Intel Processor, 1G memory, Intel card
100Mb
2x100Mb 1 Gbps
Policy Applied
Cisco 7200
100Mb
100Mb
6
Mngm_port128.40.4.159
1A6: 128.40.255.33/30
7200
fe1/0:128.40.255.34 /30128.40.255.234/30 secondary
ge0/0:128.40.169.217 /29 SW:6500
mbng1/pc58MBNG2
JANET
GEANT
GARRCabletronSS8600
1Gb
1Gb100Mb
1 Gb
GW:131.154.99.253/24
Cabletron Smart Switch Router2000
mbng1/pc58Sunlab3f
mbng1/pc58Sunlab2f
fe:131.154.99.3/24
fe:131.154.99.2/24
1A1: 131.154.99.72/24
Mngm_port: 131.154.99.73
: Gigabit1A2: 131.154.99.74/24
eth1: 128.40.255.233/30
HUB
GW:128.40.169.222
100Mbps Bottleneck !!!Policy applied 99% BE 1% LBE
7
64 100 744047 550478 193569 26.0157 N/A 110095.6 40515181
128 100 422297 422297 0 0 N/A 84459.4 74324272
256 100 226448 226448 0 0 50 45,290 86,231,398
384 100 185642 154782 30860 16.62 60 30,956 90,640,339
512 100 117481 117481 0 0 N/A 23496.2 92856982
Per frame size 100Mbps 1BEflow Throughput achieved
8
Test Layout1 LBE Flow
1 BE Flow
Frame sizes set :
64,128,256,384,512,1180,1500,1518
Load Set ;
from 10Mbps…to 100Mbps per port
100Mbps
100Mbps
9
• 0 packets dropped both in the BE and LBE output queues of the router.
• 0 packets dropped in the router input interface
• 0 packets ignored
• 8184514-4194010=3990504 frame lost in the network..48.7% of the sent packets
• 35Mbps is the Max. IP throughput for 40Mbps Link!!!! sure not for 100Mbps…..Clearly we can’t go above the associated input rate which is
[35.22Mbps]/[8*(64-18)]=95706 packets/sec!!!
NO DIFFERENTIATION: traffic can’t arrive at the router in order to congest the output diff. serv. queues……..
Traffic is seriously blocked by the network….the maximum IP level throughput for a 64 Bytes frame size flow on a 100Mbps link would be 54.7 Mbps…..and we reach 35Mbps………..
Similar router (we saw the same in bck2bck) along the path reached their Max. Input rate !!!! (95706 packets/sec)
35Mbps
64UDP 64Fr.Size 1:1 BE+LBE
0
5,000,000
10,000,000
15,000,000
20,000,000
25,000,000
30,000,000
35,000,000
40,000,000
offered wire rate load
IP R
x.t
hro
ug
hp
ut
UDP_BE
UDP_LBE
Total
UDP_BE 5,251 10,95 16,42 17,10 17,07 17,08 17,28 17,49 17,45 17,51
UDP_LBE 5,251 10,95 16,42 17,27 17,27 17,30 17,54 17,67 17,58 17,71
Total 10,50 21,90 32,85 34,37 34,35 34,38 34,83 35,16 35,04 35,22
10 20 30 40 50 60 70 80 90 100
UDP 64f.s. 1:1 BE+LBE
0
10
20
30
40
50
60
70
80
wire rate offered traffic (Mbps)
Pa
ck
et
Lo
ss
%
UDP_BE
UDP_LBE
UDP_BE 0 0 0 21.9 37.6 48 54.9 60.1 64.6 68
UDP_LBE 0 0 0 21.2 36.9 47.3 54.2 59.7 64.3 67.7
10 20 30 40 50 60 70 80 90 100
10
1. 0 packets dropped both in the BE and LBE output queues of the router.
2. 0 packets dropped in the router input interface
3. 179884 packets ignored..5,3% of all- rate packets that entered the input interface (higher % for the high rate packets)
4. 4645260 – 3380105 = 1265155 frame lost in the network..27% of the frame sent
5. At least….74.3 Mbps IS the Max. IP throughput possible for a 128f.s. on a 100Mbps link
Traffic is partially blocked by the network….the remaining is then blocked at the input interface where that size provokes a lot of packet ignored…
If the 64B packets had arrived at the router…we would have seen packets both ignored and flushes……he same thing would have happened for 128B packets…
NO DIFFERENTIATION: traffic can’t arrive at the router in order to congest the output diff. serv. queues
74.3Mbps
128UDP 128Fr.Size Throughput 1:1 BE+LBE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
wire rate offered load (Mbps)
Rx.
IP
Th
rou
gh
pu
t
UDP_BE
UDP_LBE
Total
UDP_BE 7,430 14,86 22,29 29,72 37,15 37,02 36,97 37,01 36,99 36,98
UDP_LBE 7,429 14,86 22,29 29,72 37,15 37,34 37,29 37,41 37,41 37,43
Total 14,85 29,72 44,59 59,45 74,31 74,36 74,26 74,42 74,40 74,41
10 20 30 40 50 60 70 80 90 100
11
0 packets dropped both in the BE and LBE output queues of the router.
0 packets dropped in the router input interface
48767 packets ignored….2.6% of the all-rate packets that entered the input interface (higher % for high rate packets)
2490930 - 1813031 = 677899 frame lost in the network..27% of the packets sent
86.2Mbps is the Max. IP throughput possible for a 256f.s. on a 100Mbps link
NO DIFFERENZIATION: traffic can’t arrive at the router in order to congest the output diff. serv. queues
86.2Mbps
Traffic is partially blocked by the network….the remaining is then blocked at the input interface where that size provokes packet ignored…
256UDP 256f.s. 1:1 BE+LBE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
100,000,000
wire rate offered load (Mbps)
Rx.
IP
th
rou
gh
pu
t (b
ps)
UDP_BE
UDP_LBE
Total
UDP_BE 8,617 17,24 25,86 34,49 43,11 43,10 43,06 43,05 43,05 43,06
UDP_LBE 8,617 17,24 25,86 34,49 43,11 43,23 43,22 43,25 43,25 43,28
Total 17,23 34,49 51,73 68,98 86,23 86,34 86,29 86,31 86,31 86,35
10 20 30 40 50 60 70 80 90 100
Less than for 128…..
12
1. 0 packets dropped in the BE queue BUT 232844 packets dropped in the LBE output queue of the router.
2. 0 packets dropped in the router input interface
3. 2374 packets ignored in the input interface…1.6% of the packets that entered the interface (higher % for high rate packets)
4. 1701722 - 1471758 = 229964 frame lost in the network…13.5%
5. 89.6Mbps would be the IP throughput BE should get at 200Mbps offered load according to 384f.s. and to the policy applied (99%)…
PARTIAL DIFFERENZIATION: traffic arrive at the router in order to congest the output diff. serv. Queues but it is still dropped in the network and ignored in the input interface as well…..
UDP 384 f.s.1:1 BE+LBE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
100,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
UDP_BE
UDP_LBE
Total
UDP_BE 9,049 18,11 27,17 36,23 45,29 54,35 60,08 60,06 60,00 60,05
UDP_LBE 9,051 18,11 27,17 36,23 45,29 36,26 30,63 30,68 30,63 30,69
Total 18,10 36,23 54,35 72,47 90,59 90,61 90,72 90,75 90,64 90,75
10 20 30 40 50 60 70 80 90 100
384
Loss %
0
10
20
30
40
50
60
70
wire rate offered load Mbps
% o
f lo
ss
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0 0.01 5.25 17.1 26.4 33.7
UDP_LBE 0 0 0 0 0 33.3 51.7 57.7 62.4 66.1
10 20 30 40 50 60 70 80 90 100
Less than 256
13
1. 0 packets dropped in the BE queue BUT 339579 packets dropped in the LBE output queue of the router.
2. 0 packets dropped in the router input interface
3. 0 packets ignored in the input interface
4. 1292284- 1045468 = 246816 frames lost in the network…19% of the sent frames
5. 91.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…the target throughput is 91.8 – 88.07= 3.1Mbps far…
UDP 512 Th. 1:1 BE+LBE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
100,000,000
wire rate offered load (Mbps)
Rx
IP T
hro
ug
hp
ut
(bp
s)
UDP_BE
UDP_LBE
Total
UDP_BE 9,274 18,57 27,85 37,14 46,42 55,71 64,99 74,28 83,51 88,07
UDP_LBE 9,274 18,57 27,85 37,14 46,40 37,23 27,98 18,69 9,427 4,943
Total 18,54 37,14 55,71 74,28 92,83 92,95 92,97 92,97 92,94 93,01
10 20 30 40 50 60 70 80 90 100
UDP 512 Loss 1:1 BE+LBE
0
10
20
30
40
50
60
70
80
90
100
wire rate offered load (Mbps)
Lo
ss
% UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0.01 0 0 0 0.06 5.15
UDP_LBE 0.12 0 0 0 0.04 33.2 57 74.8 88.7 94.7
10 20 30 40 50 60 70 80 90 100
UDP 512 St.Dev. 1:1 BE+LBE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
wire rate offered load (Mbps)
Sta
ndar
d de
viat
ion
(Mic
rose
c.)
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0 0 0 0 0 0
UDP_LBE 0 0 0 0 0 0 1219 5686 5281 1655
10 20 30 40 50 60 70 80 90 100
512
14
1. 464 packets dropped in the BE queue!!!!...traffic
2. arrives finally
3. 0 packets dropped in the router input interface
4. 0 packets ignored in the input interface
5. 0 frames lost in the network
6. 95.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE gets]….it’s ok!!!
UDP 1180 Th. 1:1 BE+LBE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
wire rate offered load (Mbps)
Rx.
IP th
roug
hput
(bp
s)
UDP_BE
UDP_LBE
Total
UDP_BE 9,654 19,36 29,05 38,73 48,41 58,12 67,78 77,44 87,15 95,92
UDP_LBE 9,658 19,36 29,05 38,73 48,41 38,85 29,21 19,52 9,840 1,169
Total 19,31 38,73 58,10 77,46 96,83 96,98 96,99 96,97 96,99 97,09
10 20 30 40 50 60 70 80 90 100
UDP 1180 Loss 1:1 BE+LBE
0
20
40
60
80
100
120
wire rate offered load (Mbps)
Pac
ket
Lo
ss %
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0 0 0 0.02 0 0.93
UDP_LBE 0 0 0 0 0 33.1 56.9 74.8 88.7 98.8
10 20 30 40 50 60 70 80 90 100
UDP 1180 St.Dev. 1:1 BE+LBE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
10 20 30 40 50 60 70 80 90 100
wire rate offered load (Mbps)
OW
D S
tand
ardr
Dev
iatio
n (M
icro
sec.
)UDP_BE
UDP_LBE
1180
15
1. 357 packets dropped in the BE queue
2. 0 packets dropped in the router input interface
3. 0 packets ignored in the input interface
4. 0 frames lost in the network
5. 96.52Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE get]…..it’s ok!!!
UDP 1500 Th. 1:1 BE+LBE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
UDP_BE
UDP_LBE
UDP_BE 9,717 19,49 29,24 38,99 48,74 58,53 68,24 77,99 87,73 96,65
UDP_LBE 9,712 19,49 29,24 38,99 48,74 39,15 29,43 19,72 9,947 1,221
10 20 30 40 50 60 70 80 90 100
UDP 1500 Loss. 1:1 BE+LBE
0
20
40
60
80
100
120
wire rate offered load (Mbps)
Packet
Lo
ss %
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0 0 0 0 0 0.9
UDP_LBE 0.4 0 0 0 0 33 57 75 89 99
10 20 30 40 50 60 70 80 90 100
UDP 1500f.s. St.Dev. 1:1 BE+LBE
0
5000
10000
15000
20000
25000
10 20 30 40 50 60 70 80 90 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
ev
iati
on
(M
icro
se
c.)
UDP_BE
UDP_LBE
1500
16
1. 351 packets dropped in the BE queue
2. 0 packets dropped in the router input interface
3. 0 packets ignored in the input interface
4. 0 frames lost in the network
5. 97.02Mbps would be the IP throughput BE should get at 200Mbps offered load according to 1518f.s. and to the policy applied (99%) [it also equalises what 100M BE get]….nearly 0.3 Mbps missing!!!
UDP 1518 Loss 1:1 BE+LBE
0
20
40
60
80
100
120
wire rate offered load (Mbps)
Pa
ck
et
Lo
ss
%
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0 0 0 0 0 0.9
UDP_LBE 0 0 0 0 0 33 57 75 89 99
10 20 30 40 50 60 70 80 9010
0
UDP 1518f.s. Th. 1:1 BE+LBE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
UDP_BE
UDP_LBE
UDP_BE 9,71 19,5 29,2 39,0 48,7 58,5 68,2 78,0 87,7 96,6
UDP_LBE 9,71 19,5 29,2 39,0 48,7 39,1 29,4 19,7 9,96 1,22
10 20 30 40 50 60 70 80 90 100
UDP 1518 St.Dev.1:1 BE+LBE
0
5000
10000
15000
20000
25000
10 20 30 40 50 60 70 80 90 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
ev
iati
on
(M
icro
se
c.)
UDP_BE
UDP_LBE
1518
17
Zoom 512Zoom 512f.s. Th. 1:1 BE+LBE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
w ire rate offered load (Mbps)
Rx. I
P Th
roug
hput
(bps
)
UDP_BE
UDP_LBE
UDP_BE 83,5 84,4 85,4 86,2 87,2 88,0 88,1 88,1 88,0 88,8 88,2
UDP_LBE 9,01 8,49 7,51 6,66 5,70 4,94 4,82 4,85 4,87 4,14 4,74
90 91 92 93 94 95 96 97 98 99 100
Zoom 512f.s. Loss 1:1 BE+LBE
0
20
40
60
80
100
120
w ire rate offered load (Mbps)
Pack
et L
oss
%
UDP_BE
UDP_LBE
UDP_BE 0 0.06 0 0.07 0 0.16 1.12 2.11 3.22 3.39 4.93
UDP_LBE 89.2 90 91.2 92.3 93.5 94.4 94.6 94.6 94.6 95.5 94.9
90 91 92 93 94 95 96 97 98 99 100
Zoom 512 Standard Deviation 1:1 BE+LBE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
90 91 92 93 94 95 96 97 98 99 100
w ire rate offered load (Mbps)
OW
D St
anda
rd D
evia
tion
(Mic
rose
c.) UDP_BE
UDP_LBE
1. 91.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalizes what 100M BE get]…nearly 4Mbps missing…
18
Zoom 1180
Zoom 1180f.s. OWDSt.Dev. 1:1 BE+LBE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
90 91 92 93 94 95 96 97 98 99 100
w ire rate offered load (Mbps)
OW
D S
tand
ard
Dev
iatio
n (M
icro
sec.
)UDP_BE
UDP_LBE
Zoom 1180f.s. Th. 1:1 BE+LBE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
w ire rate offered load (Mbps)
Rx.
IP T
hrou
ghpu
t (bp
s)
UDP_BE
UDP_LBE
UDP_BE 83,5 88,1 89,0 90,0 91,0 91,9 92,9 93,9 94,8 95,8 95,8
UDP_LBE 9,39 8,88 7,90 6,93 5,98 5,01 4,06 3,07 2,12 1,17 1,21
90 91 92 93 94 95 96 97 98 99 100
Zoom 1180f.s. Loss 1:1 BE+LBE
0
20
40
60
80
100
120
w ire rate offered load (Mbps)
Rx.
IP T
hrou
ghpu
t (bp
s)
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0 0 0 0 0 0 1
UDP_LBE 89 90 91 92 93 95 96 97 98 99 99
90 91 92 93 94 95 96 97 98 99 100
464 packets dropped in the BE queue!!!!...traffic arrives finally
0 packets dropped in the router input interface
0 packets ignored in the input interface
0 frames lost in the network
95.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE gets]….it’s ok!!!
19
Zoom 1500
Zoom 1500f.s. Loss 1:1 BE+LBE
0
20
40
60
80
100
120
w ire rate offered load (Mbps)
Pa
ck
et
Lo
ss
%
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0.8707
UDP_LBE 95.858 96.67 97.727 98.735 98.748
96 97 98 99 100
Zoom 1500f.s. OWDStandarDeviation 1:1 BE+LBE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
90 91 92 93 94 95 96 97 98 99 100
wire rate offered load (Mbps)
OW
D S
tan
dar
d D
evia
tio
n (
mic
rose
c.)
UDP_BE
UDP_LBE
1. 357 packets dropped in the BE queue
2. 0 packets dropped in the router input interface
3. 0 packets ignored in the input interface
4. 0 frames lost in the network
5. 96.52Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE get]…..it’s ok!!!
Zoom 1500f.s. Th. 1:1 BE+LBE
0
50000000
100000000
150000000
w ire rate offered load (Mbps)
Rx. I
P Th
roug
hput
(bps
)
UDP_BE
UDP_LBE
UDP_BE 89783117 94572941 95549875 96524438 96650112
UDP_LBE 3876912 3148954 2172019 1221168 1221168
96 97 98 99 100
20
ZooZZZ
Zoom 1518
Zoom 1518f.s. Loss 1:1 BE+LBE
0
20
40
60
80
100
120
wire rate offered load (Mbps)
Pa
ck
et
Lo
ss
%
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0 0.8711273
UDP_LBE 95.667889 96.669034 97.903274 98.724832 98.747447
96 97 98 99 100
1. 351 packets dropped in the BE queue
2. 0 packets dropped in the router input interface
3. 0 packets ignored in the input interface
4. 0 frames lost in the network
5. 97.02Mbps would be the IP throughput BE should get at 200Mbps offered load according to 1518f.s. and to the policy applied (99%) [it also equalises what 100M BE get]….nearly 0.3 Mbps missing….
Zoom 1518f.s. Th. 1:1 BE+LBE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
UDP_BE
UDP_LBE
UDP_BE 93,626,400 94,603,200 95,248,800 96,552,000 96,679,200
UDP_LBE 4,056,000 3,151,200 2,004,000 1,231,200 1,221,600
96 97 98 99 100
21
Interleaving(1)Interleaving frame sizes below 512 with frame size 1500
• 64 : NO for both combinations
• 128 : NO for both combinations
• 256 : “YES” (Great improvement!!)…but only if BE is 1500 and LBE is 256…….see next slide…..
• 384 : PERFECT for both combinations
• 512 : PERFECT for both combinations
22
Interleaving(2)UDP BE 1500 LBE 256
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
100,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (M
bp
s)
UDP_BE
UDP_LBE
UDP_BE 38,99 48,74 58,49 68,23 77,97 86,49 87,51
UDP_LBE 34,49 43,11 34,51 25,87 17,29 8,581 8,877
40 50 60 70 80 90 100
UDP BE 1500 + LBE 256
0
10
20
30
40
50
60
70
80
90
100
wire offered rate (Mbps)
pac
ket
Lo
ss %
UDP_BE
UDP_LBE
UDP_BE 0 0 0 0.02 0.03 1.43 10.24
UDP_LBE 0 0.01 33.3 57.13 74.93 88.94 89.71
40 50 60 70 80 90 100
UDP BE 256 + LBE 1500
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
40 50 60 70 80 90 100
wire rate offered load (Mbps)
Rx.
IP
Th
rou
gh
pu
t (M
bp
s)
UDP_BE
UDP_LBE
UDP BE 256 + LBE 1500
0
10
20
30
40
50
60
70
80
90
wire rater offered load (Mbps)
Pa
ck
et
Lo
ss
%
UDP_BE
UDP_LBE
UDP_BE 0 0 1.01 0.02 4.46 15.09 23.51
UDP_LBE 0 0 34.22 57.06 70.37 73.73 76.38
40 50 60 70 80 90 100
23
Tests with disproportion :
10 flows per Smartbits port ( overall 20 flows)
• Case 1:1(again..)
5 BE + 5 LBE flows per port (i.e. X=Y=5)
• Case 7:3
7 LBE + 3 BE flows per port (i.e. X=3;Y=7)
• Case 9:1
9 LBE + 1 BE flows per port (i.e. X=1;Y=2)
X BE + Y LBE Flows
X BE + Y LBE Flows
24
1500f.s. 5:5 (5LBE+5BE per port sharing the BW)
1500f.s. Th. 1:1(5,5) LBE-BE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 73,104,09677,986,397 82,873,44087,724,91592,619,072 96,624,029
LBE 24,563,26119,699,930 14,834,227 9,935,328 5,098,080 1,218,797
75 80 85 90 95 100
1500f.s. Loss 1:1(5,5) LBE-BE
0
20
40
60
80
100
120
wire rate offered load (Mbps)
Pa
ck
et
Lo
ss
%
BE
LBE
BE 0 0 0 0.01 0 0.88
LBE 66.4 74.74 82.1 88.68 94.5 98.75
75 80 85 90 95 100
1500f.s. OWD St.Dev. 1:1(5,5) LBE_BE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
ev
iati
on
(b
ps
)BE
LBE
[5/(5+5)= 0.5] * 200Mbps * [(1500-18/1500+20)=0.975] = 97.5Mbps(*) (*) this cannot be achieved.....99%(97.5)=96.52Mbps can be achieved ……It is Fine!!!!!!!!!!!!!!!!
25
1500f.s.7:3 (7LBE+3BE per port sharing the BW)
1500f.s. Th. 7:3 LBE-BE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 43,862, 46,786, 49,721, 52,640, 55,557, 58,488,
LBE 53,814, 50,900, 47,957, 45,040, 42,138, 39,214,
75 80 85 90 95 100
1500f.s. Loss 7:3 LBE-BE
0
10
20
30
40
50
60
70
80
wire rate offered load (Mbps)
Pa
ck
et
Lo
ss
%
BE
LBE
BE 0 0.02 0 0 0.03 0
LBE 47.42 53.38 58.67 63.33 67.5 71.27
75 80 85 90 95 100
1500 St.Dev. 7:3 LBE-BE
0
5000
10000
15000
20000
25000
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
ev
iati
on
(M
icro
se
c.)
BE
LBE
[3/(3+7)= 0.3] * 200Mbps * [(1500-18/1500+20)=0.975] = 58.5Mbps…it is fine!!!!……..obvious…....there is no loss!!!
A : 200Mbps = 140Mbps LBE + 60Mbps BE offered load
26
1500 9:1 ( 9LBE+1BE per port sharing the BW)
1500f.s. 9:1 LBE-BE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 14,030, 15,597, 16,574, 17,542, 18,523, 19,496,
LBE 79,684, 82,088, 81,132, 80,118, 79,190, 78,206,
75 80 85 90 95 100
1500f.s. Loss 9:1 LBE-BE
0
10
20
30
40
50
60
w ire rate offred load (bps)
Pa
cket
Lo
ss %
BE
LBE
BE 0 0 0 0.03 0 0
LBE 39.44 41.52 45.61 49.27 52.5 55.43
75 80 85 90 95 100
1500f.s. St.Dev. 9:1 LBE-BE
0
2000
4000
6000
8000
10000
12000
14000
16000
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
ev
iati
on
(M
icro
se
co
nd
s)
BE
LBE
[1/(9+1)= 0.1] * 200Mbps * [(1500-18/1500+20)=0.975] = 19.5Mbps…it is fine!!.........obvious…….there is no loss!!!!!
It is even better than the 7:3 case…
200Mbps = 180Mbps LBE + 20Mbps BE offered load
27
1180f.s. 5:5 (5LBE+5BE per port sharing the BW)
1180 Th. 1:1 (5,5) LBE-BE
0
20,000,000
40,000,000
60,000,000
80,000,000
100,000,000
120,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 72,620,3577,441,2582,306,7887,140,7091,937,4495,947,73
LBE 24,377,8319,525,3114,685,829,837,0275,014,2621,169,437
75 80 85 90 95 100
1180f.s. Loss 5:5(1,1) LBE-BE
0
20
40
60
80
100
120
wire rate offered load (mbps)
pa
ck
et
los
s %
BE
LBE
BE 0 0.02 0 0 0.04 0.91
LBE 66.43 74.79 82.16 88.71 94.55 98.79
75 80 85 90 95 100
1180f.s. St.Dev. 5:5(1,1) LBE-BE
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
dar
d d
evia
tio
n (
Mic
rose
c.)
BE
LBE
[5/(5+5)= 0.5] * 200Mbps * [(1180-18/1180+20)=0.968] = 96.8Mbps(*) (*) this cannot be achieved.....99%(96.8)=95.832Mbps can be achieved…. It is fine!!!
28
1180f.s. Th. 7:3 LBE-BE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 43,572,211 46,472,563 49,384,070 52,271,408 55,184,774 58,096,282
LBE 53,394,365 50,523,760 47,606,675 44,674,717 41,798,534 38,909,338
75 80 85 90 95 100
1180f.s Loss 7:3 LBE-BE
0
10
20
30
40
50
60
70
80
wire rate offered load (Mbps)
Pa
ck
et
los
s %
BE
LBE
BE 0 0 0 0.02 0 0
LBE 47.48 53.41 58.69 63.38 67.54 71.3
75 80 85 90 95 100
1180f.s. OWD St.Dev. 7:3 LBE-BE
0
500
1000
1500
2000
2500
3000
3500
4000
4500
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
ev
iati
on
(M
icro
se
c.)
BE
LBE
1180f.s. 7:3 (7LBE+3BE per port sharing the BW)
[3/(3+7)= 0.3] * 200Mbps *[(1180- 18/1180+20)=0.968] = 58.08Mbps…..it is fine!!!!......obvious……...there is no loss!!!
200Mbps = 140Mbps LBE + 60Mbps BE offered load
29
1180f.s. 9:1 (9LBE+1BE per port sharing the BW)1180f.s. Th. 9:1 LBE-BE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 13,934,704 15,490,854 16,457,638 17,428,141 18,387,488 19,365,427
LBE 79,116,397 81,505,469 80,503,360 79,549,590 78,562,355 77,640,192
75 80 85 90 95 100
1180f.s. Loss 9:1 LBE-BE
0
10
20
30
40
50
60
wire rate offered load (Mbps)
Pa
ck
et
Lo
ss
%
BE
LBE
BE 0 0 0.02 0 0.04 0
LBE 39.47 41.54 45.66 49.28 52.55 55.45
75 80 85 90 95 100
1180f.s St.Dev. 9:1 LBE-BE
0
100
200
300
400
500
600
700
800
900
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
dar
d D
evia
tio
n (
Mic
rose
c.)
BE
LBE
[1/(9+1)= 0.1] * 200Mbps * [(1180-18/1180+20)=0.968] = 19.36Mbps……it is fine!!!.......obvious……..there is no loss!!!It is even better than the 7:3 case…
200Mbps = 180Mbps LBE + 20Mbps BE offered load
30
512f.s. 5:5 (5LBE+5BE per port sharing the BW)
512 Th.1:1 (5,5) BE+LBE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
100,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 69,642, 74,281, 78,881, 83,568, 88,138, 89,332,
LBE 23,285, 18,692, 14,015, 9,394,6 4,816,6 3,669,0
75 80 85 90 95 100
512 Loss 1:1 (5,5) BE+LBE
0
20
40
60
80
100
120
wire rate offered load (Mbps)
pac
ket
loss
%
BE
LBE
BE 0 0 0.05 0 0.08 3.79
LBE 66.56 74.84 82.24 88.76 94.54 96.05
75 80 85 90 95 100
512 Standard Deviation 1:1 (5,5) BE+LBE
0
2000
4000
6000
8000
10000
12000
14000
16000
75 80 85 90 95 100
wire rate offered load (Mbps)
Rx.
IP T
hrou
ghpu
t (bp
s)BE
LBE
[5/5+5= 0.5] * 200Mbps * [(512-18/512+20)=0.928] =100Mbps BE * 0.928 = 92.8Mbps(*) this cannot be achieved.....99%(92.8)=91.8Mbps can be achieved……The target throughput is 91.8. – 89.3 = 2.5Mbps far……but this is not a news….see 512 1:1….
31
512 f.s. 7:3 (7LBE+3BE per port sharing the BW)
512 Th. 7:3 LBE-BE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
wire rate offered load (Mbps)
Rx
. IP
Th
rou
gh
pu
t (b
ps
)
BE
LBE
BE 41,775,011 44,569,075 47,334,685 50,117,683 52,780,541 52,635,107
LBE 51,133,347 48,405,677 45,563,398 42,825,453 40,228,198 40,366,518
75 80 85 90 95 100
512 Loss 7:3 LBE-BE
0
10
20
30
40
50
60
70
80
wire rate offered load (Mbps)
pa
ck
et
Lo
ss
%
BE
LBE
BE 0.02 0 0.04 0.05 0.27 5.53
LBE 47.56 53.45 58.76 63.4 67.42 68.95
75 80 85 90 95 100
512 St.Dev. 7:3 LBE-BE
0
100
200
300
400
500
600
700
800
900
1000
75 80 85 90 95 100
w ire rate offered load (Mbps)
OW
D St
anda
rd D
evia
tion
(Mic
rose
c.) BE
LBE
[3/(3+7)= 0.3] * 200Mbps * [(512-18/512+20)=0.928] = 60Mbps BE * 0.928 = 55.68Mbps... The target Throughput is 55.68 – 52.63 = 3.05Mbps (5%loss) far due to the presence of 140Mbps LBE traffic……SEE BLUE ARROW IN SLIDE 13
A : 200Mbps = 140Mbps LBE + 60Mbps BE offered load
A
B
B : 190Mbps = 133Mbps LBE + 57Mbps BE offered load
[3/(3+7)= 0.3] * 190Mbps * [(512-18/512+20)=0.928] = 57Mbps BE * 0.928 = 52.896Mbps... The target Throughput is 52.896-52.780 = 116kBps (0.27%loss) far due to the presence of 140Mbps LBE traffic……INTERPOLATE THE VALUES IN SLIDE 13 FOR 57Mbps BE STANDALONE
32
512 9:1 (9LBE+1BE per port sharing the BW)
512 Th. 9:1 LBE-BE
0
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
60,000,000
70,000,000
80,000,000
90,000,000
wire rate offered load (Mbps)
Rx
. IP
th
rou
gh
pu
t (b
ps)
BE
LBE
BE 13,928,429 14,851,616 15,784,288 16,713,798 17,102,675 17,082,915
LBE 78,998,109 78,106,538 77,130,394 76,246,726 75,894,998 75,939,261
75 80 85 90 95 100
512 Loss 9:1 LBE-BE
0
10
20
30
40
50
60
wire rate offered load (Mbps)
Pa
cke
t L
oss
%
BE
LBE
BE 0 0.03 0 0 3.06 8.01
LBE 36.98 41.58 45.71 49.31 52.2 54.57
75 80 85 90 95 100
512 St.Dev. 9:1 LBE-BE
0
200
400
600
800
1000
1200
1400
75 80 85 90 95 100
wire rate offered load (Mbps)
OW
D S
tan
da
rd D
evi
ati
on
(M
icro
sec
on
ds
)
BE
LBE
[1/(9+1)= 0.1] * 200Mbps * [(512-188/512+18)=0.928] = 20Mbps BE Wire rate * 0.928 =18.56Mbps……The target Throughput is 18.56 – 17.05 = 1.51Mbps (8%loss) far… due to the presence of 180Mbps LBE traffic……SEE RED ARROW IN SLIDE 13
[1/(9+1)= 0.1] * 190Mbps * [(512-188/512+18)=0.928] = 19Mbps BE Wire rate * 0.928 =17.632Mbps……The target Throughput is 17.632 – 17.102 = 1/2Mbps (3%loss) far… due to the presence of 171Mbps LBE traffic….……..INTERPOLATE THE VALUES IN SLIDE 13FOR 19Mbps BE STANDALONE
A
B
A : 200Mbps = 180Mbps LBE + 20Mbps BE offered load
B : 190Mbps = 171Mbps LBE + 19Mbps BE offered load
33
The network noise for 512 f.s. doesn’t affect the results significantly because it is absent in point 1 and not dominant in point 2.
1. Slide 7 and 13 demonstrates that 512 frame size flow can reach the theoretical target throughput when the offered wire rate is 60,57,20,19 and 100 Mbps respectively.
2. The network noise provokes an equal drop probability for both classes…(Geant transparency)……so…if it was dominant we should have seen a smaller drop % for BE when its load was smaller….however, we saw quite the reverse, that is, a higher drop % when BE load was smaller.
34
64 95706pps input rate exceeded
128 NO
256
384 NO-ISH
512 YES-ISH
1180 YES-VERY GOOD
1500
1518 YES
1:1 Diff. Serv. Evaluation(Cisco7200)
35
XXXXXXXXXX
XXXXXXXXXX
XXXXXXXXXX
XXXXXXXXXX
XXXXXXXXXX
XXXXXXXXXX
XXXXXXXXXX
XXXXXXXXXX
Very Bad
Very Very Bad
Bad
Good
Very Good
Very Very Good
1:1 5:5 7:3 9:1
1500, 1180
512
36
StandaloneThroughput
80
82
84
86
88
90
92
94
96
# of flows
TC
P T
hro
ug
hp
ut
BE Throughput
LBE Throughput
BE Throughput 85.98 92.26 93.72 93.86 94.54 95
LBE Throughput 87.68 93.14 93.32 93.88 94.04 95.06
1 2 4 8 12 20
Throughput Vs. # of flows
0
20
40
60
80
100
120
# of flows per each class
TC
P T
hro
ug
hp
ut
(Mb
ps)
BE Throughput
LBE Throughput
aggregate throughput
BE Throughput 79.02 86 88.3 90.94 93.54 94.2
LBE Throughput 12.62 9.76 7.16 6.345 7.304 12.96
aggregatethroughput
91.64 95.76 95.46 97.285 100.84 107.16
1 2 4 8 16 32
Loss %
0
0.5
1
1.5
2
2.5
# of flows per each class
Loss
% %Be loss
%LBE loss
%Be loss 0.00155 0.0042 0.01538 0.05454 0.16679 0.58784
%LBE loss 0.05703 0.42317 0.78629 1.91059 1.19613 1.51166
1 2 4 8 16 32
•As regards BE, a greater # of flows lose more because more TCP flows have to adapt…
•BE aggregate gets….:
1BE+1LBE: 79.02/85.98=91%
2BE+2LBE : 86/92.26=93%
4BE+4LBE:88.3/93.72=94%
8BE+8LBE:90.94/93.86=96.8
16BE+16LBE:93.54/94.77= 98%
32BE+32LBE:94.2/95 = 99%
37
32 flows BE TCP + 1 flow UDP LBE 512&1500
It seems that, for 512f.s., beyond 80x2=160 Mbps LBE offered load…the input interface goes crazy again….and while BE TCP backs-off…LBE UDP fill the gaps created by TCP; This is why LBE “steals” BW to LBE…….
I think this is the situation where the LBE BW. percentage assigned has to be 0%...but CISCO doesn’t support it
Bear in mind that when LBE is 160Mbps, the load at the input interface is constituted by the TCP load as well which is around 94Mbps
UDP LBE 512f.s. + 32flows TCP BE
0
20
40
60
80
100
120
LBE UDP per port Load (Mbps)
Th
rou
gh
pu
t(B
lue
&P
ink
)) A
ND
Lo
ss
%(Y
ell
.)
TCP BE Throughput
LBE Loss % (512)
LBE IP Throughput (512)
32 TCP Standalone
TCP BEThroughput
92.5 93.3 92.3 49 26.22
LBE Loss %(512)
94.78 95.64 96.42 72.72 60.94
LBE IPThroughput(512)
5.815 5.662 5.315 45.593 72.534
60 70 80 90 100
UDP LBE 1500f.s. + 32flows TCP BE
0
20
40
60
80
100
120
60 70 80 90 100
LBE UDP per port Load (Mbps)
Th
rou
gh
pu
t(B
lue
&P
ink
) A
ND
Lo
ss
%(Y
ell
.)
32 TCP BE Throughput
LBE Loss (1500f.s.)
LBE IP Throughput (512)
32 TCP Standalone
38
Better reaction for the 64 aggregate with respect to the 32 one……..the higher the # of flows the higher the reactivity!!!!!!!!!!!
UDP LBE 512f.s. + 64flows TCP BE
0
20
40
60
80
100
120
LBE UDP per port Load
Th
rou
gh
pu
t(B
lue&
Pin
k) A
ND
L
oss
%(Y
ell.
)
TCP BE Throughput
LBE Loss (512f.s.)
LBE IP Throughput (512)
64 TCP Standalone
TCP BEThroughput
93.4 93.7 90.03 63.908 39.045
LBE Loss(512f.s.)
97.89 98.93 97.23 79.79 64.21
LBE IPThroughput(512)
2.349 1.396 4.111 33.771 66.464
60 70 80 90 100
64 TCP flows instead of 32
39
Conclusions:
UDP• Router input interfaces suffer a lot from high input rate…it
prevents traffic from arriving at the Diff. Serv. output interfaces.
………so…..small frame sizes LBE traffic has to be carefully rate- limited, when necessary, in the router input interface.
• For small frame sizes BE is not protected from LBE attack…the bigger the disproportion the worse the situation…it is likely that this is going to be a problem for Gbps speed….
......so….such form of rate-limiting has to be function of the traffic disproportion present in the system as well….
TCP• The policy works better when the number of flows increases
40
Future work
TCP + UDP
• Try N flows TCP BE + UDP LBE with and without burstiness and check if BE TCP is protected
• Apply a 0% BW reservation to LBE UDP traffic
TCP
• Investigate whether, in the case of a small number of TCP flows, it is actually the unreliability of the measurements obtained with a very small number of TCP flows which is behind the bad performance of the policy, rather than the policy itself not working with a small number of flows.
41
THANK YOU………….