![Page 1: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/1.jpg)
Networking update and plans(see also chapter 10 of TP)
Bob Dobinson, CERN, June 2000
![Page 2: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/2.jpg)
GE Ethernet
• Prices dropping • see following graph
• GE twisted pair available (RAL)• As GE is getting so cheap
• Why not equip everywhere for GE?• Run some areas initially at 100 MBPS
– Nice up grade path later
• To be looked at in more detail
![Page 3: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/3.jpg)
Cost Evolution
0
500
1000
1500
2000
2500
3000
1995 1996 1997 1998 1999 2000Year
FE NIC projected
FE NIC actual
FE port projected
FE port actual
GE NIC
GE port
![Page 4: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/4.jpg)
An all optical network solution?• Optical fibre now very cheap
– Total installation cost is 20-50% more than CAT 5 TP same price as CAT 6– “VF-45 connector joins 2 fibres in less than 2 mins for less than $5 per
connector”
• A more reliable better total system, says 3M, (especially for raw packets)
– No interference– Lighter, more robust, easier to install– 20 year guarantee!
• To be looked at further– Think a 20 year timescale
![Page 5: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/5.jpg)
Programmable NICs(Alteon)
• Nice results obtained recently ( see later)• Applications
• Switch testing at line speed (Gigabit)• RoI and event building• Pre processing• Protocol processing
![Page 6: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/6.jpg)
Alteon ACENIC
PCI BUS
PCI interface
RISC2
RISC1
MAC
Ext Mem 0.5/1 Mbyte
Mem
Mem
DMA1
DMA2
PhyGE
TIGON CHIP
![Page 7: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/7.jpg)
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 15000
10
20
30
40
50
60
70
80
90
100
Efficiency (%)Max. Effiency (%)
Data Pay load (by tes)
Effi
cien
cy (
%)
SEND ONLY
Data payload
% o
f 1 G
bps (
effic
ienc
y)
![Page 8: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/8.jpg)
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 15000
10
20
30
40
50
60
70
80
90
100
Efficiency (% )Max. Effiency (%)
Data Pay load (by tes)
Y a
xis
title
SEND PLUS CONCURRENT RECEIVE SINGLE PROCESSOR
% o
f 1G
bps (
effic
ienc
y)
Data payload
![Page 9: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/9.jpg)
Alteon contacts
• Meeting next Tuesday with the company• New chips?
• Would make a nice ROB component?
• New NICs?• Future of PCI?
![Page 10: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/10.jpg)
Tests with Foundry Big Iron
• Tests with 16 port switch at CERN• High throughput, but some flow control
problems• This model switch can support 64 ports, we
need about 128• So we can almost buy what we need• 128 certainly in the pipeline
![Page 11: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/11.jpg)
Switch testing
• GE, use Alteon boards programmed as traffic generators
• FE use 32 node FPGA based traffic generators– Doubles as ROB emulator– Note/talk by Micheal Levine
• Used to evaluate switches and calibrate modeling
![Page 12: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/12.jpg)
Switch under test
GE NIC GE NIC GE NIC GE NIC GE NIC GE NIC GE NIC GE NIC
Clock Mem
CPU
Cache Controller
PCI BUSFE NIC
Expansionto multiple units
NICs housed in PCs with 14/17 PCI slots
Global clock
![Page 13: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/13.jpg)
![Page 14: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/14.jpg)
Modeling
• Continuing progress• Adding QoS, trunking and TCP/IP stack
• Model various models for Trigger/DAQ networks
• Two separate• Integrated
![Page 15: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/15.jpg)
Geographic distribution of computing resources
• At the pit• On CERN site• At remote locations
• Farms at home
![Page 16: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/16.jpg)
Network: a global view and possible integration?
• Level 2• DAQ/EF• DCS• Front end ROD crates
![Page 17: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/17.jpg)
Ethernet SW evolution
• See talk of Marc• Also will revisit the problem of high
performance Ethernet I/O– Will look at the TCP/IP co-existence problem– How to best deliver fibre performance to the application?– Scalability– Fault tolerance– QoS– Trunking
![Page 18: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/18.jpg)
Present PC I/O architecture
![Page 19: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/19.jpg)
Future PC I/O Architecture
![Page 20: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/20.jpg)
Infiniband Trade Association• Major players
» IBM» INTEL» Microsoft» Lucent» SUN» Compaq» Hitachi» Fujitsu» Dell
• A significant development! Need to know more. Spec available soon.
![Page 21: Networking update and plans (see also chapter 10 of TP) Bob Dobinson, CERN, June 2000](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b437f8b9ab0599a25de/html5/thumbnails/21.jpg)
Is this the end of PCI?
• I think not!• As ESA bus became a sub system on PCI, so PCI
will likely be a sub system hanging off Infiniband• May gain a nice way of supporting high bandwidth
from multiple PCI buses into a host