4th e-vlbi workshop, sydney, july 2005, a. szomoru, jive e-vlbi and expres arpad szomoru
TRANSCRIPT
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
e-VLBI and EXPReS
Arpad Szomoru
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Outline
• Current status, recent results
• Network test
• e-VLBI and the EXPReS proposal
• Conclusions, future
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Why e-VLBI ?
• Reliability – real-time feedback to the telescopes
• Sensitivity – sustained high data rates possible
• Logistics – No media management
• Rapid science results:• Geodesy
• Precision spacecraft navigation
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
e-VLBI Proof-of-Concept Project
• DANTE/GÉANT Pan-European Network• SURFnet Dutch NREN• GARR Italian NREN • UKERNA UK NREN• PSNC Polish NREN • DFN German NREN• KTHNOC/NORDUnet Nordic NREN • Manchester University Network application software • JIVE EVN Correlator• Westerbork telescope Netherlands• Onsala Space Observatory Sweden• MRO Finland• MPIfR Germany• Jodrell Bank UK• TCfA Poland • CNR IRA Italy
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
0SISISISI
GRGRGRGR
CZCZCZCZ
IEIEIEIE
LULULULU
EEEEEEEESKSKSKSK
ILILILILLTLTLTLTCYCYCYCY
ESESESES
HUHUHUHUITITITIT
FRFRFRFR
UKUKUKUK
BEBEBEBE
NLNLNLNLDEDEDEDE SESESESE CHCHCHCH
PTPTPTPT
ATATATAT
GEANTGEANTGEANTGEANT
2.5 G 1.2 G 622M
155 M
BGBGBGBGLVLVLVLVRORORORO
HRHRHRHR
PLPLPLPL
45 M34 M
310 M
GÉANT:Access of NRENs to GÉANT
EVN telescope
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
• for the EVN and JIVE
• Standards:- Protocols, parameter tuning, procedures at telescope and correlator.
• New Capabilities:- Higher data rates, improved reliability, quicker response.
• for GÉANT and the NRENs• To see significant network usage with multiple Gbit streams
converging on JIVE.• Minimum is three telescopes (not including Westerbork)• Must be seen to enable new science and not just solve an existing
data-transport problem• Real-time operation is seen as the ultimate aim, buffered operation
accepted as a development stage.
POC targets:
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Fibre pair from Amsterdam
Optical Splitters
GE lines to correlator
LX Optics converters
Cisco ONS 15252
Correlator Interface
Current status:
•JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps
•1 Gbps connections to Westerbork, Torun, Onsala, Jodrell
•155 Mbps to Arecibo, 128 to Cambridge
•Medicina being connected at 1 Gbps now, Metsahovi 2006
Current status:
•JIVE: 6 lambdas via Netherlight, each capable of 1 Gbps
•1 Gbps connections to Westerbork, Torun, Onsala, Jodrell
•155 Mbps to Arecibo, 128 to Cambridge
•Medicina being connected at 1 Gbps now, Metsahovi 2006
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Network testing
• Use existing protocols on currently available hardware
• TCP maximal reliability• Not really required
• Sensitive to congestion• Lot of fine-tuning necessary
• And possible
• UDP connectionless• Unaccountable
• Tailor made protocols?
• Lambda switching
• Internet weather• Hard to quantify
• Hard to pinpoint bottlenecks
0
100
200
300
400
Dw- BoBo- DwDw- Bo*Bo- Dw*
0
200
400
600
800
Dw- Bo
Dw- Tr
Dw- OnM
bps
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
• February 2005: network transfer test (BWCTL) employing various network monitoring tools involving Jb, Cm, On, Tr, Bologna and JIVE
Onsala-Jive
0
200000000
400000000
600000000
800000000
1000000000
1200000000
1 3 5 7 9 11
13
15
17
19
21
23
25
TCP
UDP
e-VLBI transfer tests
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
e-VLBI transfer tests
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Memory-memory Disk2net-net2disk In2net-net2disk
In2net-net2out
UDP TCP UDP TCP TCP TCP
Bench via patch 930 250 256
Idem, jumbo frames 960 544 512
Bench via Amsterdam 500 360 256
Idem, jumbo frames 341 456
Westerbork-JIVE 867 680 256 256
Idem, jumbo frames 249 378
Bologna-JIVE 670 128 307
Jodrell-JIVE 50 70 64 256
Arecibo-JIVE 88 64
Torun-JIVE 800 260 128
Onsala-JIVE 177 256 256
JIVE-Haystack 612 71
Test results
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
POC results
• Demonstration of feasibility
• Identification of problems
• Has led to closer ties with networking community and generated political interest
• Has laid the foundation for the next step forward (EXPReS)
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
The Future… e-VLBI & EXPReS!
• I3 proposal to the EC (Communication & Network Development Call)
• Ranked first out of 43 proposals; nearly fully funded to an amount of 3.9 MEuro.
EXPReS = EXpress
Production
Real-time
e-VLBI
Service
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
EXPReS aims
• Upgrade EVN to e-EVN• Last mile problem at many telescopes
• Including connections on a global scale
• 16 Gb/s lines into Dwingeloo to JIVE correlator
• Software in field and correlator to become real real-time
• And look beyond 1 Gb/s• More capacity on digital sampling, more bandwidth
• As being implemented for e-MERLIN in UK
• Hardware (PC-based) and protocols for transport
• Correlator with more capacity: distributed correlation
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Expanding the e-VLBI Network
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
New Antennas
40m antenna at Yebes, SpainRadio and mm frequencies
Sardinia Radio Telescope64m; radio to millimeter
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Current “e-VLBI” Telescope Status Telescope Status Comments Cost (Meuro)
JIVE 6 x 1 Gbps 16 x 1 Gbps upgrade ……
WSRT (NL) 1 Gbps - -
Onsala (SE) 1 Gbps 10 Gbps upgrade 0.15
Torun (PL) 1 Gbps -
Jodrell Bank &
Cambridge (UK)
2.5 Gbps 10 Gbps upgrade
(7 x 30 Gbps e-MERLIN**)
0.12
Arecibo (USA) 155 Mbps 1 Gbps upgrade 0.4
Medicina (IT) 1 Gbps Summer 2005 -
Metsahovi (FI) 1 Gbps 2006 0.33
** Crucial part of the plan is to combine e-MERLIN and e-VLBI Telescopes transparently.
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Telescope Status Comments Cost (Meuro)
Yebes (ES) NYO 1 Gbps planned 2006/7 0.53
VIRAC (LV) NYO 1 Gbps planned 2006/7
VLBI Capable 2006 ?
0.53
Shanghai (CN) NYO 0.1- 1 Gbps planned
Last mile: 25 km
Poor connectivity between Europe & China (ORIENT)
0.35
Miyun (CN) & Kunming (CN)
NYO 0.1-1 Gbps planned
Last mile: few km
0.3
Urumqi (CN) NYO Fibre installed – 2 Mbps
0.1-1 Gbps planned
0.1
Hartebeesthoek (SA) NYO 1 Gbps planned as part of SKA ambitions
Poor connectivity between South Africa & Europe
-
TIGO (Chile) NYO Fibre installed -
Noto (IT) NYO Last mile problem 3.1
Sardinia (IT) NYO Last mile 1.0
Effelsberg (DE) NYO Last mile 5 km but… 2.0
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Towards “real”real-time
• Reliability of operation
• Robustness
• Feedback to stations
• Status display at stations and Jive
• Ease of operation
• Continuous monitoring
• Better protocols?
• Software consistency
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Protocol, buffer selection
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
FABRICFuture Array of Broadband Radio-telescopes on Internet Computing • Will need a new correlator for 4 Gb/s
• Current EVN Mk4 based on 16x16x1Gb/s
• Implemented on 1024 special chips
• Next generation will most likely use standard CPUs• Current EVN Mk4 processor equivalent 40 T-ops (2bit)
• LOFAR BlueGene ≈ 27 Tflops
• 32 station x 4 Gb/s 640 T-ops⇒• And requirement to route 32x32 4 Gb/s input stream
• Possible solution: distribute the computing • Use the Internet as the cross-switch
• SETI@home gets 59 Tflops
• Proposed to do pilot on the Grid
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Telescopes will do multicast transmission
Compute nodes will be cluster size
Transported data may be time/frequency slices
Resource allocation from central control
Output data stored in central archive, proprietary rights for users
Central astronomical scheduling
4th e-VLBI workshop, Sydney, July 2005, A. Szomoru, JIVE
Conclusions, future• POC has been a success, but, progress has been both fast
and slow• Bottlenecks hard to identify, packetloss does not equate
congestion, fairness is a problem, relation packetloss—correlation poorly understood
• Geant2, Surfnet6 will be hybrid networks • lambda switching• Bandwidth on demand
• New protocols, software/hardware interfaces• 4—10Gbps transfer—new hardware?• e-VLBI is moving ahead; e-EVN will soon be a reality