high rate internet data transfer for evlbi
DESCRIPTION
High Rate Internet Data Transfer for eVLBI. Ralph Spencer, Richard Hughes-Jones and Simon Casey The University of Manchester All Hands Sept 2005. Radio Astronomy. The study of celestial objects at 1m wavelength. Sensitivity for continuum sources - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/1.jpg)
High Rate Internet Data Transfer for eVLBI
Ralph Spencer, Richard Hughes-Jones and Simon Casey
The University of ManchesterAll Hands Sept 2005
![Page 2: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/2.jpg)
Radio Astronomy• The study of celestial objects at <1 mm to >1m
wavelength.• Sensitivity for continuum sources• B=bandwidth, integration time.• High resolution achieved by interferometers.
Some radio emitting X-ray binary stars in our own galaxy:
B/1
GRS 1915+105 MERLIN
SS433MERLIN and European VLBI
Cygnus X-1VLBA
![Page 3: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/3.jpg)
GRS 1915+105: 15 solar mass BH in an X-ray binary: MERLIN observations
receding
600 mas = 6000 A.U. at 10 kpc
![Page 4: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/4.jpg)
Very Long Baseline Interferometry
![Page 5: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/5.jpg)
How interferometry works
• Interferometer: signals from each telescope brought together coherently for correlation (multiplication)
• Resolution /D << that possible from a single telescope, e.g the pimples on a golf ball at 3000 miles at the highest resolution
• Output of the correlator is one component of the Fourier transform of the sky brightness
Dcos
X
D
![Page 6: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/6.jpg)
Earth-Rotation Synthesis
.
The baseline between each pair of telescopes forms an ellipse as the Earth rotates
![Page 7: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/7.jpg)
The Aperture Plane (u,v)
• Need ~ 12 hours for full synthesis, not necessarily collecting data for all that time.
• Telescope data correlated in pairs: N(N-1)/2 baselines
• NBTrade-off between B and t for sensitivity ~sqrt(Bt)
• Data are calibrated, Fourier inverted and then deconvolved to give the images.
EVN
MERLIN
![Page 8: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/8.jpg)
The European VLBI NetworkEVN
• Detailed radio imaging uses antenna networks over 100s-1000s km
• At faintest levels, sky teems with galaxies being formed
• Currently use disk recording at 512Mb/s (MkV)
• real-time connection allows greater – response– reliability– sensitivity
– Need Internet eVLBI
![Page 9: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/9.jpg)
WesterborkNetherlands
Dedicated
Gbit link
EVN-NREN
OnsalaSweden
Gbit link
Jodrell BankUK
DwingelooDWDM link
CambridgeUK
MERLIN
MedicinaItaly
Chalmers University
of Technolo
gy, Gothenbu
rg
TorunPoland
Gbit link
![Page 10: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/10.jpg)
eVLBI Milestones
• January 2004: Disk buffered eVLBI session:– Three telescopes at 128Mb/s for first eVLBI
image– On – Wb fringes at 256Mb/s
• April 2004: Three-telescope, real-time eVLBI session.– Fringes at 64Mb/s– First real-time EVN image - 32Mb/s.
• September 2004: Four telescope real-time eVLBI– Fringes to Torun and Arecibo– First EVN, eVLBI Science session
• January 2005: First “dedicated light-path” eVLBI– Australia to JIVE via Canarie: data from Huygens
descent transferred at ~450Mb/s
![Page 11: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/11.jpg)
• 20 December 20 2004• connection of JBO to Manchester by 2 x 1 GE• eVLBI tests between Poland Sweden UK and Netherlands at 256 Mb/s
• February 2005• TCP and UDP memory – memory tests at rates up to
450 Mb/s (TCP) and 650 Mb/s (UDP)• Tests showed inconsistencies between Red Hat
kernels, rates of 128 Mb/s only obtained on 10 Feb• Haystack (US) – Onsala (Sweden) runs at 256 Mb/s
• Regular tests with VLBI data every ~6 weeks • 128 Mpbs OK, 256 Mpbs often,• But not 512 Mbps – WHY NOT?
• Correlator can cope with large error rates• up to 2 %• but need high throughput for sensitivity• implications for protocols
![Page 12: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/12.jpg)
Gnt5-DwMk5 11Nov03-1472 bytes
0
2
4
6
8
10
12
0 5 10 15 20 25 30 35 40Spacing between frames us
% P
acket
loss
Gnt5-DwMk5
DwMk5-Gnt5
Throughput vs packet spacing Manchester: 2.0G Hz Xeon Dwingeloo: 1.2 GHz PIII Near wire rate, 950 Mbps UDPmon
Packet loss
CPU Kernel Load sender
CPU Kernel Load receiver 4th Year project
Adam Mathews Steve O’Toole
Gnt5-DwMk5 11Nov03/DwMk5-Gnt5 13Nov03-1472bytes
0
200
400
600
800
1000
1200
0 5 10 15 20 25 30 35 40Spacing between frames us
Recv W
ire r
ate
Mbits/s
Gnt5-DwMk5
DwMk5-Gnt5
Gnt5-DwMk5 11Nov03 1472 bytes
020406080
100
0 5 10 15 20 25 30 35 40Spacing between frames us
% K
erne
l S
ende
r
Gnt5-DwMk5 11Nov03 1472 bytes
020406080
100
0 5 10 15 20 25 30 35 40Spacing between frames us
% K
erne
l R
ecei
ver
UDP Throughput Oct-Nov 2003 Manchester-Dwingeloo Production
![Page 13: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/13.jpg)
Throughput (JbGig1 - Mark608 @ JIVE via production)
500
600
700
800
900
1000
25/0
1/20
0500
:00
25/0
1/20
0512
:00
26/0
1/20
0500
:00
26/0
1/20
0512
:00
27/0
1/20
0500
:00
Time of day
Wir
e r
ate
(M
b/s
)
12us
18us
20us
% Packets lost (JbGig1 - Mark608 @ JIVE via production)
0
0.5
1
1.5
2
25/0
1/20
0500
:00
25/0
1/20
0512
:00
26/0
1/20
0500
:00
26/0
1/20
0512
:00
27/0
1/20
0500
:00
Time of day
% L
os
t
12us
18us
20us
January 2005 repeated UDP over 2 days
![Page 14: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/14.jpg)
ESLEA
• Packet loss will cause low throughput in TCP/IP
• Congestion will result in routers drooping packets: use Switched Light Paths!
• Tests with MB-NG network Jan-• JBO connected to JIVE via UKLight
in June (thanks to John Graham, UKERNA)
• Comparison tests between UKLight connections JBO-JIVE and production (SJ4-Geant)
![Page 15: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/15.jpg)
UKLight Switched light path
![Page 16: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/16.jpg)
Tests on the UKLight switched light-path Manchester : Dwingeloo
• Throughput as a function of inter-packet spacing
• Packet loss • Maximum size packets can
reach full line rates with no loss, and there was no re-ordering (plot not shown).
gig03-jiveg1_UKL_25Jun05
0100200300400500600700800900
1000
0 10 20 30 40Spacing between frames us
Rec
v W
ire r
ate
Mbi
t/s
50 bytes
100 bytes
200 bytes
400 bytes
600 bytes
800 bytes
1000 bytes
1200 bytes
1400 bytes
1472 bytes
gig03-jiveg1_UKL_25Jun05
0.0001
0.001
0.01
0.1
1
10
100
0 10 20 30 40Spacing between frames us
% P
acke
t lo
ss
50 bytes
100 bytes 200 bytes
400 bytes 600 bytes
800 bytes 1000 bytes
1200 bytes 1400 bytes
1472 bytes
![Page 17: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/17.jpg)
Tests on the production network Manchester : Dwingeloo.
• Throughput
• Small (0.2%) packet loss was seen
• Re-ordering of packets was significant
gig6-jivegig1_31May05
0.0001
0.001
0.01
0.1
1
10
100
0 10 20 30 40Spacing between frames us
% P
acket
loss
50 bytes
100 bytes 200 bytes
400 bytes 600 bytes
800 bytes 1000 bytes
1200 bytes 1400 bytes
1472 bytes
![Page 18: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/18.jpg)
UKLight using MkV terminals
![Page 19: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/19.jpg)
The GÉANT2 Launch June 2005
![Page 20: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/20.jpg)
Jodrell BankUK
DwingelooDWDM link
MedicinaItaly Torun
Poland
e-VLBI at the GÉANT2 Launch Jun 2005
![Page 21: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/21.jpg)
UDP Performance: 3 Flows on GÉANT• Throughput: 5 Hour run 1500 byte MTU
• Jodrell: JIVE2.0 GHz dual Xeon – 2.4 GHz dual Xeon670-840 Mbit/s
• Medicina (Bologna): JIVE 800 MHz PIII – Mk5 (623) 1.2 GHz PIII 330 Mbit/s limited by sending PC
• Torun: JIVE 2.4 GHz dual Xeon – Mk5 (575) 1.2 GHz PIII
245-325 Mbit/s limited by security policing
(>600Mbit/s 20 Mbit/s) ?
• Throughput: 50 min period• Period is ~17 min
BW 14Jun05
0
200
400
600
800
1000
0 500 1000 1500 2000Time 10s steps
Rec
v w
ire ra
te M
bit/s
JodrellMedicinaTorun
BW 14Jun05
0
200
400
600
800
1000
200 250 300 350 400 450 500Time 10s steps
Rec
v w
ire ra
te M
bit/s
JodrellMedicinaTorun
![Page 22: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/22.jpg)
18 Hour Flows on UKLightJodrell – JIVE, 26 June 2005
• Throughput:• Jodrell: JIVE
2.4 GHz dual Xeon – 2.4 GHz dual Xeon
960-980 Mbit/s
• Traffic through SURFnet
• Packet Loss– Only 3 groups with 10-150 lost
packets each– No packets lost the rest of the
time
• Packet re-ordering– None
man03-jivegig1_26Jun05
0
200
400
600
800
1000
0 1000 2000 3000 4000 5000 6000 7000
Time 10s steps
Rec
v w
ire r
ate
Mbi
t/s
w10
man03-jivegig1_26Jun05
900910920930940950
960970980990
1000
5000 5050 5100 5150 5200
Time 10s
Recv w
ire r
ate
Mbit/s w10
man03-jivegig1_26Jun05
1
10
100
1000
0 1000 2000 3000 4000 5000 6000 7000
Time 10s steps
Packet
Loss
w10
![Page 23: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/23.jpg)
Conclusion and Future directions• eVLBI needs to get to rates of 512 Mpbs – not there yet.
• Rates of 256 Mpbs on production networks possible now using TCP/IP in good conditions, but not higher rates.
• G2 launch showed continuous long term flows as in VLBI have different characteristics to those in short term tests (Iperf, UDPmon) : more investigation needed – collaborating with DANTE and SURFnet
• Packet loss is a killer for TCP – new protocols needed (ESLEA) - VSI-E standard
• The End Host Problem– The performance of Motherboards, NICs, RAID controllers and
Disks matter: Mk V units (Conduant)– MkV-B 2 Gbps data recording in 2 yrs?
• Next generation correlator 10 Gbps in 5-10 yrs
![Page 24: High Rate Internet Data Transfer for eVLBI](https://reader035.vdocuments.us/reader035/viewer/2022062520/56815907550346895dc639f7/html5/thumbnails/24.jpg)
Thanks to
• Paul Burgess JBO• Steve Parsley, Arpad Szomoru, Cormac Reynolds JIVE• Nicola Pezzi, John Graham, Colin Greenwood, Peter
Clarke ESLEA• EVN observatories staff