casper signal processing workshop 2009 ska signal processing (preliminary) wallace turner domain...
TRANSCRIPT
Casper Signal Processing Workshop2009
SKA Signal Processing(Preliminary)
Wallace TurnerDomain Specialist for Signal Processing
SPDO
Example Configuration (Phase 2)
2 of 35
Memo 100 identifies the following options:70-200MHz: Sparse AA200-500MHz: Sparse AA
500MHz-10GHz: 3000 15m dishesOr500MHz- 10GHz: 2000 15m dishes with PAFs plus WBSPFOr500MHz-10GHz: 250 Dense AA plus 2400 15m dishes/ WBSPF
Note: On going discussions 15m vs 12m dishes
Example Configuration with Dense AA + SPF
SPDO
Dishes+Single Pixel Feeds
4 of 35
American: 6m Hydroformed Dish South Africa: 15m Composite Dish
Canadian: 10m Composite Dish
Note:On going discussions 12m vs. 15m dishRequired sensitivity 10,000 m2K-1
Correlator processor and dump rate proportional to Nant
2 ADC likely to be at antenna (4 bit ?)O/Prate = fs.4bits = 160 G bits/s per antenna
Where fs = sample rate likely to be split into smaller basebands
SPDO
AGN Science Chapter Example
5 of 35
0
200
400
600
800
1000
1200
1400
1600
30 35 40 45 50 55 60
No. dishes
Achieved Tsys K
No. of 15 m dishes needed for AGN and Star Formation for various aperture efficiencies with single pixel feeds
50%
55%
60%
65%
AGN Science ChapterSurvey Speed: 1 x 105 m4K-2deg2
Frequency range 500 MHz to 8GHzTsys now: Not achieved over bandwidthTarget Tsys: 35K
SPDO
Dishes+Phased Array Feeds
6 of 35
Apertif Netherlands: Vivaldi Array ASKAP Australia: Checkerboard Array
PHAD Canada: Vivaldi Array
Note:Some Channelization and Beamforming likely to be at antenna.Maximum Field of View limited by Array size and focal length of dish.Achievable field of view limited by network bandwidth.
SPDO
Example Number PAFs Required
7 of 35
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
40 50 60 70 80 90 100
No. dishes
Achieved Tsys K
No. of 15 m dishes needed for Wide Field Polarimetry for various aperture efficiencies with PAF
50%
60%
70%
80%
Wide Field Polarimetry Science ChapterSurvey Speed: 5 x 108 m4K-2deg2
Max frequency 2GHz
WBSPF would require over 15,000 dishes
Tsys now: 100 KTarget Tsys: 50 K
SPDO
Simplistic View of PAF Processing
• Consider Frequency Domain Beamforming• Channelisation: Nchan ~ 64
– 12 taps gives < 60db aliasing between channels– Processing load ~ (Ntaps + 3*log2(Nchan)) x Nel x 2pol x fs
– Ntaps = 12, Nel = 96 x 2pol & fs = 1.4 GHz for 700MHz bandwidth
Processing load = 8 x 1012 MACS• Beamformer (per antenna):
– Average beams per channel 30– Processing Load = Nbms.Nel.2pol.fs. 4Multiplies = 3 x 1013
Macs
• O/Prate= Nbms.2pol.fs.4bits.8B10B = 420 G bit/s– 42 10 G bit/s optical cables per antenna– Includes 25 % extra bandwidth required for 8B/10B
Encoding• 2000 dishes with PAFs (total 840 T bits/s)Note :*FFT implementation dependent
8 of 35
SPDO
Sparse Aperture Arrays
9 of 35
LOFAR: Netherlands et al
LWA: USA MWA: USA & Australia
Note:Two types of sparse AA required:70MHz – 200 MHz200MHz – 500 MHz
Only solution for EoRHI Science Chapter
SPDO
Dense Aperture Array Station
10 of 35
Assumed Dense Aperture Array~256 tiles x 256 elements per tile2 polarisations per elementSample rate ~ 2.5 G Hz4 bits/ sample56 m diameter250 stations Tsys now 120KTarget 35K Memo 100
Processing Bunker
Dense AA Detail300MHz to 1GHz i.e. 700MHz bandwidthfs=2 x 700 MHz56m diameter array =>2463 m2
44.4 2pol elements per m^2 (30 cm wavelength) = 110,000 elements x 2pol per station, about 2 x 64k elements.Array efficiency 80%, Bore efficiency 75%& Tsys = 35K -> 250 stations for 10,000m2K-
1 sensitivity
SPDO
Simplistic View of Dense Aperture Array Processing
• Consider Frequency Domain Beamforming– Delay is implemented as a phase slope in frequency
domain.– Alternative time domain with tuned lengths of co-ax.
• Channelisation: Nchan 1024,– 12 taps gives < 60db aliasing between channels– Ntaps = 12, Nant = 64k x 2pol & fs = 1.4 GHz for 700MHz
bandwidth– Processing ~ (Ntaps + 3*log2(Nchan)) x Nant x 2pol x fs**=8 x 1015
MACS• Beamformer (per station):
– Average 1437 beams per channel to cover 250 sq degrees FoV
– Processing Load = Nant.2pol.Nbms.fs. 4MACS = 1 x 1018 MACS
• O/Prate**= Nbms.2 pol.fs.4bits.8B10B = 20 T bit/s– Over 2000 10 G bit/s optical cables, 8B/10B Encoding factor
1.25• Up to 250 Dense AA Stations (total 10 peta bits/s)Note :*FFT implementation dependent **Ignores upsampling of channelizer
11 of 35
SPDO
Correlator Processing Loads• Channelisation (700MHz bandwidth):
– SPF 4 x 1014 (105 channels)– PAF 4 x 1012 (4096 fine channels giving ~ 105 total)– Dense AA 6 x 1015 (128 fine channels ~ 105 total)
• Correlation load (700 MHz bandwidth)– SPF load = 6 x 1016 MACS– PAF load = 1 x 1018 MACS– Dense AA = 2 x 1018 MACS
• Correlator Dump Rate (Dish Solution)– 2280 15 m dishes + 40 x 18 dish stations and 3000 km
baseline:– Integration time ~ 200ms for < 1% smearing5 x 106/2 baselines x 105 channels x 4 bytes x 5 Hz = 9 T
Bytes/sNot calculated for other configurations yet
12 of 35
SPDO
Signal Processing Overview Memo 100 Option a*
13* Sparse AAs + 3000 15-m dishes with SPFs
Sparse AAChannelizer Full Stokes Correlator
Pulsar & Transient Processing
Timing
xx,xy,yx,yy
Multipier
Opto-Electrical
Conversion
Optical Rx
Timing
Monitoring & Control
Visibilities
PulsarCandidate
Data
1680Dishes
SparseAA
Local Monitoring
IntegratorDelays
PulsarSearch
& TimingProcessing
Opto-Electrical
Conversion
Opto-Electrical
Conversion
Poly-PhaseFilter
Candidates
Opto-Electrical
Conversion
70 MHz 500 MHz
Full Stokes Correlatorxx,xy,yx,yy
MultipierVisibilities
Integrator
500 MHz 10 GHz
Opto-Electrical
Conversion
Opto-Electrical
Conversion
Poly-PhaseFilter
DishChannelizer
Opto-Electrical
Conversion
Poly-PhaseFilter
Beamformer (core)
BeamFormer
Opto-Electrical
Conversion
Optical Rx
SPF Dish600 CoreDishes
SPF Dish40
Stations
Delays
TBA10 G bit/s
OpticalConnections
7424010 G bit/s
OpticalConnections
TBA10 G bit/s
OpticalConnections
18 Dishes per
station
SPDO
Signal Processing OverviewMemo 100 option b*
14 of 35* Sparse AAs + 2400 15-m dishes with PAFS & WBSPF
Sparse AAChannelizer Full Stokes Correlator
Pulsar & Transient Processing
Timing
xx,xy,yx,yy
Multipier
Opto-Electrical
Conversion
Optical Rx
Timing
Monitoring & Control
Visibilities
PulsarCandidate
Data
1080PAF
SparseAA
Local Monitoring
IntegratorDelays
PulsarSearch
& TimingProcessing
Opto-Electrical
Conversion
Opto-Electrical
Conversion
Poly-PhaseFilter
Candidates
Opto-Electrical
Conversion
70 MHz 500 MHz
Full Stokes Correlatorxx,xy,yx,yy
MultipierVisibilities
Integrator
500 MHz 10 GHz
Opto-Electrical
Conversion
Opto-Electrical
Conversion
PAF/WBSPFChannelizer
Opto-Electrical
Conversion
Optical Rx
SPF Dish600 Core
PAF
SPF Dish40
Stations
Delays
TBA10 G bit/s
OpticalConnections
144,48010 G bit/s
OpticalConnections
TBA10 G bit/s
OpticalConnections
Poly-PhaseFilter
18 PAFS per
station Possibly include a beam former for the core
SPDO
Signal Processing OverviewMemo 100 option c*
15 of 35* Sparse AAs + 250 Dense AA + 2000 15-m dishes with SPFs
Sparse AAChannelizer Full Stokes Correlator
Pulsar & Transient Processing
Timing
xx,xy,yx,yy
Multipier
Opto-Electrical
Conversion
Optical Rx
Timing
Monitoring & Control
Visibilities
PulsarCandidate
Data
SparseAA
Local Monitoring
IntegratorDelays
PulsarSearch
& TimingProcessing
Opto-Electrical
Conversion
Opto-Electrical
Conversion
Candidates
Opto-Electrical
Conversion
70 MHz 500 MHz
Full Stokes Correlatorxx,xy,yx,yy
MultipierVisibilities
Integrator
300 MHz 1 GHz
Opto-Electrical
Conversion
Opto-Electrical
Conversion
Dense AAChannelizer
Opto-Electrical
Conversion
Optical Rx
SPF Dish Delays
TBA10 G bit/s
OpticalConnections
1,000,00010 G bit/s
OpticalConnections
TBA10 G bit/s
OpticalConnections
250Dense AA
1280Dishes
Opto-Electrical
Conversion
Optical Rx Full Stokes Correlatorxx,xy,yx,yy
MultipierVisibilities
Integrator
1 GHz 10 GHz
Opto-Electrical
Conversion
4224010 G bit/s
OpticalConnections
40Stations
Delays
Opto-Electrical
Conversion
Poly-PhaseFilter
Dish WBSPFChannelizer
Poly-PhaseFilter
Poly-PhaseFilter
SPDO
Technology Options• FPGA
– Virtex 6 (available 2010): 2016 x DSP slices clocked at 600 MHz -> 1200 G
MACS
~ 25 G MACs per Watt
1018 MACS requires ~ 106 FPGAS
=> 48 W per device and ~ 48 M Watts for 1018 MACS
Operating cost 1$ per Watt per year => $48M per annum
Plus cost of cooling and delivering power• ASIC
– 22nm (available 2010): 2.5 nW/MHz/Gate > 40 T MACS (4 bit) per device => 25,000
devices Assuming < 50 % gates switching at any one
time: 600kWOperating cost $600k per annum
16 of 35
SPDO
What would F or X unit look like?
17 of 35Baseline Board (front)
Station Board Baseline Board (rear)
Pictures courtesy Brent Carlson
EVLA style boards might be an optiton ?64 ASICS or FPGAs on board (~1.5 kW card)~ 190 boards for Dense AA ASIC correlator14 cards per shelf -> 14 shelvesIs production yield an issue? Could use smaller 8 processor chip board As per ASKAP or UniboardInter-board Communication links increase
SPDO
Multichip Module
18 of 35
• SKADS have developed a promising Multichip Module:4 x 4 antenna array currently,Current RFI Protection shows -57dB per M (in air)
Picture courtesy of Kris Zarb Adami
Could be developed and used in several areas of the SKA(Note that the key components are ADC and Optical I/O, although the others could be useful in some applications.)
SPDO
Correlator Centre Build Cost
19 of 35
0 20000 40000 60000 80000 100000 1200000
10000000
20000000
30000000
40000000
50000000
60000000
70000000
80000000
90000000
100000000
Data Centre Build Cost
50W/sq ft
100W/ sq ft
400W sq ft
Total sq feet of data centre
Bu
ild C
os
t $
ASIC Solution
Kevin Wohlever 2006
ASIC solution• 1 cabinet per 30 sq
ft• Between 80 & 160
cabs?• Factor of 4 for air
con &PSU units, offices
• ~ 50 W per sq ft
SPDO
SPDO Team
Project Director Richard Schilizzi
Project Engineer Peter Dewdney
Executive Officer Colin Greenwood
Project Scientist Joe Lazio
System Engineer Kobus Cloete
Domain Specialist Receptors Neil Roddis
Domain Specialist Signal Transport Roshene McCool
Domain Specialist Computing & Software Duncan Hall
Domain Specialist Signal Processing Wallace Turner
Site Engineer Rob Millenaar
Project Management Officer Billy Adams
Industry Relations Manager Phil Crosby
Office Manager Lisa Bell
20