the alma correlator gianni comoretto, j.c. webber, a. baudry, c.m. broadwell, r. p. escoffier, j.h....
TRANSCRIPT
The ALMA Correlator
Gianni Comoretto, J.C. Webber, A. Baudry, C.M. Broadwell,
R. P. Escoffier, J.H. Greenberg, R.R. Treacy, P. Cais, B Quertier,
P. Camino, A. Bos, W Gunst,
Workshop on New Generation Correlators
Groningen, 27th-29th June, 2006
Contents
• The ALMA interferometer
• The correlator
• Time and frequency division modes
• Characteristics and operating modes
• First test results
• Summary
The Atacama Large Millimeter Array
Interferometer
• Joined NRAO-Canada-Eso-Japan project
• 64 elements connected interferometer
• 5000 m altitude site, 1.5 mm precipitable H2O
• Total collecting area 7000 m2
• Baseline spacing => 12 Km: 25mas @ 200 GHz
• 30-950 GHz receivers in 10 frequency windows
• Compact array with smaller antennas
12m dish at VLA test site
Chainantor Site
The Atacama Large Millimeter Array
Interferometer
• 8 GHz total bandwidth – dual polarization
• 4 independently tunable IF channels
• Fringe stopping at 2nd LO
• 3 bit sampler, 2-4 GHz band
• Variable clock phase for fine delay compensation
• Digital fiber data transmission: 3 bit x 8 channels x 4 GS/s on
multicarrier fiber
• Minimum dump time:
– 1ms for autocorrelation data
– 16 ms for visibility data in continuum mode
– 500 ms for full spectroscopic modes (128M visibility
points, 1GB/s data rate)
• Low resolution spectroscopic mode needed for continuum
observations to remove spectral lines Dual sampler
The ALMA correlator
• 4 quadrants, each one processing single 2 GHz IF channel
• 4 polarization products, 64 antennas
• Correlator chip: 4K lags at 125 MHz clock, 2 bit sampling
• Correlation card: 32x32 antennas, 4 polarizations, 125 MHz clock
• 4 cards process 1/32 of the IF bandwidth: 512 cards total
• Time division and frequency division modesOne quadrant
Station rack Correlator rackpowersupply
Computer
System design evolution
• Initial design: pure lag correlator, with band selecting digital filter
– Time division architecture: each correlator plane analyzes a time segment
of the input data
– Limited frequency resolution at full BW: 64 points/IF (31 MHz)
– Full resolution (2K points) at 1/32 BW
• Retrofitted with digital filterboard, to implement a frequency division FXF
architecture
– Filterbank performs 32 point frequency division
– Each correlator plane analyzes one frequency segment
– Resolution increases by up to 32 (1 MHz at full BW)
– Filterboard mechanically, electrically and cost compatible with old digital
filter
• Full BW time division mode mantained for “continuum” observations
Correlator structure
• Antenna based design
• Antenna unit: delay compensation, filtering,
recirculating memory
• Correlator unit: 64x64x4 correlators, Long
Term Accumulator
• Minimum number of inteconnections:
– Each antenna unit sends 2 pol. data to 32
correlator units
– Each correlator unit receives data from 64
antenna units
– Point-to-point interconnections, without
data duplication
– LVDS interconnections with 250 MS/s
data rate
The ALMA correlator chip
• Full custom ASIC at 125 MHz clock
• Conventional lag type real correlator
• 2 bit full multiplication table. 4 blocks can be joined for 4 bit operation
• Individual block: 2 polarization per antenna
– 4x64 lags at full polarization (HH, HV, VH, VV)
– 2x128 lags at dual polarization (HH, VV)
– 256 lags with one polarization
• No cascading of blocks. Larger delays implemented using station based
buffer memory
• 20 bit integration and 16 bit secondary storage
Station (Memory) Card
• Antenna based, dual polarization
• Implements:
– bulk (static) geometric delay,
– correlation delay, to synthetize
large lags without inter-block
connections
– Time multiplexing modes
• Circular memory buffer: 1 ms being written while previous ms being read
• Write buffer free running, value stored at beginning of each ms
• 2 read buffers for each correlator plane.
• Different planes can correlate different data, different time interval, and/or different
delay range, changing only read pointer offset
Time division mode
• 1 ms data block divided into (up to) 32 shorter blocks
• Each block sent at 125 MS/s to one correlator plane
• At 2 GHz BW, 32 correlator planes needed: 64 spectral points total (4 pol. mode)
• At reduced BW, more planes available: each plane processed data with different
delay offset => more lags
• No interconnections between planes, all “routing” performed in memory board
Frequency division mode
• Digital filter to adapt input band to correlator band
• Each correlator plane analyzes 1/32 of the input band
• More correlator planes may analyze one slice (subchannel) with increased
resolution (zoom modes)
• Each subchannel can be arbitrarily positioned in the input band
• Different band/resolutions possible on different (or the same) portions of
the band
• Huge increase in correlator output: minumum integration time ~0.5 s
• For fast mapping in continuum mode (16 ms dump time), original time
division mode retained
Subchannel stitching
• Stitching together many spectra
with few points: Edge effects
important
• Real correlator: -> 0 at ->0 and
-> max
• Shift of spectral point barycentre:
error on gradients and line position• Shifting in frequency by ½ channel
improves phase response near
edges• Need to delete edge points :
overlap of sub-channels required
Polyphase filters not easily usable
RealImaginary
Tunable Filter Bank
• Array of digital receivers
• Each receiver selects and
downconverts a 62.5 MHz frequency
slot in the input 2 GHz band
• Delay compensation for short term
geometric delay variations
• Implemented using 16 FPGAs
(Altera Stratix2 EP2S30)
• Prone to single-event upset in
configuration RAM: internal CRC
monitor to detect configuration faults
Tunable Filter - Block diagram
• Configurable parameters: bandshape (via 2nd stage filter taps, only 2 sets used), central frequency, scale factor (gain) before final requantization
• Mode flags: bypass mode, ½ band mode, 4 bit mode, oversampling
Bandpass filter
• 2 stage FIR filter:
– decimating 1/32 band,
8 bit tap coefs
– Half band
bandshaping filter, 9
bit tap coefs
• > 55 dB stopband
• Average stopband > 60dB
• 0.3 dB passband ripple
• 93% useful bandwidth
Dynamic range and noise
• Dynamic range
– Polyphase digital LO/mixer using 6 bit
lookup tables: 0.9% loss, > 52 dB SFDR
– Filter: 55 dB stopband rejection
• Excess noise:
– 3 bit + 2 bit quantizations increase quant.
noise
– Possibility of trading BW for sensitivity
Sampler Mixer Total
Bypass 2 bit 13.6 13.6
Bypass 3 bit 4.1 4.1
Freq. division:
2 bit 4.1 0.1 13.6 18.3
4.1 0.1 ~8.0 ~12.0
2nd Quant
Modes for increased sensitivity
• Normal 2 bit
• Double Nyquist: +6%
• Requires 2 filters
• Output processed by 2 correlator planes
• In ½ bandwidth mode, single stream at 125MS/s processed by 1 corr. plane
• 4 bit: +13%
• Requires 2 filters
• Output processed in 4 correlator planes
Implementation
• Signal distribution direct point-to-point
LVCMOS
• Connections only between adjacent
chips
• Interboard connections point-to-point
PECL
• Signal integrity checkers on each data
path. Programmable signal phase
• Clock distribution using internal FPGA
PLL.
• Inter-rack connections 250 MHz LVDS
cables
• Power distribution using onboard DC-
DC converter, 48V busDigital filterboard
First tests with simulated data
• Pseudo random noise + 2 sine
lines + their harmonics due to
finite quantization
• 10 second integration
• Line strength 15 dB
• >40 dB spurious free dynamic
range
• Good alignment of subchannels
(< 0.1%) despite different power
level
Problems
• Thermal: 100C Tj, better cooling
schemes needed
• Mechanical: stiffners or higher
(2mm) board
Summary
• Connected element interferometer, using digital fiber link interconnection• Spectral line output, full polarization (4 pol channels)• Total BW 8+8 GHz, digitized as 4+4 channels 2 GHz, 3 bit• Resolution channels mode over 2 GHz IF (single quadrant):
– 31 MHz 64 fast continuum mode, full polarization – 1 MHz 2048 Frequency division mode, full band & polarization– 250 KHz 8192 Full band, single polarization– 3.8 KHz 8192 minimum band (31 MHz/IF), single polarization
• 64 antennas: 64 ACF, 2K CCF, with up to 1K frequency and polarization channels per baseline,
total of up to 128M lags • Dump time, limited by total data rate (1 GB/s): 16 ms / 500 ms. Typ. up to minutes• Dynamic range: ~50 dB• Technology: full custom ASIC for correlator, FPGA for filterbank • Correlator board size limits n. of antennas. All ACF and CCF on one board group. Scalable in
bandwidth and resolution, adding “planes”• FXF with first F stage performed using tunable filterbank