Technology and Functionality of the EVLA Correlator(Next Corr Workshop, June 27, 2006)

OutlineEVLA project overview.

Correlator capabilities.

Correlator architecture.

Technology.

Schedule.

EVLA Project OverviewNRAO:

Upgrade VLA from the current ~100 MHz BW to 16 GHz (8 GHz/poln).

New receivers: 1-50 GHz contiguous frequency coverage.

Replace waveguide system with digital fiber system.

3-bit (8x2 GHz) and 8-bit (4x1 GHz) sampling.

New correlator (NRC Canada).

Correlator Capabilities16 GHz/antenna (8 GHz/poln). 32-antenna systemexpandable.

Can tradeoff BW for #ants and #beams; can proc multiple narrow (

Correlator Capabilities16k channels/baseline wideband; up to 4 Mchannels/baseline narrowband with recirculation.

Up to 144 tunable multi-stage digital filters per antennaslot restrictions in 1st stage of filter.

Use filter logic for sub-band multi-beaming and/or narrowband delay.

~60 dB spectral dynamic range.

WIDAR technique (hybrid).

Correlator CapabilitiesHigh-speed pulsar phase binning; 2000 bins/product, min ~15 usec bin width.

High-speed dumping. 1 Gbps Ethernet/board standard. Up to 10 Gbps/board. 25 Mbytes/sec to archive.

All digital phased output1 GHzexpandable to full BW.

Unlimited sub-arrays.

Multiple VSI I/O.

Configurable for virtually any radio telescope configurationincluding auto-correlation.

Uses VLBI standard frequencies (256 MHz). Can incorporate VLBA into excess correlator capacity.

Correlator ArchitectureStation Board: 6U Fiber receiver mezzanine card (NRAO).Cross-bar switch FPGA.Delay module mezzanine card (+/-0.5 samples delay).Wideband autocorrelator.Multi-stage digital filters (1 FPGA per sub-band).Cross-bar switch and sub-and multi-beaming delay memory.4 stages128 MHz31.25 kHz BW out.Power measurements for sub-band stitching.Real-time RFI blanking per sub-band.CPU-settable scaling factors/re-quantization for max sensitivity/dynamic range.State counts, lag-0 power, phase-cal.Narrowband sub-sample delay with 16-step FIR interpolation.Cross-bar switch and mux to 1 Gbps for data transport to Baseline Board.

FIR Interpolator 1

FIR Interpolator 2

FIR Interpolator 3

FIR Interpolator 4

FIR Interpolator 5

FIR Interpolator 16

Fractional delay model

Output

Input

Correlator ArchitectureFanout Board:Dual 1-8 fanout of signals (4-wafer stack, 1 sub-band, all basebands, one antenna) for baseline rack signal distribution.

Baseline Board:8x8 matrix of corr chips fed by 8 X and 8 Y FPGAs (recirculation, phase generation, 7-bit handling, cross-bar).Corr chip: 2048 c-lags, in 16 128 c-lag cells. Full 4-bit multiply, no trunc, 10-500 sec int. Cells can be concatenated.Dedicated LTA FPGA +256 Mbit RAM for each corr chip; high-speed phase binning and recirculation.Output 1 Gbps Ethernet via FPGA.Can set corr chip and board for autocorr mode: 64 independent 2048 channel, 128 MHz autocorrelations.Can process up to 32 stations (1024 correlations), 128 MHz, 1 poln product, at 64 channels/baseline.

0

17

k

0

17

k

0

17

0

17

Filter Bank1(18 sub-band filters)

Filter Bank0(18 sub-band filters)

DelayModule

DelayModule

Fiber Receiver Module

Station Board

0

17

k

0

17

k

0

17

0

17

Filter Bank1(18 sub-band filters)

Filter Bank0(18 sub-band filters)

DelayModule

DelayModule

Fiber Receiver Module

Station Board

RC

RC

RC

CC

RC

Baseline Board iin the kth sub-band correlator

wafer 0

Filter Bank1(18 sub-band filters)

Filter Bank0(18 sub-band filters)

DelayModule

DelayModule

Fiber Receiver Module

Station Board

0

17

k

0

17

k

0

17

0

17

Filter Bank1(18 sub-band filters)

Filter Bank0(18 sub-band filters)

DelayModule

DelayModule

Fiber Receiver Module

Station Board 0 of quad n

0

17

k

0

17

k

0

17

0

17

4 wafer cablebundle from station n, physical output k, sub-band s

s

s

4 wafer cablebundle from station m, physical output k, sub-band s

wafer 1

wafer 2

wafer 3

BB-0,1

BB-2,3

BB-4,5

BB-6,7

LTA

COTS GigE Switch i"

GigE FPGA

BaselineBrd i",koutput

k+1

BE CPU

BE CPU

Antenna n, BB-0,1

X

Y

TechnologyUse Xilinx Virtex-IV, and Altera Stratix II/GX/Cyclone FPGAs.

Standard cell correlator chip, 4 million gate, 0.13 m. PD~3.7 W each @ 256 MHz, 1 V. Use Accel point-of-load reg to minimize power, track chip speed with time.

BGA packages throughout, some gull-wing memories/drivers.

Technology8 different boards in total:

Station Board, Baseline Board: 12U x 400 mm, 28 layer, 0.0035 trace-n-space, 0.125 thick. Phasing Board: ~6U x 160 mm(?).

Fanout Board: 6U x 100 mm, with re-sync Stratix II FPGA.

Common Backplane/midplane (3U x 25 mm); signal I/O, power, IDused everywhereno monolithic backplanes. HM 2.0 mm straight-thru. All press-fit.

Delay module (~5x3.5)mezzanine card: FPGA+DDR SDRAMs

PC/104+ COTS CPU + PCMC card.

RPMIB (very simple; diodes, optos) for rack/fan power monitor and control

Technology1 Gbps station-baseline data transport on high-density Meritec 2 mm hardmetric cable assemblies. Make correlator highly configurable for various correlator configurations (e.g. e-MERLIN).

Use Alteras 1 Gbps mux/demux with dynamic phase alignmentbuilt into Stratix chips.

About 500 rack-to-rack cables, 2500 intra-rack cables (

Baseline Board (12Ux400 mm) PCB-back

Baseline Board PCB-front

Thermal vias pocket for corr chip heatsink

TechnologyUse central -48 VDC COTS power supply, DC-DC supplies on each board. Provides battery backup (5 min req).

24, 24 racks (2.5 x 3 x 7), each one holding up to 16 large boards in two crates, and Fanout Boards in 6U crate.

All hot-swappable, including fans.

On-line detection of communications errors, temps, voltagesremote shutdown/power cycle of any board through central PXI chassis/CPU.

Thermal mock-up test rack

TechnologySoftware:Embedded PC/104+ COTS CPU on each large board. Cheap, fast, easily available, easily replaceable, many vendors.RT LinuxUnix-style device drivers.Systemboardchip-level GUIs for initial testing. Useful during operations for continued debugging, low-level access.MCCCcentral host computer maps high-level requests from EVLA M&C to embedded processors, via XML messages.CPCCcentral power control computer.Backendarray of COTS PCs connected to Baseline Boards via commercial GigE switches.

ScheduleSlipped about 1.5 yrs from original 2001 schedulevarious reasons.

10 proto corr chips arrived June 20/06.

Baseline Boardprobably August/06some PCB fab problems have delayed delivery.

Station Boardsigned-off design for fab quotationprobably get assembled board late September/06.

Depending on how well testing goes:

1st prototype corrs at VLA/Jodrell Bankmid 07.Full production mid 2008. Full installation early 2009.Turn off old correlator end of 2009.

Summary ResponseType of correlations: allOutput req: spectral + continuumSpecial processing req: tunable sub-bands, high spec res, pulsar, 4/7-bit corr, RFI blanking/robust; RFI excision in backend CPUs eventually; possible RT RFI cancel.Input BW + digitization: 8x2GHz 3-bit; 4x1GHz 8-bit; dedicated fiber I/F on mezzanine card; Other BW/digitization supported; VSI I/O.#spec channels at max BW: 16k, increases by 2X with each decrease in BW by .#baselines correlated: 528 baselines, 163,840 cross-corrsIntegration times: min ~15 usec with binning. 11 msec standardscalable with backend and O/P link; unlimited max.Dynamic range: ~60 dB spectral.Scale: large, near limits of technology but not bleading edge.Technology: FPGAs, custom hardware, 1 ASIC, COTS CPUs/network, standard form factor.Scalability: unlimited but high flexibility not necessarily best approach for large N. 4X baselines possible with new ASIC, same architecture.Flexibility: high: BW/Nant/Nchan/Nbeam/Nbits; RT, nRT VLBI, auto, connected element, integration/dumping independent of system timing.Architecture: WIDAR (hybrid XF).RFI mitigation: Nbits, RT blanking, high dynamic range.