low-frequency, all-sky imaging with the paper array using aipy · movable (unburied tv cable) !...

Low-Frequency, All-SkyLow-Frequency, All-Sky

Imaging with the PAPERImaging with the PAPER

Array Using AIPYArray Using AIPY

A. Parsons1, D. Backer1, G. Foster1,

R. Bradley2,4, C. Parashare2, N. Gugliucci2,

E. Mastrantonio4, J. Aguirre3, D. Jacobs3,

C. Carilli5, A. Datta5,

M. Lynch6, D. Herne6, T. Colegate6

1 U. of California, Berkeley, 2 U. of Virginia, 3 U. of Pennsylvania,4 NRAO, Charlottesville, 5 NRAO, Socorro,6 Curtin U., Perth, Australia

The PAPER ArchitectureNon-tracking Crossed DipolesWide Bandwidth (125-205 MHz) !

Movable (unburied TV cable) !Smooth Beam

Flexible FPGA-based

Packetized CorrelatorFull-Stokes

Large # Ants (scalable)!Wide Band (up to 200 MHz)!

2048 Channel PolyphaseFilter Banks

4-bit Cross-Multipliers

AIPY: Model-based Imaging/CalibrationBootstrap Imaging/Calibration

W Projection, Multi-frequency Synthesis

Python-Based, Ties in MIRIAD, HEALPIXMeasurement Equation-based Parameter Fit

PAPER Antennas/Analog ElectronicsDeveloped by the Charlottesville (NRAO,UVA) Team

! Smooth beam (spatially and spectrally) ! Gain sensitive to ambient temperature

PAPER/CASPER Packetized Correlator

Two Dual 500 Msps ADCs + IBOB

Berkeley Wireless Research Center's BEE2

Two Calibration/Imaging Paths...

Bootstrap: Direct, fast, imperfect

Model-Fit: Clean, correct, expensive

“Sometimes you can't get started because you can't get started” – Don Backer

Parameter Space:

AIPY:AIPY: AAstronomicalstronomical

IInterferometry in nterferometry in PyPythonthon

an open-source toolkit targetingan open-source toolkit targeting

low-frequency interferometrylow-frequency interferometry

http://pypi.python.org/pypi/aipyhttp://pypi.python.org/pypi/aipy

What AIPY Is (and Isn't)!AIPY is:

AIPY isn't:

! A module adding tools for interferometry to Python (not visa versa)!! An amalgam of pure-Python and wrappers around C++ and Fortran! Object-Oriented! A collection of low- to mid-level operations (you write the program) !

! A toolkit (i.e. a grocery store, not a restaurant) !! In beta release (frequent updates/bug-fixes/changes) !

! A solution (it just helps you find it)!! A one-stop shop (it makes use of other open-source projects)!

! A replacement for other interferometry packages! Wedded to a file format (but only MIRIAD-UV, FITS currently supported) !

Philosophical Pedantry:

! Follow the (abbreviated) Zen of Python:! Explicit is better than implicit! Simple is better than complex! Complex is better than complicated

! Special cases aren't special enough to break the rules! Practicality beats purity! There should be one (and preferably only one) obvious way to do it, although that way may not be obvious at first unless you're Dutch! Now is better than never

! In the face of ambiguity, refuse the temptation to guess! Don't handicap the programmer; allow them all the rope they want! Don't hide data; return it (or at least grant access) at every turn

The ToolboxMODULES:

ant/sim/fit

const

coord

deconv

healpix

img

loc

map

miriadoptimize

src

----3rd party----

ephem

pyfits

numpy

pylab/matplotlib

geometry, phase / gain, amplitude / parameter fitting

physical constants

coordinate transforms (eq, ec, ga, precession, etc.)!

deconvolution (clean, lsq, mem, anneal) !

wrapper around healpix C++ (Max-Planck-Institut) !

imaging (gridding/weighting, W projection, beam calc) !

machinery for importing user-defined AntennaArrays

generation of all-sky maps, faceting

interface to MIRIAD filesparameter fitting, optimization code

lists of known sources and parameters

------------------------------------------------------------------------------

astrometry (relied upon heavily)!

reading/writing FITS files

numerical python (fundamental to everything)!

plotting, map projection

and example scripts for:

manipulating/visualizing MIRIAD files, flagging RFI, filtering out crosstalk,

fitting parameters, simulating arrays, making/viewing all-sky maps, phasing to/filtering out sources, imaging, etc.

Building a Simulator

AntennaArray

! set_jultime(...)!

! gen_phs(...) !

! gen_uvw(...) !

! sim(...) !

Antenna

! x,y,z

! dly,off

! bp_r,bp_i

Beam

! response(...) !

! <coeffs>

SrcCatalog

! compute(...) !! get_crd(...) !

RadioFixedBody

! compute(...) !

! ra,dec

! str,index

! angsize

RadioSpecial

! compute(...) !

! str,index

! angsize

Antenna

! x,y,z

! dly,off

! bp_r,bp_i

YourModule.py

! get_aa(...) !

! get_catalog(...) !

PAPER DeploymentsPGB: PAPER, Green Bank PWA: PAPER, Western Australia

38:25:59.24 N-79:51:02.1 W

Green Bank, WV

-26:44:12.74 S

116:39:59.33 EBoolardy, WA

! PGB-4: 2004-6, ants w/o flaps, 1 brd correlator! PGB-8: 2006-8, upgraded ants, packet correlator

! PGB-16: 2008-?, testing & characterization! Prototype facility for PWA deployments

! PWA-4: 2007, ants w/o flaps, 1 brd correlator! (PWA-32): 2009, ants w/ flaps, packet correlator

! (PWA-128): 2010?! Low-RFI, full-capability facility

Antenna Configuration

! Inner circle is PWA-4 configuration ! Outer circle is PGB-8 configuration! Shaded circles have radius = 10x elevation (dark is negative) !

Sample Data: 1 Day, 1 Baseline, PGB

TV/Aircraft TXFringes (Cas,Cyg,Sun)!

CrosstalkBeating sources

Intermittent TX

Satellite TX

Phase Amplitude (log)!

1-D “Delay” ImagingXRFI:! Remove model sky! Statistical thresholding! Manual flagging

DELAY TF:! iFFT of passband! Convolved by “delay beam”

1D CLEAN:! Compensate forholes in passband! Retain passband shape for self-cal

Delay/Delay-Rate Transform

! 1 hour of data with Cas A, Cyg A, Tau A! Phase to a source (here, Cas A)!! FFT of frequency axis = “Delay Image”! FFT of time axis = “Delay/Delay-Rate”! Cas A is confined to a region near origin! PSF determined by bandpass + time variabliity! Can recover bandpass + beam functions

Delay/Delay-Rate Transform! Two sources can show up at same delay for a given baseline! Over a time interval (say, 1 hr), can be differentiated by fringe rates (see below)!

! To prevent excessive blurring (changing fringe rate w/ time) best to phase to source before filtering! Clean delay axis before iFFT of time axis! Clean fringe rate axis to account for flagging of entire integrations

! Top to bottom: 3 baseline orientations! Left to right: delay bins, delay-rate bins, and combined

filter bins! DDR filters have 2 lobes of response: one that tracksyour source, one that sweeps across the sky

Recovering Bandpass and Beam

Kernels from DDR Transform

! Top plot: ! '+' = uncleaned bandpass recovered! heavy dots = cleaned bandpass! light dots = autocorr for comparison

! Bottom plot:! '+' = uncleaned delay image! line = cleaned delay image

! Cas A response versus time, Cyg A removed, 143MHz (top) and 165 MHz (bottom)!! '+' = beam response recovered using delay filter! dots = beam response recovered using DDR! lines = prediction of beam model

Modeling Beam of Dipole + Flaps

40dB zenith to horizon, 60 degree FWHM

Smooth spatially and vs. frequency

138 MHz 156 MHz 174 MHz

Model Beam vs. Source Strengths

Black: response toward Cas A

Blue: response toward Cyg A

Cyan: response toward Tau AGreen: response toward Vir A

Red: response toward Sun

Imaging with Non-Tracking Beams

Position-Dependent Weighting in Data Dirty Beam with Incorrect Weighting

! Only possible to weight for one track through primary beam! Dirty beam weighted for phase center imperfectly deconvolves away from center! Can evade problem with snapshot imaging

Using:8 Antennas at pos dec, 4 at negative3 days of data, 14s integrationsW projected, ~15 deg phase centers

138.8-174.0 MHz PWA-4/PGB-8

! Scale: log(Jy/beam) !! black->cyan = -1 to 1

! yellow >= 2, red >= 3

! Equatorial Mollweide Projection, RA=0 on right! Confusion at 80mJy/beam RMS at positive dec

! Sun at (1:10,+7:27), removed with DDR filter! Cyg,Cas,Vir,Tau peeled with ionospheric-dependent

solutions.

150 MHz PGB-8

! 1 day of 8-element single polarization data! Spans 138.8 to 174.0 MHz! Model subtraction of Cyg,Cas,Vir,Crab! Fit for ionospheric defraction

! Delay/F subtraction of Sun! Map has constant flux scale! Changing noise level due to beam response

RA,DEC = 12:00, 40:00

< "T > = 625 mK (10 mJy/bm)!

PGB-8 Thermal Noise Map

RMS Noise Map, log10(Jy/bm)!

Tsys = 250K

#FoV = 0.425 strradians

$0 = 162.5 MHz%$ = 35.2 MHz

#bm = 2.2e-5 strradians

Nant = 8 antennas

&int = 3 day observing = 3.5 hrs/pointing

Integrations added with alternating signs

PGB-8 System Temperature

Tsys (and predicted synchrotron) !

Tsky (confusion + synchrotron)!

Thermal noise (3 day integration)!

Using RMS pixel values from 4 patches nearRA=12:00,DEC=40:00 as a function of frequency, we

determine a perceived Tsky and thermal noise level.

Backing out integration, this gives us Tsys

Tsky

Thermal noise

“Beware the Jabberwock, my son!The jaws that bite, the claws that catch!”

-Lewis Carrol (1872)!

Power Spectra and Foregrounds

21 cm

EoR (z

=9.5) !

CMB Background

Confusion

Noise

Galactic Free-Free

Extragalac

tic Free-F

ree

Galactic Synchrotron

21 cm

EoR (z

=9.5) !

Power spectra at 146.6, 155.7, 164.5, and 173.3 MHz

(top to bottom w/ error bars), from 4 fields nearRA=12:00,DEC=40:00.

Point source dominated, starting to see synchrotronat lower frequencies, wavemodes

Summary!16 antennas deployed in PGB, 32 in PWA this fall!Analog system working well, going into production

!Packetized Correlator enabling build-out in staged deployments

! Working on QADC to lower digitizing cost, reducing to 100 MHz BW! Lowering integration time to ~8s

!AIPY-based processing pipeline maturing! Available at http://pypi.python.org/pypi/aipy

! Working on cluster/parallel computing

!Ionospheric refraction noticeable in data, compensated per source!Have a confusion/synchrotron-limited map (8.4K), ready for next tier of foreground removal

!Thermal noise in map (600 mK) continuing to integrate down

!Source removal currently limited by primary beam model! Using satellites to characterize per-antenna beam shapes

!Have not started investigating polarization or galactic synchrotron

Confusion NoiseNeed to balance array configuration between resolving and removing confusing sources (longbaselines), and optimizing for a power spectrum detection (concentrated core). Below, a map with nosources removed (left) and with the top 5 sources removed (right), with corresponding pixel histograms.

IonosphereTime scale: 10s Length scale: few km Refraction scale: few arcminutesPushes up data rates, need for instantaneous UV coverage (snapshot imaging) !Avoid dawn/dusk when ionosphere is highly variableAffects degree to which confusion noise can be suppressed

IonosphereTime scale: 10s Length scale: few km Refraction scale: few arcminutesPushes up data rates, need for instantaneous UV coverage (snapshot imaging) !Avoid dawn/dusk when ionosphere is highly variableAffects degree to which confusion noise can be suppressed

Black: Cas A, Magenta: Cyg A, 3 days of data, nighttime is shaded

38:25:59.24 N

-79:51:02.1 WGreen Bank, WV

38:25:59.24 N

-79:51:02.1 WGreen Bank, WV

-26:44:12.74 S

116:39:59.33 EBoolardy, WA

-26:44:12.74 S

116:39:59.33 EBoolardy, WA

Radio Frequency InterferenceSolution 1: Discretion is the better part of valor (run away) !Solution 2: Statistical thresholding of raw dataSolution 3: Manually flag contaminated channelsSolution 4: Delay/Delay-Rate FilteringSolution 5: Statistical thresholding after model subtraction

Boolardy, Australia Site Characterization

Light: PWA, 3 days of dataDark: PGB, 3 days of data

Blue:Peak, Green:Average, Red:Minimum

Interferometry FundamentalsOur Parameterized Measurement Equation:

Flat Field (Small Angle) Approximation:

! (u,v) coordinates represent the Fourier transform of the angular sky coordinates (l,m) !! Angular power spectrum directly measured in UV (visibility) domain! Inverse 2D FFT of gridded UV visibilities yields dirty image (convolved with synthesizedPSF) !! Synthesized PSF is inverse 2D FFT of visibility weights

Comparing with Other Experiments

MWA PAPER LOFAR 21CMA

Advantages/Philosophies:! Hands on: collect data early, address problems as they are encountered in data! Flexibility to adapt to new theory and new knowledge about foregrounds! Optimize smoothness of parameter space to allow faster, more precise fitting! Wide simultaneous bandwidth, full correlation of large numbers of antennas! Both fully functional prototype environment and low RFI location

Disadvantages:! Collecting area! Simulation

Unsure:! Spectral resolution vs. bandwidth! Group size

Sky |Vis| ø(Vis)!

Actu

al

Pri

ma

ry

Be

am

Re

str

icte

d

Vis

ibili

tie

s

For incoherent signalsummation, scaling of SNR with# samples depends on SNR ofsamples:

Sky-Vis Relationships

Implications formeasuring PS

Field of view (FoV)determines coherence regionin visibility domain.

Signal strength depends onFoV

Sample UV Optimized for 3 Scales

UV coverage Dirty Beam

Nested Triangular Configuration

with Outlying Calibrators

2008 May 13 Harvard-Smithsonian: 21cm Cosmology 38

Slices Through Image

Galactic Plane/Her AConfusion Sun Confusion

Virgo A

Power SpectraWhat an interferometer measures:

Related measures used in literature:

v

u

!̂

Sensitivity in Visibility DomainTsky ~ 160K#beam ~ 60 deg x 60 deg

%$ ~ 5 MHz#patch ~ 10 deg x 10 degNsamples ~ 81 baselines&s ~ 40 min (varies with patch size and baseline orientation) !

To hit SNR~1 in a UV pixel:' ~ 2 months for 0.5 mK

Minimum to constrain cosmic variance:- 3 sigma detection ~ 9 pixels around circle. - Using nested triangles with 3x3 antennas at each virtex,~ 64 antennas (being clever).

- Want to sample multiple rings to select for shape of power spectrum, so more like 128 antennas.- Need calibrator/imaging antennas, so more like 160 antennas.

BUT...

** UV pixel sampled is frequency dependent,undermining ability to subtract foregrounds.

What We Can Learn

What we can learn in first experiments:! Time of EoR degenerately constrains ! Dark Energy/Dark Matter (time for structure to collapse) + star formation rate! Confirm inside-out structure formation! X-ray heating by first quasars

What we can learn with SKA-class study:! Matter power spectrum through Dark Ages! Constrains dark energy, dark matter

(growth factor in collapse) !! Constrains time of radiation-matter

equality (transfer function) !! Geometry of Reionization! Star formation! Galaxy formation! Supermassive black hole formation! Feedback of reionization on

structure formation

“Richest of all datasets”

Inte

raction D

ista

nce S

cale

(lo

g)!

Time (log) !

Density P

ert

urb

ation (

log

)!

Inflationary Radiation Dominated Matter Dominated...

Recombination

Matter-Radiation Equality

Evolution of Perturbations

Simulated Power Spectra

Wouthuysen Field Effect

21 cm Spin Temp

Bounds on EoR (6<zEoR<12) !Gunn-Peterson Absorptionin high-z galaxies, (Becker, Fan)!

CMB Polarization from Thompson Scattering (Page, Spergel) !

Raw Sensitivity in Image DomainTsys = Tsky + Trx ~ 170K + 110K%$ ~ 1 MHzN ~ 256Aeff ~ 2#obs ~ 1 to 0.1 degrees

For necessary sensitivity:' ~ 1 month for 1 mK at 1 degree (l ~ 100)!

or more likely:' ~ 10 yrs for 1 mK at 0.1 degree (l ~ 1000)!

Have added flaps to! increase collecting area! reduce sky temp! suppress sidelobes of sources ! supress RF 21

cm EoR

(z=9.

5) !

CMB Background

Confusion

Noise

Galactic Synchrotron

Galactic Free-Free

Extragalac

tic Free-F

ree

Challenge: Starting from Scratch

Bootstrapping: ! Direct, imperfect, iterative! Do not rely (excessively) on priors

! Take advantage of wide bandwidth! Address degeneracies one at a time

“Sometimes you can't get started because you can't get started” – Don Backer

! Self-calibration needs single-source data! Source isolation requires calibration

! Parameter fitting needs a good starting guess! How do we get to first base?

The Problem:

Polarized Galactic Synchrotron

325 MHz Measured Polarization

150 MHz Power Spectrum Predicted from Komolgorov Turbulence

Faraday Rotation Screen:

! Smooth with frequency, but improperly calibrated linearpolarization can produce frequency-dependent structure.! Smooth power spectrum can allow further rejection.! Need a factor of 1000 suppression