tdp calibration and processing group (cpg): activities and...

TDP calibration and processing group (CPG): activities and status

Athol Kemball (for CPG)University of Illinois at Urbana-Champaign

[email protected]

US SKA Nov 08

Primary CPG goal

Provide a quantitative cost and feasibility model for calibration and processing elements of the US TDP LNSD SKA design.

Contribute to a construction-ready Phase 1 SKA design and proposal.

US SKA Nov 08

SKA design & development phase (2007-2011)

SKA science requirements

SKA technical specifications, e.g. sensitivity, survey speed, time, spectral, and angular

resolution, etc.

SKA reference design, e.g.

receptor, array configuration, receptor and

receivers, signal transport and

correlation, cal & processing, data

management

Antenna design elements e.g.

receptor, receivers,

antenna etc..

Calibration and processing

design elements e.g. calibration and imaging algorithms,

scalability, etc..

Design drivers• Hardware design choices will define

calibration and processing performance (e.g. dynamic range), cost, and feasibility.

• In turn, need to identify calibration and processing constraints on hardware designs.

US SKA Nov 08

Calibration and processing design elements

CPG goals• Determine feasibility of calibration and

processing required to meet SKA science goals;

• Determine quantitative cost equation contributions and design drivers as a function of key design parameters (e.g. antenna diameter, field-of-view, etc)

• Measure algorithm cost and feasibility using prototype implementations

• Demonstrate calibration and processing design elements using pathfinder data.

US technology development emphases• Large-N small-diameter (LNSD) parabolic

antenna design, wide-band single-pixel feeds, mid to high frequency range.

• Close liaison with current international and national efforts.

US SKA Nov 08

Pathfinder demonstrations

Need complementary approaches:• Pathfinder demonstration; each

pathfinder illuminates different calibration and processing problems.

• Design studies and simulations to scale.

Special emphases• Strong ties to most SKA pathfinders,

national and international.• TDP will work substantially with ATA,

EVLA, MWA for pathfinder demonstration

• Agreement with EVLA/NRAO regarding EVLA test data requests

(ATA)

(EVLA)

US SKA Nov 08

Current CPG membership

• Athol Kemball (Illinois) (Chair)• Sanjay Bhatnagar (NRAO)• Geoff Bower (UCB)• Jim Cordes (Cornell; TDP PI)• Shep Doeleman (Haystack/MIT)• Joe Lazio (NRL)• Colin Lonsdale (Haystack/MIT)• Lynn Matthews (Haystack/MIT)• Steve Myers (NRAO)• Jeroen Stil (Calgary)• Greg Taylor (UNM)• David Whysong (UCB)

Calgary.... .

. .Cornell

NRLUIUC MIT

NRAOUCB UNM→

→

→

→

US SKA Nov 08

CPG event timeline Oct 07 – Sep 08

OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEP

CPG formation

Project execution plan

Execution plan implementation

F2F planning meeting (Urbana)

URSI 2008 LNSD session

CPG meeting

CPG F2F meeting (Perth)

SKA CALIM 08

CPG F2F meeting (Washington DC)

URSI GA meetings (Chicago)

US SKA Nov 08

CPG event timeline Sep 08 →

SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

Implementation

SAC review

PDRA Lynn Matthews starts

PrepSKA WP2 planning meeting

US SKA Madison

URSI 08 NC (Boulder)

PDRA David Whysong starts

Student conference (CT)

SKA DR conference (CT)

US SKA Nov 08

CPG project execution planCPG work breakdown structure • WBS 2.0: General• WBS 2.1: Signal transport• WBS 2.2: Calibration algorithms• WBS 2.3: Imaging, spectroscopy, &

time-domain imaging• WBS 2.4: Scalability, & high-

performance computing• WBS 2.5: RFI• WBS 2.6: Surveys• WBS 2.7: Data management

Cross-cutting goals• LNSD feasibility:

– e.g. dynamic range error budget• LNSD cost equation contributions (per

calibration and processing technology)

OVERALL TDP GOALS

WBS 2 CAL & PROCESSING GOALS

1. Feasibility

2. Cost equation contribution

3. Design driver identification

4. Pathfinder demonstration

Calibration & processing issue #1




… X Design parameters (e.g. diameter, mid-range upper cutoff frequency, amongst others)

Calibration and processing issues Design parameters

PrepSKA/CDIT SKA pathfinders

US SKA Nov 08

CPG FTE resources

PostdocSenior

• Directly TDP-funded over 4 years:– Postdoctoral fellows (11.0 FTE)– Senior personnel or summer

salary support (2.4 FTE)– Total: 13.4 FTE

• In-kind contributions via partnerships and collaborations.

• Hiring postdoctoral fellows in this area has been challenging:

– FTE deficit in year #1

• Hiring progress has been made:– MIT postdoc started 09/08 (Lynn

Matthews)– UCB postdoc will start 11/17

(David Whysong)– UIUC postdoc in process for fall

2008.

00.5

11.5

22.5

33.5

4

FTE

Year 1 Year 2 Year 3 Year 4

PlannedHired

US SKA Nov 08

WBS 2.0.1: Science goal mapping to C&P requirements

• UCB lead (G. Bower)• Hired postdoc: David Whysong:

– Currently NRAO postdoc– Moving to Berkeley in Nov 08– Whysong spent one week in

Berkeley to learn about ATA & TDP (July 08)

• Science requirements– Bower &Whysong met with

Gaensler to discuss requirements in SKA Memos 100 (July 08); emphasis on dynamic range requirements

– Bower, Lazio, Lonsdale, Kemball met to discuss joint approach between TDP & SKA SWG and translation of results into engineering requirements(Aug 08)

US SKA Nov 08

WBS 2.1.x: Signal transport

• WBS 2.1.1: Real-time imaging and calibration (UCB/MIT):

– Prototyping real-time imaging (ATA: Wright).

– Simultaneous in-beam calibrator observations for precision VLBI astrometry and faint source detection (ATA).

– Global Sky Model, decimation, and image-based archiving (MWA)

• WBS 2.1.2: Sidelobe stability & beam-forming (UCB):

– Test stability, calibration, interference rejection of phased array signals for SKA station use (ATA)

US SKA Nov 08

WBS 2.0.2: Instrumental simulation infrastructure

• MIT lead (S. Doeleman)• CPG Postdoc position filled (9/08): Lynn

Matthews• Summer student (2008) • MAPS (MIT Array Performance Simulator)

– Extended to process SKA configurations.

– All-sky image capability.– Time-variable station beam-shape

implementation.– In partnership with MWA program.– Functioning package for CPG

simulations. – Developing suite of sample skies for

simulation.– Provides data sets for FOV shaping

algorithm development.– Postdoc recently joined to work on

this.(MWA schematic)

US SKA Nov 08

• LNSD data rates (Perley & Clark 2003):

where D = dish diameter, B = max. baseline, ∆ν = bandwidth, and ν = frequency

• Wide-field imaging cost ~ O(D-4 to -8) (Perley & Clark 2003; Cornwell 2004; Lonsdale et al 2004).

• Full-field continuum imaging cost (derived from Cornwell 2004):

• Strong dependence on 1/D and B. Data rates of Tbps and computational costs in PF are readily obtained from underlying geometric terms.

• Spectral line imaging costs exceed continuum imaging costs (further multiplier )

• Possible mitigation through FOV tailoring (Lonsdale et al 2004), beam-forming, and antenna aggregation approaches (Wright et al.)

WBS 2.3.1: Imaging cost equation contributions

tNNN

TBpsV antantchanvis

∆−

=⎟⎟⎠

⎞⎜⎜⎝

⎛1210

)1(20

⎟⎠⎞

⎜⎝⎛ ∆

⎟⎠⎞

⎜⎝⎛

⎟⎟⎠

⎞⎜⎜⎝

⎛ −

νν2

1410~DNB

TBpsVvis

1500273.02

2 2103.22−

⎟⎠⎞

⎜⎝⎛≈⎟

⎠⎞

⎜⎝⎛ D

B

ant DBN

PFC

0

500

1000

1500

2000

D=12.5m, B=5km

D=12.5m,B=35km

D=6m, B=5km

TB per hour

0

2

4

6

8

10

D=12.5m,B=5km

D=12.5m,B=35km

D=6m,B=5km

Peak PF

chanN

(Kemball et al. (2008), CPG Memo #1)

US SKA Nov 08

WBS 2.1.3: Central processor architecture & transport (MIT): FOV shaping• FOV shaping reduces sidelobe contamination. • Initial tests use 4 station L-band data from MERLIN of 3c343/3c343.1

pair (29’ separation)

No Convolution Convolution• Jinc function convolution of UV data removes 3c343.1 source and

sidelobes.

US SKA Nov 08

WBS 2.1.3: Central processor architecture & transport (MIT): FOV shaping

• New MERLIN Data Set– MERLIN has proper baseline length dynamic range to support FOV

shaping by UV convolution.– New 6-station observations completed -this week - and at Haystack

for processing: observed 3c343/3c343.1 and M31 (to test algorithm with more background sources).

– Software package written to extract correlated data and perform convolution.

– Involves undergraduate student and newly arrived postdoc.– Tests on new data set:

• effects of RFI on algorithm (will imperfect convolution kernel compromise FOV shaping?)

• Tests of optimal convolution kernel (Gaussian, Jinc, Spheroidal …)

US SKA Nov 08

SKA dynamic range assessment – beyond the central pixel

• Current achieved dynamic ranges degrade significantly with radial projected distance from field center, for reasons understood qualitatively (e.g. direction-dependent gains, sidelobe confusion etc.)

• An SKA design with routine uniform, ultra-high dynamic range requires a quantitative dynamic range budget.

• Strategies:– Real data from similar pathfinders (e.g.

ATA, EVLA) are key.– Simulations are useful if relative dynamic

range contributions or absolute fidelity are being assessed with simple models.

– New statistical methods:• Statistical resampling (Kemball et al

2005, 2008) and Bayesian methods (Sutton & Wandeldt 2005) offer new approaches.

– Best-practice survey: Underway (Kemball)

Feasibility: dynamic range assessment

( )S ρ

Noordarm et al 1982

3C84 WSRT 1.4 GHz 10,000:1

Geller et al 2000 1935-692 ATCA 1.4 GHz 77,000:1

de Bruyn & Brentjens 2005

Perseus WSRT 92 cm 400,000:1

de Bruyn et al, 2007

3C147 WSRT 1.4 GHz 1,000,000:1

Dynamic range

(de Bruyn and Brentjens, 2005)

US SKA Nov 08

Direction-dependent variance estimation methods

M1: Np=1; ∆t = 60 s

M2: Np=1; ∆t = 150 s

M3: Np=1; ∆t = 300 s

M4: Np=2; ∆t = 900 s

S1: delete frac. 12.5%

S2: delete frac. 25%



MC M1 M2

M3 M4 S1

S2 S3 S3

(Kemball et al. (2008), AJ, (joint TDP paper))

Truth from MC simulation Other estimates from statistical methods

US SKA Nov 08

WBS 2.4: Scalability• Sustained petascale calibration and imaging

performance for SKA requires:– Demonstrated mapping of SKA calibration

and imaging algorithms to modern HPC architectures, and proof of feasible scalability to petascale: [O(105) processor cores].

– Remains a considerable design unknown in both feasibility and cost.

• Need to leverage existing NSF support and existing leading-edge HPC experience in the physical sciences:

– Radio imaging efforts at UIUC (incl. SKA) will move to new Center for Extreme-scale Computing

• Matching UI investment in NSF-funded petascale system “Blue Waters”.

• Up to ten interdisciplinary EECE / CS and applications faculty

• Collective access to pre-cursor petascale systems and engineering and computing teams

0

20000

40000

60000

80000

100000

10 TF 100 TF 1 PF

No processors

US SKA Nov 08

Joint SKA calibration and processing efforts

US SKA Nov 08

• Meeting goal: define PrepSKA process & structure; refine WBS for WP2• Initial PrepSKA computing program task orientation:

• Proposed (and adopted) reorganization:

• Also P9:T2 becomes:

PrepSKA WP2 liaison meeting 11/08 (SPDO)

• Rationale:– Calibration and processing

technologies can be decomposed optimally by receptor type and implied frequency

• Matches regional activities more closely, so reduces management complexity

– New focus in T2 to deal with important SKA problem and to leverage available additional resources

– Better mix of tasks and reflection of TDP resources committed

US SKA Nov 08

OVERALL PREPSKA DESIGN AND COST MODEL

PREPSKA WP2 CALIBRATION AND PROCESSING DESIGN AND COST

MODEL

1. Feasibility

2. Cost equation contribution

3. Design driver identification

4. Pathfinder demonstration

Calibration & processing

issue #1



Calibration & processing

issue #4

Design parameters (e.g. diameter, mid-range upper cutoff frequency, amongst others)

Calibration and processing issues Design parameters

Design and/or evaluation studiesSKA pathfinders PrepSKA needs

good coverage of design parameter space

Priority of individual calibration and processing design elements depends on factors such as receptor type, frequency, etc.

Central, synthesized and updated design model

US SKA Nov 08

PrepSKA integration issues• Next steps for CPG:

– Refine statements of work in PrepSKA WP2 / P9– Ensure existing CPG WBS matches TDP SOW

• Good agreement in scope, but prioritization needs attention

• Challenge:– PrepSKA has important technology decision point in mid-2010– 18 months earlier than end-point of TDP/CPG delivery– So, can’t deliver full CPG WBS on PrepSKA time-scale. Options:

• Iterative refinement of LNSD CPG cost-feasibility model already adopted in CPG earlier this year; deliver best model available mid-2010

• Abbreviate or truncateTDP PrepSKA work plan

US SKA Nov 08

CPG 2009 planning

• Detailed milestones already discussed and filed with NSF at time of last annual report

• Immediate focus is first iteration (v0.1) of LNSD cost-feasibility model

• In implementation phase of project execution plan, but have been affected by hiring delays. 0

0.51

1.52

2.53

3.54

FTE

Year 1 Year 2 Year 3 Year 4

PlannedHired

US SKA Nov 08

END

tdp calibration and processing group (cpg): activities and...

Documents