national radio astronomy observatory may 10, 2006 – evla advisory committee meeting...
DESCRIPTION
National Radio Astronomy Observatory May 10, 2006 Organization Management IPT Organization G. van Moorsel ((e)VLA/VLBA) B. Glendenning (ALMA), Michael Rupen (EVLA) & Debra Shepherd (ALMA) – Project Scientists for Computing J. McMullin, K. Golap (Project Management) S. Myers (SSG Project Scientist) Priorities (ALMA/EVLA) Scientific Completeness/Correctness, Usability Robustness, Performance Resources NRAO : 6.0 FTEs (including management; split 50/50 with ALMA) ALMA : 5.5 FTEs (Application development, Data Model, Data Model Interfaces, Data Capture process) NSF grant : 1.0 FTE (visualization; concludes )TRANSCRIPT
National Radio Astronomy ObservatoryMay 10, 2006 – EVLA Advisory Committee Meeting
Post-processing Overview
Bryan Butler (& Joe McMullin)(NRAO-AOC)
National Radio Astronomy Observatory May 10, 2006
Science Software Group
Mission• Primary purpose is to provide
facilities for post-observational scientific reduction of ALMA/EVLA data
• Requirement documents: • ALMA Offline Data Processing
Requirements• AIPS++ Audit• EVLA Data Post-Processing
Software requirements• Uses the legacy AIPS++ code
base for development• AIPS used for validation,
algorithm development (e.g., automated flagging)
Requirement Key Areas
• General Requirements and Interaction
• Interface• Data Handling• Calibration and Editing• Imaging• Data Analysis• Visualization• Special Features
• Spreadsheet/tracking on the combined projects’ requirements (common to both projects and the deltas between projects)
National Radio Astronomy Observatory May 10, 2006
Organization
Management• IPT Organization
• G. van Moorsel ((e)VLA/VLBA)• B. Glendenning (ALMA), • Michael Rupen (EVLA) & Debra
Shepherd (ALMA) – Project Scientists for Computing
• J. McMullin, K. Golap (Project Management)
• S. Myers (SSG Project Scientist)• Priorities (ALMA/EVLA)
• Scientific Completeness/Correctness, Usability
• Robustness, Performance
Resources
• NRAO: 6.0 FTEs (including management; split 50/50 with ALMA)
• ALMA: 5.5 FTEs (Application development, Data Model, Data Model Interfaces, Data Capture process)
• NSF grant: 1.0 FTE (visualization; concludes 2007.8)
National Radio Astronomy Observatory May 10, 2006
Resources: DetailsFTEs Comments
NRAO (SSG) 4.75 [5.75]
Project Scientist: Steve Myers
Sanjay Bhatnagar 0.75 Astronomer Algorithm Development (EVLA)
Kumar Golap 0.75 Astronomer Application Code Application code is common to both ALMA&EVLA
Joe McMullin 0.75 Astronomer Management
George Moellenbrock 0.75 Astronomer Application Code
David DeBonis 1.0 Software Engineer System Began 2004-05; depart:2006-01
Darrell Scheibel 1.0 Software Engineer System
Wes Young 0.75 Software Engineer System
ALMA 4.3 [5.3] 2.3 FTE in-kind
Gary Li 1.0 Software Engineer Application Code 0.5 FTE in-kind; Began 2004-03
Raymond Rusk 1.0 Astronomer Application Code
Tak Tsutsumi 1.0 Astronomer Application Code 1.0 FTE in-kind; Began 2005-03 (ACA)
Honglin Ye 0.5 Software Engineer Application Code Began 2005-07 (visualization)
Michel Caillat 0.8 Software Engineer ALMA Data model, simulation 0.3 FTE in-kind
Francois Viallefond 0.5 Astronomer ALMA Data model, simulation 0.5 FTE in-kind
Heiko Hafok 0.5 Astronomer ALMA Data Capture Began 2003-12
NSF grant 1.0
David King 1.0 Software Engineer visualization (concludes 2006-05)
Other 0.5
Urvashi Rao 0.5 Astronomer (student) Application code/Algorithm Dev (EVLA)
Began 2004-09 (9 month appt)
Total 10.05 [12.55]
National Radio Astronomy Observatory May 10, 2006
SSG Development
Status
• Multi-year plan for development of all Priority 1 ALMA science software requirements (2007.5)
• Project Office Page for development planning, accounting, transparency
• http://projectoffice.aips2.nrao.edu • 18 two-month development cycles
completed • Established strong feedback loop
with internal NRAO scientists• Established external scientist
testing/verification • Infrastructure
• RPM distribution• Framework (slide)
Milestones• Yearly NRAO VC and UC meetings• ASAC, EVLA ACM meetings
• ALMA CDR1 (Jun 03) - Passed• ALMA TST1 (Jan 04) – Passed
• Single Field Interferometry• ALMA CDR2 (Jul 04) - Passed• ALMA R2 (Oct 04) - Complete• ALMA TST2 (Nov 04) – Passed
• +Mosaics• ALMA TST3 (Apr 05) – Passed
• +synthesis/single dish combo• ALMA R2.1 (Apr 05) - Complete• EVLA TST1 (Jun 05) – Passed
• +Wide Field Imaging• ALMA CDR3 (Jul 05) - Passed• ALMA R3 (Oct 05) - Complete
• +ATF simulation (1st CASA test)• ALMA R3.1 (Mar 06) - Complete
• +User Interface Charette• Provided testing/detailed requirements
National Radio Astronomy Observatory May 10, 2006
Long-term Schedule (Presented at CDR2 in July 2004)
• 2003.5• Long-term plan
• 2004• Science: Imaging performance enhancements• Science: Flexible continuum subtraction• Framework: CVS code management• Data Interfaces: Draft AEDF document
• 2004.5• Science: SF reduction (TST1)• Science: Calibrater performance enhancements• Science: Calibrater flexibility (interpolations, transfer,
etc).• Framework: Framework requirements, plan• Data Interfaces: AEDF submitted for review, data
capture design• 2005
• Science: Mosaic reduction (TST2)• Science: Automated, interactive data editing• Framework: Library re-organization• Data Interfaces: Data Capture process
• 2005.5• Science: Single dish/interferometric (TST3)• Science: Simulator (telescope modeler, sampler,
visibility simulator)• Framework: Beta, subset, distribution with ACS• Data Interfaces: DRP filler
• 2006• Science: Complex mosaics (TST4) – changed to
ATF (single-baseline) test• Science: Full polarized primary beam imaging• Framework: Initial CLI, Python interface – added
a working group evaluation of the interface• 2006.5
• Science: Single Dish (TST5)• Science: Flexible line fitting• Framework: Testing with new framework – 1st
done in first half 2006• 2007
• Science: Simulations I (TST6)• Science: Multi-frequency synthesis• Framework: GUI development
National Radio Astronomy Observatory May 10, 2006
Coarse Timeline & Staging of Community Usage/User Support of CASA2006
• NRAO: NAUG testing (AIPS++)• Community: Project tests• ALMA: external science testing:
+single-baseline commissioning test) (CASA)
• ALMA: external science testing: +single dish reduction (CASA)
• ALMA: commissioning support (CASA)
• ALMA/EVLA: user interface review (CASA)
• EVLA: external science testing: +full polarization imaging; antenna pointing calibration
• 2006.5 AIPS++ frozen• ALMA: Pipeline Heuristics Use of
CASA2007
• NRAO: User support (CASA)• ALMA: commissiong support
(CASA)
• EVLA: external science testing: + RFI/automated flagging (CASA)
• ALMA: P1 SSRs complete2008
• EVLA: external science testing: + wide band calibration imaging (CASA)
• NRAO: user support (CASA)• ALMA: commissioning support• EVLA: P1 SSRs complete
2009-2010 • ALMA/EVLA: commissioning
support• Community: CASA
released/distributed for early ALMA/EVLA science
2011 • Community: CASA
released/distributed; full user support (ALMA/EVLA: P1/P2s)
National Radio Astronomy Observatory May 10, 2006
Framework Migration
• AIPS++• Glish interface; unknown,
unsupported outside of NRAO• Tasking system based on Glish• GUI system based on Glish/Tk;
limited widgets, not robust!• Difficult for external developers
to contribute• Multi-CD binary distribution• Large monolithic libraries with
cross dependencies• No namespace
• Freeze 2nd half of 2006
• CASA• Python interface (community
standard);IPython • Binding to Python, ACS; other
frameworks readily possible• Hierarchical set of small
libraries with clearly defined dependencies
• Namespace protection for integration with other code
• RPM distribution mechanism; auto-updates possible
• Robust• Inherits all application code
improvements in robustness and performance.
• Smaller memory footprint/startup time
National Radio Astronomy Observatory May 10, 2006
CASA usage status:
• Has been tested internally by NRAO scientists in preparation for the 1st ALMA test.
• Early demos provided bi-monthly to the NAUG at:• http://casa.nrao.edu/gettingstarted.shtml
• 6 scientists used and reported on it – enabled deployment for the ALMA test.
• Was deployed and reviewed by four ALMA testers• http://projectoffice.aips2.nrao.edu/ALMA2006.01/ALMA2006.01.html
• The user interface was reviewed and commented on by 8 NRAO scientists
• User Interface is being refined/further developed based on the user interface report:
• http://projectoffice.aips2.nrao.edu/uiwg_report.pdf• Will be used exclusively by the ALMA Pipeline Heuristics Team (second
half of 2006)• CASA will have replaced AIPS++ within NRAO (developers and NAUG
testers) this year.
National Radio Astronomy Observatory May 10, 2006
User Interface
• User Interface review• IPython interface to functionality, prototype parameter
setting interface and in-line help are in the right direction but:• Full needs are documented at:
• http://projectoffice.aips2.nrao.edu/uiwg_report.pdf
• Revised interface prototype (similar to IRAF epar environment); high priority
• Much work needed to provide astronomer-level documentation (collaboration with NAUG to develop this); high priority
• Development progress will be reviewed with the Fall EVLA test.
National Radio Astronomy Observatory May 10, 2006
Risks • Staffing
• Mitigation strategy: efficient use of existing resources; leverage off of ALMA development as much as sensible
• User Support• Mitigation strategy: Work with group of NAUG scientists such that they can
provide the front-line help.• User Interface
• Mitigation strategy: Cyclic feedback on progress and prototypes (revisited in June-July 2006; will be a review aspect of every formal/informal test)
• Requirement rot• Mitigation strategy: Update of EVLA requirements document to reflect current
planning/time tables.
• ASDM• Mitigation strategy: Multiple evaluations, early testing of needed modifications.
• Algorithm development/data rates – SB talk
National Radio Astronomy Observatory May 10, 2006
Overall Risk Mitigation
• Backup plan: AIPS• Requires:
• Continued support and targeted development• Support current VLA/VLBA users• Develop and deploy automated flagging algorithms to
community for testing/review