alma integrated computing team coordination & planning meeting #1 santiago, 17-19 april 2013...

ALMA Integrated Computing Team

Coordination & Planning Meeting #1Santiago, 17-19 April 2013

Telescope Calibration Planning

Dominique Broguiere

ICT-CPM1 17-19 April 2013

Presentation layout

Current status

Work in progress (Recession period)

Plans for next releases (R10.2 and R10.x)

Conclusions


Overall status : TelCal Group

ICT TelCal : 1.3 FTE Dominique Broguiere, lead, 0.8 FTE Jean-Christophe Roche, developer, 0.5 FTE

Subsystem Scientist Baltasar Vila-Vilaro

Deputy Subsystem Scientist Vincent Pietu (IRAM)

Other contributions Robert Lucas

• holography package

• Scientific support

•

Alexis Tejeda • TelCal contact in JAO group : help to identify Telcal problems

• Installation of Telcal standalone package on reduction machines


Overall status : Online calibration

Calibration Delivered Planning meeting 2012 Status 2013

Phase Cal x

Ampli Cal x - Need flux in ASDM- Use model for Solar System objects

- delayed until flux issue solved- done

Bandpass x

Delay x - Use any reference antenna - done

Pointing x - improved TotalPower pointing on planets (using disk size)

Focus x

Atmosphere x - Improve efficiency by using sky only scans

-implementation done, not tested

WVR x - On-line validation required - on-line validation in progress

Sideband ratio x - Using side band rejection, needs a little work in side band separation mode

- delayed until new sideband separation mode available

Skydip x - Needs autocorrelation on horizon source (work-around)

- nothing done; not used


Overall status : Offline calibration

Calibration Delivered Planning meeting 2012 Status 2013

Tower Holography x

Astro Holography x - Use on-line WVR correction - done

Pointing Model x

Antenna Positions x

Primary Beam x - To be done in CASA - done


Overall status (continuation)

Other delivered features : Include CLIC and RadioPoint in TelCal standalone package 64-bit TelCal standalone libraries for reduction machines ASDM update changes

New Bulk Data receiver Performance improvements in some engines (Pointing, Bandpass) and in

some offline tasks (asdmplot, RadioPoint) TelCalSpy improvements

Bug fixes/Tracking issues a lot of time (~ 50%) spent to analyze problems of fix issues


Work in progress (Recession period)

The features for R9.1.x have already been delivered. This page describes the work in progress to finish the validation of some important features as BulkData NT or to fix some issues.

New bulk data system (high priority) Analyze the possible causes of TelCal slowing down the data flow and fix them Deliver 64-bit version to remove the 2GB limitation on block sizes

Missing WVR data for arrays without 12m antennas (ACA) TelCal should be able to use a value of PWV provided by the

TelCalParameterTuning interface or retrieve the value determinated by another array, using the TelCalPublisher interface.

For offline reduction, it is necessary to add a parameter in tc_atmosphere task

Bug fixes CSV-2290 (baseband inversion in ACA atmospheric calibration \) Etc…


Plan for release R10.2

For this release, currently due for 05 July, a limited number of new features is planned because the priority is to consolidate existing things, to fix bugs and to keep some time to implement improvements proposed by CSV/DSO.

Improvements for off-line mode Goal : make data reprocessing easier Enable appending results to the current ASDM for all calibration intents Use casa rules for the definition of list of parameters (antenna list, bbpairs list, etc…) Keep the current dataset in memory to avoid time consuming, each time a task is run on

the same dataset

Improvements for on-line mode Priorities to be fixed with CSV (cf TelCal-CSV meeting) Possible improvements:

• On-line wvr correction : use the real frequency range of WVR spectral channels


Plan for releases R10.x (x>2)

The schedule and the priorities of the R10.x (x>2) features are not strictly defined. They will probably be updated following priority changes or new problems raised.

Improving code performances (computation time) Becomes more critical as the number of antenna increases Critical engines

• Pointing and WVR calibration : CONTROL is waiting for these results

Several possibilities studied by Alexis (June 2011)• Code optimization • parallelism• more efficient compilers (intel compiler)

Current status• Pointing and WVR engine performance still acceptable• Little work done on this topic due to other priorities

– optimization done in some engines– Parallelism (openmp) difficult to introduce due to incompatibilities with some libraries (slatec)


TelCal Performance analysis

Measurements issued from Telcal cpp container log (January-April 2013) Other time consuming operations in TelCal processing, outside Telcal code :

metaData retrieval from DataCapturer : 0.4-1.0 s ASDM build from its IDL representation : 1.0-2.0s

Engine Computation time Number of antennas Comments

Pointing 0.5s 40

WVR2s 32

Atmosphere 10s 32 Full resolution data

SideBandRatio 0.4s 32

AmpliCal 0.5s 32

PhaseCal 6s 32

Bandpass 10s 32

Delay 6s 32

Focus0.6s

40


Plan for releases R10.x (continuation)

Features dependant on Control schedule shadowing flag sideband separation subarrays (no specific development, just testing) multi-resolution mode (no specific development, just testing)

Improvements in calibration algorithms Need analysis and work from astronomers Topics :

• on-line WVR correction

• long baselines

• high frequencies

Error calculations Error calculations are missing in some results (.I.e Sideband ratio) Reduced chi square calculation gives incorrect results in Pointing engine

Flag noisy basebands/ polarization in calibrations Add the ability to flag some noisy baseband(s) or polarization in calibrations

Documentation : Engines and User guide


ALMA Phasing Project: TelCal involvement

Context Phasing algorithm will be implemented in TelCal by APP group Phasing calculation and atmospheric coefficients determination already available

in TelCal code

Support of following activities definition of new ASDM tables sequence diagram Phasing algorithm (using existing code of TelCal)


Conclusions

All the required functionalities are in principle already in TELCAL but some features still need to be tested or improved.

Communication between TelCal and other groups is important CSV/DSO : priorities for new developments or bugs fixing CONTROL : synchronization of TelCal developments or tests with CONTROL deliveries

(scan sequence, shadowing flag, etc…) ACS : new releases, bulkdata NT CASA : casa used in offline mode (task interface) and in online mode (solar objects

ephemeris) Etc…

Testing improvements TelCal Unit tests must be carefully updated and maintained Integrated tests or simulations are often not useful for TelCal because the data (0 or

random values) cannot be used for real calibrations. Would it be possible to use data from existing datasets?

alma integrated computing team coordination & planning meeting #1 santiago, 17-19 april 2013...

Documents