fiorentino & rosa wavelength calibration in physical model based calibration pipelines....

Fiorentino & Rosa

Wavelength calibration in physical model based calibration pipelines.

Astronomical Data Analysis III S. Agata sui due Golfi, Naples, April 2004

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Overview

• IPMG at ST-ECF - Who we are. • HST Spectrographs & traditional pipelines. • Predictive calibration based on:

1. physical model of the instrument

2. simulated annealing technique for optimization

1. Show how we implement this into the science data pipeline.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

IPMG at ST-ECF

• Comprehensive empirical calibration pipeline already exists for the HST STIS Spectrograph

• We aim to improve those components which benefit from physically motivated corrections

• Current work includes 1. Wavelength Calibration

2. Calibration lamp line list - measurements at NIST

3. Detector Model repairing the Charge Transfer (CTE)

Fiorentino & Rosa 04/30/04 ADA III - Napoli

What is STIS ?

• STIS is the HST imaging spectrograph.– spatially resolved spectroscopy from 1150 Å to 10,300 Å at

low to medium spectral resolution– echelle spectroscopy (high resolution) in the ultraviolet.– time tagging of photons in the ultraviolet (high time

resolution).• Since 1997 on board HST • Unlikely to be replaced during the remaining HST lifetime

Fiorentino & Rosa 04/30/04 ADA III - Napoli

STIS optical layout

Fiorentino & Rosa 04/30/04 ADA III - Napoli

STIS Pipeline ‘calstis’

• calstis for spectra - series of modules that1. Control the data flow through the pipeline

2. Basic 2-D image reduction (e.g. bias subtraction)

3. Reject cosmic rays from CCD data

4. Process the contemporaneously obtained wavecal data to ascertain zero point shifts in the spectral and spatial directions

5. Extract 1 dimensional spectra – need to know geometry

6. Perform spectroscopic wavelength and flux calibration

7. Sum any CR-SPLIT and REPEATOBS exposures.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Pipeline Flow for Spectroscopic Data

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Where the empirical wavelength calibration is currently used.

• Determine MSM offset from wavecal. • Its purpose is to find the offset of the spectrum from the

expected location, owing to non repeatability of the MSM.

• Spectroscopic Calibration and Extraction.• 1-D spectral extraction. A spectrum is extracted along a

narrow band, summing over the cross-dispersion direction and subtracting background values to produce a 1-D array of fluxes for each spectral order.

• In order to calculate the offsets and to assign wavelengths the empirical pipeline uses bi-dimensional polynomial dispersion solutions. Therefore it can only apply linear translations (offsets) , but not rotations.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

STIS Auto Wavecals

A standard wavecal is usually only a few seconds long.•X and Y displacements based on a few lines.

• X and Y are not the same on the whole detector because, the differential rotation (“splaying‘”) of individual echelle orders resulting from the combined effects of the echelle and cross-dispersing elements, cause different orders to be differentially rotated (“splayed”).

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Short and Long Wavecal

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Short and long wavecal (detail)

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Traditional Pipeline’s accuracyvs. Enhanced calibration.

•1)Image shift (-3,+3) pixels due to the MSM.

•2)Thermal effects cause the spectrum to drift by about 0.1 pixels up to 0.35 pixels per orbit.

•3)Shift not always precisely determined due to, for instance, a short wavecal.

1)The Absolute Wavelength zero points shifts are not predicted with the traditional calibration (errors in E140H up to 1.3km/s 0.5-1.0 Pixel).

We aim to reach 0.1 pixel precision.

2)We will have an homogenous calibration for each mode and overall the lifetime of STIS.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

The alternative: predictive calibration

“The calibration of astronomical data can be significantly improved by constructing instrument models which incorporate as fully as possible a knowledge of optical and detector physics”

• A typical example is the wavelength calibration – • empirical dispersion relations should be replaced by a physical

model (simple ray trace) of the spectrograph• This usually yields better than 0.1 % accuracy (1 pix in 1000)

straight away• Distortions may be added to go to sub-pixel accuracy

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Predictive Calibration: Echelle model & Simulated Annealing.

•Mathematical model with about 35 parameters which need to be optimized. Derivatives cannot be easily formulated and analytical inversion is impossible. •Simulated Annealing (SA) is one of the technique which cope with such a problem.•Although easy in principle, its implementation may not be trivial.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Simulated Annealing.

• SA exploits an analogy between the way in which a metal cools and freezes into a minimum energy crystalline structure and the search for a minimum in a more general system.

• SA don’t get trapped at local minima.

• The algorithm accepts also changes that increase objective function f with a probability following the Boltzmann probability distribution.

• Not all sets of parameters which minimize the cost function are physically acceptable therefore our SA algorithm will make those configurations extremely costly.

•

ADA III - Napoli

Randomize according to the T

Better than the current solution ?

Start

Store it. Yes

Accept or reject based on Boltzmann Probability

Distribution.

NO

Random tries > Max_Iteration ?No

Decrease Temperature

Min Temperature reached ?

Yes

Yes

Exit

No

SA Data Flow

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Fitlines

Mode,Slit,Central wavelength,Catalog lines

Wavecal (FITS Image)

Calculates the predicted X,Y lines positions on the detector.

Output: observed X,Y lines positions on the detector.

Center a box around the predicted lines positionsand estimate the exact observed positions by fittingthe line shape with a Gaussoid.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

STIS Anneal

Annealing Algorithm to optimizethe set of parameters for each configuration.

repeat the annelingprocess.

If config file is good store it

If not good

One or more sets oflines positions from

Fitlines.

One or more configuration

file each with 35 parameters

ADA III - Napoli

Master CatalogMode, CenWave,

SlitPos, Config File What else?

ExtractSubCatalog

Fitlines

SubCatalog

Wavecal exposure

STISAnneal

(X,Y)measured

Store it or not Store it ?

New Cfg File

If good store it

Not Yet good ?

New Master ?

If not good

Reject Bad lines, Change Weights,Intensity thresholds

Not possible to anneal all parameters at the same time therefore needs to identify set of them to be annealed.

Learning curve for a new instrument.

Reference Files Data Flow

Fiorentino & Rosa 04/30/04 ADA III - Napoli

SA into the Science pipeline

• Once all the reference files have been determined we will be able to predict, for a given configuration and for each order and lambda, the position on of the corresponding line on the detector.

• However, in order to cope with the non repeatibility of the MSM, another SA need to be run each time a science exposure is taken.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

SA into the Science Pipeline

Final set of config files for each mode, Central wavelength, slit, Epoch.

User selects a mode, Central wavelength …

Config File extracted but MSM positions may no longer be accurate.

Run Fitlines + Fast-Annealin order to calculate the the actual MSM position

Run Calstis

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Discover Dependencies

Run Fitlines +Anneal

Select a mode and fetch all the wavecals.

N Wavecals extracted.

N Config files

Analyze config files against environmental conditions.

Relation T, Focal Length ?

Enhance the model

Number of config files reduced.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Modeling Echelle Spectrographs

At the ST-ECF we are currently implementinga STIS model based on first optical principles. Itincorporates off-plane grating equations and 3D rotations inorder to account for line tilt and order curvature.Similar formalism had already been partially implementedand applied for FOS(HST), UVES, CASPEC pipelines withsignificant science improvement.See Ballester and Rosa A&AS 126, 563-571 (1997).

www.stecf.org/poa/pcrel/scicase.htmlwww.eso.org/observing/dfo/quality/Messenger/UVES_Messenger_101.html

Fiorentino & Rosa 04/26/04 ADA III - Napoli

Good only for HST spectrographs ?

• Predictive calibration can be applied to any spectrograph.

• We aim to implement the STIS pipeline such that can be easily re-used for other spectrograph (i.e. Object oriented code).

• Although this is just a part of a pipeline…

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Status of the STIS implementation

• Prototype implementation finished (C++).• Wavelength calibration translated into C in order to import

into the existing IRAF/C STIS pipeline. • Reference files production is in C++ and does not need to

be translated since it is an offline tools. • Future items:• Analyze science cases in order to test the CE_CALSTIS. • Enhance the model (MSM model).

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Referenced articles & URL links

• Ballester and Rosa Astron. & Astrophysic. Suppl.Ser 126, 563-571 (1997).• Ballester & Rosa ADASS XIII, Instrument Modeling in Observational Astronomy. • Kirkpatrick, S., C. D. Gelatt Jr., M. P. Vecchi, "Optimization by Simulated

Annealing",Science, 220, 4598, 671-680, 1983.• Metropolis,N., A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, "Equation of

State Calculations by Fast Computing Machines", J. Chem. Phys.,21, 6, 1087-1092, 1953.

• URL links:• www.stecf.org/poa/pcrel/scicase.html• www.stecf.org/poa/index2.html• www.eso.org/observing/dfo/quality/Messenger/UVES_Messenger_101.html

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Science Improved: FOS case.

• Effect of the improved dispersion relation. • We looked at the interstellar absorption lines imprinted on

the spectrum of a low red-shift quasar (PG 1115+407, PI B. Wills).

• There were two separate FOS observations red and black dots. All measurements have been reduced to barycentric velocities.

• The solid line is the weighted average of HI 21 cm line observations with the dashed lines indicating the range of velocities found in the line of sight.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Standard Calfos dispersion solution

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Improved dispersion solution.

Fiorentino & Rosa 04/30/04 ADA III - Napoli

STIS Spectroscopic Capabilities

•

Fiorentino & Rosa 04/30/04 ADA III - Napoli

Traditional Pipeline’s accuracyvs. enhanced calibration.

•Image shift (-3,+3) pixels due to the MSM.

•Thermal effects cause the spectrum to drift of about 0.1 pixels up to 0.35 pixels per orbit.

•Shift not always precisely determined due to, for instance, a short wavecal.