spire spectrometer pipeline descriptionherschel.esac.esa.int/twiki/pub/spire/science... ·...

SPIRE Spectrometer PipelineDescription

SPIRE-BSS-DOC-002966

Trevor Fulton

Issue 1.205 December 2008

SPIRE Spectrometer Pipeline DescriptionTrevor Fulton

Table of Contents1. Introduction ...................................................................................................................... 1

1.1. Acronyms .............................................................................................................. 11.2. Symbols ................................................................................................................ 21.3. Scope of Document ................................................................................................. 21.4. Documents ............................................................................................................. 2

1.4.1. Applicable Documents ................................................................................... 21.4.2. Reference Documents .................................................................................... 3

1.5. Document History ................................................................................................... 31.6. Change Record ....................................................................................................... 4

2. SPIRE Spectrometer Pipeline Overview ................................................................................. 72.1. Spatial Sampling ..................................................................................................... 72.2. Spectral Resolution .................................................................................................. 7

3. SPIRE Spectrometer Building Block Pipeline .......................................................................... 93.1. Modify Timelines. ..................................................................................................10

3.1.1. First Level Deglitching .................................................................................113.1.2. Removal of Electrical Crosstalk ......................................................................123.1.3. Detector Non-Linearity Correction ..................................................................123.1.4. Clipping Correction ......................................................................................133.1.5. Time Domain Phase Correction ......................................................................143.1.6. Removal of correlated noise due to bath temperature fluctuations ..........................15

3.2. Create Interferograms .............................................................................................163.2.1. Interferogram Creation ..................................................................................16

3.3. Modify Interferograms ............................................................................................183.3.1. SCAL and Telescope Correction .....................................................................193.3.2. Interferogram Baseline Correction ..................................................................203.3.3. Second Level Deglitching ..............................................................................203.3.4. Channel Fringe Correction .............................................................................213.3.5. Phase Correction .........................................................................................213.3.6. Apodization ................................................................................................23

3.4. Transform Interferograms ........................................................................................243.4.1. Fourier Transform ........................................................................................24

3.5. Modify Spectra ......................................................................................................263.5.1. Flux Conversion ..........................................................................................273.5.2. Removal of Optical Crosstalk .........................................................................273.5.3. Spectral Averaging .......................................................................................28

4. SPIRE iFTS Spectral Mapping ............................................................................................304.1. SPIRE iFTS Spatial Regridding ................................................................................30

A. Appendix .......................................................................................................................33A.1. First Level Deglitching Description ...........................................................................33A.2. Radiation Incident on the SPIRE Spectrometer Detectors ..............................................34A.3. Double-sided and Single-sided Interferograms ............................................................38

A.3.1. Double-sided Interferograms .........................................................................38A.3.2. Single-sided Interferograms ..........................................................................38

A.4. Available Apodizing Functions ................................................................................39

iii

Chapter 1. Introduction1.1. Acronyms

Table 1.1. Acronyms

Short Form Full Name

ADC Analog to Digital Converter

BSM Beam Steering Mirror

FFT Fast Fourier Transform

FT Fourier Transform

FTS Fourier Transform Spectrometer

iFTS imaging Fourier Transform Spectrometer

ILS Instrument Line Shape

IPAC Infrared Processing and Analysis Center

JPL Jet Propulsion Laboratory

LHS Left Hand Side

LPF Low Pass Filter

LVDT Linear Variable Differential Transformer

MPD Mechanical Path Difference

NHKT Nominal HouseKeeping Timeline Product

OPD Optical Path Difference

QCP Quality Control Pipeline

RC Resistor-Capacitor

RHS Right Hand Side

RMS Root Mean Square

RSRF Relative Spectral Response Function

SBS Spectrometer Beam Splitter

SCAL Spectrometer Calibrator

SDI Spectrometer Detector Interferogram Product

SDS Spectrometer Detector Spectrum Product

SDT Spectrometer Detector Timeline Product

SLW Spectrometer Long Wavelength Detector Array

SMEC SPIRE Spectrometer Mechanism

SMECT Spectrometer Mechanism Timeline Product

SOF Spectromter Observation Format

SPG Standard Product Generation

SPIRE Spectral and Photometric Imaging REceiver

SPP SPIRE Pointing Product

SSW Spectrometer Short Wavelength Detector Array

TBD To Be Determined

TBW To Be Written

WTMML Wavelet Transform Modulus Maxima Lines

ZPD Zero Path Difference

1

1.2. Symbols

Table 1.2. Description of symbols used throughout this document

Symbol Description

I(σ) Vector of calibrated flux as a function of wavenumber

Ii(σ) Vector of calibrated flux as a function of wavenumber for a given detector, i

In-i

(σ) Vector of calibrated flux as a function of time for a given detector, i, for a given mechanism scan,n

Ii(σ

k) The kth calibrated flux sample for a given detector, i

n(t) Vector of scan numbers as a function of time

P(t) Vector of pointing positions as a function of time

Pi(t) Vector of pointing positions as a function of time for a given detector, i

σ Regularly-sampled wavenumber vector

t Regularly-sampled time vector

V(t) Vector of voltages as a function of time

Vi(t) Vector of voltages as a function of time for a given detector, i

Vn-i

(t) Vector of voltages as a function of time for a given detector, i, for a given mechanism scan, n

Vi(t

k) The kth voltage sample for a given detector, i

V(σ) Vector of voltages as a function of wavenumber

Vi(σ) Vector of voltages as a function of wavenumber for a given detector, i

V(x) Vector of voltages as a function of optical path difference

Vi(x) Vector of voltages as a function of optical path difference for a given detector, i

x Regularly-sampled optical path difference vector

z(t) Vector of spectrometer mechanism positions as a function of time

1.3. Scope of DocumentThis purpose of this document is to present an outline of the processing steps in the SPIRE spectro-meter pipeline. The processing modules presented in this document follow those presented in Sec-tion 3 of [AD01], which describes the steps that are common to both the SPIRE spectrometer andphotometer pipelines.

1.4. Documents1.4.1. Applicable Documents

AD01 M. Griffin, The SPIRE Analogue Signal Chain and Photometer Detector Data Pro-cessing Pipeline, SPIRE-UCF-DOC-002890, Issue 5, 01 August 2008.

AD02 C. Pearson, SPIRE Pipeline Description, SPIRE-RAL-DOC-002437, Issue 1.0, 02August 2008.

AD03 SPIRE Observers' Manual, HERSCHEL-HSC-DOC-0789, Version 1.2, 11September 2007, http://herschel.esac.esa.int/Docs/SPIRE/html/spire_om.html.

AD04 K. J. King, SPIRE Data Product Definition

AD05 Operating Modes for the SPIRE Instrument, SPIRE-RAL-DOC-0000320, Issue3.3, 24 June 2005.

Introduction

2

AD05 Polehampton, E., "SPIRE FTS Mapping Modes", SPIRE-RAL-DOC-002801, Is-sue 1.0, 16 January 2007.

AD07 SPIRE Pipeline Mask Policy, SPIRE-BSS-DOC-003127, Issue 1.1, 09 October2008.

1.4.2. Reference Documents

RD01 Jean-Paul Baluteau, PFM3b data: some SMEC or FTS performances, Presentationto SDAG 15, 10 July 2006

RD02 Sensitivity of the SPIRE Detectors to Operating Parameters, SPIRE-UCF-DOC-002901, 14 November 2007

RD03 D. A. Naylor and M. K. Tahic, "Apodizing functions for Fourier transform spec-troscopy," J. Opt. Soc. Am. A 24, 3644-3648 (2007)

RD04 F. Pinsard, "HERSCHEL/SPIRE Detector Control Unit Design Document",SPIRE-SAP-PRJ-001243, Issue 1.0, 11 July 2005.

RD05 Knutsson, H. and Westin, C. F., "Normalized and Differential Convolution Meth-ods for Interpolation and Filtering of Incomplete and Uncertain Data," Proc. Com-puter Vision and Pattern Recognition, 512-523, (1993).

RD06 Spencer, L. D., et. al., "Performance evaluation of the Herschel/SPIRE instrumentflight model-imaging Fourier transform spectrometer," in Space Telescopes andInstrumentation I: Optical, Infrared, and Millimeter Wave, 7010, Proc. SPIE,(2008)

RD07 Fulton, Trevor, "SPIRE Spectrometer Time Constant Derivation,", SPIRE-BSS-DOC-000XXXX, Draft 0.1, 24 October 2008.

RD08 Fulton, Trevor, "Obliquity in the SPIRE Spectrometer,", SPIRE-BSS-DOC-000XXXX, Draft 0.3, 29 April 2008.

RD09 Davis, P., Fulton, T., and Kennedy, P., "Removing the baseline of interferogramsfrom the SPIRE imaging FTS,", SPIRE-BSS-NOT-002996, Issue 1.0, 09 Novem-ber 2007.

RD10 Davis, Peter, "Apodizing SPIRE Interferograms,", SPIRE-BSS-REP-003108, Draft0.4, 29 May 2008.

1.5. Document History

Table 1.3. Version and Date

Issue Date

Draft 0.1 27 March 2007

Draft 0.2 02 May 2007

Draft 0.3 (was Version 1.0) 09 May 2007

Draft 0.4 (was Version 1.1) 27 June 2007

Draft 0.5 (was Version 1.2) 10 July 2007

Draft 0.6 (was Version 1.3) 31 August 2007

Draft 0.7 (was Version 1.4) 26 September 2007

Draft 0.8 (was Version 1.5) 02 October 2007

Introduction

3

Issue Date

Draft 0.9 (was Version 1.6) 10 April 2008

Issue 1.0 23 May 2008

Issue 1.1 04 August 2008

1.6. Change Record

Issue 1.0 to Is-sue 1.1 • Changed Chapter 2 to include a summary of the SPIRE Spectrometer AOTs.

• Changed Chapter 3 to describe the building block pipeline.

• Modified the "Clipping Correction" section so that it describes its current im-plementation.

• Modified the "Time Domain Phase Correction" section for clarity by splittingit into two sections; identification and correction. Some of the equations werealso changed for clarity.

• Modified Section 3.3.3 "Second Level Deglitching" for clarity.

• Modified Section 3.3.5 "Phase Correction" for clarity.

• Corrected Equation 3.37 in Section 3.4.1 "Fourier Transform".

• Corrected Equation 3.40 in Section 3.5.1 "Spectral Response Correction".

• Corrected Equation 3.45 in Section 3.5.1 "Flux Conversion".

• Corrected Equation 3.47 and Equation 3.48 in Section 3.5.3 "Spectral Aver-aging".

• Added Chapter 4 "SPIRE iFTS Spectral Mapping".

Issue 1.1 to Is-sue 1.2 [FTS-1] Changed Table 2.1: last entry now reads " Raster, full".

[FTS-2] Added a reference to AD05 and changed the SLW spacings forIntermediate and Full Sampling.

[FTS-3] Fixed typo in Table 2.2, regarding HR spectral resolution.

[FTS-4] Deleted first sentence in Section 2.3.

[FTS-5] Defined V(t), z(t), n(t), P(t), V(x), B(σ), and I(σ).

[FTS-6] Modified Section 3.1 to be more constistent with the text andwith [AD02].

[FTS-7] Removed Figure 2.1 and added calibration products toFigure 3.2, Figure 3.2, Figure 3.3, Figure 3.4, Figure 3.5, andFigure 3.6, .

[FTS-8] Corrected text in second paragraph of Chapter 3.

[FTS-9] Renamed Section 3.1 to be more constistent with other portionsof this document.

[FTS-38] Three different glitch reconstruction methods are now describedin Section 3.1.1.

Introduction

4

[FTS-39] Cannot rename Section A.1 to Appendix.

[FTS-40] Deglitching steps in Section 3.1.1 and Section 3.3.3 now includea reference to SPIRE Masks [AD07];

[FTS-58] Changed references to "missed" samples to "clipped" samples.

[FTS-59] Changed the paragraph that emphasizes the Nyquist criteria.

[FTS-60] Fixed typo in Section 3.1.4; imput becomes input.

[FTS-61] Fixed typo in Section 3.1.4; fo becomes of.

[FTS-62] Included a note in Section 3.1.4 referring to why the detector sig-nals might be clipped in the first place.

[FTS-63] Clarified Equation 3.4 and Equation 3.5 in Section 3.1.4.

[FTS-70] Changed to ω and defined this quantity in Section 3.1.5.

[FTS-71] Redefined ωThemal

in Section 3.1.5.

[FTS-72] Clarified usage of ωs in Equation 3.6- Equation 3.8 of Sec-tion 3.1.5.

[FTS-73] Added a citation to Section 4 of AD01 with respect to fast andslow time constants.

[FTS-80] Non-linearity correction now appears before the Clipping correc-tion and Time Domain Phase Correction modules.

[FTS-81] Noted that there will be a calibration table per bias setting in Sec-tion 3.1.3.

[FTS-96] Pointing is now computed per scan per detector in the Interfero-gram Creation step (Section 3.2.1).

[FTS-97] An explicit reference between obliquity in the SPIRE FTS[RD08] and the scaling factor, f, has been drawn. (Section 3.2.1).

[FTS-98] Modified the first paragraphs in both Section 3.2 and Sec-tion 3.2.1 for clarity.

[FTS-99] Clarified the usage of subscripts in Equation 3.17.

[FTS-103] Changed Figure 3.5 and Section 3.3.1 to SCAL, Telescope, andBeamsplitter correction.

[FTS-104] Made explicit reference to fact the the correction in Section 3.3.1will be applied independently to data from the two mechanismscan directions.

[FTS-105] Stated explicitly the baseline assumption of a single SCALthrough the mission lifetime.

[FTS-106] Revised the second paragraph of Section 3.3.1 to make clear thatthe reference observation will precede the on-source observation.

[FTS-113] Both Fourier and polynomial fitting methods are now describedin Section 3.3.2.

[FTS-115] Section to Appendix reference cannot be changed as requested inDocBook.

Introduction

5

[FTS-116] Description of polynomial correction method has been clarified.

[FTS-117] Description has been changed as requested.

[FTS-122] An explicit reference has been made to the effect that the identi-fication portion of 2nd level deglitching (Section 3.3.3) will beperformed on scan direction basis.

[FTS-123] The minimum number of scans per observation building blockhas been changed from six to four in Section 3.3.3.

[FTS-126] Clarified the description section of Section 3.3.4 to note that Ap-odization is the method of choice which means that there existsno module specifically dedicated to fringe correction.

[FTS-127] Removed redundant sentence in Section 3.3.4.

[FTS-129] Added a reference to the expected phase in Section 3.3.5 andmade a note that a full fit may be required during the PV phase.

[FTS-130] Added a note in Section 3.3.5 to the effect that the residual phasewill be monitored.

[FTS-134] Added as appendix the list of available apodization functions,(Section A.4). A reference to this list has been included in Sec-tion 3.3.6.

[FTS-135] A note stating that two spectral products, one apodized one not,will be produced was made in Section 3.3.6.

[FTS-136] Added a reference to [RD10] in Section 3.3.6.

[FTS-141] Clarified the link between processing modes and resolutionmodes in Section 3.4.1.

[FTS-148] Added a statement differentiating between point and extendedsource calibration in Section 3.5.1.

[FTS-149] Merged the Spectral Response and Flux Conversion steps.

[FTS-151] Added a note on the potential BSM dependency in Section 3.5.1.

[FTS-152] Corrected the symbol nomenclature in Section 3.5.1.

[FTS-157] The assumption that optical crosstalk will be negligible has beenstated in Section 3.5.2.

[FTS-158] Added a reference to PV phase measurements in Section 3.5.2.

[FTS-160] Added references to computing the average on a per directionbasis and to outlier identification in Section 3.5.3 and that out-liers would not factor in to neither the resultant average nor theresultant standard deviation.

[FTS-161] Added a quality control reference with respect to the number ofoutliers in Section 3.5.3.

[FTS-162] Added a note in Section 3.5.3 to the effect that all spectra for agiven detector within an observation that share the same pointingvalues will be used to compute the average.

[FTS-169] A note was added to Section 4.1 stating that both fixed and mov-ing coordinates can be processed.

Introduction

6

Chapter 2. SPIRE SpectrometerPipeline Overview

The data processing pipeline for the Spectral and Photometric Imaging Receiver (SPIRE) imagingFourier Transform spectrometer (iFTS) contains processing modules commonly used to processFTS data, such as phase correction and the Fourier transform. The SPIRE iFTS pipeline also con-tains processing steps unique to SPIRE, such as the correction for the Herschel Telescope and Spec-trometer Calibrator (SCAL).

The SPIRE iFTS pipeline has been designed to be consistent with the astronomical observation tem-plates (s) that are available to the users of the SPIRE spectrometer [AD05]. The final data productsgenerated by the Spectrometer pipelines will in all cases consist of hyperspectral data; two spatialdimensions representing the astronomical region under study and one spectral dimension. The de-gree to which the hyperspectral data product is sampled both spatially and spectrally depends on thetype of observation chosen.

2.1. Spatial SamplingThe spatial sampling in the final hyperspectral cubes depends on a combination of the number of re-quested pointing positions of the Herschel Telescope and the number of jiggle positions of theSPIRE Beam Steering Mirror (BSM) selected. The number of Herschel Telescope pointing posi-tions, n, will depend on the observing area requested and is limited by the maximum observing timefor one (18 hours [AD03]). A list of the spatial sampling options available to astronomical observersis shown in Table 2.1.

Table 2.1. SPIRE Spectrometer spatial sampling options [AD05]

Spatial Sampling Number ofHerschel

Telescope Po-sitions

Number ofBSM Posi-

tions

Total Num-ber of Point-ing Positions

Beam Spa-cing, SSW

Band [arcsec]

Beam Spa-cing, SLW

Band [arcsec]

Single, Sparse 1 1 1 32.5 50.5

Single, Intermediate 1 4 4 16.3 28.1

Single, Full 1 16 16 8.1 14.1

Raster, Sparse n 1 n 32.5 50.5

Raster, Intermediate n 4 4n 16.3 28.1

Raster, Full n 16 16n 8.1 14.1

2.2. Spectral ResolutionA SPIRE iFTS observation building block is defined as a set of equal-length scans of the SPIRESpectrometer Mechanism (SMEC) at a single pointing position of the Herschel Telescope andSPIRE BSM. The spectral resolution of these scans is determined by the maximum optical path dif-ference (OPD) that the instrument can achieve by displacing the SMEC from the point of symmetry,also known as the position of zero path difference (ZPD). The spectral resolution options availableto astronomical observers for the SPIRE iFTS are shown in Table 2.2.

Table 2.2. SPIRE Spectrometer spectral resolution options

Spectral Resolution Scan Length (OPD) [cm] Spectral Resolution [cm-1]

Low 0.5 1.0

7

Spectral Resolution Scan Length (OPD) [cm] Spectral Resolution [cm-1]

Medium 2.0 0.25

High 12.5 0.04

SPIRE Spectrometer Pipeline Overview

8

Chapter 3. SPIRE SpectrometerBuilding Block Pipeline

The block diagram of the SPIRE iFTS data processing pipeline is shown in Figure 3.1. The structureof the data processing pipeline specific to the SPIRE spectrometer follows the observation buildingblocks. This structure was chosen as it allows the processing modules that modify the signal data ofthe SPIRE spectrometer detectors to take advantage of symmetries and redundancies that are presentfor a series of FTS scans of the same astronomical target.

Figure 3.1. SPIRE iFTS data processing block diagram.

9

The data processing pipeline of the SPIRE spectrometer is split into two sections. The first of thesesections (Chapter 3) describes the modules that make up the building block pipeline (seeFigure 3.1). The processing steps that combine the building block products are then described inChapter 4.

The SPIRE spectrometer data processing pipeline described here operates on a single observationbuilding block at a time. The building block pipeline consists of six major processing groups (seeFigure 3.1).

1. Common Photometer/Spectrometer Processing modules. The first steps of the SPIRE spec-trometer and photometer pipelines are identical and are described in Section 3 of [AD01].

2. Modify Timelines The processing modules in this group perform time domain operations onthe spectrometer detector samples.

3. Create Interferograms The processing modules in this group merge the timelines of the spec-trometer detectors and spectrometer mechanism into interferograms. The spectrometer detectorsamples are split into different sets, with each set defined by a single scan of the spectrometermechanism.

4. Modify Interferograms The processing modules in this group perform operations on the spec-trometer detector interferograms. These operations differ from those in the "Modify Timelines"group in that they are designed to act on spatial domain data rather than time domain data.

5. Transform Interferograms The processing modules in this group transform the interfero-grams into a set of spectra.

6. Modify Spectra The processing modules in this group perform operations on the spectrometerdetector spectra.

3.1. Modify Timelines.After application of the processing steps common to both the photometer and spectrometer detectors[Section 3 of AD01], the raw samples for each one of the 66 spectrometer detectors, labeled i, willhave been converted into RMS voltage timelines, V

RMS-i(t). These quantities are contained in the

Level 0.5 Spectrometer Detector Timeline Product (SDT).

The processing modules described in the following sections are applied to the timelines for eachspectrometer detector. Each of the processing steps contained in this processing block (see Fig-ure 3.2) accepts a Level-0.5 SDT product as input and delivers an SDT product as output.

SPIRE Spectrometer Building Block Pipeline

10

Figure 3.2. Modify Timelines block of the SPIRE Spectrometer building block pipeline

3.1.1. First Level DeglitchingGlitches due to cosmic ray hits or other impulse-like events in the detectors will be removed usingan algorithm based on a wavelet-based local regularity analysis (see Section A.1). This process iscomposed of two steps: the first step detects glitch signatures over the measured signal; the secondstep locally reconstructs a signal free of such glitch signatures.

1. Glitch Identification. Glitches are detected in the input SDT product by wavelet analysis as-suming that the glitch signature is similar to the signature of a Dirac delta function. Eachsample that is identified as a glitch will have its mask modified in accordance with the SPIREpipeline mask policy [AD07].

2. Glitch Reconstruction. Currently, three methods of glitch reconstruction are being investig-ated:

a. Wavelet Analysis. Each sample flagged as a glitch by the preceding step is contains local-ized wavelet coefficients specific to the glitch. These coefficients are removed and a local,inverse wavelet transform is performed to create an SDT product that is free of glitches.


11

b. Polynomial Interpolation. The samples idetified as glitches are replaced by way of poly-nomial interpolation.

c. Average Replacement. This method would leave the replacement of the samples flaggedas glitches to the 2nd level deglitching module, where the glitches would be replaced by theaverage of the glitch-free samples at the same OPD position (see Section 3.3.3).

Regardless of the chosen reconstruction method, each sample that is corrected will have itsmask modified in accordance with the SPIRE pipeline mask policy [AD07].

The output of this module is the deglitched voltage timeline, V1-i

(t) for detector i.

3.1.2. Removal of Electrical CrosstalkAfter de-glitching, let the timeline for bolometer i be denoted as V

1-i(t). This timeline might contain

contributions that depend on the signals from other detectors due to either electrical or opticalcrosstalk. Electrical crosstalk arises after the detector and is due to capacitative or inductive coup-ling between the detector readout channels. Optical crosstalk occurs before the detector and is due todiffraction or aberrations in the optical system causing some of the power from an astronomicalsource to fall on inappropriate detectors.

Electrical crosstalk can be removed if the coupling between the detectors is known, and it is appro-priate to do it at this stage.

Here it is assumed that:

1. electrical crosstalk is linear, so that the effects can be characterised by a crosstalk matrix, Celec

,with constant elements;

2. electrical crosstalk from one detector to another involves negligible diminution of the signal inthe primary detector;

3. there is no crosstalk between different arrays.

For a particular time-step, the vector of electrical crosstalk-corrected signals is given by V2

= Celec

V1. As an illustration, with three detectors, the matrix equation would be

Equation 3.1.

The electrical crosstalk matrix may be implemented as a calibration file. Determination of the ele-ments of the electrical crosstalk matrix is a difficult problem. One possibility is to use the occasionalionizing radiation hits that the bolometers will experience. Ideally, a single event in a bolometer pro-duces a spike only in its own output; crosstalk results in this being accompanied by lower-level re-sponses from other detectors.

3.1.3. Detector Non-Linearity CorrectionEven though bolometric detectors are commonly fabricated with highly linear response characterist-ics, the detectors of the SPIRE spectrometer will be subject to a wide dynamic range which makes anon-linear response likely. A dedicated non-linearity correction is designed to account for changesin the response of the detectors to strong signals. The form of this correction will be a function thatis dependent on the amplitude of the signal itself as in Equation 3.2:


12

Equation 3.2.

where f(V), the real detector responsivity, Vr

is a reference voltage, and V0

is a fixed bolometervoltage. The normalized value of f(V) is derived as in Equation 3.3:

Equation 3.3.

A set of calibration tables, one per bias voltage, will each contain the values for V0, K

1, K

2, and K

3for each detector. Initially, the quantities in these calibration tables will be based on model predic-tions.

3.1.4. Clipping CorrectionThe purpose of this processing step is to correct clipped signals in the input SDT product of themeasured signals due to the limited range of the detector ADCs.

Clipped signals in the voltage timelines of the SDT are problematic as they are essentially missedsamples. The presence of clipped samples depends on the source strength and the detector offset set-ting. If left uncorrected, clipped samples can lead to further complications in particular when thetimelines are converted into interferograms (Section 3.2.1).

Clipped samples can be corrected in a given SDT timeline are reconstructed using an eigth-orderpolynomial. While the only theoretical limit to the order of the polynomial is the total number ofsamples, the quality of its reconstruction has been found to depend on the number of clippedsamples.

Tests have revealed that a clipped signal of 40% of the theoretical peak (i.e. 60% of signal amp-litude is left over) corresponds to four clipped samples (at ZPD) and to eight clipped samples (forsecondary peaks) per scan. In this case, the restored signal has an RMS error of 2%-3% of signalamplitude at ZPD. That corresponds to less than 1% RMS error on the final reconstructed spectrumcontinuum. The same tests showed that a clipped signal that results in an 80% reduction of the ZPDamplitude results in an RMS error of up to 6% on the reconstructed interferogram signal.

The process by which clipped timelines are corrected is described below.

1. Identify the clipped samples in the SDT timelines. Let V3-i

(tk) denote those samples that

have been flagged as being clipped, let V3-i

(tj) represent all other samples, and let i represent a

given spectrometer detector.

2. Interpolate the modified SDT timeline. A polynomial of degree eight (8) is applied to thefive points before and after those identified as being clipped.

3. Replace the SDT timeline. Replace those samples that had been identified as clipped in theoriginal detector timeline, V

3-i(t

k) with the results of the polynomial fit, V

fit(t

k).

Equation 3.4.


13

Samples that were not identified as being clipped, V3-i

(tj) are simply propagated to the resultant

timeline, V4-i

(t).

Equation 3.5.

Note

Further study into the number of consecutive clipped points that can be successfully corrected us-ing this algorithm is warranted. For now, an dditional step will have the Quality Control engineerinspect any SDT that contains clipped signals to see if valid astronomical products can be derived.

3.1.5. Time Domain Phase CorrectionThe purpose of the Time Domain Phase Correction module is to correct the detector timelines,V

3-i(t), for delays induced by the filters in the readout electronics and the thermal response of the de-

tectors themselves.

3.1.5.1. Determination of the Induced Phase Shift

The SPIRE spectrometer detector chain contains a 6-pole Bessel low pass filter (LPF) as well as anadditional RC LPF (see Section 3.5 of AD01) The transfer function of which is shown in Equa-tion 3.6,

Equation 3.6.

, where ω=2πf and f is the frequency of the recorded signal.

In addition to the electronic LPF, the thermal response of the SPIRE bolometers needs to be takeninto account. As was shown in Section 4 of AD01, the frequency response of the SPIRE bolo-menters may be modeled as a combination of a fast, τ

1, and a slow τ

2detector-specific time con-

stant. The overall response thereby taking the form shown in Equation 3.7,

Equation 3.7.

.

While ground-based studies of the thermal response of the SPIRE spectrometer detectors suggestthat the fast time-constant by itself sufficiently charaterizes their behaviour [RD07] (i.e. a

i=0, τ

2=0),

this aspect will be monitored over the course of the mission.

These two effects may be combined into a single detector transfer function, given by Equation 3.8:

Equation 3.8.


14

.

The combined response of the electronic LPFs and the thermal behaviour of the SPIRE bolometerdetectors, H

TOTAL(ωs), will affect both the magnitude (Equation 3.9) and the phase (Equation 3.10)

of the signals recorded by the SPIRE detectors.

Equation 3.9.

Equation 3.10.

According to Fourier theory, a change of phase in the spectral domain corresponds to a time shift inthe temporal domain. This effect is particularly problematic for the SPIRE spectrometer in scanningmode (SOF1 and SOF2 [AD05]), where the delay induced by the electronic and thermal phase canlead to errors in the interpolation of the detector signals (see Section 3.2.1).

The shift in the time domain, TDPCF, is quantified as the inverse Fourier Transform of the fre-quency domain phase shift as in Equation 3.10,

Equation 3.11.

.

3.1.5.2. Correction of the Induced Shift

The input timelines (V4-1

) are corrected by way of time convolution with the TDPCF, resulting in aset of corrected timlines, V

5-1.

Equation 3.12.

3.1.6. Removal of correlated noise due to bath temper-ature fluctuations

To first order, bath temperature fluctuations will influence all detectors in an array coherently -- thetemperature and corresponding output voltages will go up and down in synchronism. The bath tem-perature, T

o, may fluctuate due to temperature drifts within the instrument, and a set of timelines,

Vth-i(t), must be generated to correct for this. The most important effect of bath temperature vari-ations for the level of fluctuations expected in SPIRE, will be the direct response of the detector out-put voltage. Fluctuations in T

oare expected to be much faster than a single, high-resolution scans of

the spectrometer mechanism, so that this correction will be needed for such observations.


15

A correction timeline for each detector, Vth-i

(t), will be generated by way of a suitable algorithm us-ing thermometry data (to be based on evaluation carried out on data by IPAC/JPL). This timeline isthen subtracted from that detector's signal timeline, V

5-i(t). In order to avoid introducing additional

noise to the corrected detector timeline, the Vth-i

(t) timeline will need to be significantly less noisythan the bolometer signals. It will therefore need to be averaged over a period of time such that it be-comes a negligible fraction of the detector noise. This will require a suitable averaging period(JPL/IPAC assessment of the data indicates that a period on the order of a second or a few secondsis suitable). Thermal fluctuations on timescales shorter than this will not be tracked.

The output of this module is a set of spectrometer detector voltage timelines corrected for low-frequency thermal drifts: V

6-i(t) for detector i.

3.2. Create InterferogramsThe pipeline modules listed to this point describe the operations that will be performed on the de-tector timelines in the Level 0.5 SDT product. At this point, in the SPIRE iFTS building blockpipeline, three additional Level 0.5 products are required to proceed: the Spectrometer MechanismTimeline product (SMECT); the Nominal Housekeeping Timeline product (NHKT); and the SPIREPointing product (SPP) (see Figure 3.3).

Figure 3.3. Interferogram creation block of the SPIRE Spectrometer pipeline

3.2.1. Interferogram CreationA single building block of a SPIRE spectrometer observation in scanning mode consists of a seriesof scans of the spectrometer mechanism while the instrument is pointed at a given target. Thesampling of the SPIRE spectrometer detectors and the spectrometer mechanism is decoupled; thetwo subsystems are sampled at different rates and at different times. In order to derive the sourcespectrum from the measured data, the spectrometer detector samples must be linked with the posi-tion of the SMEC in the form of interferograms. Additionally, the SMEC positions onto which thespectrometer detector signal samples are to be interpolated should be regularly-spaced in terms ofoptical path difference (OPD).

The process by which interferograms are created involves two steps, each of which is described be-low.

1. Interpolation of the SMEC timeline. This step converts the spectrometer mechanism timelinefrom one that is non-uniform in position to one that is uniform in position.

a. Establish a common OPD position vector. This step creates a common vector of OPDpositions that will be the basis of the interferograms for all of the spectrometer detectors


16

and for all of the scans in the observation. This common position vector will containsamples that are uniformly-spaced in terms of OPD position as well as a sample at the po-sition of zero-path-difference (ZPD).

The step size of the common OPD vector is chosen is such a way as to match the samplingrate of the spectrometer detector signal samples. For an SDT sampling rate s [Hz] and aSMEC scanning speed v

SMEC[cm/s MPD], the position step size, δMPD in units of cm; is

given by:

Equation 3.13.

δMPD = vSMEC

/ s

This step is then converted such that it is in terms of OPD by the following relation

Equation 3.14.

δOPD = FLOOR[4δMPD]

where FLOOR[] denotes that the step size is rounded down to the nearest integer in unitsof µm OPD and the factor of four is the nominal conversion between MPD and OPD for aMach-Zehnder FTS. Using the nominal SPIRE spectrometer settings -- s=80Hz,v

SMEC=0.05cm/s -- this results in an OPD step size of 25 µm.

b. Map the common OPD position vector to a SMEC position vector for each spectro-meter detector. This step maps, for each spectrometer detector, the common OPD posi-tions established in the preceding positions in units of mechanical path difference. Thisstep involves: a scaling factor, f [RD08], that takes into account the obliquity in the SPIREFTS; and a shifting factor, ZPD, which establishes the position of zero optical path differ-ence. Since these quantities are unique to each spectrometer detector, i, this mapping isperformed on a detector-by-detector basis and is shown in Equation 3.15.

Equation 3.15.

c. Parse the measured SMEC timeline into discrete scans. This step splits the full SMECtimeline (z(t

SMEC)) from the input SMECT product into a series of discrete timelines,

zn(tSMEC). Each of the discrete timelines, z

n(t

SMEC), represents one spectrometer scan. The

delineation of the SMEC timeline is accomplished by comparing consecutive SMEC posi-tion samples and finding those samples where the motion of mirror mechanism changeddirection.

d. Interpolate the measured SMEC timelines onto the mapped SMEC timelines. Thenext step is to determine, on a detector-by-detector and scan-by-scan basis, the times whenthe spectrometer mechanism reached the mapped SMEC positions. Since, for each detect-or, there is a 1:1 relationship between the mapped SMEC positions and the regularly-spaced OPD positions, this step effectively determines the times when the SMEC reachedthe regularly-spaced OPD positions for each detector.

Equation 3.16.

2. Merge the spectrometer detector and the mapped SMEC timelines. This step combines the


17

signal samples from the signal timeline of a given spectrometer detector (V6-i

(ti)) with the

mapped SMEC timelines.

a. Interpolation of the spectrometer detector timelines. The input spectrometer detectorsignal samples, V

6-n-i(t

i), are mapped onto the times corresponding to the regular MPD

(tMPD-i) positions by way of interpolation. Since there is a 1:1 relationship between thesetime samples, t

MPD-i, and the regular MPD positions, MPD

i, this interpolation effectively

maps, for each detector, the signal samples to the regularly-spaced MPD positions.Moreover, since there is a 1:1 relationship between the regular MPD positions for each de-tector and the common OPD positions (as shown in Equation 3.15), this step accomplishesthe mapping of the signal samples for each detector to the common OPD positions, whichis the resultant interferogram that is desired, V

7-n-i(OPD).

Equation 3.17.

b. Affix a pointing value to the resultant interferogram. Upon creation of each interfero-gram, V

7-n-i(x), the pointing timeline for each detector i, P

i(t) is evaluated and its time-

averaged value is affixed to each spectrometer detector for each interferogram in thebuilding block.

3.3. Modify InterferogramsThe pipeline modules described in this section perform operations on the interferograms created inthe previous step. Each of the processing steps contained in this processing block accepts an SDIproduct as input and delivers an SDI product as output (see Figure 3.4).


18

Figure 3.4. Interferogram modification block of the SPIRE Spectrometer pipeline

3.3.1. SCAL and Telescope CorrectionThe equation for the total intensity of the radiation incident upon the spectrometer detectors showsthat, in addition to radiation from the astronomical source, the detectors record a modulated signalfrom the Herschel Telescope, signals from each of the components of the spectrometer calibrator(SCAL), and a signal from the emission of the first beamsplitter in the SPIRE FTS. This processingstep removes these extra components from the measured interferogram.

Equation 3.18.

The method employed in the simple pipeline to correct for the Herschel Telescope, the SCAL com-ponents, and the SPIRE beamsplitter is to subtract from the measured interferogram (V

7-i(x)) an in-

terferogram derived from a reference observation wherein the the SPIRE instrument is pointed atblank sky (V

7-ref-i(x)). During the on-source observation, the SCAL components (SCAL2 and

SCAL4) will be set to the same temperature settings as was the case for the reference observation.

Note

Since the current baseline assumption is that the components of SCAL will be kept at a single oper-ating temperature through the lifetime of the mission, the stipulation that the SCAL settings be thesame for the reference and on-source observations should be easily met. direction.

Equation 3.19.

Equation 3.20.

Equation 3.21.

Note

In order to avoid any potential systematic errors, the reference interferograms will be split into twosets: one for each mechanism scan direction. Thus, the operations described inEquation 3.19-Equation 3.21 should be taken as to apply independently to the interferograms de-rived from the SPIRE mechanisms forward and reverse scans.

In the model-based pipeline, removal of the contributions due to SCAL, the Herschel Telescope, andthe SPIRE spectrometer beamsplitters is performed by subtraction of a model their contributions de-rived from the measured temperatures of each element and the measured transmission through thespectrometer for each emitting element. This method has the advantage of not requiring a blank-skycalibration observation though the precision of such a method cannot be evaluated until flight condi-tions are observed.


19

The result of this correction step will be a set of interferograms for each spectrometer detector(V

8-i(x)); these interferograms are stored in an SDI product. The principal component of the result-

ant interferograms is the radiation from the astronomical source.

3.3.2. Interferogram Baseline CorrectionAccording to the equations presented in Section A.2, the overall intensity incident on the SPIREspectromter detectors can be separated into two components: an offset or baseline portion that is afunction of OPD; and a component that is modulated as a function of OPD.

On a detector-by-detector and scan-by-scan basis, the baseline correction algorithm evaluates andremoves the baseline portion of the derived interferogram (V

8-i(x)). The baseline correction pro-

cessing step provides two methods to evaluate the baseline of the derived interferograms: polynomi-al fitting; and fitting by way of the Fourier transform.

1. Polynomial fitting. One of the methods that may be employed to evaluate the interferogrambaseline is to fit each of the derived interferograms with a fourth-order polynomial (Equa-tion 3.22).

Equation 3.22.

2. Fourier transform fitting. Alternately, the interferogram baseline may be evaluated as thatportion of the measred interferograms whose Fourier components are less than 4cm-1 (Equa-tion 3.23).

Equation 3.23.

Regardless of the method used to evaluate the interferogram baseline, once evaluated, the baseline isremoved from the measured interferogram by subtraction (Equation 3.24).

Equation 3.24.

Note

The nominal method of interferogram baseline evaluation that will be employed by this processingmodule will be the fourth-order polynomial fitting method. This method was chosen as it has beenfound to be the most robust when evaluated using data [RD09].

3.3.3. Second Level DeglitchingLocalized artifacts in the interferograms, glitches, pose a serious problem for Fourier TransformSpectrometer observations. As such, a glitch that affects as few as one interferogram sample can ad-versely affect each and every spectral component. Glitches in an interferogram must therefore beidentified and removed prior to transformation in order to avoid unwanted spectral artifacts.


20

Glitches are identified for each spectrometer detector, i, by comparing, on a OPD-posi-tion-by-OPD-position basis (i.e. each x

k), the samples from one scan, j, to those from all other scans

in the same observation building block derived from the same mechanism scan direction (scan num-ber n, where n ≠ j and all n and j are from the same mechanism scan direction). The samples that de-viate more than a prescribed amount from the median are flagged as glitches and will have itssample mask modified in accordance with the SPIRE pipeline mask policy [AD07].

The samples that are identified as glitches are then replaced. For a glitch at a given position for agiven spectrometer detector, the value of the replacement sample is determined by the average of thenon-glitch samples from the other observed interferograms at that position. Each sample that is re-placed will have its sample mask modified in accordance with the SPIRE pipeline mask policy[AD07].

Equation 3.25.

Note

The two steps of the interferogram deglitching module rely on a statistical analysis of the measuredinterferograms. As such, a minimum number of four interferograms (two per scan direction) perobservation building block will be required so that these statistics will be meaningful.

3.3.4. Channel Fringe CorrectionThe effect of channel fringes on spectrometer data is similar to that of glitches (Section 3.3.3). If leftuncorrected the channel fringes will contaminate the measured spectrum. Currently, three differentalgorithms are being evaluated as possible methods to correct for channel fringes. These are:

1. Apodization. The application of an apodization function (Section 3.3.6) is currently thebaseline method for fringe correction. While apodization is effective at removing the spectralartifacts due to the channel fringes, its application results in a reduction of the observed spectralresolution. Given that the fringe features appear at the extreme high-resolution OPD end for theSLW array and at the extreme medium-resolution end for the SSW array, however, the reducedresolution is not expected to be significant for those observing modes.

2. Interferogram Truncation. This method of channel fringe removal involves truncating the in-terferogram prior to the channel fringe region. The drawback to this method is similar to thatfor apodization; reduced spectral resolution.

3. Iterative Subtraction. This method of channel fringe correction is performed by removingsuccessive copies of the interferogram in the region near ZPD from the wing portion of the in-terferogram.

Note

Apodization (Section 3.3.6) has been chosen as the preferred method of fringe correction. As such,there is no specific task dedicated to fringe removal in the current implementation of the SPIREFTS data processing pipelines.

3.3.5. Phase CorrectionThe symmetry of a Fourier Transform spectrometer theoretically implies that interferograms recor-ded by the spectrometer will exhibit even symmetry. Since the spectrum of an evenly symmetric in-terferogram contains only real components, it is therefore expected that the phase should be zero for


21

all spectral components.

The presence of dispersive elements and the possibility that the position of zero path difference notbeing sampled can, however, result in a interferogram whose signal samples are not symmetricabout ZPD. In this scenario, the phase that is expected would take the form shown in Equation 3.26:

Equation 3.26.

where the φLin

(σ) term represents the mis-sampling of the position of ZPD, the φNL

(σ) term repres-ents the effects of the dispersive elements, and φ

Random(σ) represents any phase due to noise.

The phase correction module is separated into two components: the first step identifies whether anyphase is present in the measured interferogram and also quantifies this phase; the second step re-moves the phase from the measured interferograms.

1. Phase Identification. This step of the phase correction process is to identify whether the meas-ured interferograms contain any non-zero phase. In order to make this determination, the spec-trum of the double-sided (see Section A.3.1) portion is computed for each interferogram (Equa-tion 3.27)

Equation 3.27.

where L represents the extent of the interferogram about the ZPD position. The phase of thecomputed spectrum may be evaluated for each spectral component or wavenumber as in Equa-tion 3.28.

Equation 3.28.

2. Phase Removal. The phase in the measured interferograms having been identified, the nextstep in the process is phase removal. Based on the expected phase given in Equation 3.26, a lin-ear fit is made to the measured in-band phase in order to quantify the degree to which the posi-tion of ZPD was missampled (Equation 3.29):

Equation 3.29.

Next, a non-linear phase derived from calibration measurements is added to the linear phase asshown in Equation 3.30.

Equation 3.30.


22

Note

During all phases of the mission, this processing step will be monitored to ensure that no systemat-ic residual phase remains after its application.

A phase correction function (PCF) is then derived from the fitted phase for each interferogramas in Equation 3.31).

Equation 3.31.

The derived PCFs are then applied to spectra computed from each of the interferograms in theinput SDI product by way of multiplication (Equation 3.32).

Equation 3.32.

If the observing mode is low- or medium-resolution, the above represents the final step in thephase correction process. High-resolution observations require an extra step. Phase correctionof high-resolution interferograms, proceeds in the interferogram (spatial) domain wherein aconvolution of the measured interferogram and the the inverse FT of the PCF is performed (seeEquation 3.33).

Equation 3.33.

where l represents the extent of convolution kernel, i.e. the inverse transform of the PCF.

Note

During the initial stages of the mission, the non-linear portion of the phase referred to in Equa-tion 3.30 may not be sufficiently quantified. During this period, a fourth-order polynomial fit willbe used to quantify the in-band phase (see Equation 3.34) and will provide the basis for the phasecorrection function.

Equation 3.34.

3.3.6. Apodization


23

The natural instrument line shape (ILS) for a Fourier Transform spectrometer is a cardinal sine, orSinc function. If the source signal contains features at or near the resolution of the spectrometer, theILS can introduce secondary maxima in the spectra. The apodization functions available within thismodule may be used to reduce these secondary maxima at the cost of reducing the resolution of theresultant spectrum. The apodization module in the SPIRE spectrometer data processing pipeline of-fers a number of apodizing functions that to allow for an optimal trade-off between reduction in thesecondary maxima and reduced resolution [RD03, RD10].

Apodization is performed by multiplying the input interferograms (Vn-11-i

(x)), on a detector-by-detector and on a scan-by-scan basis with a tapering or apodizing function.

Equation 3.35.

A list of the avaliable apodization functions is given in Section A.4. Also shown in Section A.4 isthe default apodization function that is to be used for Standard Product Generation.

The baseline approach for Standard Product Generation will be to provide two spectral products perobservation building block: one in which the default apodization function has been applied to the re-corded interferograms; the other where no apodization has been applied.

3.4. Transform InterferogramsAt this point in the building block pipeline, the operations that are best performed in the interfero-gram domain have been implemented. From this point on, further processing can take place in thespectral domain. As such, this presents the opportunity to transform the interferograms for each de-tector, V

n-i(x), to the spectral domain. The section describes the process by which the interferograms

contained in the Level 1 SDI product created by the preceding steps are transformed to spectra thatwill be contained in a Level 0.5 Spectrometer Detector Spectrum (SDS) product (see Figure 3.5).

Figure 3.5. Interferogram transformation block of the SPIRE Spectrometer pipeline

3.4.1. Fourier TransformThe purpose of the Fourier Transform module is to transform the set of interferograms from aSPIRE spectrometer observation into a set of spectra. This processing module is capable of trans-forming both double-sided and single-sided interferograms (see Appendix A for the definition ofdouble-sided and single-sided interferograms).

Double-sided Transform. For the double-sided transform, each interferogram in the SDI is ex-amined and only the double-sided portion of the interferogram is used to compute the resultant spec-trum. The resultant spectra will contain both real and imaginary components.

Equation 3.36.


24

In this case, the discrete fourier transform that is used to compute the spectral components takes theform shown in Equation 3.37.

Equation 3.37.

Single-sided Transform. In the case of the single-sided transform, only those interferogramsamples to one side of the position of zero path difference are considered. The spectra that resultfrom the single-sided transform therefore contain only real components.

Equation 3.38.

The discrete fourier transform that is used to compute the spectral components for single-sided inter-ferograms takes the form shown in Equation 3.39.

Equation 3.39.

Wavenumber Grid. For both the single-sided and double-sided transforms the wavenumber gridonto which the spectrum is registered is calculated based on the interferogram sampling rate(∆OPD) and on the maximum OPD displacement from the position of ZPD, L.

The Nyquist frequency (σNyquist

), the maximum independent frequency in the output spectrum, isgiven by:

Equation 3.40.

The spacing between independent spectral samples (∆σ) is given by:

Equation 3.41.

The spacing between spectral samples can be modified by padding the interferogram with zeroes.This procedure does not add any information to the spectrum but allows for an easier comparison


25

between observations. In this case, a zero-padded interferogram (V12-ZP-i

) is given by:

Equation 3.42.

The corresponding spectral sampling interval is given by:

Equation 3.43.

and the resultant spectrum of the zero-padded interferogram is given by:

Equation 3.44.

The scan lengths and resultant spectral sampling intervals for the three distinct spectral resolutionsfrom Section 6.6 of [AD03] are given in Table 3.1.

Table 3.1. Interferogram Padding

Spectral Resolution[AD03]

Sampling Interval(OPD) [µm]

Nyquist Wavenum-ber [cm-1]

Padded ScanLength (OPD) [cm]

Spectral SamplingInterval [cm-1]

Low 25 200 2.0 0.25

Medium 25 200 10.0 0.05

High 25 200 50.0 0.01

This processing step will create a Level-1 Spectrometer Detector Spectrum (SDS) product that willbe available to observers.

Note

Both the double-sided transform and the single-sided transform are applied to data from the SPIREFTS regardless of the spectral resolution mode. The double-sided transform is applied beforephase correction (Section 3.3.5) so that the phase may be evaluated. The single-sided transform isapplied after apodization (Section 3.3.6) -- when all of the asymmetric components of the interfero-grams have been removed.

3.5. Modify SpectraThe pipeline modules that follow in this section describe the operations that will be performed onthe Level-0.5 SDS products that were created in the preceding step. The end result of these spectralmodifying processing steps will be a Level-1 SDS product that contains a single, flux-calibrated, av-eraged spectrum for each spectrometer detector, I

i(σ). The spectral modification creation block is

shown in Figure 3.6.


26

Figure 3.6. Spectral modification block of the SPIRE Spectrometer pipeline

3.5.1. Flux ConversionThe flux conversion module translates each of the measured spectra in the input SDS product fromvoltage quantities with units of [Volts/(cm-1)] (V

n-13-i(σ)) to optical power quantities with units of

[either Watts/m2/(cm-1) or Janskys] (In-14-i

(σ)). This conversion is applied on a wavenumber-by-wavenumber basis as in Equation 3.45.

Equation 3.45.

The exact manner by which the conversion curves, fi(σ), will be derived is still TBD but the current

baseline is to perform a calibration observation of a source with a known flux. The spectrum derivedfor each spectrometer detector V

13-i(σ) will then be used as the conversion curve for this module.

Note

The flux conversion method that is to be applied during Stadard Product generation will involvecalibration with a point rather than an extended source. Many scientific observations will liebetween these two extreme cases and observers are advised to apply an additional extended sourcecorrection.

It is possible that the calibration tables used in this step will depend on the orientation of the BSM.This potential will be evaluated during the PV phase of the mission.

3.5.2. Removal of Optical CrosstalkOptical crosstalk is here defined as power from the astronomical sky that should be incident on onedetector actually falling on another. It is important to note that in the case of SPIRE, neighboring de-tectors are separated by approximately twice the of the incident beam, therefore even if a source ison-axis for a given detector, a small fraction of the source power will be incident on the neighboring


27

detectors due to telescope diffraction. Non-neighbouring detectors are sufficiently far apart that theyshould not pick up any power from an on-axis source.

Optical crosstalk will be characterised by a crosstalk matrix, Copt

, analogous to the electricalcrosstalk matrix described in Section 3.1.2. Let I

14be the input vector of spectral flux densities.

The vector of optical crosstalk-corrected spectral flux densities is then given by I15

= Copt

I14

As an illustration, if there are three detectors, the matrix equation would be

Equation 3.46.

Unlike the case of electrical crosstalk (Section 3.1.2), the diagonal elements are not necessarilyequal to unity since optical crosstalk involves loss of power from the primary detector.

The optical crosstalk matrix may be implemented as a calibration file. The method by which thecomponents of the optical crosstalk matrix, C

opt, is TBD.

Note

In the absence of optical crosstalk, or if the crosstalk correction is to be omitted, the non-diagonalcoefficients of C

optare set to zero and the diagonal coefficients are set to unity.

Note

The initial assumption is that any effect due to optical crosstalk will be negligible (i.e. the initialstate of C

optwill be equal to the identity matrix). This assumption will be verified during the PV

phase of the mission, which includes observations intended to characterize this phenomenon.

3.5.3. Spectral AveragingThis module computes, on a wavenumber-by-wavenumber basis for each spectrometer detector, theaverage of the spectral intensities across all scans (see Equation 3.47). Nominally, this processingmodule will operate on spectral data from a single observation building block. In the case of obser-vations where data from multiple building blocks share the same pointing coordinates -- when thebuilding blocks are separated by PCAL flashes, or when the building blocks are part of a repeatedjiggle pattern -- all spectra for a given detector that share the same pointing coordinates will be usedto compute the average.

Equation 3.47.

This module also computes, on a wavenumber-by-wavenumber basis for each spectrometer detector,the uncertainty in the spectral average. The uncertainty is calculated as the standard deviation of thespectral components as in Equation 3.48.

Equation 3.48.


28

Note

The average and standard deviation calculations listed in Equation 3.47 and Equation 3.48 abovewill be performed taking the mechanism scan direction into account. That is, spectral samples fromforward scans will be averaged together and spectral samples from reverse scans will be averagedtogether.

Prior to the evaluation of the average and standard deviation for each spectral bin, outliers thatreside an as yet to be determined number of standard deviations from the mean or median absolutedeviations from the median will be removed. These outlier will therefore not affect ether the finalaverage or standard deviation values. The processing module will provide, by way of quality con-trol information, the number of outliers that it identifies.


29

Chapter 4. SPIRE iFTS SpectralMapping

The pipeline modules that follow in this section describe the operations on the Level-1 SDSproducts created in the preceding step. The end result of these spectral modifying processing stepswill be a Level-2 Spectrum Cube product.

4.1. SPIRE iFTS Spatial RegriddingThe SPIRE iFTS building block pipeline produces one spectrum per detector. The spatial distribu-tion of the spectra in the astronomical region of interest from a single pointing follows approxim-ately a honeycomb pattern as per the design of the detector arrays (Figure 4.1).

For an observation performed at intermediate or full spatial sampling (Figure 4.2, Figure 4.3), theset of Level 1 SDS products is interpolated onto a hyperspectral data cube that is equidistantlysampled in the two spatial dimensions while leaving the equidistant grid along the spectral dimen-sion unchanged. This operation will not be applied in the sparse spatial sampling mode (Figure 4.1)as the spatial sampling in that mode will not meet the Nyquist criteria.

Figure 4.1. Astronomical footprint of the SPIRE detector arrays; sparse spatial sampling mode. [AD03]

30

Figure 4.2. Astronomical footprint of the SPIRE detector arrays; intermediate spatial sampling mode.[AD03]

Figure 4.3. Astronomical footprint of the SPIRE detector arrays; full spatial sampling mode. [AD03]

An algorithm for the interpolation of spectral data collected at non-uniformly sampled locations hasbeen identified and a normalized convolution algorithm has been implemented [RD05]. The al-gorithm iterates along the spectral dimension and evaluates a two-dimensional convolution of themeasurements at given sky positions with a separable two-dimensional kernel describing the field ofview of the detectors. Ground-based measurements have shown that the of the beam of the SPIRE

SPIRE iFTS Spectral Mapping

31

spectrometer detectors vary in a non-linear fashion with frequency between 15.5 and 17.5 arcsec forthe SSW and between 31 and 41 arcsec for the SLW band [RD06]. Th interpolation algorithm iswell set up to take such a frequency-dependent beam size into account. It does, however, assumethat the beam sizes of all detectors are identical. The suitability of this procedure remains to be veri-fied.

Note

The spatial regridding algorithm will take into account bot fixed coordinates (RA and Dec) as wellas moving coordinates (e.g. Solar System objects).

SPIRE iFTS Spectral Mapping

32

1 C.Ordenovic, C. Surace, B. Torresani, A. Llebaria, "Glitches detection and signal reconstruction using Holder and wavelet analysis.",ADAIV preprint, 5 March 2007.

Appendix A. AppendixA.1. First Level Deglitching Description

1. Glitch Identification. Glitch signatures are detected by performing a local regularity analysis(Holderian analysis) over the wavelet transform modulus maxima lines (WTMML) of the sig-nal.

Let H be the Holder exponent, s the scale of decomposition, Xi(s) the time (or OPD) domain

coordinate of the maxima line for the scale s, then when the scale s goes to zero the corres-ponding wavelet coefficient, W(X

i(s),s), is given by:

Equation A.1.

W(Xi(s), s) <= CsH

where C is a real constant.

The scale of decomposition, s, may be expressed over a logarithmic scale as:

Equation A.2.

s = 2o 2v/nV

where positive integers o, nV, and v (with v < nV) are respectively called octave, number ofvoices, and voice of the decomposition 1, respectively.

On each maxima line, the regularity degree of the signal is estimated by computing the slope ofthe linear regression over the set of points (log

2(|W|), log

2(s)) over the range of scales

[scaleMin, scaleMax]. If the relation is linear, i.e. if the square of its correlation coefficient C isgreater than the threshold coefficient thresholdCorr then the Holder exponent H can be estim-ated by the measure of the slope of the relation. Glitches are detected as they are similar to dir-ac-like signatures and show a Holder exponent (i.e. regularity degree) close to -1, ie in a range[thresholdHolder, hMin] centered over -1.

Noise can generate false detections (it can be shown that the Holder exponent of a gaussiannoise has a value (in mean) of 0.5). In order to minimize the likelihood of these false positives,contraints are applied to the wavelet coefficients. By considering a gaussian noise of standarddeviation σ, it can be shown that at the lowest scale of decomposition, the following threshold :

Equation A.3.

|W| <= σ √(2 ln N)

where N is the size of the signal.

The noise standard deviation σ on the signal can be estimated using the Donoho estimator; atthe lowest scale, σ = 0.6745 × med |W|

For each maxima line, if the value of the wavelet coefficient for the first scale value is greaterthan the previous threshold an estimate is made of the regularity degree.

Glitch Removal. Each glitch flagged by the preceding step is characterized by a Holder expo-nent H and by a maxima line X giving for each scale s the location of the maximum of themodulus of the wavelet coefficient X(s). For a glitch signature (the Holder exponent H ~ -1), itcan be shown that the wavelet coefficients W

gare given by:

33

Equation A.4.

Wg

= ψ( (b-X(s)) / s)

ψ being the wavelet function.

The glitch coefficients, Wg, are then subtracted from the wavelet coefficients, W, and the signal

is locally reconstructed by performing a wavelet transform over the corrected wavelet coeffi-cients.

The parameters that follow are optional and have been optimzed for the SPIRE spectrometer detect-ors. SPIRE PFM1 data that, by visual inspection, contained 29 glitches was used as a basis for thisoptimization.

• scaleMin, scaleMax: The scale range used for the linear regression. Optimal values arescaleMin = 2 and scaleMax = 8.

• thresholdHolder and Hmin

: The Holder exponent range used to select a glitch. Optimal valuesare thresholdHolder = -0.6, H

min= -1.3.

• thresholdCorr : The square threshold correlation that defines linear behaviour. The optimalvalue is thresholdCorr = 0.985.

• voices : The number of voices used for the scale decomposition. he optimal value is voices = 5.

A.2. Radiation Incident on the SPIRE Spectro-meter Detectors

The radiation path through the SPIRE spectrometer is illustrated for one case in Figure A.1.

Appendix

34

Figure A.1. Radiation from an astronomical source through the SPIRE spectrometer

As shown in Figure A.1, the first beamsplitter (SBS1) divides the incoming electric field (ES) into

two components (ESr1c

and ESt1). These two components pass through the interferometer and then

are split further at the second beamsplitter (SBS2). The upper beam from Figure A.1 then passes tothe SSW detectors while the lower beam passes to the SLW detectors. The electric fields incident onthe SSW and SLW detectors are given by the following equations:

Equation A.5.

Equation A.6.

At the detectors, the intensity recorded is the time-average of the square of the incident electric field.Using the SSW detectors for illustration, the measured intensity for radiation from an astronomicalsource at the detectors is given by the following:

Equation A.7.

Appendix

35

where:

Equation A.8.

Equation A.9.

Combining the above results in the following equations for the measured intensity at the SSW andSLW detectors for radiation from an astronomical source:

Equation A.10.

Equation A.11.

In addition to the astronomical source, radiation from the Herschel telescope and the three compon-ents of SCAL (SCAL2, SCAL4, and SCAL) is incident on the SPIRE spectrometer detectors. Forthe telescope radiation, its path through the SPIRE spectrometer is the same as that for the astro-nomical source. The path for the SCAL emitters is slightly different (see Figure A.1). The equationsfor the radiation incident on the SSW and SLW detectors are given by the following:

Equation A.12.

Appendix

36

Equation A.13.

Taken together, the overall intesity of the radiation measured by the SPIRE spectrometer detectors isgiven by the following:

Equation A.14.

1. SSW Detectors.

Equation A.15.

2. SLW Detectors.

Equation A.16.

Appendix

37

2 Some implementations of the Fourier Transform may require an even number of points (NTOTAL

even). In this case, the RHS of thedouble-sided interferogram will contains an extra point to render the total number of points even.

A.3. Double-sided and Single-sided Interfero-grams

The terms double-sided and single-sided as used in this document describe the two types of inter-ferograms that can be measured with a Fourier Transform Spectrometer.

A.3.1. Double-sided InterferogramsDouble-sided interferograms are defined as those interferograms or that portion of measured inter-ferogram where the sample positions are symmetric about the position of zero path difference(ZPD). That is, a double-sided interferogram is one that contains an equal number of samples beforeand after the ZPD sample 2. An envelope of a double-sided interferogram is shown in Figure A.2.

Figure A.2. Envelope of a double-sided interferogram

A.3.2. Single-sided InterferogramsSingle-sided interferograms are defined as those interferograms that contain more samples on oneside of ZPD than the other. An envelope of a single-sided interferogram is shown in Figure A.3.

Appendix

38

Figure A.3. Envelope of a single-sided interferogram

A.4. Available Apodizing FunctionsThe following table lists the apodizing functions that may be applied using the apodization pro-cessing module (Section 3.3.6, their keyword abbreviations, and the mathematical operations theyperform on interferometric data.

Table A.1. Apodization Functions

Apodization Function Keyword Formula

HAMMING aHM_150 0.54 + 0.46 cos(π x)

HANNING aHANN 0.50 + 0.50 cos(π x)

Gaussian aGAUSS exp(x2/2)

Norton-Beer 1.1 aNB_11 0.701551 - 0.639244 (1-x2) + 0.937693 (1-x2)2




Norton-Beer 1.5 aNB_15 0.077112 + 0.703371 (1-x2)2 + 0.219517 (1-x2)4

Norton-Beer 1.6 aNB_16 0.039234 + 0.630268 (1-x2)2 + 0.234934 (1-x2)4 + 0.095563 (1-x2)6

Norton-Beer 1.7 aNB_17 0.02007835 + 0.4806674 (1-x2)2 + 0.386409 (1-x2)4 + 0.1128451(1-x2)6



Norton-Beer 2.0 aNB_20 0.002267285 + 0.1404125 (1-x2)2 + 0.4871719 (1-x2)4 + 0.2562002(1-x2)6 + 0.1139479 (1-x2)8

Appendix

39