The processing aspects of the interferometric imaging spectrometer

(EE391 project report)Ying-zong Huang

Joint work with Ulrich Barnhofer and Sameer Bhalotra

1 Introduction

A natural scene contains not merely“colors”but spec-tral emissions that are not fully detectable by conven-tional imaging systems, such as a digital camera. Theusual means to extract spectral information is witha hyperspectral imager, which filters light at differ-ent wavelengths and records their intensities. Theinterferometric imaging spectrometer (IIS) is an ex-perimental device that extracts spectral informationa different way, by generating and capturing light in-terference patterns.

The IIS embeds an interferometer within a con-ventional imaging system. Therefore, a key featureof the IIS is that it detects spectral information at alllocations of a 2D scene in parallel. Coupled with per-pixel processing implementable using simple analogcircuits, the IIS is capable of directly extracting froma scene specific spectral properties of interest – forexample, the widths of spectral peaks. Detecting thewidth of spectral peaks has been the focal point ofefforts with the IIS. We examine the processing stepsnecessary for that purpose and show experimental re-sults and derived applications.

2 System operation & analysis

The interferometric imaging spectrometer is shown inFigure 1 and diagrammed in Figure 2. Light fromeach point in the scene enters an imaging lens (Lens1) and is split into two paths by the Beam Splitter.One path is reflected at Mirror 1 and the other atMirror 2. Mirror 1 is held fixed, while a piezoelectricdevice attached to Mirror 2 moves it in a periodicpattern along the optical axis. Light from the twopaths is recombined at the Beam Splitter and is sent

Figure 1: Interferometric imaging spectrometer.

Absorption

Emission

Mirror 1

Mirror 2

Lens 1

Lens

2

Beam Splitter

Shift

Figure 2: System schematic

1

Figure 3: Example of an interferogram

through a final imaging lens (Lens 2) for detection bya high speed CMOS imager [1].

2.1 Interferometric pattern

When Mirror 2 is at the zero-point, where the opticalpath lengths of the two paths through the system areequal, then the image that forms at the CMOS im-ager is a conventional intensity image of the scene. IfMirror 2 is at any other point, an optical path lengthdifference develops between the two paths throughthe system, and an interference image forms at theCMOS imager.

As Mirror 2 sweeps back and forth from one side ofthe zero-point to the other, the CMOS imager cap-tures a set of images. The imager’s capturing se-quence is triggered by the driving waveform to thepiezoelectric device attached to Mirror 2. At one endof the mirror motion, the imager is triggered and atthe other end of the mirror motion, the imager ceasescapturing. Each pixel on the imager thus captures aninterferogram (Figure 3) and the imager as a wholecaptures a 2D interferogram, which is a sequence ofimages in time (Figure 4). The finite range of dis-placements scanned by Mirror 2 limits the spectralresolution of the system.

2.2 Spectral emission & interferogram[2, 4]

Given that the length of photons is many orders ofmagnitude longer than the maximum optical pathlength difference encountered in the IIS, we can treat,for the calculation of the interferograms, all photonwave functions as having infinite extent. We also as-sume that our image sensor is ideal in photon de-

tection (flat frequency response and 100% quantumefficiency).

A photon of length L is represented by its wavefunction

ψ(z, t) =1√Leik(z−ct)

where k is the wave number and c is the speed of light.A photon passing through the IIS where Mirror 2 is atdisplacement x with respect to the zero-point has thewave function (after a slight change in coordinates):

ψ(z, t) =1

2√L{eik[(z−x)−ct] + eik[(z+x)−ct]}

=1

2√Leik(z−ct)(eikx + e−ikx)

=1√Leik(z−ct) cos(kx)

Thus, the probability of detection of this photon is

|ψ(z, t)|2 =cos2(kx)

L

∫ L2

−L2

∣∣∣e2ik(z−ct)∣∣∣2

dz

= cos2(kx)= cos2(2πνx)

A point of emission has a spectrum defined by theaverage photon flux spectral density function s(ν),ν ∈ [0,∞). If instead the spectrum is given in σ(λ),we can convert it by the equivalence s(ν) = λ2σ(λ) =ν−2σ(ν−1). Also define the modified spectrum s(ν) =12 [s(ν) + s(−ν)]. Then the expected photon flux atMirror 2 displacement x is

φ(x) =∫ ∞

0

s(ν) |ψ(z, t)|2 dν

=∫ ∞

0

s(ν) cos2(2πνx)dν

=∫ ∞

0

s(ν)[12

+12

cos(4πνx)]dν

=12

∫ ∞

0

s(ν)dν +12

∫ ∞

−∞s(ν)ei4πνxdν

=12Φ +

12

∫ ∞

−∞s(ν)ei2πν(2x)dν

=12Φ +

12F−1 {s(ν)} (2x)

=12Φ +

12F {s(ν)} (2x)

2

Figure 4: A few frames from a 2D interferogram

Clearly, φ(x)∆t, where ∆t is the integration time onthe imager, is the interferogram for that point. Thelast line above establishes that the basic relationshipbetween the emission spectrum of a point (ν-domain)and its interferogram (x-domain) is the Fourier trans-form.

Specifically, since s(ν) is a real and even function,φ(x) is also real and even. The first term of φ(x) isequal to half the total photon flux Φ at the point,whereas the second term is half the Fourier trans-form of the modified spectrum s(ν), evaluated at theoptical path difference 2x.

2.3 Peak width & coherence

From the interferogram decomposition expression forφ(x), we see that each peak in ν-domain correspondsroughly to, in the x-domain, the sum of (1) an offsetof the conventional intensity generated by the peak;and (2) a cosine at the center frequency modulatedby an enveloped that is the Fourier transform of thepeak shape. Therefore, a narrowband peak results ina slowly decaying sinusoidally varying interferogramand a wideband peak results in a quickly decaying si-nusoidally varying interferogram. By linearity, multi-ple peaks (or multiple light sources) in the ν-domainwill also see their interferograms summed in the x-domain. The amount by which the interferogram en-velope persists away from the zero-point is a quantitytermed coherence.

2.4 Different peak widths

Figure 5 shows details of three sources. On the leftare their emission spectra (of the σ(λ) type). On theright are interferograms simulated using the expres-sions derived here. The three sources have differentpeak widths and the interferograms have different en-velope structure.

2.5 Different peak frequencies

Figure 6 shows three LEDs of different center wave-lengths. With substantially the same peak widths,their interferograms have the same envelope struc-ture, which is invariant to shifts in center frequency.

3 Coherence detection

Distinguishing between narrowband peaks and wide-band peaks in spectra is one way to detect certaintypes of sources. By measuring coherence, we obtaina measure of the selectivity of the dominant peakin the spectrum. The simple relationship betweenpeak width and coherence means that low complex-ity methods to detect peak widths using the IIS arerealizable.

Several circuit-implementable processing methodsto detect coherence were developed. We describe aparticularly straightforward version.

3

400 500 600 700

Wavelength nm

-6 -4 -2 0 2 4 6

Mirror shift�m

0

1000

2000

3000

4000

5000

400 500 600 700

Wavelength nm

-6 -4 -2 0 2 4 6

Mirror shift�m

0

1000

2000

3000

4000

5000

400 500 600 700

Wavelength nm

-6 -4 -2 0 2 4 6

Mirror shift�m

0

1000

2000

3000

4000

5000

20 40 60 80 100 120 140 160 180 200 2200

5

10

15

20

20 40 60 80 100 120 140 160 180 200 2200

10

20

30

40

50

60

20 40 60 80 100 120 140 160 180 200 2200

10

20

30

40

50

60

70

80

90

Halogen

LED

Laser

Figure 5: Spectra and interferograms of three source types

4

400 500 600 700Wavelength �nm�

-6 -4 -2 0 2 4 6Mirror shift �Μm�

0

1000

2000

3000

4000

5000

400 500 600 700Wavelength �nm�

-6 -4 -2 0 2 4 6Mirror shift �Μm�

0

1000

2000

3000

4000

5000

400 500 600 700Wavelength �nm�

-6 -4 -2 0 2 4 6Mirror shift �Μm�

0

1000

2000

3000

4000

5000

Figure 6: Spectra and interferograms of three LEDs. From top to bottom: GaP green LED, InGaN greenLED, blue LED.

5

Figure 8: Demonstration of coherence detection. Each column displays information on one type of source.The top row shows interferograms measured from the IIS. The middle row shows the same after AC couplingand taking absolute value. The bottom row shows interferogram envelopes after low-pass filtering, with redbars indicating the locations of the selected zero-point and x0.

Separate ResultIntensity

Figure 7: Three light sources. From left to right:halogen, LED, laser.

First, if we are not interested in the center fre-quency of the light or its intensity, we can ignorethe non-varying intensity offset and the sinusoidal fre-quency in the interferogram. To extract the envelopeon the sinusoid, a standard AM demodulation cir-cuit suffices to demodulate the interferogram at eachpixel by AC coupling, taking the absolute value, andlow-passing. Now, suppose for each pixel-wise inter-ferogram of the 2D interferogram, φ(x) denotes theenvelope of the interferogram φ(x) thus obtained. Weuse φ(x0)

φ(0)as the coherence measure. The zero-point

location is selected ahead of time by maximum

Method 2

Figure 9: Coherence measure φ(x0)/φ(0).

detection on the interferogram envelopes fromacross the 2D interferogram; x0 is at an appropriateoffset from the zero-point, tunable to the envelopedecay rates (hence spectra peak widths) of interest.

4 Experiments & applications

4.1 Three sources

Figure 7 is a scene with the same three sourcesshown in Figure 5. They differ in spectral peakwidths. We demonstrate the ability to distinguishbetween them by coherence detection.

Figure 8 gives the measurements from the IIStaken at locations in the scene corresponding to thesources and the results of the pixel-wise processing

6

400 500 600 700Wavelength �nm�

-6 -4 -2 0 2 4 6Mirror shift �Μm�

0

1000

2000

3000

4000

5000

400 500 600 700Wavelength �nm�

-6 -4 -2 0 2 4 6Mirror shift �Μm�

0

1000

2000

3000

4000

5000

Lambertian surface emitting white light

Method 2Figure 10: Hidden tag: a“barcode” surface emitting a narrowly peaked spectrum (top spectrum) is concealedin a surrounding surface that emits a broadband spectrum (middle spectrum). While the spectra emittedfrom the two Lambertian surfaces are different, both emit the same white color (same color coordinates) andthe same intensity. Thus a conventional imager sees a homogenous surface (bottom left), while the IIS withcoherence detection reveals the secret (bottom right), i.e. the “barcode.”

7

described in the previous section. A map of the pro-cessed coherence measure over the scene is given inFigure 9. Regardless of the original relative sourceintensities, the three sources are easily distinguish-able.

4.2 Hidden tags

An important application of the IIS is invisible tag-ging. Specifically, coherence detection using the IIScan recover intentionally hidden spectral informationnot visible to a conventional imager (even a colorone).

For example, a conventional imager is unable todistinguish between a broadband green light and anarrowband green LED. Indeed, in this case, the sub-tle spectral difference would be invisible even to theeye. We can take advantage of this by placing secretnarrowband tags or beacons into natural broadbandscenes. Only a 2D hyperspectral device, such as theIIS with coherence detection, can reveal the tags orbeacons by specifically distinguishing them from thescene based on peak width differences. See Figure10.

4.3 Hyperspectral imaging

By inverting the 2D interferogram measured usingthe IIS, we can recreate the full spectrum at eachpixel. This allows us to generate a high dynamicrange, hyperspectral 2D image (and possibly video)of the scene from a grayscale imager.

First, we remove skew in the experimental inter-ferograms due to the nonlinear mirror motion at theendpoints of each sweep. Then, we remove the DCcomponent of the interferogram (assumed to be 1

2Φ)and apply the Fourier transform, inverting the pro-cess for simulating interferograms based on spectra.The noisy and window-limited interferogram mea-surement puts a fundamental restriction on the reso-lution and accuracy of the recovered spectrum. How-ever, we already obtain a good facsimile even with-out more sophisticated processing. Figure 11 showsa scene rendered from recreated spectra and Figure12 shows the process for one pixel of the image.

Figure 11: Scene of color (top) and white (bottom)LEDs rendered from a 2D interferogram captured onthe IIS.

0 100 200 300 400 50020

40

60

80

100

0 0.5 1 1.5 2

x 10−5

20

40

60

80

100

0 5 10

x 106

0

1

2

3

200 400 600 8000

2

4

6

8x 10

12

Figure 12: Inverting an interferogram. The raw cap-tured interferogram (top) at a pixel on one of thewhite LEDs is compensated for skew due to mirrormotion (middle) and inverted to obtain s(ν) (bottomleft) and σ(λ) (bottom right). The actual spectrumis the middle panel of Figure 10.

8

5 Conclusion

The interfermoetric imaging spectrometer combinesthe compact spectral acquisition capabilities of an in-terferometer with the 2D parallel operation of an im-age sensor array to form a 2D hyperspectral imager.The x-domain in which it operates and in which in-terferogram images are generated, is well suited forsimple yet powerful processing methods to detect thespectral peak widths anywhere in a 2D scene usingcoherence detection. These methods also lend them-selves well to feasible implementations. This enablesa number of imaging scenarios and valuable applica-tions such as the hidden tags.

Further work continues to characterize the system,including analyzing its noise properties and the exacteffects of mirror motion on spectral resolution.

6 Acknowledgement

I would like to thank Ulrich Barnhofer and SameerBhalotra for providing much guidance and for havingbuilt and troubleshooted a device sensitive to evensmall misalignment. As this is a joint work, manyideas described are attributable to them. I also thankProfessor Miller for letting me sign up for EE391 towork on this.

References

[1] S. Kleinfelder, S. H. Lim, X. Q. Liu, and A. ElGamal, “A 10,000 frames/s CMOS digital pixelsensor,” in IEEE Journal of Solid State Circuits,Vol. 36, No. 12, Pages 2049-2059 (2001).

[2] E. Hecht, Optics, Addison-Wesley, Reading, MA(1987).

[3] J. E. Chamberlain, The principles of interfero-metric spectroscopy, John Wiley & Sons, NewYork (1979).

[4] R. J. Bell, Introductory Fourier transform spec-troscopy, Academic Press, New York (1972).

9