hswg andy harvey:[email protected] spectral imaging at heriot watt university dr andy r...

25-May-05 1HSWG Andy Harvey:[email protected]

Spectral Imaging at Heriot Watt University

Dr Andy R Harvey

School of Engineering and Physical Sciences

Heriot Watt University

Edinurgh, EH14 4AS

Tel +0131 451 3356

[email protected]


Some Heriot Watt spectral imaging solutions

• Birefringent 2D Fourier-transform imaging spectrometer (FTIS)

• Snapshot 2D foveal imaging spectrometer (OFIS)• Snapshot 2D imaging spectrometer (IRIS)


• Conventional FTIS offers• High SNR in low flux

• MWIR, twilight

• Very high spectral resolution• Wide spectral range

• But conventional time-sequential interferometry in real-world applications is highly problematic

BirefringentFourierTransformImaging Spectrometer

Fixedmirror

Scanning mirror

Detector array


Birefringent FTIS

• Mechanical sensitivity of conventional FTIS makes real-world applications almost impossible

• Introduce temporal path difference with scanning Wollaston prisms

• Inherently vibration insensitive since path difference due by birefringence within a single crystal and common path

• Optical gearing reduces required accuracy of movement by a factor ~200

221 tantantan2 hdnn oe


Movie of spectral image cube

Colour image


Foveal hyperspectral imaging in 2D

OpticalFibre-coupledImaging Spectrometer

• Real-time hyperspectral imaging in 2D would require excessive information throughput• GVoxel/sec

• Bottlenecks include• detector – 20 MVoxel/sec• Computer processing

• Biological systems with this problem employ a scanning fovea….


Foveal hyperspectral imager: OFIS Schematic

Dispersive 1D

Imaging spectrograph

14x14 pixels

Panchromatic detector

Fibre bundle

Intermediate Image plane

Composite panchromatic image with hyperspectral

fovea

Scene

196 pixels


OFIS: Hardware & raw data

The hyperspectral fovea assembly:• Custom fibre optic image refromatter • 1D dispersive hyperspectral imager • CCD camera

Spatial extent

Wavelength

400 nm

700 nm

Fir

st

fib

re

La

st

fib

re

• Raw image at CCD prior to reformatting


OFIS: Movie demonstrating real-time spectral ID with simple recognition

• Colour image


Snapshot spectral imaging in 2D

ImageReplicationImaging Spectrometer


Image Replication Imaging Spectrometer: IRIS• Single image multiplexed onto 2N passband

images• ‘100%’ optical efficiency• Snapshot image

• no temporal misregistration• Trade spectral resolution for FoV

• Low resolution, wide FoV• High resolution, small FoV• Gas detection

• High spectral resolution• Few Bands• Modest FoV

• Conceptually related to Lyot filter• World’s only snapshot,

2D spectral imager (almost !)

Large formatdetector

SpectralDemultiplexor


• Wollaston prism polarisers replicate images• Each Wollaston prism-waveplate pair provides both cos2 and sin2 responses

• All possible products of spectral responses are formed at detector

Exploded view of N Wollaston prisms N wave plates

2N spectral images at detector Field

stop

Input polarizer

)(sin

)(cos2

2

)2(sin

)2(cos2

2

)4(sin

)4(cos2

2

IRIS snapshot spectral imager:


Components & Assembly

• 8 channel system• 3 Quartz retarders• 3 Calcite Wollaston prisms


Absolute total transmission

• Bandpass filter & polariser dominate losses

• Improved system: T>80%

• Theoretical throughput is 2n times higher than for other techniques!

• Demonstrated 96% transmission for IRIS-only components

0

25

50

Re

sp

on

se

(%

)

Absolute response curves in polarised light


An example medical application:

Blood oxymetry in the retina


Requirements for a snapshot technique: retinal imaging

• Improved calibration

• Patient patience

• Remove misregistration artefacts; imperfect coregistration arises due to

• Distortion of eye ball with pulse

• Variations in imaging distortion between images

• Similar issues with other in vivo applications

• Imaging epithelial cancers

PC15


Blood oximetry

• Optimal spectral band for retinal oximetry• Vessel thickness ~ optical depth• 570-615 nm• Eight bands approximately equally spaced

0

2

4

6

8

10

12

14

16

18

20

565 575 585 595 605 615 625

Wavelength (nm)

Tra

nsm

issi

on (

%)

80

40


Spectral Retinal Imaging • Difficult imaging conditions render application of traditional HSI

techniques problematic• IRIS enables real-time and snapshot spectral imaging

Canon CR4-45NMCR4-45NM


Video sequence recorded with low-power, CW tungsten illumination


Retinal image recorded with flash illumination


574581585592595603607613

Coregistered and PCA images

PC1PC2PC1 & PC2


Application to microscopy:Imaging of multiple fluorophors

• IRIS fitted to conventional epi-fluorescence microscope

• Germinating spores of Neurospora crassa stained with• GFP – nucleii fluoresce at 510 nm• FM4-64 – membranes fluoresce at >580 nm0

25

50

Re

sp

on

se

(%

)

Thi

s do

cum

ent g

ives

onl

y a

gene

ral d

escr

iptio

n of

the

prod

uct(

s) o

r se

rvic

es a

nd

exce

pt w

here

exp

ress

ly p

rovi

ded

othe

rwis

e sh

all n

ot fo

rm p

art o

f any

con

trac

t.

May 2005 Hyperspectral Working Group 24

MWIR IRIS

Consists of: COTS Phoenix MWIR Camera Specac Polariser IRIS II Optical Telescope


Conclusions

• The transfer of spectral imaging from scientific to military and laboratory applications must address the needs of high SNR, accurate coregistration and logistics.

• No single technique can satisfy all requirements simultaneously

• ‘Horses for courses’

• New techniques such as described here illustrate how these requirements can be satisfied

• Similar issues occur in both military and civilian (eg medical) applications introducing significant scope for dual use.


Additional information

• Linked by previous slide buttons


The co-registration problem

• Co-registration required for time sequential direct and FT imaging

• Not for snapshot/fully-staring

• Misregistration of spectral images distorts spectral basis sets

• Video spectrum frame rates insufficient to freeze motion from most aerial platforms

Target


The magnitude of the co-registration problem

• Co-registration should be better than 1/20 - 1/50 of a pixel

• Deployment of time sequential DIS and FTIS will normally require ‘step and track’

0.01 0.02 0.05 0.1 0.2 0.5 1offsetHpixelsL0.0001

0.001

0.01

0.1

1

10

lanoitcarfrorre

0.00010.5 1 1.5 2


Bandpass functions

• Bandpass are overlapping bell shapes• Can be optimised by adjusting waveplate thickness and dispersion

18000 20000 22000 24000 26000 28000Wavenumber Hcm -1L

0.2

0.4

0.6

0.8

1

THîL


Spectral discrimination

2 3 4 5 6 7 8Spectral bin

0.2

0.4

0.6

0.8

1

desilamronesnopser

Conif

Conc

Paint

Al

2 3 4 5 6 7 8Spectral bin

1

2

3

4

5

6

7desilamron

esnopser

Conif

Conc

Paint

Al

• Bell-shaped IRIS transmission functions tend to smooth spectra• Typically 6% reduction

in separation in 8D spectral space

• 8x improvement in SNR

Contiguous ‘top-hat’

IRIS


1D image x path difference

Fixedmirror

Scanning mirror

Detector array

N

NxNy(t)

N

NxNy(t)

FTFT

N(t)

NxNy

N

NxNy(t)

Direct Imaging Spectrometry (Fourier) Transform Imaging SpectrometryT

emp

oral

ly s

can

ned

Sn

apsh

ot/f

ull

y st

arin

g

N(t)

NxNy

FT

N

NxNy

• No temporal coregistration problem

• The traditional technique for 1D remote sensing

• 2D very immature….• IRIS• OFIS

Summary and novel HWU techniques in red• Very high spectral resolution• Highest SNR in low-light conditions • The optimum technique for MWIR• Unsuitable for poorly controlled

environments...• FTIS

• Mature• The traditional

technique for 2D static spectral imaging

• Low MPLX efficiency


1 10 100 1000 10000Tspect � T�

0.5

1

2

5

10

20

RNSSITF�RNS SID

4

8

16

32

64

128

256

512

DIS

FTIS

SNR

SNR

TTtot /

Ratio of SNRs in 3-5 m band -temporal scan

1500 m nadir path

40 Hz,10 bands

Zero range

1 Hz,10 bands


1 10 100 1000 10000Tspect � T�

0.5

1

2

5

10

20

RNSSITF�RNS SID

4

8

16

32

64

128

256

512

DIS

FTIS

SNR

SNR

TTtot /

Ratio of SNRs in 8-14 m band - temporal scan

Zero range

40 Hz, 10 bands

1500 m nadir path


IRIS:FTIS SNR

1 10 100 1000 10000Tspect � T�

0.02

0.05

0.1

0.2

0.5

1

RNSSITF�RNS SIRI

4

8

16

32

64

128

256

512

Zero range

40 Hz, 10 bands1500 m nadir path


Lyot filter: principle of operation

n=1 � l Cos2@pîDDCos2@pîDDCos2@2pîDDCos2@pîDDCos2@2pîDDCos2@4pîDDCos2@pîDDCos2@2pîDDCos2@4pîDDCos2@8pîDD

PolariserWaveplate


Optical scaling laws

Hamamatsu

ORCA-ER

Inputs:

FoV

Sub image size on CCD

CCD pixel size

Primary lens magnification & F#

Collimating lens back focal distance, focal length & front element diameter

Prism birefringence

Outputs:

Field stop size

Collimating lens rear element diameter

Splitting angles, apertures & depths of prisms

Apertures of retarders, polarisers and filters

Imaging lens focal length & front element diameter

Field stopCollimating

lens

Bandpass

filter

Imaging

lens

Camera

Polariser, retarders & Wollaston prisms

(index matched)Primary lens


Spectral retinal Imaging• By 2020 there will be 200 million visually-

impaired people world wide• Glaucoma, diabetic retinopathy, ARMD• 80% of those cases are preventable or

treatable • Screening and early detection are

crucial • Spectral imaging provides a non-invasive

route to monitoring retinal biochemistry• Blood oximetry, lipofuscin accumulation

800nm

Diabetic Retina

Normal Retina


Measured & predicted spectral responses


Imaging Concepts Group

• Research Group• Head

• Dr Andy Harvey• PDRA

• Dr Colin Fraser• Dr Eirini Theofanidou• Bertrand Lucotte

• PhD Students• Alistair Goreman• Asloob Mudassar• Gonzalo Muyo• Sonny Ramachandran• Ied Abboud• Beatrice Graffula

• External PhD students • Ruth Montgomery (NPL)• Robert Stead (Thales)

• Funders/Collaborators• AstraZeneca • AWE • BAE Systems• DSTL • EPSRC• NATO• NPL • QinetiQ• Royal Society• Scottish Enterprise• South Glos. NHST• SAAB• Thales


Research areas

• Imaging Concepts Group• Spectral imaging • Retinal imaging • Wavefront coding • Aperture synthesis imaging (optical and mm-wave)• Optical encryption for communications• mm-wave imaging• Biophotonics• Insect flight dynamics


Overview

• Introduction to spectral imaging• Spectral imaging techniques at Heriot-Watt University

• FTIS• Inherently robust FT imaging spectrometer

• IRIS• Snapshot, ‘100%’ optical throughput imaging spectrometer

• OFIS• Foveal hyperspectral imaging spectrometer

• An example application• Spectral imaging of the retina

• Conclusions


What are the issues

• High SNR required• >100

• No spatial or spectral multiplexing desirable• Fourier-transform

• in some conditions

• Accurate coregistration required (<1/20 pixel)• Snapshot spectral imaging preferred

• Spectral resolving power matched to requirement• 100s for data acquisition• ~10 for many applications

• As few as two if clutter allows (eg spectral lines)

• Detector is ‘information bottleneck’• 20 MVoxel/second per tap

hswg andy harvey:[email protected] spectral imaging at heriot watt university dr andy r...

Documents