smart ccd image sensors for optical metrology and machine vision.pdf

8/11/2019 Smart CCD image sensors for optical metrology and machine vision.pdf

1/4

Smart CCD Image Sensors for Optical Metrology and Machine Vision

T. Spirig, P. Seitz,

0

Vietze

Paul Schrerrer Institute Zurich

Badenerstrasse 569, CH-8048 Zurich

e-Mail: Thomas. Spirig psi.ch; [email protected]; Oliver.Vietze@ psi.ch

Abstract -Tw o types of CCD image sensors are described.

The first sensor is a two-dimensional, synchronous

detector/demodulator (Lmk-In CCD) of spatially

modulated light fields for applications in heterodyne

interferometry

and

time-of-flight range imaging.

Simultaneous measurements of amplitude, phase and

background level are carried out at e ach pixel site. This

is made possible by the principle of synchronized,

periodic multi-tap sampling and photo charge

accumulation. The second sensor, the Convolver CCD

is capable

of

performing image acqu isition and real-time,

parallel convolution with an arbitrary kernel. Tap weight

accuracies of typically 2 of the largest tap values have

been obtained

for

a variety of linear filters that are

commonly

used

in machine vision. The sensors have been

realized by using a comm ercially available multi-project

wafer CMOS/CCD process. For both sensors, the

principle, design, operation and measurement results are

presented and discussed.

I. INTRODUCT ION

Charge-coupled devices (CCDs) are used in most of

todays image sensing applications, although the primary

motivation for their development was charge storage [I].

Soon, it became apparen t that the CC D principle is also well

suited for the processing of analog signals in the charge

domain [2]. Using modem semiconductor technologies, it is

possible to fabricate photosensors whose geometry and

functionality are adapted to specific sensing tasks of various

optical measurement techniques. In the present work,

two

types of such sensors are presented. The first type, called

lock-in CCD, is a

two

dimensional array of pixels, each of

which is a synchronous detectoddemodulator for spatially

modulated light fields. This is made possible by a

synchronous photo-charge detection a nd storage s proposed

in an incomplete form with

a

modified CCD in Ref. [3]. Th e

Lock-In CCD is described in Section

2.

The second type of

sensor, described in Section 3, is an image sensor capable

of

carrying out convolutions with arbitrary kemels during the

exposure. The possibility of realizing such a device

was

initially suggested by Beaudet [4] oth sensors have been

realized using Orbits Foresight 2pm N-well CMOYCCD

process [ 5 ] In the concluding Section 4 the obtained results

are discussed and possible applications are presented.

11. LOCK-IN CCD

In typical image sensing applications a scene is

optically projected onto the image sensor, which has to

detect the local light intensity of an essentially stationary

scene. There are circumstances, however, in which the

sought information is not the local light intensity, but is

encoded in a modulation parameter of an oscillating

waveform. One might be interested in the local phase of the

modulated light, the amplitude and the background

illumination. The well-established measurement technique

by which local phase, amplitude and offset of a sinusoidal

electrical signal are measured, is called lock-in technique.

Conventional synchronous detectors (lock-in detectors),

often based on phase-locked loop (PLL) technology, are

essentially restricted to the analysis of one temporal signal at

a time. If modulated light is de tected with 3 or more samples

per period, its mean brightne ss level B (offset), phase cp and

amplitude

A

can

be

determined unambiguously. Figure

1

illustrates the measurement principle for the

so

called four-

bucket technique. Th e charge is integrated during time

intervals I1 to I4 of equal length At within one modulation

period T. Four charge packets per pixel, o to a3 are then

obtained, which have to

be

stored at four different spatial

locations. The charge signals a0 to a3 are converted into the

t

T

Fig.

1

Measurement principle

of

the Lock-in CCD. During one

modulation period, four charge packets *a3 are created

by

integrating the photo-generated charge. With these

four

packets

the phase

9

he offset B and &he amplitude can be determined.

(0-7803-2943-0)

11


2/4

sought amplitude

A,

phase cp and mean brightness level B

by using the relationships, described in Reference

[6] .

The architecture of one pixel is illustrated

schematically in Figure 2. One pixel consists of a photo-gate

(PG),

a

dump-gate (DG), a dump-diffusion (DD) and

4

transfer gates (TGo-TG3). In our implementation the area of

the photogate

is

10.5p m x 41.5 pm. Two vertical four-phase

CCD lines are located adjacent to each pixel. The four

transfer-gates act as sw itchable connections between PG and

the four-phase CCD. The four-phase CCD is covered by a

metal layer in order to prevent charge smear during read out.

Tests have been carried out using a n image sensor with

10

x

fo

Fig.

2.

Schematic layout of one pixel.

It

consists

of a

photo-gate

(PG), four transfer-gates (TGo-TGs),

a

dump gate (DG) and

a

Dump diffusion (DD). The charge packets are stored under the

gates fp of the two adjacent CCD lines.

15

pixels, with a horizontal pitch of 92.75 pm and a vertical

pitch of

80 pm.

The op eration occurs in

two

stages, signal integration

and storage,

followed by read out. During the signal

integration phase, the modulated light intensity produces a

modulated photo-current, which is integrated on the

MOS

capacitor of the photosite. Synchronously with the master

oscillator this integrated charge is transferred and

accumulated in one of the shielded storage sites by

appropriately clocking the transfer gates and the photogate.

Th e exposure time and the location where the signal charg e

is stored can be chosen by programming an appropriate

clocking sequence. The pixel can be cleared, e.g. for

shuttering operations, by pulsing the dump -gate high and

the photo-gate low. The charge is then transported to the

dump diffusion.

In the read out phase, the electrodes of the line

transfer CCD are clocked sequentially, transporting the four

charge packets per pixel

in

shielded CCD lines to the output

stage. The signal charge is converted to a voltage by use of a

floating diffusion output stage with a single on-chip p-

channel

MOSFET

acting

as

a source follower.

The CCD has been tested using a light source with a

modulated intensity. The center wavelength is a t 630 nm

and

the beam was focused on the CCD

as

described in Reference

[7] in m ore detail. The m odulation frequency was chosen to

be

100.4

kHz.

This is

a

typical frequency for the intended

application of the lock-in CCD in heterodyne interferometry.

The charge was collected over

100

modulation periods.

Figure 3 shows the measured phase vs. the true master

oscillator phase for 12 different phases starting at

(pmaster = 0

and increasing with a step size of 30 . An individual offset

for all outputs per pixel was subtracted and the resulting

signals were normalized. The r.m.s deviation of measured

vs. true master oscillator phase was

3.1

degrees (relative

phase error of

0.009).

The relative deviation from true

amplitude was

0.05.

C 0 81

measured values

l

average

master osc illator phase [deg]

Fig.

3.

Amplitude measurements obtained with the Lock-in pixel

CCD, exposed to a sinusoidally oscillating LED light source. The

modulation frequency

was 100.4

kHz. The relative amplitude

measurement deviation from the true master oscillator amplitude

was 0.05.

LI. CONVOLVER CCD

any signal and image processing functions are

linear, i.e. one-or two dime nsional convo lutions with specific

kemels

[8].

The two-dimensional result

of

the convolution is

determined

s

a linear combination of neighboring pixel

intensities with suitable weights, as described in Reference

An interpretation of the convolution is that the

picture is shifted laterally in both dimensions and for each

position the pixel values are multiplied with a different

weight and accumulated to the sum, resulting in the new

pixel value.

As

the CCD is capable

of

shifting charge

laterally in two dimensions, this interpretation leads to a

naturally parallel implementation of the convolution using a

CCD.

The

weighting can be realized by varying the exposure

times for the different lateral

shifts.

This principle holds

only for kemels with positive weights. The problem with

negative weights can be solved by performing two

convolutions each with only positive weights and performing

the difference either on or off-chip. The convolution CCD

consists

of

bidirectional CCD columns and bi-directional

rows and at the intersections of rows and columns the

~71.

12


3/4

photosites are located. Each pixel has its

own

associated

storage area, realized as an additive CCD column with

separate gates in parallel to the main CCD column. In our

implementation, the area of the photo-gates is

30

x

30

pm2

and the pixel pitch is

63 pm

horizontally and 65 pm

vertica lly leading to a fill factor of

22 .

First tests have been

performed w ith

16

x 16pixels.

The Point spread function (PSF) can

be

meas ured by

illuminating a single pixel with a

20

pm diameter light spot.

Four types of filters have been chosen, which are commonly

used in machine vision. These are: I)nisotropic Gaussian

filters; (11) Canny filters (the first deviation of Gaussian

filters); (111) Lap lacian of Gau ssian filters (L OG -filters) and

(IV) quadrature

pair

filters (Gabor filters).

A

set of

7 x

7

filter coefficients was calculated for each of these four

continuos functions. They were used to set the exposure

times. As the image is a single spot, there is a direct

correspondence between the calculated coefficients and the

measured ou tput voltage. To obtain a fig ure of acc uracy , the

image was offset corrected and normalized to fit the filter

coefficients and an r.m.s. dev iation is obtained. We found

excellent correspondence between measured values and true

filter values. Typical r.m.s. deviations from the ideal filter

characteristics are between 1-2% of the largest kernel tap

value. As an example the tw o-dimensional point spread

function of a Laplacian of Gaussian filter

is

illustrated in

Figure

4

The following three compo nents contribute to the

error:

(1)

The read-out noise of the output amplifier,

(2)

the

dark current noise, a shot noise compo nent that depends on

the exposure time and

3)

the non-ideal charge transfer

efficiency (CTE). In our case, most of the error is attributed

to the imperfections of ou r measuremen t technique, largely

due to the non-linearity of the output stage. In addition

filters without negative elements in their filter mask have

slightly smalle r r.m.s. erro rs since only one convolution was

Fig.

4

Two-dimensional point spread function of

a

Laplacian

of Gaussian filter measured

with

the convolution CCD. The

r.m.s deviation of the obained point spread

function from the

ideal filter was less than 2 .

performed and hence the expo sure time

for

these filters was

smaller.

ILL DISCUSSION

Two image sensors have been presented with on-chip

signal processing capabilities. These sensors have been

fabricated with a commercially available CM OSK CD

process. The Lock-in CCD accurately measures the phase

and the amplitude of light modulated at 100

kHz

This is

ideal for the intended application in heterodyne

interferometry, higher modulation frequencies are possible,

since the charge coupled transfer principle has been

demonstrated at frequencies up to 32 5 MHz in silicon [9].

Interesting applications for the convolver CCD are in

real time object recognition using m atched filter techniques.

By providing two or more storage sites per pix el, it would be

possible to perform two convolutions and hence to detect

keypoints in real-time

[lo]

Either sensor could

also

find

application in motion detection due to its capability of

storing two (convolver CCD)

or

four (Lock-in CCD)

successive pictures before readout. It is noted that spatial

resolution of the CCDs, i.e. the modu lation transfer function

MTF) is degraded due to carrier diffusion in the

semiconductor sensor

Ell],

[12]. Light with a long

wavelength produces a large penetration depth of the

photons in

the

semiconductor, leading to a larger cross-talk

and hence a reduced spatial resolution for the convolver

CCD and a significantly reduced dynamic range for

measurements with the lock-in CCD .

The presented CCD structures offer an interesting

alternative to the active pixel sensors

A P S ) ,

where the

signal processing is performed with a conventional CMOS

circuitry. The CCD approach has the advantage that the

signal processing is carried out in the charge domain,

without any noise contributions from the device other than

dark noise. The novel device structures presented here are

merely an exam ple of possibilities that CCD pro cessing has

to offer in the field of optical metrology and m achine vision.

ACKNOWLEDGEMENT

We gratefully acknowledge the invaluable help from

many people at the Paul S cherrer Institute in Zurich. Special

thanks goes to M.Kuhn, who provided the design and the

realization of the o ptical setup,

ro

P.Metzler, who designed

and fabricated the trapezoidal clock drivers and to

J.M.

Raynor for his invaluable help with analog and digital

electronic problems. We would also acknowledge

stimulating discussions

wt

0.Kubler and F.Heitger at the

Federal Institute of Technology (ETH) Zurich. This work

was supported in part by the Swiss priority programme

Optique, project number

524

13


4/4

REmRENCES

[41

[71

W.S. Boyle and G E Smith, Charge Coupled

Semiconductor Devices, Bell Syst. Tech. Jour. Vol.

C.H. Shuin and M.F. Tompsett, Charge Transfer

Devices,

Academic Press

Inc., New York, 1975.

H.Pove1, H. Aebersold and J.O. Stenflo,

Chargcoupled device image sensor as a demodulator

in a 2-D polarimeter with a piezoelastic modulator,

Applied Optics

Vol. 29, pp. 1186-1190, 1990.

P.R.Beaudet, Time and Space Multiplexing Focal

Plane Convolvers, Proc. SPZE Vol. 107

1

pp. 90-99,

1989.

Orbit Semiconductor Inc., 1230 Bordeaux Drive,

Sunnyvale, California, 94089, USA.

T.Spirig, P.Seitz, 0 Vietze and F.Heitger, The

Lock-In CCD - Two-Dimensional Synchronous

Detection of Light, submitted to

IEEE J Quantum

Electronics in print (1995).

P. Seitz, T. Spirig, 0 Vietze and K. Engelhardt,

Smart sensing using custom photo-ASICs and CCD

technology,

Optical Engineering

to appear in

August 1995.

W.K.Pratt, Digital Image Processing, 2nd Edition,

Wiley Sons, New York, 1991.

A.L. Lattes, S.C. Munro e and M.M. Seave r,

Ultrafast Shallow-Buried-C hannel CCDs with Built-

In Drift Fields,

IEEE Electr. Dev. Lett.

ol. 12, pp.

F. Heitger, L. Rosenthaler, R. von der Heydt, E.

Peterhans and 0 Kubler, Sim ulation of neural

contour mechanisms: From simple to end-stopped

cells,Vision Research

32, pp. 963-981, 1992.

D.H.Seib, Carrier diffusion degradation

of

modulation transfer function in charge coupled

imagers, IEEE Trans. Electron Devices vol ED-21,

E.G. Stevens and J.P. Lavine, An analytical,

aperture, and two-layer carrier diffusion

MTF

and

quantum efficiency model for solid-state image

sensors, IEEE Trans. Electron Devices vol. 41, pp.

49, pp. 587-593, 1970.

104-107, 1991.

pp. 210-217,

Mar.

1974.

1753-1760, Oct. 19 94

14

smart ccd image sensors for optical metrology and machine vision.pdf

Documents