18. November 2007

Johannes Fürtler

100 kHz Color Line Scan Cameraxposure 1300c

Johannes Fürtler

Booth 4A08Austrian Research Centers GmbH – ARC

smart systems divisionHigh Performance Image Processing group

28. November 2007

Johannes Fürtler

Austrian Research Centers GmbH (1/3)

HQ: Vienna10 sites in Austria>1,000 employees

38. November 2007

Johannes Fürtler

Austrian Research Centers GmbH (2/3)

The four interdisciplinary research areas

48. November 2007

Johannes Fürtler

DivisionFunctional Materials

DivisionBiogenetics

DivisionLife Sciences

DivisionBiomedical Eng.

DivisionRadiation Safetyand Applications

LKR{Kplus - Light metals}

ECHEM {Kplus -Electrochemistry &

Surface Techn.}

DivisionSmart Systems

DivisionMobility

{arsenal research}

Advanced Computer Vision

{Kplus ACV}

DivisionEnergy

{arsenal research}

DivisionNano-System-Technologies

DivisionSystems Research

DivisionResearch Studios

Personnel & Legal Affairs Finance & Controlling R&D Strategy

Strategic DivisionHealth

Technologies

Strategic DivisionMaterials

Technologies

Strategic DivisionInformation

Technologies

Strategic DivisionMobility

& Energy

ARC Management

Nuclear Engineering Seibersdorf

{NES}

Corporate Staff

TechnologyTransfer

Austrian Research Centers GmbH (3/3)

High Performance Image Processing group

58. November 2007

Johannes Fürtler

Overview xposure 1300c

High speed color camera (monochrome model available)USP: color line-scan camera (1,280 px @ 100 kHz)Area scan mode

Camera features1. Multiple Xposure (MX) method2. Image preprocessing3. Application specific processing

68. November 2007

Johannes Fürtler

How to build a 100 kHz color line-scan camera

High speed CMOS area sensorpartial read out of few rows 2 us per line possibleRGB Bayer pattern

High-end FPGAInterface

78. November 2007

Johannes Fürtler

CMOS area image sensorarbitrary read-out of sensor data - sophisticated sensor control possible global shutterno pixel bloomingsimultaneous read-out during next exposure

88. November 2007

Johannes Fürtler

Compact camera integrationFPGA: sensor control and complex image processing

FPGA … field programmable gate arraySRAM … sttica random access memoryDAC … digital to analog converterFLASH … non volatile memory deviceCL0, CL1 … Camera Link™ connector

98. November 2007

Johannes Fürtler

Need for speed (1/3)

ExampleIndustrial quality inspection of printed surfaces (e.g., banknotes, stamps and web inspection)

Different classes of print flaws are to be distinguishedlack of the print, blotches, smears, scratchers, …

108. November 2007

Johannes Fürtler

Need for speed (2/3)

Sheet transportation and imagingSeveral cameras (e.g., different spectral sensitivity)

118. November 2007

Johannes Fürtler

Need for speed (3/3)

Image acquisition with line-scan camerasresolution d (in transport direction, e.g., 0.1 mm)transport speed V (e.g., 10 m/s)line rate = V / d

Requirements100 kHz line rateexposure time <10 µs

128. November 2007

Johannes Fürtler

Multiple Xposure method

138. November 2007

Johannes Fürtler

100 kHz => 10 us period4 lines can be read out within 10 us (including 2 us for pipeline setup)Innovation: Multiple Xposure (MX) method

Light sensitivity

Image sensor

e.g., 1,280 pixels widthe.g

., 1,02

4 pixe

ls he

ight e.g., 4 lines

148. November 2007

Johannes Fürtler

Multiple Xposure principleFor every object point a number of exposures are summed up (e.g., two exposures as shown below)

o00 o01 o02 o03 o04 o05 o06 o07

o10 o11 o12 o13 o14 o15 o16 o17

o20 o21 o22 o23 o24 o25 o26 o27

o30 o31 o32 o33 o34 o35 o36 o37

o40 o41 o42 o43 o44 o45 o46 o47

o50 o51 o52 o53 o54 o55 o56 o57

p00 p01 p02 p03 p04 p05 p06 p07

p10 p11 p12 p13 p14 p15 p16 p17

o00 o01 o02 o03 o04 o05 o06 o07

o10 o11 o12 o13 o14 o15 o16 o17

o20 o21 o22 o23 o24 o25 o26 o27

o30 o31 o32 o33 o34 o35 o36 o37

o40 o41 o42 o43 o44 o45 o46 o47

o50 o51 o52 o53 o54 o55 o56 o57

p00 p01 p02 p03 p04 p05 p06 p07

p10 p11 p12 p13 p14 p15 p16 p17

Object

Sensor

Object’s movement

158. November 2007

Johannes Fürtler

for every object four exposures are summed upthree adders and three delay stagessingle output row

Example: MX for 4 monochrome stages (MX-4)

168. November 2007

Johannes Fürtler

Bayer filter matrix: two different types of rows (GR and BG)output is one double row (GR + BG)color information is increased two timesvalue for green is exactly known for every object point

Obj

ect’s

Mov

emen

t

Example: MX for 2 color stages (MX-2c)

178. November 2007

Johannes Fürtler

0

5

10

15

20

25

30

0 2 4 6 8 10 12 14 16 18

MX Stages

digital responsivity

RMS read noise

longer effective exposure time: improved light sensitivityelectronic shutter: arbitrary exposure timesno bloomingimproves signal to noise ratio

Advantages of Multiple Xposure

188. November 2007

Johannes Fürtler

Traditional TDI (CCD-TDI)implemented in CCD (charge coupled device) image sensors onlyintegration of photoelectric charges before conversion to digital numbersno shutter, hence difficult exposure control (fixed object speed is necessary)CCD tends to blooming (charge spills over into neighboring pixels)

Multiple Xposure vs traditional TDI (1/2)

198. November 2007

Johannes Fürtler

Our innovationfrom the analog domain to the digital domainwe generalized the original CCD-TDI principleFPGA is used for implementation of MX algorithmCMOS area image sensor is used for image acquisition

Multiple Xposure overcomes CCD-TDI drawbacksarbitrary exposure time (global shutter)no pixel blooming

Multiple Xposure vs traditional TDI (2/2)

208. November 2007

Johannes Fürtler

Image preprocessing

218. November 2007

Johannes Fürtler

Image preprocessing (1/2)

MX FFC CFA CTR BinningCLD

Analog OffsetAnalog Gain

(Ref. Voltages)global Number of MX Stages

FFC OffsetsFFC Gains

Mode-Switch (Color or Monochrome)

Color TransformationMatrix (3x3)

Correction Table of Lens Distortion Values

FPGA

Ext. Line Trigger

Binning Format

RGBimage

2*MX rows per trigger

1 double row per trigger

(R, G0, G1, B)

1 double row per trigger

(R, G0, G1, B)

1 row of RGB-triplesper trigger

1 row of RGB-triplesper trigger

1 row of RGB-triplesper trigger

1 row of RGB-triplesper trigger

image quality improvement

228. November 2007

Johannes Fürtler

Flat field correction, FFCColor filter demosaicing, CFA

interpolation of missing R and B valuesimproved algorithm as G is known for every pixel

Color transformation3x3 matrix multiplication

Correction of lens distortionsBinning

Image preprocessing (2/2)

238. November 2007

Johannes Fürtler

configurable transfer functionone dimension

Example:

Compensation of geometric distortions

Abbildung 6: (a) original image, (b) corrected image, (c) graph of the second term.

( ) 2SRC 12804-127963)( ⋅⋅⋅+= iiiix

248. November 2007

Johannes Fürtler

the xposure 1300c supports measurement and digitalization of an analogue signalthe actual value can be included in line header “pixels”e.g., exposure sensor or temperature sensor, etc. which can be used for automatic calibrationin addition index information can be added to the image data (line counter, etc.)

Image data header

258. November 2007

Johannes Fürtler

Application specific processing

268. November 2007

Johannes Fürtler

High-end FPGA enables complex image algorithms to be implemented within the cameraExtreme parallel processing possibleHigh performance

Application specific processing (1/3)

278. November 2007

Johannes Fürtler

Example: Feature extraction and data reduction in the camera

G3x3 … 3x3 GaussianG5x5 … 5x5 GaussianSH … horizontal SobelSV … vertical SobelGrad … gradientMux … multiplexer

Application specific processing (2/3)

288. November 2007

Johannes Fürtler

Example: Austrian stamp “Am Steinernen Meer”* Data reductionColor Feature Edge Feature Luminance Feature 1.200 dpi scan

* A.Tuma, 2002

Application specific processing (2/3)

298. November 2007

Johannes Fürtler

xposure 1300c enables acquisition of high quality color images from objects moving with high speedEasy integration into industrial production facilitiesApplication as line-scan or as area-scan camera (color or monochrome operation possible)Easy installation through electronic adjustmentApplication of cost-effective lenses, since systematic errors can be compensated within the cameraApplication specific processing can be integrated

Conclusions (1/2)

308. November 2007

Johannes Fürtler

Multiple Xposure methodHigh quality color images @ high speed 100 kHz color line-scan camera (monochrome model available)Higher light sensitivity due to longer effective exposure timeFree choice of exposure time Improved SNR

Conclusions (2/2)

100 kHz color line-scan camera

high sensitivity

MX method

318. November 2007

Johannes Fürtler

The patented MX method enables direct camera improvements due to advances in sensor and FPGA technologyxposure 4000c coming soon (>2,300 pixels line width)

Outlook

328. November 2007

Johannes Fürtler

Thank you for your attention! Questions?

More questions? Visit us at booth 4A08!