image signal processing verhaert

20.10.2006

IMAGE SIGNAL PROCESSINGIMAGE SIGNAL PROCESSING

Slide 1

ww

w.m

aste

rsin

inno

vatio

n.co

m

IMAGE SIGNAL PROCESSING

Frederik Wouters

[email protected]

www.verhaert.com

VERHAERTINNOVATIONDAY – OCTOBER 20th, 2006

www.mastersininnovation.com

Commercially confidence – This presentation contains ideas and information which are proprietary of VERHAERT, Masters in Innovation®*, it is given in confidence. You are authorized to open and view the electronic copy of this document and to print a single copy. Otherwise, the material may not in whole or in part be copied, stored electronically or communicated to third parties without prior agreement of VERHAERT, Masters in Innovation®*.

* VERHAERT, Masters in Innovation is a registered trade name of Verhaert Consultancies N.V.

20.10.2006


Slide 2

ww

w.m

aste

rsin

inno

vatio

n.co

m

Agenda

Spec’s & constraints

Technical solutions Costs

System DefinitionApplication&

Market

ManagementChallenges

20.10.2006


Slide 3

ww

w.m

aste

rsin

inno

vatio

n.co

m

Product Development

Integrated Sensors

Control Systems

Mechanisms & Robotics

Cabinets & Housings

20.10.2006


Slide 4

ww

w.m

aste

rsin

inno

vatio

n.co

m

Application & Market

- 1m/s- Working temperature range 15 – 40°C- Movement tire ± 1 cm- Req. precision < 20 µm

20.10.2006


Slide 5

ww

w.m

aste

rsin

inno

vatio

n.co

m

Specification Definition

6760.166100339.1108461.7100508.1 22537 +⋅⋅+⋅⋅−⋅⋅ −−− xxx

20.10.2006


Slide 6

ww

w.m

aste

rsin

inno

vatio

n.co

m

Range Precision Repeatability (2 sigma)

Toe +/- 10° +/- 0,5’ +/-0,1’

Camber +/- 10° +/- 1’ +/-0,1’

Caster 0°-10° +/- 3’ +/- 1’

SAI 0°-20° +/- 0,3° +/- 0,2°


20.10.2006


Slide 7

ww

w.m

aste

rsin

inno

vatio

n.co

m - Pricing

- Options- 2 x DSP with IEEE 1394 -> custom design- 1 x Floating point DSP + 1 FPGA with IEEE 1394 -> COTS- 1 x Fixed point DSP + 1 FPGA with IEEE 1394 -> custom design- ...

Pricing

0

2.000

4.000

6.000

8.000

Giga Ethernet Firewire

Euro

ProcessingCameraTotal

Target pricing


20.10.2006


Slide 8

ww

w.m

aste

rsin

inno

vatio

n.co

m

Thermal Challenges- Thermal expansion of distance CCD – light source i.e.

- Normal distance of 250 mm- Working temperature range 15 – 40°C- I.e. 3 micron error introduced

20.10.2006


Slide 9

ww

w.m

aste

rsin

inno

vatio

n.co

m

Optical Challenges

- Optical errors- Speckle - Optical elements

- Aberration- Non linearity

20.10.2006


Slide 10

ww

w.m

aste

rsin

inno

vatio

n.co

m

Processing Challenges- Hard Realtime vs. Soft Realtime

RTOS Real-time Performanceby John A. Carbone, VP,

MarketingExpress Logic, Inc

Carnegie Mellon University18-849b Dependable Embedded SystemsKanaka Juvva http://www.ece.cmu.edu/~koopman/des_s99/real_time/

20.10.2006


Slide 11

ww

w.m

aste

rsin

inno

vatio

n.co

m

System Definition

- Hard Realtime vs. Soft Realtime- guaranteed worst-case response times

- 1m/s 30 fps (1image 10 Mb per 33,3 ms)

- 1280*1024 , 8 bit, 30 fps 300 Mbps

- 300 Mbps 5 x fixed point

6760.166100339.1108461.7100508.1 22537 +⋅⋅+⋅⋅−⋅⋅ −−− xxx

32 bit integer

20.10.2006


Slide 12

ww

w.m

aste

rsin

inno

vatio

n.co

m

Algorithms

20.10.2006


Slide 13

ww

w.m

aste

rsin

inno

vatio

n.co

m

- Noise filtering - Smoothing

- Binning - Denoising- Compensate for saturation

- Erode - Decrease noise (Shrinking object)

- Dilate - Increase signal (Object growing)

- Multiply - AND operation

- Classification - Which points belong together ?

- Fitting - Polynominal fit

50 100 150 200 250 300 350 400 450

50

100

150

200

250

300

350

400

45050 100 150 200 250 300 350 400 450

50

100

150

200

250

300

350

400

45050 100 150 200 250 300 350 400 450

50

100

150

200

250

300

350

400

45050 100 150 200 250 300 350 400 450

50

100

150

200

250

300

350

400

45050 100 150 200 250 300 350 400 450

50

100

150

200

250

300

350

400

450

Algorithms

20.10.2006


Slide 14

ww

w.m

aste

rsin

inno

vatio

n.co

m

- Dilate & Erode

Algorithms

20.10.2006


Slide 15

ww

w.m

aste

rsin

inno

vatio

n.co

m

- Noise filtering - fixed, non recursive, line based

- Binning - fixed point or integer, no recursive, line based

- Erode - fixed point or integer, no recursive, line based

- Dilate - fixed point or integer, no recursive, line based

- Multiply - binary operation, pixel based

- Classification - recursive search algorithm, pixel based

- Fitting - floating point

Algorithms

20.10.2006


Slide 16

ww

w.m

aste

rsin

inno

vatio

n.co

m

ProcessingMatlab - CPU Power

Noise FilteringBin by thresholdingBin by local maximaErode & DilateMultiplicationClassifyPolynomial fit

Comparison of Processing speed

1.00

10.00

100.00

1000.00

MATLAB -PC

PC Single DSP Dual DSP DSP &FPGA

Seco

nds

Log

20.10.2006


Slide 17

ww

w.m

aste

rsin

inno

vatio

n.co

m

Processing Requirements- MMACS – Million Multiply and Accumulate operations per second

(example)- Noise filtering convolution with 16 x 1 Byte window filter- 1280 * 1024 @ 30 fps- I.e. 1280 * 1024 * 30 * 16 = 630 MMAC’s per second.

- Floating Point Multipliers in hardware- PC & ARM: 1 multiplier - DSP: upto 6 multipliers- FPGA: built to requirement

- Test of noise filtering on PC P4 2.8GHZ, 1Gbyte RAM- Processing time = [13.22 – 18.78] ms for a 492x460 image - I.e. for a 1280*1024 image will this be about 100ms.

20.10.2006


Slide 18

ww

w.m

aste

rsin

inno

vatio

n.co

m

PowerPC- 4000 - 8000 MFLOPS- 1 Floating point multiplier- 1 Fixed point multiplier- RAM access upto 128 bit @ 200 Mhz

20.10.2006


Slide 19

ww

w.m

aste

rsin

inno

vatio

n.co

m

Floating Point DSP

- 1800 – 3600 MFLOPS- Upto 6 Floating point multipliers @ 32 bit- 2 Fixed point multipliers- RAM access 32 bit @ 100 Mhz- 2 MB (TI) upto 3 MB (Analog) memory onboard

20.10.2006


Slide 20

ww

w.m

aste

rsin

inno

vatio

n.co

m

Fixed Point DSP- 3200 – 8000 MIPS - 4000 MMACs @ 32 bit - 2 MB (TI) upto 3 MB memory onboard (Analog)- RAM access 64 bit @ 133 Mhz (TI)

20.10.2006


Slide 21

ww

w.m

aste

rsin

inno

vatio

n.co

m

Memory requirements- Bandwidth to External Memory - Theoretical limits:

- ARM interface to external RAM is 32 bit @ 400 Mhz

- Floating Point DSP interface to external RAM is 32 bit standard @ 100MHz:- 3200Mbps: 320 images/sec can be read or written to the external memory. At 30fps:

(320/30)/ 2 (read/write) = 5 image handlings per cycle.

- Fixed Point DSP interface to external RAM is 64 bit standard @ 133MHz:- 8512 Mbps: 851 images/sec can be read or written to the external memory. At 30fps

(851/30) / 2 (read/write) = 14 image handlings per cycle.

- FPGA interface external RAM is DDR2 32 bit @ 400 Mhz- 12800Mbps: 1280 images/sec can be read or written to the external memory. At 30fps

(12800/30)/ 2 (read/write) = 21 image handlings per cycle.

20.10.2006


Slide 22

ww

w.m

aste

rsin

inno

vatio

n.co

m

Processing selection

1.0

10.0

100.0

1000.0

Noise Filte

ring

Bin by th

resho

lding

Bin by lo

cal m

axima

Erode &

Dilate

Multiplica

tion

Classify

Polynom

ial fit

Total

MATLABPCDSPDSP & FPGA

- DSP for floating operations

- Algorithm steps for FPGA parallel implementation :- Denoising with filter, binning steps and erode & dilate image processing- AND functions of 2 or more images- (optionally) classification

Gain of 10 ms due to fixed

op’s in FPGA

20.10.2006


Slide 23

ww

w.m

aste

rsin

inno

vatio

n.co

m

Management- Trade off Performance – Cost – Development Risk- Algorithm vs. Processing hardware

- Camera’s: 1 Mpixels, 2 Mpixels- More precision in algorithms & calibration- More subpixel resolution 1/25 1/50

less smoothing, better peak detectors, better fittingsmore floating point operations, more MMACs required, ....

20.10.2006


Slide 24

ww

w.m

aste

rsin

inno

vatio

n.co

m

- Trade off Performance – Cost – Development Risk- Frame rate vs. CPU power

- Camera’s: 15 fps 30 fps 60 fps- Doubling required processing power, ....- More statistical algorithms that improve accuracy

Management

20.10.2006


Slide 25

ww

w.m

aste

rsin

inno

vatio

n.co

m

Management of Development

- Multi core DSP vs. DSP and FPGA

- COTS image processing board vs. Custom image processing board

- TI, meanwhile, has a roadmap in place to quickly move the 'C6x floating platform to 3,000 MMAC, said Rick Rienhart, 'C6000 product line manager at TI in Houston.

- TI expects to produce devices achieving speeds of 3 trillion instructions per second by 2010.

- In the future processing power will as good as for for free (More’s Law)

20.10.2006


Slide 26

ww

w.m

aste

rsin

inno

vatio

n.co

m

Conclusions

- Choice of DSP + FPGA on COTS board

- Waiting for DSP implementations with GMAC processing

- Algorithms is mathematics

- Manage development risks by early stage breadboarding.

- System cost impacts earning capabilities in our prior application and its transfer opportunties to other market applications, thus interact with business development from the start of the program.

20.10.2006


Slide 27

ww

w.m

aste

rsin

inno

vatio

n.co

m

Verhaert New Products & Services nvHogenakkerhoekstraat 219150 KruibekeBelgiumTel +32 (0)3 250 19 00 Fax +32 (0)3 254 10 [email protected]

www.mastersininnovation.com

image signal processing verhaert

Technology

image signal processing

mbps5 x fixed point

soft realtime

fixed point dsp

verhaert consultancies

proprietary of verhaert

temperature range

floating point dsp