4th European Micro-UAV Meeting

Sept. 15-17, 2004Generic Visual Perception Processor

GVPP

www.gvpp.org

Topic

- Real time processing- Detect persons, objects and events by temporal

coincidence with unsupervised collective decision - Track persons and objects in the image with anticipation

of their motions - Learning by example.

CAMERA

GenericVisualPerceptionProcessor

Action

4 adaptive properties

Introduction

Imaging processing• Perception

• Understanding• Action

GVPP

Perception Understanding Action

Power

Nb. frames/seconds

IP

GVPP

Method - 1Method - 1

Sequencer

SpatialDomain

Angle 1Angle

Angle n

Temporal Domain

Mot

ion

Col

or

Sensor

1Input signal segmentation in sequences and

sub-sequences with relation:- sequences to temporal domain

- sub-sequences to spatial domain

Subsequences

Sequence

Spatio-Temporal Neuron bloc modelisationby electronic implementation

P.P. 2002

Method - 2

Sequencer

SpatialDomain

Angle 1Angle

Angle n

Temporal Domain

Mot

ion

Col

or

Sensor

2The

sequenceis divide in

3 steps

1Input signal segmentation in sequences and

sub-sequences with relation:- sequences to temporal domain

- sub-sequences to spatial domain

INITCOMPUTATIONRESULTS

Subsequences

Sequence

Spatio-Temporal Neuron bloc modelisationby electronic implementation

P.P. 2002

Method - 3

Sequencer

SpatialDomain

Angle 1Angle

Angle n

Temporal Domain

Mot

ion

Col

or

Sensor

WHERE

Motion

D

Reg.

X/Y

D

Reg.

WHAT

2The

sequenceis divide in

3 steps

1Input signal segmentation in sequences and

sub-sequences with relation:- sequences to temporal domain

- sub-sequences to spatial domain

INITCOMPUTATIONRESULTS

Subsequences

Sequence

3Fonctional group:

temporal and spatialinput parameters

STN STN

Spatio-Temporal Neuron bloc modelisationby electronic implementation

TemporalDomain

SpatialDomain

P.P. 2002

Method - 4

Sequencer

SpatialDomain

Angle 1Angle

Angle n

Temporal Domain

Mot

ion

Col

or

Sensor

WHERE

Motion

D

Reg.

X/Y

D

Reg.

WHAT

2The

sequenceis divide in

3 steps

1Input signal segmentation in sequences and

sub-sequences with relation:- sequences to temporal domain

- sub-sequences to spatial domain

histogramcomputation

Validation

Reg.

Parameter

4Biological properties

- population analysis- majority vote- time coincidences amplification- prediction

INITCOMPUTATIONRESULTS

Subsequences

Sequence

3Fonctional group:

temporal and spatialinput parameters

STN STN

STN

Spatio-Temporal Neuron bloc modelisationby electronic implementation

TemporalDomain

SpatialDomain

P.P. 2002

Decisionmaking

Tim

eC

oinc

iden

ces

Histogram

MIN MAX

Registers

RMAX

POSMOY POSRMX

RMAX/2

NBPTS

A generic Spatio-Temporal Neuron

REG REG

REG

RegistersRegistersAPI

Bus

par.

Time-Coincidences Bus

PARAM

D

FoGReg.

API

F: automatic ClassificationG: Anticipation

STN block

Self Action

STN

MVT

STN

X-YFoG

Initial ROI Final ROI

NBPTS

Z

TEMPORAL DOMAIN SPATIAL DOMAIN

MOTION

WHAT and WHERE*

Motion Perception

Self Organization - 1

• Motion PerceptionX/Y

D

Reg.

MVT

D

Reg.

Z 0 MVT 0

Z 0

BAR z0

LOW SPATIAL RESOLUTION

Spatial Domain Temporal Domain

Self Organization - 2

• Recruitment

Color Analysis

X/Y

D

Reg.

MVT

D

Reg.

Col.

D

Reg.

Z 0

MVT 0

C 0

Z 0

BAR z0

BAR z0

LOW SPATIAL RESOLUTION

Spatial Domain Temporal Domain

Self Organization - 3

• Main Color Found

• Inhibition

No Main Color Analysis

On the Main Area

• Tree generation

For Labeling

X/Y

D

Reg.

Col.

D

Reg.

X/Y

D

Reg.

Z 0

BAR z0

Z 1

BAR z1 01

Z 0

C 0

BAR z0

BAR z1

01

LOW SPATIAL RESOLUTION

Spatial Domain Temporal Domain

HIGHER SPATIAL RESOLUTION

Spatial Domain

Self Organization - 4

• Improvement

X/Y

D

Reg.

X/Y

D

Reg.

Col.

D

Reg.

X/Y

D

Reg.

Z 0

BAR z0

Z 1

BAR z1 01 02

Z 2

Z 2

Z 1

Z 0

C 0

Z 0

C 0

Z 1

BAR z0

BAR z1 BAR z2

01 02

MVT

D

Reg.

MVT 1

LOW SPATIAL RESOLUTION

Spatial Domain Temporal Domain

HIGHER SPATIAL RESOLUTION

Spatial Domain Temporal Domain

Self Organization - 5

• Face Organization

Labeled and learned

• Perception/SynthesisX/Y

D

Reg.

X/Y

D

Reg.

Col.

D

Reg.

X/Y

D

Reg.

BAR z0

BAR z1 BAR z2 BAR z3

01 02 03

MVT

D

Reg.

MVT 1

X/Y

D

Reg.

MVT

D

Reg.

MVT 2

Z 0

BAR z0

Z 1

BAR z1 01 02

Z 2

Z 3

03

LOW SPATIAL RESOLUTION

Spatial Domain Temporal Domain

HIGHER SPATIAL RESOLUTION

Spatial Domain Temporal Domain

Eyes Tracking

Generic Visual Perception Processor

GVPP – System on a Chip

A LONG STORY

• 1986 2000• One statistic computation One System on Chip with 23 statistics computations

2004 GVPP7-B

ANTICIPATION

Lines Perception

X/Y

D

Reg.

OE

D

Reg.

Z 0 OE 0

LOW SPATIAL RESOLUTION

Spatial Domain Temporal Domain

OE

Q 1 2

EDGE ORIENTATION FLOW

CURVES FLOW

Horizon Perception

STABILIZATION

Automotive Application

Game on TV

GVPP7-BMemoryVRAM

CMOS Imager

PC

I

M

Screen

VIDEO BUS

I2C

RS

232

I/O

Quartz

MemoryFlash

CK

Res

et

Pow

er

Re

tro

-act

ion

D.

GVPPChip

GVPP-7B

4

2

2 2 8 1 1 4 50 12+4 40

20x17 159 beams

Debug option

CMOSImager

GVPP-7B

PRELIMINARY INFORMATION

• Array Format (max): 800Hx600V max• Frame Rate: 0-100 VGA frames per second progressive-scan• Interface Mode: Master/Slave• Data Rate (max): 40 mega pixel per second• Dynamic Range: 10-bits• Parameters: Luminance, Hue, Saturation, Motion (orientation, velocity),

Spatial lines orientation,curves,corners• Computation: 64 STN blocks• Multi-scales possibilities• Internal OS• C language• 0.5 Watt 3.3 Volts @ 13.5MHz• Interface: PCI, I2C, RS232

GVPP road map

FIRSTCHIP

GVPP-6100 mm²

GVPP-7176 mm²

GVPP-7B

Next Generation50 mm²

Commercial use0,35

0,25

0,18

25mm²

85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10YEARS

1 3 7 23 64 STN

100

50

128

STN

The Robotic Future

1970 1980 1990 2000 2010 2020 2030

Analog TV, VCR

(PC)

Mobile PhoneMP3

PPDARobotics

$ 30Milliars

PERSPECTIVES

www.gvpp.org

Thanks you for your attention