bio-inspired active vision - nvidiaon-demand.gputechconf.com/gtc/2013/presentations/s3104... ·...
TRANSCRIPT
![Page 1: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/1.jpg)
Martin Peniak, Ron Babich, John Tran and Davide Marocco
Bio-inspired Active Vision
![Page 2: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/2.jpg)
GPU Computing Lab
![Page 3: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/3.jpg)
3
Traditional Computer Vision
![Page 4: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/4.jpg)
Specific template or computational
representation is required to allow object
recognition
Must be flexible enough to account with all
kinds of variations
4
Traditional Computer Vision
“Teaching a computer to classify objects has proved much harder than was originally anticipated” Thomas Serre - Center for Biological and Computational Learning at MIT
![Page 5: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/5.jpg)
5
Biological Vision
“Researchers have been interested for years in trying to copy biological vision systems,
simply because they are so good” ~ David Hogg - computer vision expert at Leeds University, UK
Highly optimized over millions of years of
evolution, developing complex neural structures
to represent and process stimuli
Superiority of biological vision systems is only
partially understood
Hardware architecture and the style of
computation in nervous systems are
fundamentally different
![Page 6: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/6.jpg)
6
Biological Vision
![Page 7: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/7.jpg)
Seeing is a way of acting
7
![Page 8: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/8.jpg)
Inspired by the vision systems of natural organisms that have
been evolving for millions of years
In contrast to standard computer vision systems, biological
organisms actively interact with the world in order to make sense
of it
Humans and also other animals do not look at a scene in fixed
steadiness. Instead, they actively explore interesting parts of the
scene by rapid saccadic movements
8
Active Vision
![Page 9: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/9.jpg)
Evolutionary Robotics Approach
Creating Active Vision Systems
9
![Page 10: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/10.jpg)
New technique for the automatic creation of autonomous robots
Inspired by the Darwinian principle of selective reproduction of
the fittest
Views robots as autonomous artificial organisms that develop
their own skills in close interaction with the environment and
without human intervention
Drawing heavily on biology and ethology, it uses the tools of
neural networks, genetic algorithms, dynamic systems, and
biomorphic engineering
10
Evolutionary Robotics
![Page 11: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/11.jpg)
11
...
...
...
Population (Chromosomes)
Evaluation (Fitness)
Selection (Mating Pool)
Genetic operators
Artificial neural networks (ANNs) are very powerful brain-inspired
computational models, which have been used in many different
areas such as engineering, medicine, finance, and many others.
Genetic Algorithms (GAs) are adaptive heuristic search
algorithm premised on the evolutionary ideas of natural
selection and genetic. The basic concept of GAs is
designed to simulate processes in natural system
necessary for evolution.
![Page 12: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/12.jpg)
12
Related Research Mars Rover obstacle avoidance (Peniak et al.)
![Page 13: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/13.jpg)
13
Related Research Koala robot obstacle avoidance (Marocco et al.)
![Page 14: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/14.jpg)
14
Related Research Autonomous driving car (Floreano et al.)
![Page 15: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/15.jpg)
15
Related Research Object recognition (Floreano et al.)
![Page 16: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/16.jpg)
Designing active vision system for real-world object recognition
Going Further
16
![Page 17: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/17.jpg)
Design active vision system that can learn to recognize the
following objects
17
Task
![Page 18: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/18.jpg)
18
Method
Evolution of the active vision system for real-world object recognition training the system in a parallel manner on multiple objects viewed from many different angles and under different lighting conditions
Amsterdam Library of Object Images (ALOI) provides a color image collection of one-thousand small objects
recorded for scientific purposes
systematically varied viewing angle, illumination angle, and illumination color
Active Vision Training trained on a set of objects from the ALOI library
each genotype is evaluated during multiple trials with different randomly rotated objects and under varying lighting conditions
evolutionary pressure provided by a fitness function that evaluates overall success or failure of the object classification
trained on increasingly larger number of objects
Active Vision Testing robustness and resiliency of recognition of the dataset
generalization to previously unseen instances of the learned objects
![Page 19: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/19.jpg)
19
Experimental Setup
Recurrent Neural Network
Inputs: 8x8 neurons for retina, 2 neurons for proprioception (x,y pos)
No hidden neurons
Outputs: 5 object recognition neurons, 2 neurons to move retina (16px max)
Genetic Algorithm
Generations: 10000
Number of individuals: 100
Number of trials: 36+16 (object rotations + varying lighting conditions)
Mutation probability: 10%
Reproduction: best 20% of individuals create new population
Elitism used (best individual is preserved)
![Page 20: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/20.jpg)
20
Experimental Setup
Each individual (neural network) could freely move the retina and
read the input from the source image (128x128) for 20 steps
At each step, neural network controlled the behavior of the
system (retina position) and provide recognition output
The recognition output neuron with the highest activation was
considered the network’s guess about what the object was
Fitness function = number of correct answers / number of total steps
![Page 21: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/21.jpg)
GPUs were used to accelerate:
Evolutionary process – parallel execution of trials
Neural Network – parallel calculation of neural activities
21
GPU Accelerating GA and ANN
![Page 22: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/22.jpg)
22
Results
Fitness can not reach 1.0 since it takes few time-steps to recognize an object
All objects are correctly classified at the end of the each test
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
fitn
ess
generations
best fitness average fitness
![Page 23: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/23.jpg)
23
Evolved Behavior
![Page 24: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/24.jpg)
Extending the current system and applying it to the real-world
problems requiring fast and energy-efficient pattern recognition
Implementing island model GA on GPU
AXA Research Fund
Active Vision in search and rescue scenarios
24
Future Work
![Page 25: Bio-inspired Active Vision - NVIDIAon-demand.gputechconf.com/gtc/2013/presentations/S3104... · 2013-03-20 · Specific template or computational representation is required to allow](https://reader033.vdocuments.us/reader033/viewer/2022042311/5ed92dd06714ca7f47694b09/html5/thumbnails/25.jpg)
Thank you!
25
"Imagination is the highest form of research" Albert Einstein