gpus enable deep neuroevolution for vision-based...
TRANSCRIPT
![Page 1: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/1.jpg)
GPUs Enable Deep Neuroevolution for Vision-Based Autonomous
Driving
Faustino Gomez, CEO NNAISENSE SA
Lugano, Switzerland
![Page 2: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/2.jpg)
SUPERVISED LEARNING
Learned Model
+Training set
prediction
target
errorinput
• Labeled training set
• Data is fixed and know a priori
• Learn from prediction error
• Regression / Classification
• Particularly amenable to GPUs
![Page 3: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/3.jpg)
REINFORCEMENT LEARNING
sense act
S1 S2 S3 Sn…
sense actsense act
a1 a2 a3
• Sequential decision task
• Data determined by learning agent
• No targets (i.e. teacher)
• Learn good control policy from sparse reward signal
• Partial obersvability
Environment
Learning Agent
reward
![Page 4: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/4.jpg)
NEUROEVOLUTION
Environment
fitness
evaluate
sensors action
fitness
Evolutionaryalgorithm
![Page 5: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/5.jpg)
Neuroevolution: Advantages
• No linearity assumptions
• Can cope with high-dimensional input/output
• Can use history of sensor readings of unknown depth (short-term memory)
• Can incorporate arbitrary constraints
• Behavior is learned not programmed
• Does not require knowledge of what constitutes optimal performance, i.e. reference signal
![Page 6: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/6.jpg)
SUCCESSFUL TEST CASE A U T O M AT E D D R I V I N G U S I N G V I S I O N
Jan Koutnik, Giuseppe Cuccu, Juergen Schmidhuber, and Faustino Gomez (2013). Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO, Amsterdam).
Million weight RNN learns to drive car using vision WITHOUT A TEACHER
![Page 7: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/7.jpg)
AUDI:Autonomous Parking
Objective: Learn to park car elegantly in more general conditions using only local sensors
• Reinforcement-learn continuous non-linear control • No global information • High-dimensional, noisy input • Closing reality gap (forward model) • Computationally intensive (physical model on GPU) • Timeline: build system for scratch in 6 months for NIPS
Challenges:
![Page 8: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/8.jpg)
AUDI PLAYCAR DEMMO
![Page 9: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/9.jpg)
Couldn’t do without GPU
• GPUs used to train all components
- CNN localizer
- Forward car model
- RNN controller
• Enable running 20K simulations parallel
• 100x speedup over multicore CPU cluster
![Page 10: GPUs Enable Deep Neuroevolution for Vision-Based …on-demand.gputechconf.com/gtc-eu/2017/presentation/2… · · 2017-10-27AUTOMATED DRIVING USING VISION Jan Koutnik, ... Evolving](https://reader031.vdocuments.us/reader031/viewer/2022030419/5aa5c7f07f8b9ab4788da0fd/html5/thumbnails/10.jpg)
THANK YOU