the 2011 simulated car racing championship @ evo*-2011
DESCRIPTION
These slides report the results of The 2011 Simulated Car Racing Championship @ Evo*-2011TRANSCRIPT
2011 Simulated Car Racing Championship @ EVO*-2011
The 2011 Simulated Car Racing
Championship @ Evo*-2011Daniele Loiacono, Luigi Cardamone,
Martin V. Butz, and Pier Luca Lanzi
2011 Simulated Car Racing Championship @ EVO*-2011
2011 Simulated Car Racing Championship9 races during 3 conferences
Develop a driver for TORCS(hand-coded, learned, evolved, …)
Drivers will be awarded based on their score
in each conference competition
At the end, the team with highest overall scorewins the championship
2
2011 Simulated Car Racing Championship @ EVO*-2011
Roadmap of the 2011 Championship
1. EVO*-2011, Torino (Italy)April 27-29, 2011 (Now!)
2. ACM GECCO-2011, Dublin (Ireland)July 12-16, 2011
3. IEEE CIG-2011, Seoul (South Korea)August 31 September 3, 2011
2011 Simulated Car Racing Championship @ EVO*-2011
Motivations
Proposing a relevant game-based competition
more representative of commercial games AI
more similar to a real-world problem
Proposing a funny and exciting competition
you can see and play with the entries of this competition
human players can interact with AI
a lot of programmed AI available for comparison
Proposing a challenging competition
computationally expensive
real-time
on-line learning
noisy sensors
The Open Racing Car Simulator
2011 Simulated Car Racing Championship @ EVO*-2011
The Open Racing Car Simulator
TORCS is a state of the art open source simulator written in C++
Main features
Sophisticated dynamics
Provided with several cars, tracks, and controllers
Active community of users and developers
Easy to develop your own controller
OS Support
Linux: binaries and building from sources
Windows: binaries and “limited” bulding from sources support
OSX: legacy binaries and no building from sources support
2011 Simulated Car Racing Championship @ EVO*-2011
The Open Racing Car Simulator & the Competition Software
TORCS
BOT BOT BOT
TORCS
PATCH
SBOT SBOT SBOT
BOT BOTBOT
UDP UDPUDP The competition server
Separates the bots from TORCS
Build a well-defined sensor model
Works in real-time
2011 Simulated Car Racing Championship @ EVO*-2011
Sensors and actuators
Rangefinders for edges on the track and opponents (with noise)
Speed, RPM, fuel, damage, angle with track, distance race, position on track, etc.
Six effectors: steering wheel [-1,+1], gas pedal [0, +1], brake pedal [0,+1], gearbox {-1,0,1,2,3,4,5,6}, clutch [0,+1], focus direction
Structure of the Race
2011 Simulated Car Racing Championship @ EVO*-2011
What is the structure of a race?
Three stages: warm up, qualifiers, actual race
During warm-up, each driver can explore the track and learn something useful
During qualifiers, each driver races alone against the clock (the best 8 drivers move to the race)
During the race all the drivers race together
12
Competitors
2011 Simulated Car Racing Championship @ EVO*-2011
The competitors
Four entries in the first leg:
Mr. Racer, TU Dortmund
Mariscal & Fernández, Málaga University
Ready2Win, Slovak University of Technology - FIIT
Powaah, Blekinge Institute of Technology
Three more entries from the 2009 championship:
AUTOPIA, Madrid and Granada
COBOSTAR (Lönneker & Butz, University of Würzburg)
POLIMI (Cardamone, Politecnico di Milano)
2011 Simulated Car Racing Championship @ EVO*-2011
Jan Quadflieg, Tim Delbruegger and Mike Preuss TU Dortmund
Mr Racer
2011 Simulated Car Racing Championship @ EVO*-2011
Mr. Racer 2011
Mostly described in the Evo* Paper “Driving Faster Than a Human Player“
13 Variables learned offline with the CMA-ES
Warmup used to learn a model of the track
Noise Handling:
Low pass filter on sensor values
2 Regression polynoms are fitted to each side
Resample the polygons
2011 Simulated Car Racing Championship @ EVO*-2011
Mr. Racer 2011 - Opponent Handling
Noise filtering with a lowpass filter
Observer module interprets filtered sensor readings
This leads to a recommended speed and an overtaking line (created on the fly!)
Planning module incorporates recommended target speed and overtaking line into the plan
Lots of nummerical parameters not yet tuned:
left for Gecco Competition
2011 Simulated Car Racing Championship @ EVO*-2011
2011 Simulated Car Racing Championship @ EVO*-2011
Mariscal & Fernández
General approach: expert systems improved by multi-objectives evolutionary computation techniques
Steering: based on a simple rule follow the track sensor with biggest distance.
2011 Simulated Car Racing Championship @ EVO*-2011
Mariscal & Fernández - Throttle/Brake
Implemented via a parameterized expert system whoseparameters were optimized via a Multi-ObjectiveEvolutionary Algorithm (NSGA-II)
Two objectives:
Maximize distance.
Minimize damage.
Linear function (Ci are parameters to optimize):
C1 (Speed) + C2 (Distance to the next turn) + C3 (Turn degrees)
We obtain three sets of parameters
(i.e., controllers) to be used under
specific conditions of the race
(1) To run fast (2) To run safe(3) To run on dirtytracks.
2011 Simulated Car Racing Championship @ EVO*-2011
Mariscal & Fernández - Overtaking
Expert system with four groups of rules:
Other strategies are implemented (via expert
systems) such as collision avoidance and slipstream overtakings.
2011 Simulated Car Racing Championship @ EVO*-2011
2011 Simulated Car Racing Championship @ EVO*-2011
Ready2Win
Modular architecture:
Driving module:
• The speed is computed using the maximum braking distance
Overtaking module
Recovery module
ABS module
Track learning during the warm-up:
First lap, driven slow, to identify turns (start, end, entry position, curvature) and learn the track model
Other laps, speed adaptation:
• Increasing according to lateral speed
• Reducing when the car goes off-road
2011 Simulated Car Racing Championship @ EVO*-2011
Tim Uusitalo Blekinge Institute of Technology
Supervisor: Stefan Johanson
Powaah
2011 Simulated Car Racing Championship @ EVO*-2011
Powaah
Based on Artificial Potential Field
Every sensor in the span (-40, +40) has a charge
The charge is determined by how far the sensor measures the track
Coulomb’s law is used to calculate potentials in the field
Pot = (C1 * C2 ) / d2
The sensor with the highest potential is used for computing how much steering is needed
2011 Simulated Car Racing Championship @ EVO*-2011
Industrial Computer Science Department.
Centro de Automática y Robótica
Consejo Superior de Investigaciones Científicas
Madrid, Spain
Contact:E. Onieva ([email protected])
AUTOPIA
2011 Simulated Car Racing Championship @ EVO*-2011
AUTOPIA
Fuzzy Architecture based on three basic modules for gear, steering and speed control
optimized with a genetic algorithm
Learning in the warm-up stage:
Maintain a vector with as many real values as tracklengthin meters.
Vector initialized to 1.0
If the vehicle goes out of the track or suffers damage then multiply vector positions from 250 meters before the current position by 0.95.
During the race the vector is multiplied by F to make the driver more cautious in function of the damage:
F=1-0.02*round(damage/1000)
2011 Simulated Car Racing Championship @ EVO*-2011
COBOSTAR
Thies Lönneker and Martin V. Butz
University of Würzburg
http://www.coboslab.psychologie.uni-wuerzburg.de
2011 Simulated Car Racing Championship @ EVO*-2011
COBOSTAR
One of the best controller of the 2009 Championship
Parameterized controller optimized with CMA-ES
Dynamically saves crash points
2011 Simulated Car Racing Championship @ EVO*-2011
CIG-2008 Champ
Luigi Cardamone
Politecnico di Milano
2011 Simulated Car Racing Championship @ EVO*-2011
Luigi Cardamone
Winner of the CIG-2008 Car Simulated Competition
The controller is based on a neural network evolved with NEAT
Very simple policy for overtaking
Qualifying
2011 Simulated Car Racing Championship @ EVO*-2011
Scoring process: Warm-up & Qualifying
Scoring process involves three tracks:
OldPeak
Polimi
HillRoad
The tracks are not distributed with TORCS:
Generated applying evolutionary computation
The competitors cannot know the tracks
Each controller raced for 100000 game ticks in the warm-up stage and then its performance is computed in the qualifying stage as the distance covered within 10000 game ticks
Dirt-1
OldPeak
2011 Simulated Car Racing Championship @ EVO*-2011
Qualifying: OldPeak
7441,21
4606,09
8159,67
5062,1
6799,89
7372,23
8202,92
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Ready2Win
Powaah
Mr. Racer
Mariscal&Fernandez
Cobostar
Cardamone
Autopia
Dirt-1
Polimi
2011 Simulated Car Racing Championship @ EVO*-2011
Qualifying: Polimi
6343,6
3891,01
8570,81
7775,64
6694,85
96,1469
8307,57
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Ready2Win
Powaah
Mr. Racer
Mariscal&Fernandez
Cobostar
Cardamone
Autopia
Dirt-1
HillRoad
2011 Simulated Car Racing Championship @ EVO*-2011
Qualifying: HillRoad
7528,29
10075,3
10250,3
10547,8
10006,7
8937,33
11088,4
0 2000 4000 6000 8000 10000 12000
Ready2Win
Powaah
Mr. Racer
Mariscal&Fernandez
Cobostar
Cardamone
Autopia
2011 Simulated Car Racing Championship @ EVO*-2011
Qualifying summary
Driver OldPeak Polimi HillRoad Total
Autopia 10 8 10 28
Mr. Racer 8 10 6 24
Mariscal&Fernandez 3 6 8 17
Cobostar 4 5 4 13
Ready2Win 6 4 2 12
Cardamone 5 2 3 10
Powaah 2 3 5 10
2011 Simulated Car Racing Championship @ EVO*-2011
How much does noise affect the performance?
2011 Simulated Car Racing Championship @ EVO*-2011
HillRoad with and without noise
0 2000 4000 6000 8000 10000 12000
Ready2Win
Powaah
Mr. Racer
Mariscal&Fernandez
Cobostar
Cardamone
Autopia
NO-NOISY NOISY
The Race
2011 Simulated Car Racing Championship @ EVO*-2011
Three Tracks
For each track we run 7 races with random starting grids
Each race is scored using the F1 point system (10 to first, 8 to second, 6 to third, …)
Two points to the controller with lesser damage
Two points for the fastest lap of the race
46
2011 Simulated Car Racing Championship @ EVO*-2011
Race: OldPeak
Competitor Score
Autopia 12
Mr. Racer 8
COBOSTAR 6
Cardamone 5
Ready2Win 5
Mariscal&Fer
nandez 4
Powahh 2
2011 Simulated Car Racing Championship @ EVO*-2011
Race: Polimi
Competitor Score
Autopia 10
Mariscal&Fer
nandez 8
Mr. Racer 8
COBOSTAR 7
Cardamone 4
Powahh 3
Ready2Win 3
2011 Simulated Car Racing Championship @ EVO*-2011
Race: HillRoad
Competitor Score
Autopia 12
Mariscal&Fer
nandez 8
Mr. Racer 6
Powahh 5
COBOSTAR 4
Cardamone 3
Ready2Win 2
2011 Simulated Car Racing Championship @ EVO*-2011
Final Results
Competitor OldPeak Polimi HillRoad Total
Autopia 12 10 12 34
Mr. Racer 8 8 6 22
Mariscal&Fernandez 4 8 8 20
COBOSTAR 6 7 4 17
Cardamone 5 4 3 12
Powaah 2 3 5 10
Ready2Win 5 3 2 10
What about the race results?
Race and Qualifying have similar outcomes(with the exception of Ready2Win)
AUTOPIA still very competitive
Opponent manegement is crucial on race start. Very difficult with noise!
Thank you!
Looking forward to see your controllers @ Gecco-2011