the 2010 mario ai championship
TRANSCRIPT
![Page 1: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/1.jpg)
The 2010 Mario AI Championship
Learning, Gameplay and Level Generation tracksWCCI competition event
Sergey Karakovskiy, Noor Shaker,Julian Togelius and Georgios Yannakakis
Sunday, July 25, 2010
![Page 2: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/2.jpg)
How many of you saw the paper about the 2009 Mario AI
Competition yesterday?
Sunday, July 25, 2010
![Page 3: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/3.jpg)
What was theMario AI Competition?• A competition based on “Super Mario
Bros”...
• ...designed to test and rank AI methods for game playing...
• ...where researchers submitted their best Mario-playing controllers...
• ...and the best Mario AI won?
Sunday, July 25, 2010
![Page 4: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/4.jpg)
Why bother?• Problem faced (voluntarily) by hundreds of
millions of gamers around the world since 1985
• Games are designed to challenge human cognitive skills and learning abilities
• Could help improve game design/development (e.g. for PCG)
• Similarity to robotics problems etc.
Sunday, July 25, 2010
![Page 5: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/5.jpg)
Competition objectives
• Ease of participation
• Transparency
• Ease of finding a winner
• Depth of challenge
Sunday, July 25, 2010
![Page 6: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/6.jpg)
• by Markus Persson
• quite faithful SMB 1/3 clone
• in Java
• random level generation
• open source
Infinite Mario Bros
Friday, September 11, 2009Sunday, July 25, 2010
![Page 7: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/7.jpg)
Making a benchmark
• Control loop rewritten
• Tunable FPS, up to 1000 times faster than real-time
• Removed stochasticity
• Created and interface for agents/controllers
Sunday, July 25, 2010
![Page 8: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/8.jpg)
Sunday, July 25, 2010
![Page 9: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/9.jpg)
Last year’s winner
• Robin Baumgarten of Imperial College, UK
• Approach based on A*
• Goal: get to the right edge of the screen
• Internal model of the game’s physics
Sunday, July 25, 2010
![Page 10: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/10.jpg)
A* IN MARIO: CURRENT POSITION
current node
Goal:right border
of screen
Friday, September 11, 2009Sunday, July 25, 2010
![Page 11: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/11.jpg)
A* IN MARIO: CHILD NODES
current node right, speed
left, jump, speed
right, jumpjump
Friday, September 11, 2009Sunday, July 25, 2010
![Page 12: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/12.jpg)
A* IN MARIO: BEST FIRST
current node right, speed
Friday, September 11, 2009Sunday, July 25, 2010
![Page 13: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/13.jpg)
A* IN MARIO: BACKTRACK
current node right, speed
right, jump, speed
Friday, September 11, 2009Sunday, July 25, 2010
![Page 14: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/14.jpg)
A* IN MARIO: BEST FIRST
current node right, speed
right, jump, speed
Friday, September 11, 2009Sunday, July 25, 2010
![Page 15: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/15.jpg)
A* IN MARIO: CREATE CHILDS
current node
Friday, September 11, 2009Sunday, July 25, 2010
![Page 16: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/16.jpg)
Some takeawaysfrom last year
• A* plays this type of levels very well, but not in a human-like manner
• The task did not test learning
• And was too easily beaten
Sunday, July 25, 2010
![Page 17: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/17.jpg)
The 2010 Mario AI Championship
• An attempt to...
• ...deepen the challenge:Gameplay track
• ...test learning as well as controller design:Learning track Cancelled for this event
• ...test the capabilities of CI in game design:Level generation track Demo only for this event
Sunday, July 25, 2010
![Page 18: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/18.jpg)
The 2010 Mario AI Championship
• Gameplay trackSame as last year, except that the toughest levels are much tougher.
• In particular: dead ends that force backtracking, meant to be lethal for A*
• Learning trackAllows the controller 1000 runs on each level to allow it to learn the level; scored on the 1001st attempt
Sunday, July 25, 2010
![Page 19: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/19.jpg)
The 2010 Mario AI Championship
• Level generation trackCompetitors submit level generators that output fun levels for particular players, based on measurements of playing style.Live judging!
(We need you! Do you have Java installed?)
Sunday, July 25, 2010
![Page 20: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/20.jpg)
The 2010 Mario AI Championship
• EvoStar (April, Istanbul): Gameplay track
• WCCI (July, Barcelona): Gameplay, Learning and Level Generation tracks
• CIG (August, Copenhagen): Gameplay, Learning and Level Generation tracks
• ICE-GIC (December, Hong Kong):Turing Test track
Sunday, July 25, 2010
![Page 21: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/21.jpg)
Gameplay trackSergey Karakovskiy and Julian Togelius
Sunday, July 25, 2010
![Page 22: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/22.jpg)
Agent goals
• Develop an agent that gets as far and as fast as possible...
• ...on as many levels as possible...
• ...which are previously unseen
• Scoring: progress on 40 randomly generated levels (of different difficulty, length, type) with seed 17564
• If two agents complete all the levels: tiebreakers
Friday, September 11, 2009Sunday, July 25, 2010
![Page 23: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/23.jpg)
Challenges
• Handle a large state/observation space
• Handle very different situations (now more different than before)
• Tactical tradeoffs (e.g. go back and get the power-up or continue forward?)
Sunday, July 25, 2010
![Page 24: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/24.jpg)
InterfaceFriday, September 11, 2009Sunday, July 25, 2010
![Page 25: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/25.jpg)
• 22x22 arrays describing
• landscape features (e.g. walls, cannons, gaps)
• creatures
• Fine position of Mario and creatures
• Booleans: mario is on the ground, may jump, is carrying a shell, is small/big/fire
Environment Interface
Friday, September 11, 2009Sunday, July 25, 2010
![Page 26: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/26.jpg)
benchmark outputs
Score: 13998.645Levels cleared = 9Total time left = 6780Total Kills = 87Mario mode = 32TOTAL SUM = 20906.645
action
observation
Your Agent
22x22 byte arrays
float[] positions
(0, 1, 0, 1, 1)
float[] rewards
Mario AI in a nutshell
Sunday, July 25, 2010
![Page 27: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/27.jpg)
Very simple agent
Example
Sunday, July 25, 2010
![Page 28: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/28.jpg)
Differences fromlast year
• Framework more developed, now with better support for non-Java agents (e.g. Python)
• Tighter bound on time taken/action
• More difficult levels!
• Level generator augmented to generate really hard levels on higher difficulties
• Some levels are impossible
Sunday, July 25, 2010
![Page 29: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/29.jpg)
Evaluation setup• total episodes: 126
• Main score: distance passed
• Tie-breakers: speed, creatures killed, mode
• all 3 types of levels, fixed seed, difficulties = {0, 1, 3, 5, 12, 16, 20}
• 42 ms per action (violating the limit results in disqualification for the entire level)
• tweaked JVM run to skip the GC executions.
Sunday, July 25, 2010
![Page 30: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/30.jpg)
Results
Sunday, July 25, 2010
![Page 31: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/31.jpg)
Evaluation parameters
• Seed 429766
• 576 trials
• Remember: a controller is disqualified on a level if it takes more than 42 ms (real time) in any frame
Sunday, July 25, 2010
![Page 32: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/32.jpg)
Team Members Score Disc Technique
Robin Baumgarten
1537834 312 A*
Sergey Polikarpov 1637935 1
CyberNeuron (RL)
wSUMariO-CAT
Robert Reynolds, Leonard Kinnaird-Heether, Tracy Lai
1113437 0Elman network /
cultural algorithm
Alexander Buck
991372 0 ?
Eamon Wong 972341 0 Q-learning
Mathew Erickson
634239 0 Genetic Programming
Sunday, July 25, 2010
![Page 33: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/33.jpg)
Winner of the Gameplay track:
Sergey Polikarpov
Sunday, July 25, 2010
![Page 34: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/34.jpg)
Videos
Sunday, July 25, 2010
![Page 35: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/35.jpg)
What can we learn?
• A* is not invincible!
• At least not on its own
• All entrants fail at dead ends, but fail in different ways
• We probably need to combine micro- and macro-strategy
• Still time left to compete at CIG event
Sunday, July 25, 2010
![Page 36: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/36.jpg)
Learning trackSergey Karakovskiy and Julian Togelius
Sunday, July 25, 2010
![Page 37: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/37.jpg)
Please compete!
• The interface is almost as simple as for the GamePlay track (and almost the same)
• Allows you 1000 runs to optimize your controller
• Scores you on the 1001st
• Too few entrants for the WCCI event
• Still time left to compete...
Sunday, July 25, 2010
![Page 38: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/38.jpg)
Level generation trackNoor Shaker, Julian Togelius and Georgios Yannakakis
Sunday, July 25, 2010
![Page 39: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/39.jpg)
The Goal
Submit a level generator that generates fun levels, personalized for individual players
The levels generated also has to adhere to constraints to force the generation of diverse levels
Sunday, July 25, 2010
![Page 40: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/40.jpg)
The process
• Each judge plays a test level, and his performance on this level is recorded (various metrics such as jumps, deaths etc.)
• Each level generator generates a level tailored to each judge
• The judges play “their own” levels and rank them
Sunday, July 25, 2010
![Page 41: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/41.jpg)
Setup
Sunday, July 25, 2010
![Page 42: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/42.jpg)
InterfaceGamePlay.java contains information about the level design and how the testers played
public int totalEnemies; //total number of enemiespublic int totalEmptyBlocks; //total number of empty blockspublic int totalpowerBlocks; //total number of power blockspublic int totalCoins; //total number of coinspublic int GreenTurtlesKilled;//number of Green Turtle Mario killedpublic int ArmoredTurtlesKilled; //number of Armored Turtle Mario killed
public int GoombasKilled; //number of Goombas Mario killedpublic int timeRunningLeft;//total time spent running to the leftpublic int emptyBlocksDestroyed; //number of empty blocks destroyedpublic int coinsCollected; //number of coins collected.
Sunday, July 25, 2010
![Page 43: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/43.jpg)
Interface• LevelInterface.java provides a simple interface
that should be implemented when constructing your customized level:public byte[][] getMap();public SpriteTemplate[][] getSpriteTemplates()
• Constructed levels should communicate with the simulation only through the LevelGenerator interface:public LevelInterface generateLevel (GamePlay playerMetrics);
Sunday, July 25, 2010
![Page 44: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/44.jpg)
Interface
• All submitted levels should satisfy the constraints defined in the Constraints interface. Example:
public static int levelWidth= 320;public static int gaps = 10;public static int turtles = 7;public static int coinBlocks = 10;
Sunday, July 25, 2010
![Page 45: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/45.jpg)
Instructions• Download the jar file
• Run it (double click)
• Play the first level (controls A, S, arrows)
• Play two additional levels
• Rank the levels: most and second most fun
• Turn your laptops towards me!
• Scoring: 2 point for most fun level, 1 for second most fun
Sunday, July 25, 2010
![Page 46: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/46.jpg)
http://julian.togelius.com/
competition.jar
Sunday, July 25, 2010
![Page 47: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/47.jpg)
Scores:
• Random: 12
• Optimized: 7
• Nathan: 20
• Demo winner: Nathan Sorenson
Sunday, July 25, 2010
![Page 48: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/48.jpg)
Optimized gaps level• Refinement of the original Infinite Mario
level generator
• Placement, number and width of gaps generated using a model based on preference learning from 240 players
• Christoffer Pedersen, Julian Togelius and Georgios Yannakakis (2010): Modeling Player Experience for Content Creation. IEEE TCIAIG.
• Also forthcoming AIIDE paper (Shaker et al)
Sunday, July 25, 2010
![Page 49: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/49.jpg)
Nathan Sorenson
• Simon Fraser University
• Level generator based on GA (high level) and constraint solver (low level)
• Preliminary work reported in:Sorenson, N. & Pasquier, P. (2010). "Towards a Generic Framework for Automated Video Game Level Creation", EvoGames 2010
Sunday, July 25, 2010
![Page 50: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/50.jpg)
http://groups.google.com/group/mariocompetition/
http://www.marioai.org
Sunday, July 25, 2010
![Page 51: The 2010 Mario AI Championship](https://reader031.vdocuments.us/reader031/viewer/2022020705/61fb94782e268c58cd5fdc77/html5/thumbnails/51.jpg)
Sunday, July 25, 2010