starcraft learning algorithms by logan yarnell, steven raines, and dean antel
TRANSCRIPT
StarCraft Learning Algorithms
By Logan Yarnell, Steven Raines, and Dean Antel
Abstract
The opening or initial build order during a StarCraft match is an integral part of a player’s strategy. If an opening is naively chosen it makes a player vulnerable to tactics that can end the game very quickly. We examine three algorithms for choosing a “good” opener: Bayesian network, a genetic algorithm, and a pathfinding algorithm.
About StarCraft
Real-time strategy game
Similar to a complex digital version of chess but in real-time
Expert level gameplay is very complex, often requiring over 300 actions per minute.
Considered a national sport in South Korea
About StarCraft
Playing Starcraft requires simultaneously managing several distinct competencies:
Strategy
Production
Economy
Recon
Tactics
Basic Actions
Build buildings
Build units
Attack the enemy
Gather resources/manage economy
Genetic Algorithm
Modifies some set of “solutions” to a problem in a way similar to natural selection
Has mutation, crossover, and elitism functions
Solution = a build order
Each build order is given a fitness value determined by how successful it is
The number of mutations the build has gone through is also recorded
If a build order has gone through 10 mutations and it’s fitness < 5, elitism removes that build order
Genetic Algorithm
Most important part of genetic algorithm = mutate function
Sometimes the only part implemented if computational resources are limited
Fitness value determination is also important
Considered factors in determining a build’s fitness: time taken, amount of resources used, whether the match was won or lost, number of units/structures that survived…
In the end only win/loss rate was chosen to affect fitness
Genetic Algorithm
Mutate function randomly chooses to add a new unit, reorder the build, or remove a unit
isBuildValid method checks to make sure the mutate function doesn’t produce a useless build order by placing a unit before it’s prerequisite(s)
The newly mutated build is then written to a .txt file with NumOfMutations incremented
A match is run with that build, if it wins the fitness value is incremented, if it looses the fitness is not altered
There is an element of randomness, but complexity is more or less constant
Genetic Algorithm
Genetic Algorithm
Bayesian Network
Use knowledge from earlier games to predict what type of units the opponent is likely to have.
Requires mining a large amount of data from replays
Scouting is essential to make this work effectively
Difficult due to incomplete information available during matches
Pathfinding
From what is seen, create a shortest possible build. Relies on scouting.
Dijkstra’s algorithm O(n) complexity.
O(|V|^2)
O(|E|+|V|log|V|)
Intersection O(n^2)
Pathfinding (Continued)
Conclusions
Understanding the complexity of the environment is a key element to creating a learning algorithm for it.
Complementary not competitive.
Why 380?
There are many ways to implement a StarCraft bot that include various algorithms for the different aspects of gameplay:
Finite state machines
Scripting
Dynamic scripting
Probabilistic inference
Influence maps
Neural networks
Swarm intelligence
Potential fields
Genetic programming
Future Work
Enhance unit combat
Increase the amount of units considered in build orders
Optimize building efficiency to allow multiple buildings to be constructed at the same time.
Add dynamic scouting
Questions
What is the computational complexity (O(n)) of Dijkstra’s algorithm without optimization?
Is this a constraint satisfaction problem, or an optimization problem?
What probabilistic relationship is there between nodes in a Bayesian network that represents a StarCraft tech tree?
Answers
N^2
Optimization
Conditional dependence
Sources
http://www.sscaitournament.com/
http://web.engr.oregonstate.edu/~tgd/publications/aiide2011-starcraft-model.pdf
http://arxiv.org/pdf/1111.3735.pdf
http://plankter.se/wp-content/uploads/2014/12/making_and_acting_on_predictions_on_scbw_2014.pdf