programs that play better than us
TRANSCRIPT
![Page 2: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/2.jpg)
https://en.wikipedia.org/wiki/File:ST Battle Chess.png
![Page 3: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/3.jpg)
https://en.wikipedia.org/wiki/Deep Blue (chess computer)
Deep Blue (IBM, 1996)
![Page 4: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/4.jpg)
http://afflictor.com/2012/09/11/chess-programs-regularly-play-at-good-amateur-level/
![Page 5: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/5.jpg)
Game tree
![Page 6: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/6.jpg)
Optimal play
Terminal
min player
max player
![Page 7: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/7.jpg)
Optimal play
1 01 1 1Terminal
min player
max player
![Page 8: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/8.jpg)
Optimal play
1 01 1 1
0
Terminal
min player
max player
![Page 9: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/9.jpg)
Optimal play
1 01 1 1
0 1
Terminal
min player
max player
![Page 10: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/10.jpg)
Optimal play
1 01 1 1
0 1
1
Terminal
min player
max player
![Page 11: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/11.jpg)
Chess has about 1046 states!
![Page 12: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/12.jpg)
Minimax algorithm
Cut-off
min player
max player
![Page 13: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/13.jpg)
Minimax algorithm
.7 .1 .6 .9Cut-off
min player
max player
![Page 14: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/14.jpg)
Minimax algorithm
.7 .1 .6 .9
.1
Cut-off
min player
max player
![Page 15: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/15.jpg)
Minimax algorithm
.7 .1 .6 .9
.1 .6
Cut-off
min player
max player
![Page 16: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/16.jpg)
Minimax algorithm
.7 .1 .6 .9
.1 .6
.6
Cut-off
min player
max player
![Page 17: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/17.jpg)
https://stockfishchess.org/
Stockfish
![Page 18: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/18.jpg)
https://tests.stockfishchess.org/
Testing AI changes is crucial
![Page 19: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/19.jpg)
Value functions are hard!
![Page 20: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/20.jpg)
http://mathworld.wolfram.com/Go.html
![Page 21: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/21.jpg)
http://www.remi-coulom.fr/CrazyStone/
Remi Coulom
![Page 22: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/22.jpg)
http://www.wired.com/2014/05/the-world-of-computer-go/
![Page 23: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/23.jpg)
Monte Carlo evaluations
Cut-off
min player
max player
![Page 24: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/24.jpg)
Monte Carlo evaluations
Cut-off
min player
max player
![Page 25: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/25.jpg)
Monte Carlo evaluations
Cut-off
min player
max player
![Page 26: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/26.jpg)
Monte Carlo evaluations
Cut-off
min player
max player
![Page 27: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/27.jpg)
Monte Carlo evaluations
Cut-off
min player
max player
.7
![Page 28: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/28.jpg)
Monte Carlo Tree Search (MCTS)
![Page 29: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/29.jpg)
by Google Deepmind
https://deepmind.com/research/alphago/
![Page 30: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/30.jpg)
https://gogameguru.com/alphago-races-ahead-2-0-lee-sedol/
![Page 31: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/31.jpg)
MCTS + Policy and value networks
![Page 32: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/32.jpg)
http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html
MCTS + Policy and value networks
![Page 33: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/33.jpg)
Some games have hidden information!
![Page 34: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/34.jpg)
http://magic.wizards.com/en/events/coverage/gpsin15/father-son-2015-06-27
![Page 35: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/35.jpg)
https://magarena.github.io
![Page 36: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/36.jpg)
![Page 37: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/37.jpg)
Determinization: choose a random instance of thehidden information during simulation
![Page 38: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/38.jpg)
Comparison of Minimax and MCTS
At 1s thinking time:Minimax MCTS
1 0.88
![Page 39: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/39.jpg)
Comparison of Minimax and MCTS
At 1s thinking time:Minimax MCTS
1 0.88
At 4s thinking time:Minimax MCTS
1 1.71
![Page 40: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/40.jpg)
Open problems
![Page 41: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/41.jpg)
MCTS is bad at tight tactical play.
![Page 42: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/42.jpg)
MCTS plays badly when it is behind in the game.
![Page 43: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/43.jpg)
Further readings
![Page 44: Programs that Play better than Us](https://reader031.vdocuments.us/reader031/viewer/2022030307/58e8f9231a28ab1f248b5f5d/html5/thumbnails/44.jpg)
Further readings