Upload File

princeton university cos 495 instructor: yingyu liang...game google deepmind's deep q-learning...

  • Home
  • Documents
  • Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,
26
Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang

Upload: others

Post on 22-May-2020

9 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Introduction to Deep LearningPrinceton University COS 495

Instructor: Yingyu Liang

Page 2: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

What is deep learning?

• Short answer: recent buzz word

Page 3: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Industry

• Google

• Facebook

• Microsoft

• …

• Musk

• Toyota

• Drug

• Finance

Page 4: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Industry

• Google

Page 5: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Industry

• Facebook

Page 6: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Industry

• Microsoft

Page 7: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Industry

• Elon Musk

Page 8: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Industry

• Toyota

Page 9: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Academy

• NIPS 2015: ~4000 attendees, double the number of NIPS 2014

Page 10: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Academy

• Science special issue

• Nature invited review

Page 11: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

What is deep learning?

• Longer answer: machine learning framework that shows impressive performance on many Artificial Intelligence tasks

Page 12: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Image

• Image classification• 1000 classes

Slides from Kaimin He, MSRA

Human performance: ~5%

Page 13: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Image

• Object location

Slides from Kaimin He, MSRA

Page 14: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Image

• Image captioning

Figure from the paper “DenseCap: Fully Convolutional Localization Networks for Dense Captioning”, by Justin Johnson, Andrej Karpathy, Li Fei-Fei

Page 15: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Text

• Question & Answer

Figures from the paper “Ask Me Anything: Dynamic Memory Networks for Natural Language Processing ”,by Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Richard Socher

Page 16: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Game

Google DeepMind's Deep Q-learning playing Atari BreakoutFrom the paper “Playing Atari with Deep Reinforcement Learning”,by Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou,Daan Wierstra, Martin Riedmiller

https://www.youtube.com/watch?v=V1eYniJ0Rnk
Page 17: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Game

Page 18: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

The impact

• Revival of Artificial Intelligence

• Next technology revolution?

• A big thing ongoing, should not miss

Page 19: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Questions behind the scene

• Return of artificial neural network• What’s different

• Why get great performance

• Future development• The road to general-purpose AI?

Page 20: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Goal of the course

• Introduction

• Key concepts

• Ticket to the party

Page 21: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Syllabus

• Part I: machine learning basics• Linear model, Perceptron, SVM

• Multi-class

• Training by gradient descent

• overfitting

• Part II: supervised deep learning (feedforward network)

• Part III: unsupervised learning

• Part IV: deep learning in the wild

Page 22: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Syllabus

• Part I: machine learning basics

• Part II: supervised deep learning (feedforward network)• Multiple-layer and Backpropogation

• Regularization

• Convolution

• Part III: unsupervised deep learning

• Part IV: deep learning in the wild

Page 23: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Syllabus

• Part I: machine learning basics

• Part II: supervised deep learning (feedforward network)

• Part III: unsupervised deep learning• PCA

• Boltzmann machine, Deep Boltzmann machine

• autoencoder

• Part IV: deep learning in the wild

Page 24: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Syllabus

• Part I: machine learning basics

• Part II: supervised deep learning (feedforward network)

• Part III: unsupervised deep learning

• Part IV: deep learning in the wild• Read papers on advanced topics

• Play with the code

• Presentation

Page 25: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Textbook and materials

• Deep Learning:

http://www.deeplearningbook.org/

• Suggested software framework: Tensorflow• in Python

• Easy to install/use

• Can try it on your laptop

• Other software frameworks: Theano, Caffe, Torch, Marvin, …

http://www.deeplearningbook.org/
Page 26: Princeton University COS 495 Instructor: Yingyu Liang...Game Google DeepMind's Deep Q-learning playing Atari Breakout From the paper “Playing Atari with Deep Reinforcement Learning”,

Grading

• Problem Sets (5 sets): 70%

• Design Projects: 25%

• Oral Presentation: 5%


ATARI 65XE - Zgierz 65XE_owners_manual...ATARI 65XE PERSONAL COMPUTER OWNER'S MANUAL TM. ... PART 2: PROGRAMMING WITH ATARI BASIC ... puter's operating system

Introduction to Deep Q-network - Washington State Universitytaylorm/17_580/Yunshu_DQNpresentation_10… · Deep Q-network (DQN) •An artificial agent for general Atari game playing

Atari System Reference manualmixinc.net/atari/books/Atari-System_Reference_manual.pdf · 2014. 12. 24. · Atari ∗System Reference ... 8 The numbers on top are the bit numbers

Atari Copy Protection - info-coach.frinfo-coach.fr/atari/documents/_mydoc/Atari-Copy-Protection-v12.pdf · Atari Floppy Disk Copy Protection Copyleft Jean Louis-Guérin (DrCoolZic)

Human-level control through deep reinforcement learning · Towards General Artificial Intelligence •Playing Atari with Deep Reinforcement Learning. ArXiv (2013) •7 Atari games

Nan Du Yingyu Liang Maria-Florina Balcan arXiv:1612

Atari Notator

Zaxxon - Atari 2600 - Manual - gamesdatabase · Zaxxon - Atari 2600 - Manual - gamesdatabase.org Author: gamesdatabase.org Subject: Atari 2600 game manual Keywords: Atari 2600 1982

SIT-IN DEDICATED GAME OPERATION MANUAL Francisco Rush Man… · atari _.._ .__. . atari. \.._! ; atari! > _. : _‘“,:. .__ -... __.____ atari.-

Atari Vazno

Introduction to Artificial Intelligence CS540-1 Yingyu Liang

Hard 'n Heavy - Atari ST - Manual - gamesdatabase · Hard 'n Heavy - Atari ST - Manual - gamesdatabase.org Author: gamesdatabase.org Subject: Atari ST game manual Keywords: Atari

Paper intoduction "Playing Atari with deep reinforcement learning"

ATARI® PROGRAM EXCHANGE Program... · Atari, Inc., created the ATARI Program Exchange (APX) to distribute user-written software for ATARI Home Computers. Our goal is to increase

1 Towards Playing Montezumas Revenge with Deep ... · A. Playing Atari 2600 Games Development of a general video game playing agent specif-ically for the Atari 2600 was introduced

ATARI LOGO - atarionline.platarionline.pl/biblioteka/materialy_ksiazkowe/Atari Logo... · ATARI LOGO Introduction to Programming ... A Game Project 125 ... To use the ATARI Logo Cartridge,

F-16 Combat Pilot - Atari ST - Manual - …...Title F-16 Combat Pilot - Atari ST - Manual - gamesdatabase.org Author gamesdatabase.org Subject Atari ST game manual Keywords Atari ST

Atari xbox

Dungeon Master - Atari ST - Manual - gamesdatabase...Title Dungeon Master - Atari ST - Manual - gamesdatabase.org Author gamesdatabase.org Subject Atari ST game manual Keywords Atari

A Neuroevolution Approach to General Atari Game Playing€¦ · A few example games are shown in Figure 1. From the perspective of an AI practitioner, Atari games are similar to traditional

Sorcerer's Apprentice - Atari 2600 - Manual - gamesdatabase...Title: Sorcerer's Apprentice - Atari 2600 - Manual - gamesdatabase.org Author: gamesdatabase.org Subject: Atari 2600 game

Atari 410 Program Recorder Operator's Manual (Atari)

Atari petition

Battlezone - Atari 2600 - Manual - gamesdatabase€¦ · Battlezone - Atari 2600 - Manual - gamesdatabase.org Atari 2600 1983 Atari Simulation system game manual

Star Raiders - Atari 8-bit - Manual - gamesdatabase · atari 400/800 computer adventure star raiders ata company model cxl4011 use with atari' 400'" or atari personal computer systems

Your Atari Computer, A Guide to Atari 400/800 Personal Computers

Atari Puzzlements

Fight For Life - Atari Jaguar - Manual - gamesdatabase · Fight for Life, Atari, the Atari logo and Jaguar are trademarks or registered tradermarks of Atari Corporation, This software

Playing Atari Games with Deep Reinforcement Learning · Playing Atari Games with Deep Reinforcement Learning 2 Game Videos at: amasare/cs365/project/dqrl.html Acknowledgements

Atari - Lazard

Semantic Wordfication of Document Collections Presenter: Yingyu Wu

General Atari 2600 Game Playing - GeNeura Teamgeneura.ugr.es/cig2012/slides/keynotes/MichaelBowling-Atari...General Atari 2600 Game Playing Michael Bowling ... Pitfall! Pitfall II

PLAYING ATARI WITH DEEP REINFORCEMENT ...f15ece6504/slides/L26_RL.pdfPLAYING ATARI WITH DEEP REINFORCEMENT LEARNING ARJUN CHANDRASEKARAN DEEP LEARNING AND PERCEPTION (ECE 6504) NEURAL

Sea Battle - Atari 2600 - Manual - gamesdatabase Battle - Atari 2600 - Manual - gamesdatabase.org Author: gamesdatabase.org Subject: Atari 2600 game manual Keywords: Atari 2600 …

The Atari™ Compendium - Info-Coachinfo-coach.fr/atari/software/_development/Atari... · 2014. 11. 8. · The Atari™ Compendium ©1992 Software Development Systems Written by Scott