modern artificial intelligence via deep learningmpd37/teaching/ml_tutorials/... · 2016-10-24 ·...
TRANSCRIPT
![Page 1: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/1.jpg)
Modern Artificial Intelligence via Deep Learning
S. M. Ali Eslami
October 2016
![Page 2: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/2.jpg)
Algorithms
OutputInput
Algorithm
ProgrammableComputer
Introduction
![Page 3: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/3.jpg)
Artificial Intelligence / Machine Learning
OutputInput
Algorithm
ProgrammableComputer
Introduction
?
Horse
![Page 4: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/4.jpg)
Introduction
An Analogy
Immediate Usefulness
General Applicability
![Page 5: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/5.jpg)
Introduction
An Analogy
Immediate Usefulness
General Applicability
![Page 6: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/6.jpg)
Introduction
An Analogy
Immediate Usefulness
General Applicability
![Page 7: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/7.jpg)
Introduction
An Analogy
Immediate Usefulness
General Applicability
![Page 8: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/8.jpg)
Introduction
An Analogy
Immediate Usefulness
General Applicability
?
![Page 9: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/9.jpg)
Deep Supervised Learning
![Page 10: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/10.jpg)
Computer
Horse
Cow
OutputInput
Prep
roce
ssin
g
Feat
ure
Extr
actio
n
Feat
ure
Sele
ctio
n
Lear
ned
Dis
crim
inat
ion
Calib
ratio
n
Algorithm
![Page 11: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/11.jpg)
Computer
Horse
Cow
OutputInput
Stag
e 1
Stag
e 2
Stag
e 3
Stag
e 4
Stag
e 5
Algorithm
![Page 12: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/12.jpg)
Introduction
Convolutional Neural Networks
Torch (2015)
![Page 13: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/13.jpg)
Introduction
Convolutional Neural Networks
Krizhevsky et al. (2012)
![Page 14: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/14.jpg)
Clarifai (2014)
![Page 15: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/15.jpg)
Introduction
● Optimize directly for the end loss
● End-to-end training, no engineered inputs
● With enough data, learn a big non-linear function
● Supervised labeling is often enough for transferrable representations
● Large labeled dataset + big / deep neural network + GPUs
Deep Supervised Learning
![Page 16: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/16.jpg)
Introduction
Deep Supervised Learning
Zhang et al. (2015) Simonyan et al. (2014)
Text Classification Video Classification
![Page 17: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/17.jpg)
Introduction
● Innovation continues○ Inception (Szegedy et al., 2015)○ Residual connections (He et al., 2015)○ Batchnorm (Ioffe et al., 2015)
● Performance is continuously improving
Deep Supervised Learning
Szegedy et al., (2015)
![Page 18: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/18.jpg)
Introduction
● Sequence Modelling● Unsupervised Learning● Generative Modelling● Probabilistic Modelling● ...
Beyond Supervised Learning
Hochreiter et al. (1997) Vinyals et al. (2015) Kavukcuoglu et al. (2009) Hinton et al. (2006)
Larochelle et al. (2011)Rasmus et al. (2015)
![Page 19: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/19.jpg)
Where does the data come from?
![Page 20: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/20.jpg)
Reinforcement Learning
● Supervised / unsupervised learning is important○ Gives us tractable targets○ Helps model development○ Sometimes best to do algorithmic search when gradients are not noisy○ However large labelled datasets are not enough
● Real AI requires agents that○ interpret their environments○ act in their environments to gather data○ control themselves and their environments○ form representations that generalize○ learn end-to-end with minimal engineering
Agents
![Page 21: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/21.jpg)
Reinforcement Learning
Architecture
Agent Environment
Observations
Actions
![Page 22: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/22.jpg)
Reinforcement Learning
Powered by Neural Networks
Tesauro (1989) Lange et al. (2012)
Levine et al. (2015) Schulman et al. (2015)
![Page 23: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/23.jpg)
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis
Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis (2015)
![Page 24: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/24.jpg)
Human-level control in ATARI
100+ classic 8-bit Atari games
● Observations: Raw video (~30k dimensional)● Actions: 18 buttons but not told what they do● Goal: Simply to maximize score
● Designed to be challenging and interesting for humans● Widely adopted benchmark for evaluation (Bellemare et al., 2013)● Provides a rich visual domain● Many different games emphasize control, strategy, planning, etc.
ATARI agents
![Page 25: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/25.jpg)
● Maximise total future reward:
● For a policy π the action-value function Q is:
● Measure of how good action a is in state s○ Greedy: Follow the max○ ε-greedy: Follow the max with (1-ε) probability and random otherwise
Human-level control in ATARI
The action-value function
![Page 26: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/26.jpg)
● Maximizing Qπ(s,a) over possible policies gives the optimal action-value function and the Bellman equation:
● Basic idea:○ Approximate ○ Apply the Bellman Equation as an iterative update:
Human-level control in ATARI
Value iteration
![Page 27: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/27.jpg)
Human-level control in ATARI
End-to-end reinforcement learning
Mnih et al. (2015)
![Page 28: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/28.jpg)
● We need a loss function to minimize
● So now we can do our good old SGD update:
Human-level control in ATARI
Value iteration
![Page 29: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/29.jpg)
● Experiences in a sequence are correlated○ Do not do online updates, store in replay memory○ Sample from experience replay memory to apply Q-updates
● Targets can not depend on same Θi → introduce target network
Human-level control in ATARI
Value iteration
![Page 30: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/30.jpg)
Human-level control in ATARI
Deep Q Networks
![Page 34: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/34.jpg)
Human-level control in ATARI
Evaluation
![Page 35: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/35.jpg)
1 epoch = 50,000 interactions = 30 minutes of experienceTotal experience: 10m interactions = 5 days
Human-level control in ATARI
Data ‘efficiency’
![Page 37: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/37.jpg)
Asynchronous Methodsfor Deep Reinforcement Learning
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu (2016)
![Page 38: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/38.jpg)
Reinforcement Learning
● DQN is very robust, but computationally expensive○ About a week to train on a single GPU
● Off-policy Q-Learning○ We would like a robust system for both on-policy and off-policy methods
● Discrete action space○ We want to be able to use the same method on continuous action spaces too
Asynchronous RL
![Page 39: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/39.jpg)
Reinforcement Learning
● Asynchronous training of RL agents
● Parallel actor-learners implemented using CPU threads
● No replay? Parallel actor-learners have a similar stabilizing effect
● Choice of RL algorithm○ on- or off-policy○ value or policy-based
Asynchronous RL
![Page 40: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/40.jpg)
Reinforcement Learning
Asynchronous RL
![Page 41: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/41.jpg)
Asynchronous RL
● Parallel actor-learners compute online 1-step update
● Gradients accumulated over minibatch before update
1-step Q-learning
![Page 42: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/42.jpg)
Asynchronous RL
● Q-learning with a uniform mixture of backups of length 1 through N
● Variation of Peng and Williams (1995)
n-step Q-learning
![Page 43: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/43.jpg)
Asynchronous RL
● The agent learns a policy and a state value function● Policy gradient multiplied by an estimate of the advantage● Similar to Generalized Advantage Estimation (Schulman et al, 2015)
Asynchronous Advantage Actor-Critic (A3C)
![Page 44: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/44.jpg)
Asynchronous RL
1-step Q-learning
![Page 45: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/45.jpg)
Asynchronous RL
Asynchronous Advantage Actor-Critic (A3C)
![Page 47: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/47.jpg)
Asynchronous RL
Recap
● Lightweight framework for asynchronous reinforcement learning○ Stable training with a variety of standard RL algorithms○ State-of-the-art results on a range of domains in hours on a single machine
● Async advantage actor-critic excels on:○ Both discrete and continuous actions○ Feedforward and recurrent agents○ 2D and 3D games
![Page 48: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/48.jpg)
Model-based Methods
![Page 49: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/49.jpg)
Model-based methods
Three learning paradigms
x
z
x
SupervisedLearning
Reinforcement Learning
y
z
ahorse left
![Page 50: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/50.jpg)
Model-based methods
Three learning paradigms
Model
x
z
xx
z
x
SupervisedLearning
Reinforcement Learning
GenerativeModelling
y
z
a yhorse left
![Page 51: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/51.jpg)
Model-based methods
Three learning paradigms
Model
x
z
xx
z
x
SupervisedLearning
Reinforcement Learning
GenerativeModelling
y
z
a y
(2.3, -1, 0.5, 3)
not blinkinghorse left
![Page 52: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/52.jpg)
The Shape Boltzmann MachineS. M. Ali Eslami, Nicolas Heess, John Winn (2011)
![Page 53: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/53.jpg)
Model-based methods
The Shape Boltzmann Machine
![Page 54: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/54.jpg)
Model-based methods
The Shape Boltzmann Machine
![Page 56: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/56.jpg)
Model-based methods
The Shape Boltzmann Machine
Eslami et al. (2012)
![Page 57: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/57.jpg)
Model-based methods
The Shape Boltzmann Machine
Eslami et al. (2012)
![Page 58: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/58.jpg)
Model-based methods
The Shape Boltzmann Machine
Eslami et al. (2012)
![Page 59: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/59.jpg)
Model-based methods
Modern Variational Inference
Model
x
z
x
y
Approximate p(z|x) using q(z|x)
Parameterise q(z|x) by deep network
Parameterise p(x|z) by deep network
Minimise KL[ q(z|x) | p(z|x) ] via SGD
Samples from q(z|x) can be used as codes representing the image x
![Page 60: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/60.jpg)
Recurrent Neural Networksfor Image Generation
Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, Daan Wierstra (2015)
![Page 61: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/61.jpg)
Model-based methods
Recurrent Neural Networks for Image Generation
c
x
z
p(x|c)D
ecod
ing
Gen
erat
ion
Enc
odin
gIn
fere
nce
Gregor et al. (2015)
![Page 62: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/62.jpg)
Read
ct+1
x
Write
z
Model-based methods
Recurrent Neural Networks for Image Generation
c
x
z
p(x|c)D
ecod
ing
Gen
erat
ion
Enc
odin
gIn
fere
nce
Write
Read Read
ct
x
ct+1
x
Write
p(x|cT)
z z
Gregor et al. (2015)
![Page 63: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/63.jpg)
Model-based methods
Recurrent Neural Networks for Image Generation
![Page 64: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/64.jpg)
Model-based methods
Recurrent Neural Networks for Image Generation
![Page 65: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/65.jpg)
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
S. M. Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, Koray Kavukcuoglu, Geoffrey Hinton (2016)
![Page 66: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/66.jpg)
Model-based methods
Attend, Infer, Repeat
![Page 67: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/67.jpg)
x
z
blue brick
Mod
elIm
age
Cau
se
![Page 68: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/68.jpg)
x
z
blue brick pile of bricks
x
z
Mod
elIm
age
Cau
se
not sufficient forgraspingcountingtransfergeneralisation
![Page 69: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/69.jpg)
x
z
x
z1 z2
Mod
elIm
age
Cau
se
blue brick red brickpile of bricks
![Page 70: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/70.jpg)
x
z
Mod
elIm
age
Cau
se
x
zwhat
y1
z1 zwherez1 zwhat
y2
z2 zwherez2
atty1
atty2
blue brick red brickpile of bricks blue brickabove
red brickbelow
x
z1 z2
![Page 71: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/71.jpg)
Decoder
x y
z Decoder
x y
h1 h2 h3
z1 z2 z3
x
z
x
z1 z2 z3
Mod
elIn
fere
nce
Net
wor
k
![Page 72: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/72.jpg)
Decoder
x y
h1 h2 h3
z1 z2 z3 Decoder
x y
h1 h2 h3
zpresz1 zpresz2 zpresz3zwhatz1 zwhatz2 zwhatz3zwherez1 zwherez2 zwherez3
x
zwhat
y1
z1 zwherez1 zwhat
y2
z2 zwherez2
atty1
atty2
Mod
elIn
fere
nce
Net
wor
k
x
z1 z2 z3
![Page 73: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/73.jpg)
Decoder
x y
h1 h2 h3
z1 z2 z3 Decoder
x y
h1 h2 h3
zpresz1 zpresz2 zpresz3zwhatz1 zwhatz2 zwhatz3zwherez1 zwherez2 zwherez3
x
zwhat
y1
z1 zwherez1 zwhat
y2
z2 zwherez2
atty1
atty2
Mod
elIn
fere
nce
Net
wor
k
x
z1 z2 z3
focus on representation not reconstruction
output is a setorder? count?
![Page 74: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/74.jpg)
x y
zpres
zwhatxatt yatt
hi
zwhere...
VA
E
yi
i ii
i
i
... ...
![Page 75: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/75.jpg)
Attend, Infer, Repeat
Key Ideas
1. Build in structureGet out meaning
2. Inference networks that area. recurrentb. variable-lengthc. attentive
3. End-to-end learning througha. discrete, continuous varsb. inference and model nets
Decoder
x y
h1 h2 h3
zpresz1 zpresz2 zpresz3zwhatz1 zwhatz2 zwhatz3zwherez1 zwherez2 zwherez3
x
zwhat
y1
z1 zwherez1 zwhat
y2
z2 zwherez2
atty1
atty2
![Page 77: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/77.jpg)
Attend, Infer, Repeat
Omniglot
![Page 78: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/78.jpg)
Attend, Infer, Repeat
Representational Power
6
9
no
yes
Sum? Increasing order?
![Page 79: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/79.jpg)
Attend, Infer, Repeat
Additional Structure
x
z
distributed vector that correlates with blue brick
learned
![Page 80: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/80.jpg)
Attend, Infer, Repeat
Additional Structure
x
z
distributed vector that correlates with blue brick
learned
x
z
class=brickcolour=blueposition=Protation=R
specified
![Page 81: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/81.jpg)
![Page 82: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/82.jpg)
![Page 83: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/83.jpg)
Attend, Infer, Repeat
Additional Structure
Decoder
x y
h1 h2 h3
z1 z2 z3
x
z1 z2 z3
specified
![Page 84: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/84.jpg)
Attend, Infer, Repeat
Inverse Graphics
![Page 85: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/85.jpg)
Attend, Infer, Repeat
Inverse Graphics
![Page 86: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/86.jpg)
Attend, Infer, Repeat
Policy learning
Tabl
e-to
pM
NIS
T
![Page 87: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/87.jpg)
Towards Deep Symbolic Reinforcement LearningMarta Garnelo, Kai Arulkumaran, Murray Shanahan (2016)
![Page 88: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/88.jpg)
Unsupervised Learning of 3D Structure from Images
Danilo Rezende, S. M. Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, Nicolas Heess (2016)
![Page 89: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/89.jpg)
Model-based methods
Unsupervised Learning of 3D Structure from Images
![Page 90: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/90.jpg)
Model-based methods
Unsupervised Learning of 3D Structure from Images
![Page 91: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/91.jpg)
Unsupervised Learning of 3D Structure from Images
Inferring object meshes
![Page 92: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/92.jpg)
Unsupervised Learning of 3D Structure from Images
Class-conditional samples
![Page 93: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/93.jpg)
Unsupervised Learning of 3D Structure from Images
3D structure from multiple 2D images
![Page 94: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/94.jpg)
Pixel Recurrent Neural NetworksAäron van den Oord, Nal Kalchbrenner, Koray Kavukcuoglu (2016)
![Page 95: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/95.jpg)
Generative Modelling
Goal: Learn a generative model of natural images
Research landscape:● Latent variable models (VAEs, DRAW)● Adversarial (GANs)● Fully visible (NADE, MADE, RIDE)
PixelRNN: Fully visible, probabilistic, tractable, density estimator
Pixel Recurrent Neural Networks
![Page 96: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/96.jpg)
Pixel Recurrent Neural Networks
Model
● Fully visible
● Similar to language models with RNNs
● Model pixels with Softmax
![Page 97: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/97.jpg)
Pixel Recurrent Neural Networks
Masked Convolutions
Spatially Colors
![Page 98: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/98.jpg)
Pixel Recurrent Neural Networks
Masked Convolutions
![Page 99: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/99.jpg)
![Page 100: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/100.jpg)
![Page 101: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/101.jpg)
![Page 102: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/102.jpg)
![Page 103: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/103.jpg)
![Page 104: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/104.jpg)
![Page 105: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/105.jpg)
![Page 106: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/106.jpg)
![Page 107: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/107.jpg)
Pixel Recurrent Neural Networks
Binary MNIST
![Page 108: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/108.jpg)
Pixel Recurrent Neural Networks
CIFAR-10
![Page 109: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/109.jpg)
Pixel Recurrent Neural Networks
CIFAR-10
![Page 110: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/110.jpg)
occluded
![Page 111: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/111.jpg)
occluded completions
![Page 112: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/112.jpg)
occluded completions original
![Page 113: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/113.jpg)
Elephant
Sandbar
Coral Reef
Horse
![Page 114: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/114.jpg)
Lhasa Apso (Dog)
Brown Bear
Lawn Mower
Robin (Bird)
![Page 115: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/115.jpg)
Geyser
White Whale
Hartebeest
Tiger
Alp
![Page 116: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/116.jpg)
Matching Networks for One Shot LearningOriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra (2016)
![Page 117: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/117.jpg)
Core Machine Learning
Matching Nets x1
x2
x3
q
y1
y2
y3
z1
z2
z3
qz
a3
a2
a1
yz
y1
y2
y3
![Page 118: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/118.jpg)
Core Machine Learning
Matching Nets x1
x2
x3
q
y1
y2
y3
z1
z2
z3
qz
a3
a2
a1
yz
y1
y2
y3
red implements same-class-or-notnetwork
![Page 119: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/119.jpg)
Core Machine Learning
Matching Nets x1
x2
x3
q
y1
y2
y3
z1
z2
z3
qz
a3
a2
a1
yz
y1
y2
y3
the idea is useful because it allows us to construct a classifier on the
fly without any further training
![Page 120: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/120.jpg)
Core Machine Learning
Matching Nets x1
x2
x3
q
y1
y2
y3
z1
z2
z3
qz
a3
a2
a1
yz
y1
y2
y3
R
![Page 121: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/121.jpg)
Core Machine Learning
Matching Netsy1
y2
y3
x1
x2
x3
q
y1
y2
y3
z1
z2
z3
qz
a3
a2
a1
c
R
![Page 122: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/122.jpg)
Core Machine Learning
Matching Nets
![Page 123: Modern Artificial Intelligence via Deep Learningmpd37/teaching/ml_tutorials/... · 2016-10-24 · Modern Artificial Intelligence via Deep Learning S. M. Ali Eslami October 2016. Algorithms](https://reader035.vdocuments.us/reader035/viewer/2022070809/5f07ec0a7e708231d41f6e45/html5/thumbnails/123.jpg)
● Machine Learning● Deep Supervised Learning● Deep Reinforcement Learning● Model-based Methods● Deep Variational Inference● Structured / Unstructured Generative Models● Matching Networks
Recap