deep learningce.aut.ac.ir/~keyvanrad/download/dl961/dl_1(introduction).pdf · 9/24/2017 m.a...
TRANSCRIPT
![Page 1: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/1.jpg)
Deep Learning
Mohammad Ali Keyvanrad
Lecture 1:Introduction
![Page 2: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/2.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 2
![Page 3: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/3.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 3
![Page 4: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/4.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4
Recent success with Deep Learning
• Over the last few years Deep Learning was applied tohundreds of problems. Computer vision and pattern recognition
Speech recognition and speech synthesis
Natural language processing
Computer games, robots & self-driving cars
…
• In many problems they have established the state ofthe art Often exceeding previous benchmarks by large margins
![Page 5: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/5.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 5
Learning Lip Sync from Audio(University of Washington, 2017)
![Page 6: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/6.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 6
Restore colors in B&W photos (Waseda University, 2016)
Input [Larsson et al. 2016] [Zhang et al. 2016a] [Zhang et al. 2016b]
![Page 7: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/7.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 7
Pixel restoration(Google Brain, 2017)
• Take very low resolution images and predict what eachimage most likely looks like.
![Page 8: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/8.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 8
Describing photos(Stanford University, 2015)
• Computers can automatically classify our photos Facebook can automatically tag your friends
• Deep Learning not only learned to classify the elementsin the photo, but to actually describe them.
![Page 9: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/9.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 9
Translation(Google Translate, 2015)
• Google Translate app now does real-time visualtranslation of 20 more languages. A photo taken by the phone, and Google Translate "reads"
the text and replaces it with a text in English in real-time.
![Page 10: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/10.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 10
Create new images(University of California, 2017)
• Deep Learning network to create other types of newimages
![Page 11: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/11.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 11
Reading text in the Wild(University of Oxford, 2014)
• An attempt to read text from photos and videos Search for text from BBC News videos
![Page 12: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/12.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 12
Teach a computer to play(DeepMind, 2015)
• Google's DeepMind used a Deep Learning technique toteach a computer to play Control of the keyboard while watching the score, and its
goal was to maximize the score
![Page 13: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/13.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 13
Beating people in dozens of computer games
Computer program playing Doom using only raw pixel data.
![Page 14: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/14.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 14
Self-driving cars(Tesla, 2016)
• A Tesla electric vehicle drives without humanintervention Notice how it distinguishes different type of objects,
including people and road signs.
![Page 15: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/15.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 15
Robotics(BostonDynamics, 2016)
• Deep Learning is also heavily used in robotics thesedays SpotMini and Atlas
![Page 16: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/16.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 16
Voice generation(Google WaveNet, 2016)
• Deep Learning is taking us a step closer to givingcomputers the ability to speak like humans do. Google released WaveNet and Tacotron
Baidu released Deep Speech
![Page 17: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/17.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction)
• Tacotron learns pronunciations based on phrasesemantics. “He has read the whole thing.”
“He reads books.”
• Tacotron is sensitive to punctuation. “This is your personal assistant, Google Home.”
• Tacotron learns stress and intonation. “The buses aren't the problem, they actually provide a solution.”
“The buses aren't the PROBLEM, they actually provide a SOLUTION.”
• Tacotron's prosody changes in a question. “The quick brown fox jumps over the lazy dog.”
“Does the quick brown fox jump over the lazy dog?”
17
Voice generation(Google Tacotron, 2017)
![Page 18: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/18.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 18
Restoring sound in videos(MIT, Berkeley, Google, 2016)
• Deep Learning network was trained on videos in whichpeople were hitting and scratching objects
• After several iterations learning, the scientists asked thecomputer to regenerate the sound
![Page 19: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/19.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 19
LIPNET(Oxford, DeepMind, 2016)
• LipNet reached 93% success in reading people's lipswhere an average lipreader succeeds 52% of the time.
![Page 20: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/20.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 20
Automatically writing(Stanford University, 2016)
• Let a Deep Learning network "read" Shakespeare,Wikipedia, math papers and computer code.
![Page 21: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/21.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 21
Handwriting(University of Toronto, 2014)
• Today the computer can also handwrite.
![Page 22: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/22.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 22
![Page 23: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/23.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 23
Deep Learning definition
• Deep learning is a class of machine learning algorithmsthat: Definition 1: They use a cascade of many layers of nonlinear processing units for
feature extraction and transformation Each successive layer uses the output from the previous layer as
input. The algorithms may be supervised or unsupervised. Applications include pattern analysis (unsupervised) and
classification (supervised).
Definition 2: They are part of the broader machine learning field of learning
representations of data.
![Page 24: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/24.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 24
Deep Learning definition
Definition 3: These are based on the (unsupervised) learning of multiple levels
of features or representations of the data.
Higher level features are derived from lower level features to forma hierarchical representation.
Definition 4: They learn multiple levels of representations that correspond to
different levels of abstraction; the levels form a hierarchy ofconcepts.
![Page 25: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/25.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 25
Deep Learning definition
• Common in definitions Models consisting of multiple layers or stages of nonlinear
information processing.
The supervised or unsupervised learning of featurerepresentations in each layer, with the layers forming ahierarchy from low-level to high-level features.
![Page 26: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/26.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 26
Deep Learning definition
• Deep or Shallow? Credit Assignment Path (CAP) A chain of transformations from input to output.
CAPs describe potentially causal connections between input andoutput.
CAP depth Number of hidden layers plus one
as the output layer is also parameterized
For recurrent neural networks the CAP depth is potentiallyunlimited.
a signal may propagate through a layer more than once.
![Page 27: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/27.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 27
Deep Learning definition
• Deep/shallow?
• No universally agreed upon threshold of depth divides shallowlearning from deep learning
• Most researchers agree that deep learning has multiple nonlinearlayers (CAP > 2).
![Page 28: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/28.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 28
![Page 29: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/29.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 29
History
![Page 30: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/30.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 30
Evolution of DepthLeNet
1998
7 Layers
AlexNet
2012
8 Layers
GoogLeNet
2014
22 Layers
ResNet
2016
152 Layers
![Page 31: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/31.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 31
Evolution of Depth
![Page 32: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/32.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 32
![Page 33: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/33.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 33
Course plan
• Introduction
• A Review of Artificial Neural Networks Perceptron
Stochastic Gradient Descent
Backpropagation
Rectified Linear Function
Root Mean Square Propagation
Dropout
L1 and L2 Regularization
![Page 34: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/34.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 34
Course plan
• Deep Belief Network PGM
MRF
Sampling
RBM
• Auto-Encoder Linear Auto-Encoder
Denoising Auto-Encoder
• Computational Network
![Page 35: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/35.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 35
Course plan
• Selected Applications in Object Recognition and ComputerVision Convolutional Neural Networks Region Based CNN Generative Adversarial Network GoogLeNet and Microsoft ResNet
• Selected Applications in Language Modeling and NaturalLanguage Processing Word2Vec Recurrent Neural Networks and Language Models Machine translation and advanced recurrent LSTMs and GRUs
![Page 36: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/36.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 36
Course plan
• Selected Applications in Speech and Audio Processing Speech recognition and bi-directional RNN
Speech synthesis and WaveNet or Tacotron
• Deep Reinforcement Learning
![Page 37: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/37.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 37
![Page 38: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/38.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 38
Resources
• Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning (AdaptiveComputation and Machine Learning series)”, MIT Press, 2016.
• Dong Yu, Li Deng, “Automatic Speech Recognition: A Deep LearningApproach”, Springer, 2015
• L. Deng and D. Yu, “Deep Learning: Methods and Applications”, NowPublishers Inc, 2014.
• Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006.
• Stanford (CS224n: Natural Language Processing with Deep Learning, 2017)
• Stanford (CS231n: Convolutional Neural Networks for Visual Recognition,2017)
• Related papers
![Page 39: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/39.jpg)
OUTLINE • Recent success with Deep Learning
• Deep Learning definition
• History
• Course plan
• Resources
• Grading Policy
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 39
![Page 40: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/40.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 40
Grading Policy
• Assignments: 20%
• Presentation: 15%
• Final Exam: 35%
• Final Project: 30%
![Page 41: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/41.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 41
![Page 42: Deep Learningce.aut.ac.ir/~keyvanrad/download/DL961/DL_1(Introduction).pdf · 9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 4 Recent success with Deep Learning •Over](https://reader034.vdocuments.us/reader034/viewer/2022051900/5fef17b42aab16732b2db0c2/html5/thumbnails/42.jpg)
9/24/2017 M.A Keyvanrad Deep Learning (Lecture1-Introduction) 42