large scale data analysis using deep learningukang/courses/17s-dl/l1-intro... · 2018-12-28 ·...
TRANSCRIPT
![Page 1: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/1.jpg)
U Kang 1
Large Scale Data Analysis UsingDeep Learning
Course Introduction
U KangSeoul National University
![Page 2: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/2.jpg)
U Kang 2
In This Lecture
Motivation to study deep learning
Administrative information for this course
![Page 3: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/3.jpg)
U Kang 3
Outline
Deep Learning
Course Information
![Page 4: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/4.jpg)
U Kang 4
Deep Learning as a Machine Learning
Machine Learning (ML)
Given x (predictor) and y (response), ML learns a function f() from data, such that y = f(x)
E.g., x = image, y = category
This learned function f() can be used to classify a new example x’
This is different from a typical programming where you want to compute y, given x and f()
Deep learning provides good performances in learning f() for many problems
Learns non-linear functions
![Page 5: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/5.jpg)
U Kang 5
Deep Learning as a Machine Learning
Data Size
Accuracy Deep Learning
Other machine
learning
![Page 6: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/6.jpg)
U Kang 6
Learning Tasks
Image classification
Speech recognition
Text classification
…
“Taxi”
Hello, dear
International politics
![Page 7: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/7.jpg)
U Kang 7
Main Idea
Most perception (input processing) in the brain may use one learning algorithm
Design learning methods that mimic the brain
![Page 8: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/8.jpg)
U Kang 8
Neurons In the Brain
![Page 9: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/9.jpg)
U Kang 9
Neural Network
[LeCun et al.,
Nature 2015]
![Page 10: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/10.jpg)
U Kang 10
Convolutional Neural Net (CNN)
![Page 11: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/11.jpg)
U Kang 11
Representation Learning
Typical machine learning
Deep learning
Input OutputExtract
Features
x y
Input Output
x y
Extract
Features
Extract
Features
Extract
Features
Classifier
… Classifier
![Page 12: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/12.jpg)
U Kang 12
Human Level Object Recognition
ImageNet ~ 15M labeled images, ~ 22K categories
Top-5 Error rates (1.2 million images, 1k categories) Non-CNN based method (~2012): 26.2 %
Alexnet (2012): 15.3 %
GoogLeNet (2014): 6.66 %
Resnet (2015): 3.57%Human level:
5.1% error
![Page 13: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/13.jpg)
U Kang 13
Human-Level Face Recognition
DeepFace
97% accuracy
~ Human-level
[Taigman et al.
CVPR 2014]
![Page 14: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/14.jpg)
U Kang 14
Computer Game
Deepmind
![Page 15: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/15.jpg)
U Kang 15
Computer Game
Deepmind [Nature 2015]
https://www.youtube.com/watch?v=V1eYniJ0Rnk
![Page 16: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/16.jpg)
U Kang 16
AlphaGo
[Silver et al., Mastering the game of Go with
deep neural networks and tree search, Nature 2016]
![Page 17: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/17.jpg)
U Kang 17
Neural Artist
[Gatys et al., Image Style Transfer Using
Neural Networks, CVPR 2016]
![Page 18: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/18.jpg)
U Kang 18
Machine Translation
![Page 19: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/19.jpg)
U Kang 19
Topics in This Course (tentative)
(ch. 1) Introduction (ch. 2) Linear Algebra (ch. 3) Probability and Information Theory (ch. 4) Numerical Computation (ch. 5) Machine Learning Basics (ch. 6) Deep Feedforward Networks
(ch. 7) Regularization for Deep Learning (ch. 8) Optimization for Training Deep Models (ch. 9) Convolutional Networks (ch. 10) Sequence Modeling: Recurrent and Recursive Nets (ch. 11) Practical Methodology (ch. 12) Applications
![Page 20: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/20.jpg)
U Kang 20
Outline
Deep Learning
Course Information
![Page 21: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/21.jpg)
U Kang 21
M1522.001600, Spring 2017
http://datalab.snu.ac.kr/~ukang/courses/17S-DL/
Lecture slide: at least 1 hour before the lecture
Office hour
Mon. 11:00 – 12:00
Class meets: Mon, Wed 14:00 – 15:15, 301-101
![Page 22: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/22.jpg)
U Kang 22
Textbook
Deep Learning (Ian Goodfellow, Yoshua Bengio, and Aaron Courville)
Available at http://www.deeplearningbook.org
![Page 23: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/23.jpg)
U Kang 23
Prerequisites
Basic probability
Average, std. deviation, typical distributions, MLE, …
Basic linear algebra
Rank, singular value decomposition
Programming language
Python, (C++, Java)..
Machine Learning or Artificial Intelligence
Basic understandings of machine learning
![Page 24: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/24.jpg)
U Kang 24
Grading
10% Attendance and Quiz (random)
30% Project
30% Midterm
30% Final
(+5% Participation)
![Page 25: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/25.jpg)
U Kang 25
Late Policy
For all deliverables (homework, code, …)
No delay penalties, for medical etc. emergencies (bring doctor's note)
Each person has 4 'slip days' total, for the whole semester. 10% per day of delay, after that
![Page 26: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/26.jpg)
U Kang 26
Project
A good opportunity to solve real world problems using deep learning
Team
A group of 3-4 persons
If you cannot find your team mate, discuss with the TA and/or instructor
![Page 27: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/27.jpg)
U Kang 27
Project
Topic Improve the current status-of-the-art in deep learning
Deep learning applications E.g., Novel applications
Deep learning implementations E.g., Fast implementations using GPU
Method of deep learning E.g., New regularization method E.g., Achieve the best score in object recognition competition
We will provide some candidates, but feel free to propose your own topic
Feel free to discuss with the TA and/or the instructor before the proposal
![Page 28: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/28.jpg)
U Kang 28
Project
Advice
Start your project from day 1: today!
Think of the data you are interested in (e.g. image, audio, text, graph, etc.), and how to get it
Read related papers
Find your team mates
Data should be ready very soon
In the worst case, it should be ready until the end of March
If you plan to collect the data later, you might not get it until the semester ends
![Page 29: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/29.jpg)
U Kang 29
Project
Schedule and grading
Project proposal (due April 3) : 10%
Progress report (due May 3): 20%
Final report and presentation (due June 7) : 70%
![Page 30: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/30.jpg)
U Kang 30
Advice
Read each chapter before class It is ok to encounter something you don’t understand. Just
mark it, and later you will understand it when you come back.
“Understand” intuitions of main ideas Do not memorize without understanding Improve your problem solving skills
Active participation encouraged All questions are right; ask many questions
Use office hours (instructor and/or TA)
Enjoy this course, and study hard!
![Page 31: Large Scale Data Analysis Using Deep Learningukang/courses/17S-DL/L1-intro... · 2018-12-28 · Large Scale Data Analysis Using Deep Learning Course Introduction ... AlphaGo [Silver](https://reader036.vdocuments.us/reader036/viewer/2022081607/5ecdcafcde3f0e12940a0c28/html5/thumbnails/31.jpg)
U Kang 31
Questions?