deep learning in a nutshell
TRANSCRIPT
Deep Learning in a nutshell
JohnsonChen @ HB TechFriday
Agenda
1. Introduction to Deep Learning
2. Training You Own Deep Learning Model
a. Introduction to Tensorflow
What’s deep learning
1. Deep learning is part of a broader family of machine learning methods based on learning representations of data.
2. Deep learning can be applied to binary classification problem ( ), multiclass classification problem ( ).
3. Can we make machine learn by itself?
https://en.wikipedia.org/wiki/Deep_learning#Definitions
https://www.youtube.com/watch?v=4A14mOsT6vQ
Let’s talk about classification
http://www.louisdorard.com/blog/when-machine-learning-fails
Let’s talk about classification
( X1, X2, … )
Model
Build our own model
( X1, X2 )
(1, 1) +
(1, 0) +
(0, 1) +
(0, 0) -
(1, 1)
(1, 0)
(0, 1)
(0, 0)
This is our model
(1, 1)
(1, 0)
(0, 1)
(0, 0)
(0.3, 0.3)
What if ...
(1, 1) -
(1, 0) +
(0, 1) +
(0, 0) -
(1, 1)
(1, 0)
(0, 1)
(0, 0)
To make it “non-linear”
Activation Function
(relu: rectified linear unit)
“non-linear”
So what is deep learning?
Artificial Neural Network (ANN)
http://cs231n.github.io/neural-networks-1/
http://hyinfo.bse.ntu.edu.tw/labweb/ann.htm
Deep Learning (Deep Neural Network, DNN)
Deep = many hidden layers
6 ~ 8 hidden layers
http://markus.com/deep-learning-101/
https://classroom.udacity.com/courses/ud730/
https://classroom.udacity.com/courses/ud730/
ImageNet training set
http://www.slideshare.net/AmazonWebServices/cmp305-deep-learning-on-aws-made-easycmp305
BIG DATA
BIG MODEL
How to train model?
(0, 1)
(0, 0)
(1, 1)
(1, 0)
Let’s try handwriting recognition
Most of slides are borrowed from Hung-yi Lee deep learning talk.
Handwriting recognition
https://www.tensorflow.org/versions/r0.8/tutorials/mnist/beginners/index.html#the-mnist-data
784
28 * 28 = 784
Probability
Answer
Agenda
1. Introduction to Deep Learning
2. Training you own Deep Learning model
a. use Tensorflow
Most of slides are borrowed from Dr. Chung-Cheng Chiu deep learning talk
TensorFlowDeveloped by Google Brain Team
Initial release: November 9, 2015
Used for both Google production and research.
Production: 50 different teams in dozens of commercial Google products, such as Google Voice, Gmail, Google Photos, and Search, etc
Feature:
Python, C/C++ API
support multiple CPUs and GPUs
support mobile computing platforms, including Android and Apple's iOS
TensorFlow = Tensor + Flow
Tensor: n-dimensional arrays
Flow: A sequence of tensor operations
Deep Learning is suitable for TensorFlow, but TensorFlow can do more
Tensor
They are all tensor
Implement it
784 300 100 10
w1 w2 w3
b1 b2 b3
28 * 28 = 784
0 0 0 0 0 1 0 0 0 0
Implement it
Traning Data
Implement it
1. Define variables
2. Create (deep) neural network
3. Define cost function, optimizer
4. Train model parameter
Python code
Define variables
784 300 100 10
x y
w1 w2 w3
b1 b2 b3
Create (deep) neural network
784 300 100 10
x y
w1 w2 w3
b1 b2 b3
Define cost function, optimizer
Train model parameter
Save your model
# defaults to saving all variables - in this case w and b saver = tf.train.Saver()
Code Demo
https://github.com/ChinHui-Chen/tensorflow-nn-example/blob/master/multilayer_perceptron_example_novar.py
More examples
https://www.theinformation.com/How-Deep-Learning-Works-at-Apple-Beyond
AlphaGo: Deep Learning vs
AlphaGo Google DeepMind 3 9 3 15 AlphaGo
5
12 1. 3 9 ( ) 2. 3 10 ( ) 3. 3 12 ( ) 4. 3 13 ( ) 5. 3 15 ( )
Read: Google
Conclusion
Neural network, Deep learning
Create a framework to let machine learn by itself
Sometimes it is too complex to debug
TensorFlow tool
Try to train you own model.
Define input, out, and run/train it
http://www.techinsider.io/what-is-google-tensorflow-2015-11
Thanks