ml with tensorflow lab - github pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · fei-fei li...
TRANSCRIPT
![Page 2: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/2.jpg)
https://github.com/nlintz/TensorFlow-Tutorials
![Page 3: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/3.jpg)
CNN
http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg
![Page 4: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/4.jpg)
MNIST 28x28x1 image
32 filters (3x3x1)
Convolutional layers
![Page 5: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/5.jpg)
28x28x1 image
32 filters (3x3x1)w=tf.Variable(tf.random_normal([3,3,1,32], stddev=0.01))
Convolutional layers
![Page 6: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/6.jpg)
28x28x1 image
32 filters (3x3x1)w=tf.Variable(tf.random_normal([3,3,1,32], stddev=0.01))
Convolutional layers
![Page 7: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/7.jpg)
28x28x1 image
32 filters (3x3x1)activation maps
(?, ?, 32)
Convolutional layers
![Page 8: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/8.jpg)
ReLU
![Page 9: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/9.jpg)
Pooling layer (sampling)
conv layer
l1 = tf.nn.max_pool(c1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
![Page 10: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/10.jpg)
Lecture 7 - 27 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 - 27 Jan 201655
1 1 2 4
5 6 7 8
3 2 1 0
1 2 3 4
Single depth slice
x
y
max pool with 2x2 filters and stride 2 6 8
3 4
MAX POOLING
![Page 11: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/11.jpg)
Pooling layer (sampling)
conv layer
l1 = tf.nn.max_pool(c1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
28x28x32
? x ? x 32
![Page 12: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/12.jpg)
Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)
Shape not sure? Print tensor
Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)
![Page 13: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/13.jpg)
X = trX.reshape(-1, 28, 28, 1)
![Page 14: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/14.jpg)
X = trX.reshape(-1, 28, 28, 1)
dropout
![Page 15: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/15.jpg)
CNN
http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg
![Page 16: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/16.jpg)
Fully connected net
![Page 17: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/17.jpg)
cost and optimization
https://www.tensorflow.org/versions/r0.8/api_docs/python/train.html#RMSPropOptimizer
![Page 18: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/18.jpg)
Other TF optimizers
https://www.tensorflow.org/versions/r0.8/api_docs/python/train.html#RMSPropOptimizer
![Page 19: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/19.jpg)
Train and testing
![Page 20: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/20.jpg)
Train and testing 0 128
128 256 256 384 384 512
![Page 21: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/21.jpg)
Train and testing
![Page 22: ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab11.pdf · 2017. 10. 2. · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -55 27 Jan 2016 1 1 2 4 5 6 7 8 3 2](https://reader036.vdocuments.us/reader036/viewer/2022062609/60ff4ffa7769ae3e4502baeb/html5/thumbnails/22.jpg)
https://github.com/nlintz/TensorFlow-Tutorials