applications in computer vision machine learning for · 2016. 12. 2. · flownet: learning optical...

Computer Vision Group Prof. Daniel CremersComputer Vision Group Prof. Daniel Cremers

Machine Learning for Applications in Computer Vision

Neural Networks andDeep Learning

Philip Häusser

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Recap: What is Learning?• problem (“classification”)

• data (images + labels)• model (linear regression)• cost function• optimization algorithm

2

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Recap: What is Learning?• problem (“classification”)

• data (images + labels)• model (linear regression)• cost function• optimization algorithm

3

What is “Deep”?

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Recap: What is Learning?• problem (“classification”)

• data (images + labels)• model (linear regression)• cost function• optimization algorithm

4

What is “Deep”?

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Perceptron• The human brain (1010 cells) is the archetype of neural networks

http://home.agh.edu.pl/~vlsi/AI/intro/

5

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Neuron Activations• Different type of activation functions

Threshold activation (binary classifier)

6

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Neuron Activations• Different type of activation functions

Sigmoid activation

7

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Neuron Activations• Different type of activation functions

rectified Linear Unit activation

8

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Simple Neural Network

• good reference: neuralnetworksanddeeplearning.com

9

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Simple Neural Network

Why is it good?

• inspired by brain• highly non-linear → function approximator• shown to solve complicated tasks

successfully

• “self-organizing map” / parameters are learned

• data driven

10

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Simple Neural Network

Why is it bad?

• inspired by brain• highly non-linear• high degree of freedom• shown to solve complicated tasks

successfully

• “self-organizing map”→ parameters are learned

• data driven• not much theory yet

11

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Back Propagation• Define a cost function

• Goal: Find global minimum

12

predictionground truth

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Back Propagation

13

Gradient Descent

predictionground truth

• Define a cost function

• Goal: Find global minimum

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Back Propagation

14

Learning rate

predictionground truth

• Define a cost function

• Minimize the error (derivative of the cost) with gradient descentGradient descent update rule:

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Back Propagation• Define a cost function

• Propagate the error through layers•Take the derivative of the output of the layer w.r.t the input of the

layer•Apply update rule for the parameters

15

predictionground truth

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

MNIST Digit Classification with NN

16

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Convolutional Neural Networks

17

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Convolutional Neural Networks

18

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Convolutional Neural Networks

19

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Convolutional Neural Networks

20

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Convolutional Neural Networks

21

AlexNetKrizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton."Imagenet classification with deep convolutional neural networks.”Advances in neural information processing systems. 2012.

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Convolutional Neural Networks

22

Convolutional networks take advantage of the properties of natural signals:

• local connections • shared weights

• pooling • the use of many layers

Person

FlowNet: Learning Optical Flowwith Convolutional Networks

Philipp Fischer, Alexey Dosovitskiy, Eddy Ilg, Thomas BroxPhilip Häusser, Caner Hazırbaş, Vladimir Golkov, Daniel Cremers, Patrick van der Smagt

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Flying Chairs

24

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

Data Augmentation

25

Gen

erat

edA

ugm

ente

d

• translation, rotation, scaling, additive Gaussian noise• changes in brightness, contrast, gamma and colour

25

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

FlowNetSimple

26

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

FlowNetSimple - Flying Chairs

27

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

FlowNetSimple - Sintel

28

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

FlowNetSimple + Variational Smoothing

29

Philip HaeusserComputer Vision Group

Machine Learning for Computer Vision

FlowNet: Learning Optical Flowwith Convolutional Networks

30