introduction to neural networks - in.tum.de · introduction to neural networks freek stulp. 2...
TRANSCRIPT
Introduction to Neural Networks
Freek Stulp
2
Overview
Biological BackgroundArtificial NeuronClasses of Neural Networks
1. Perceptrons2. Multi-Layered Feed-Forward Networks3. Recurrent Networks
Conclusion
3
Biological BackgroundNeuron consists of:? Cell body? Dendrites? Axon? Synapses
Neural activation :? Throught dendrites/axon? Synapses have different
strengths
4
Artificial Neuron
aj Wji
Input links(dendrites)
Unit(cell body)
Output links(axon)
ai ai =
g(ini)ini =
? ajWji
5
Class I: Perceptron
a = g(in)
in = ? ajWj
a = g(-W0 + W1a1 + W2a2) g(in) = 0, in<01, in>0{
a1
a2
-1
Ij
a
O
W0
W1
W2
Wj
6
7
Learning in PerceptronsError often defined as:
E(W) = 1/2? d? D(td-od)2
Go towards the minimum error!
Update rules:? wi = wi ??? wi
? ? wi = -? ?E/?wi
? ?E/?wi = ?/?wi 1/2? d? D(td-od)2
= ? d? D(td-od)iid
This is called gradient descent
i
8
Class II: Multi-layer Feed-forward Networks
Feed-forward:? Output links only connected
to input links in the next layer
Input Hidden OutputMultiple layers:? hidden layer(s)
Complex non-linear functions can be represented
9
Learning in MLFF NetworksFor output layer, weight updating similar to perceptrons.Problem: What are the errors in the hidden layer?Backpropagation Algorithm? For each hidden layer (from output to input):
?For each unit in the layer determine how much it contributed to the errors in the previous layer.
?Adapt the weight according to this contribution
This is also gradient descent
10
Class III: Recurrent Networks
Input Hidden OutputNo restrictions on connections
Behaviour more difficult to predict/ understand
11
Conclusion
Inspiration from biology, though artificial brainsare still very far away.
Perceptrons too simple for most problems.
MLFF Networks good as function approximators.? Many of your articles use these networks!
Recurrent networks complex but useful too.
12
Literature
Artificial Intelligence: A Modern Approach? Stuart Russel and Peter Norvig
Machine Learning? Tom M. Mitchell