associative learning. simple associative network
Post on 15-Jan-2016
251 views
TRANSCRIPT
Associative Learning
Simple Associative Network
Banana Associator
Unsupervised Hebb Rule
Banana Recognition Example
Example
Problems with Hebb Rule
• Weights can become arbitrarily large
• There is no mechanism for weights to decrease
Hebb Rule with Decay
Example: Banana Associator
Example
Problem of Hebb with Decay
Instar (Recognition Network)
Instar Operation
Vector Recognition
Instar Rule
Graphical Representation
Example
Training
Further Training
Kohonen Rule
Outstar (Recall Network)
Outstar Operation
Outstar Rule
Example - Pineapple Recall
Definitions
Iteration 1
Convergence
Boltzmann Learning• Stochastic learning process with a recurrent structure• State of a neuron is +1 or –1 and some neurons are free (adaptive state)
and others are clamped (frozen state)• Boltzmann machine is characterized by an energy function
• Free neurons change state with probability:
• The learning rule is given by:
Where kjis the correlation with neurons in clamped states and
kj is the correlation with the neurons in a frozen state
j jk
jkkj xxwE 21
)/exp(1
1)(
TExxP
kkk
kjppw kjkjkj
Hidden
Z-1
Z-1
Z-1
Z-1
Delay
Visible
Clamped