unit ii

1UNIT-2

NEURAL NETWORKS

FOR

CONTROL

www.Vidyarthiplus.com

Hopfield-Tank model

TSP must be mapped, in some way, onto the neural network

structure

Each row corresponds to a particular city and each column to

a particular position in the tour


Mapping TSP to Hopfield neural net

There is a connection between each pair of units

The signal sent along a connection from i to t j is equal to the weight Tij if i is activated. It is equal to 0 otherwise.

A negative weight defines inhibitory connection between the two units

It is unlikely that two units with negative weigh will be active or on at the same time


Discrete Hopfield Model

connection weights are not learned

Hopfield network evolves by updating the activation of each unit in turn

In final state, all units are stable according to the update rule

The units are updated at random, one unit at a time

{Vi}i=1,...,L,

L :number of units

Vi :activation level of unit i

Tij: connection weight between units i and j

tetai: threshold of unit i.


Discrete Hopfield Model (Cont.)

Energy function

Units changes its activation level if and only if the energy of the network decreases by doing so:

Since the energy can only decrease over time and the number configuration is finite

the network must converge (but not necessarily the minimum energy state)


Continuous Hopfield-Tank

Neuron function is continuous (Sigmoid function)

The evolution of the units over time is now characterized by the following differential equation :

Ui, Ii and Vi are the input, input bias, and activation level of unit I, respectively


Continuous Hopfield-Tank

Energy function

Discrete time approximation is applied to the

equations of motion


Application of the Hopfield-Tank

Model to the TSP



model to the TSP

(1)The TSP is represented as an N*N matrix

(2) Energy function

(3)Bias and connection weights are derived



model to the TSP


Results of Hopfield-Tank

Hopfield and Tank were able to solve a

randomly generated 10-city,with parameter

value :A=B=500,C=200,N=15.

They reported for 20 trails, network converge

16 times to feasible tours.

Half of those tours were one of two optimal

tours


The size of each black square indicates the value of the output of the corresponding neuron


The main weaknesses of the original

Hopfield-Tank model



Hopfield-Tank model

(d) Model plagued with the limitation of hill-

climbing approaches

(e) Model does not guarantee feasibility



Hopfield-Tank model

The positive points:

Can easily implemented in hardware

Can be applied to non-Euclidean TSPs


PROCESS IDENTIFICATION

Pattern Classification Network

5 output nodes

16 output nodes

100 input nodes


Character RecognitionLearning Curve for Training the Pattern Classification Network

The chart depicts the learning curve, as the error tolerance was successively lowered

through the values 0.01, 0.005, 0.0025, 0.001, 0.0005, 0.00025, and finally 0.0001.

The vertical axis shows the cumulative iterations needed to achieve the perfect 5 out of

5 training performance at each level of error tolerance.

The Learning Curve for Training the Pattern Classification Network

0

200

400

600

800

1000

1200

00.0010.0020.0030.0040.0050.0060.0070.0080.0090.01Error Tolerance

C

u

m

u

l

a

t

i

v

e

I

t

e

r

a

t

i

o

n

s


Training the Network

Tips for Training

Start with a relatively large error tolerance, and incrementally lower it to the desired level

as training is achieved at each succeeding level. This usually results in fewer training

iterations than starting out with the desired final error tolerance.

Training Tip #1

If the network fails to train at a certain error tolerance, try successively lowering the

learning rate.

Training Tip #2



Tips for Training

In a system to be used for a real-world application, such as character recognition, you

would want the network to be able to handle not only pixel noise, but also size variance

(slightly smaller letters), some rotation, as well as some variations in font style.

Training Tip #3

To produce such a robust network classifier, you would need to add representative

samples to the training set. For example, samples of rotated versions of a character could

be added to the training set, etc.


Weight Change = learning rate * input * error output +

momentum_parameter * previous_weight_change

Training a NetworkThe Momentum Term

Warning!Warning! Setting the momentum term and learning rate too large can overshoot a good

minimum as it takes large steps!

Smooth out the effect of weight adjustments over time.

Momentum term can be disabled by setting it to zero.

Formula:


Using Neural Networks in a Control Problem

Inverted Pendulum Problem

Input: x, v, theta, angular velocity Output: Angle

1 output node

20 output nodes

5 input nodes



Inverted Pendulum Problem (Dynamics of the Problem)(Dynamics of the Problem)

See FFBRM.EXE

Formulas:

Input: x, v, theta, angular velocity

Output: Angle

))((cos)3/4())](sin())('()())[(cos())(sin()('' 2

2

tlmmlttlmtFttmg

tb

b

+=

mttttlmtFtx b /))]}(cos()('))(sin())('()({)('' 2 +=

)(t)(' t= broom angle(with vertical) at time t (in radians)

= angular velocity

x(t) = cart position at time t ( in meters)

x(t) = cart velecity

F(t) = force applied to cart at time t (Newtons)

m = combined mass of cart and broom (1.1 kg)

mb = mass of broom (0.1 kg)

l = length of broom (pivot point to center of mass, 0.5 meters)



The equations weve seen ignore the effects of

friction. Control system failure occurs when the

cart hits either end of the track (at x= 2.4 meters),

or the angle reaches radians (180 degrees)

For practical purposes, though, unrecoverable

failure occurs when the angle reaches 12degrees in magnitude so the Neural Network

training will be restricted to this range.

State of the Cart-Broom System:x, x, theta , angular velocity -dot

Inverted Pendulum Problem (Limits of Network Training)(Limits of Network Training)



Eulers method has only its simplicity to recommend it, and is

normally not used when any amount of accuracy is required of

a numerical solution. However, it is adequate for our needs.

State of the Cart-Broom System:x, x, theta , angular velocity -dot

SIMULATION ISSUES

)(')()( txhtxhtx +=+

Eulers Method

Eulers method relates a variable and its derivative via the

simple approximation equation:

How can we simulate the complete behaviour of the system now that we know how to derive

all the state variables describing the system?

What is the cart-broom state given any time t?



SIMULATION ISSUES

Eulers Method

Heres an appplication of Eulers method

float f_theta (float frce,float th1,float th2)

{

float denom,numer,cost,sint;

cost = cos (th1);

sint = sin (th1);

denom = four_thirds * m * l - mb * l * cost * cost; /* Always > 0 */

numer = m * g * sint -

cost * (frce + mb * l * th2 * th2 * sint);

return numer/denom;

}

float f_x (float frce,float th1,float th2,float th3)

{

float cost,sint,term;

cost = cos (th1);

sint = sin (th1);

term = mb * l * (th2 * th2 * sint - th3 * cost);

return (frce + term)/m;

}

))((cos)3/4())](sin())('()())[(cos())(sin()('' 2

2

tlmmlttlmtFttmg

tb

b

+=

mttttlmtFtx b /))]}(cos()('))(sin())('()({)('' 2 +=




SIMULATION ISSUES

Eulers Method

Heres an appplication of Eulers method


void new_broom_state (float frce,state_rec old_state,state_rec

*new_state)

{

const float h = 0.02;

float th3;

/* Euler's method applied to system of equations */

/* (not known for accuracy, but good enough here !) */

new_state->theta = old_state.theta + h * old_state.theta_dot;

th3 = f_theta (frce,old_state.theta,old_state.theta_dot);

new_state->theta_dot = old_state.theta_dot + h * th3;

new_state->x_pos = old_state.x_pos + h * old_state.x_dot;

new_state->x_dot = old_state.x_dot +

h * f_x (frce,old_state.theta,old_state.theta_dot,th3);

return;

}



Feed-Forward Controller System

Analogy to a Human Controller

Suppose that a human controller has no idea how the cart-broom system is going to respond to

input forces. By randomly applying a force and observing whether or not that force helps in

balancing the broom, eventually, the controller may notice that pushing the cart in the same

direction that the broom is leaning will slide the cart back under the broom and tends to restore

a vertical angle.

With enough repetitions, the controller learns how much force to supply and how often. An

expert controller can anticipate which way the broom is going to fall and can apply a corrective

force before the broom goes far in that direction.

How can we teach the Network to learn broom balancing?



Feed-Forward Controller System


The approach in teaching the network is similar to the process of teaching a human controller.

The networks will learn the dynamics of the cart-broom system by observing numerous

repetitions of random-force applications at different broom states.

The trained network will then be a model of the cart-broom dynamics. Like the human expert,

the network can predict the next broom statepredict the next broom state, and with this knowledge the correct force can be the correct force can be

appliedapplied.

Run the cart-broom system through 100 random force

applications and collect max-min data.

Collection of Training Data

All input data should be normalized. Each parameter will

have its own [min, max] values.


Feed-Forward Controller System (BANG-BANG Control)

Running the Network

The controller operates by looking ahead, like an experienced controller would. The trained

network emulates the broom dynamics, so that the controller can ask: What will happen if I do

nothing (zero force)?

The trained network answers this question by supplying the broom angle that would result on

the next iteration if zero force were applied. Once the angle is predicted, the appropriate

action can be taken.




Alternative Solution to the Inverted Pendulum Problem

See FFBRM.EXE

Input: x, v, theta, angular velocity Output: Force, direction

2 output nodes

16 (4+4)

hidden nodes

4 input nodes

Noise factor could be added during training (e.g. 0.05) to make the

network more robust.


The following could be done after collecting the raw data for Neural Net

training.

1. Calculate max, min values of each input parameter.

[min, max] for each variable (x, x-dot, theta, theta-dot)


Network Inputs


The following could be done after collecting the raw data for Neural Net

training.

1. Calculate max, min values of each input parameter.

[min, max] for each variable (x, x-dot, theta, theta-dot)


Network Inputs

2. Calculate xFactor.

3. see pdf file for more details on this.


unit ii

Documents

tank hopfield

hopfield network

comresults of hopfield

hopfield neural netthere

comhopfieldtank model

units i

energy function units

number of units vi