what is pattern recognition (lecture 6 of 6)

ERI SUMMER TRAININGCOMPUTERS & SYSTEMS DEPT.

Dr. Randa ElanwarLecture 6

Contents

� Network implementation on MATLAB

� Network type creation

� Input definition

� Parameters tuning

2

ERI Summer training (C&S) Dr. Randa Elanwar

� Parameters tuning

� Activation functions

� Network training

� Network testing (simulation)

Network implementation on MATLAB3

� To simulate a NN on MATLAB "neural networks toolbox“ you have to first make three main choices (1) the network type and topology, (2) the activation function type, and the (3) training mode.

� The main steps to simulate a NN on matlab are:1. Creating the network (of your defined type, structure and

parameters)

2. Training the network by the training pairs you have (<input, output>)

3. Testing the network


Important Note4

� Each of the three steps is done via a set of Matlabinstructions that update (name, arguments) from version to version.

� So I will focus on the instruction target rather than the instruction syntax.


Network type creation5

� Step 1: Network creation

� In this step you have to decide:

1. the network type (feed forward, radial basis, etc.),

2. the activation function,2. the activation function,

3. the internal structure (the number of nodes in the input layer and the number of nodes into the output layer),

4. the input stream type (serial, concurrent), and

5. parameters values (weights, bias, delay).



� And before creation you have to settle upon the data sets (input vectors and their corresponding output values) for training and the test data sets (input vectors only).

� Neural network types creation� Neural network types creation

� Matlab offers creation of a variety of neural networks types: Perceptrons, Feed-forward neural network, Recurrent neural network, Probabilistic neural network, Radial basis neural networks, Self-organizing, Time-delay neural network, etc.

� Each has a creation function with a set of input arguments to define its structure: inputs, number of nodes, number of layers, etc.



� All what you need is to call the function and specify values for these main arguments in the required data structure format (scalar, vectors, matrices, etc.)

� Neural network creation functions� Neural network creation functions

� The names might change with newer Matlab versions so this screen shot is just to illustrate the capabilities of Matlab to simulate the different neural networks types. The creation functions can be found in the "Neural networks toolbox", where you can click on the required function name (hyperlink) to navigate to the full description with examples.


Input definition9

� Not all networks take input data vectors of length n-by-1, some take data as single input with or without delay between input instances. For example:

� Concurrent:� Concurrent:

net = newlin([1 3;1 3],1);

� This command creates a new custom network with linear transfer function and specifies the range of neuron input p1 as [1 3] and input p2 as [1 3] also, i.e., two inputs and a single output ‘1’.


Input definition10

� Suppose that the network simulation data set consists four concurrent vectors

� Concurrent vectors are presented to the � Concurrent vectors are presented to the network as a single matrix:

P = [1 2 2 3; 2 1 3 1];

� We can now simulate the network:

A = sim(net,P)

A = 5 4 8 5


Input definition11

� Sequential:

�When a network contains delays, the input to the network would normally be a sequence of input vectors that occur in a certain time order.

� The following commands create this network:

net = newlin([-1 1],1,[0 1]);

� This command limits the input value from -1 to 1 with 1 output and a delay that is limited from 0 to 1.


Input definition12

� Suppose that the input sequence is p1 = [1], p2 = [2], p3 = [3] and p4 = [4]. Sequential inputs are presented to the network as elements of a cell array:

P = {1 2 3 4};

� We can now simulate the network:

A = sim(net,P)

A = [1] [4] [7] [10]


Input definition13

� Important note: usually you have to specify the upper and lower limit expected for the values of input elements.

� Some training algorithms generally works best when the network inputs and targets are scaled so that they fall approximately in the range [-1,1].approximately in the range [-1,1].

� But! If your inputs and targets do not fall in this range, you can use the functions "premnmx", or "mapstd", to perform the scaling and processes input and target data by mapping its mean and standard deviations to 0 and 1 respectively. Then use "poststd" to post-process data which has been pre-processed by "mapstd". It converts the data back into unnormalized units.


Parameters tuning14

� Usually the network is created with default values for its parameters, but you can change this either by resetting then assigning new values or direct assignment of new values.

� Example:

� Adjusting weights and bias for a concurrent input network

net = newlin([1 3;1 3],1);net = newlin([1 3;1 3],1);

net.IW{1,1} = [1 2];

net.b{1} = 0;

� Adjusting weights and bias for a sequential input network

net = newlin([-1 1],1,[0 1]);

net.biasConnect = 0;

net.IW{1,1} = [1 2];


Parameters tuning15

� In this example we only have the weights of the input layer but if the network has multiple layers, then a certain notation should be used to the assign the weights of the other layers:


Activation functions16

� There is a variety of activation functions that you can chose from for your network:



� Activation function name argument in Matlab

� These activation functions names are set as an input argument to the network creation function

� Example:

net = newelm([0 1],[3 2],{'tansig','purelin'});

� All you need is to read more about the recommended uses of each network type to design the classifier topology and internal structure you want to implement.


Network training19

� Matlab offers you a variety of learning rules and methods to train the network you designed.

� For example you can use "train" function and set the � For example you can use "train" function and set the parameters of (training mode, learning rate, number of epochs and the error limit), or use standard training function like "trainb" for example for batch training mode.


Network training20


Network training21

� Here newff is used to create a two-layer feed-forward network. The network has one hidden layer with ten neurons.

net = feedforwardnet(10);

net = configure(net,p,t);

y1 = sim(net,p)y1 = sim(net,p)

� The network is trained for up to 300 epochs to an error goal of 0.1 and then re-simulated.

net.trainParam.epochs = 300;

net.trainParam.goal = 0.1;

net.trainFcn = 'trainb';

net = train(net,p,t);

y2 = sim(net,p)ERI Summer training (C&S) Dr. Randa Elanwar

Network training22

� There is a nice option to know when had the training converged, if you set the parameter "show" before you call the training function

net.trainParam.show = 25;

In such case the error value will appear on your work space � In such case the error value will appear on your work space every "25" iterations like this:

TRAINB, Epoch 0/100, MSE 0.5/0.1.

TRAINB, Epoch 25/100, MSE 0.181122/0.1.

TRAINB, Epoch 50/100, MSE 0.111233/0.1.

TRAINB, Epoch 64/100, MSE 0.0999066/0.1.

TRAINB, Performance goal met.


Network training23

� It's important also to note that some training functions are designed to have different (adaptive) values for the learning rate to help faster convergence like "traingda"


Network testing (simulation)24

� After training the network you can test the performance on a test set, simply you can call the "sim" function giving your trained network and the test samples as input arguments.

� This is the fun part, Enjoy it


what is pattern recognition (lecture 6 of 6)

Science

probabilistic neural

recurrent neural network

network type feed

networkeri summer training

training mode

training pairs

activation function

matlab neural networks