application of neural network-function approaximation
DESCRIPTION
Application of neural network-FUNCTION APPROAXIMATIONTRANSCRIPT
FUNCTION FUNCTION APPROXIMATIONAPPROXIMATIONSarbjeet SinghNITTTR chandigarh
CONTENTCONTENT
Learning ParadigmsLearning Paradigms
• Training data: A sample from the data source with the correct Training data: A sample from the data source with the correct classification /regression solution already assigned.classification /regression solution already assigned.
• Two Types of Learning-Two Types of Learning-– SUPERVISEDSUPERVISED– UNSUPERVISEDUNSUPERVISED
• Supervised learning Supervised learning = Learning based on training data.= Learning based on training data.• Two steps:Two steps:• 1. Training step: Learn classifier /regressor from training data.1. Training step: Learn classifier /regressor from training data.• 2. Prediction step: Assign class labels/functional values to test data.2. Prediction step: Assign class labels/functional values to test data.• Example:- Perceptron, LDA, SVMs, linear/ridge/kernel ridge regression Example:- Perceptron, LDA, SVMs, linear/ridge/kernel ridge regression
are all supervised methods.are all supervised methods.
Learning Paradigms Contd..Learning Paradigms Contd..
• Unsupervised learningUnsupervised learning: Learning without training data.
• Examples:
• Data clustering. (Some authors do not distinguish between clustering and unsupervised learning.)
• Dimension reduction techniques.• Data clustering: Divide input data into groups of similar points.• → Roughly the unsupervised counterpart to classification.• Note the difference:
• Supervised case: Fit model to each class of training points, then use models to classify test points.
• Clustering: Simultaneous inference of group structure and model.
Learning TasksLearning Tasks
• There are Six learning There are Six learning – Pattern AssociationPattern Association
– Pattern RecognitionPattern Recognition
– Function ApproximationFunction Approximation
– ControllingControlling
– FilteringFiltering
– Beam formingBeam forming
Function ApproximationConsider a non linear input – output mapping
described by the functional relationship
where
Vector x is input.
Vector d is output.
The vector valued function f(.) is assumed to be unknown.
xfd
Function Approximation
To get the knowledge about the function f(.), some set of examples are taken,
A neural network is designed to approximate the unknown function in Euclidean sense over all inputs, given by the equation
Niii dx 1,
xfxF
WhereΕ is a small positive number.Size N of training sample is large enough
and network is equipped with an adequate number of free parameters,
Thus approximation error ε can be reduced.
The approximation problem discussed here would be example of supervised learning.
Function Approximation
UNKNOWUNKNOWN SYSTEMN SYSTEM
ΣΣix
id
iy
ie
Input Input VectorVector
SYSTEM SYSTEM IDENTIFICATIONIDENTIFICATIONBLOCK DIAGRAMBLOCK DIAGRAM
NEURAL NEURAL NETWORK NETWORK
MODELMODEL
System Identification
• Let input-output relation of unknown memoryless MIMO system i.e. time invariant system is
• Set of examples are used to train a neural network as a model of the system.
WhereVector denote the actual output of the
neural network.
xfd
Niii dx 1,
iy
System Identification
denotes the input vector. denotes the desired response. denotes the error signal i.e. the difference between and .
This error is used to adjust the free parameters of the network to minimize the squared difference between the outputsof the unknown system and neural network in a statistical sense and computed over entire training samples.
ix
id
ie
iyid
INVERSE INVERSE MODELINGMODELING
UNKNOUNKNOWN WN
SYSTEMSYSTEMf(.)f(.)
ΣΣInput Input VectorVector
ixixid
iy
ie
SysteSystem m OutputOutput
Model Model OutputOutput
ErrorError
BLOCK DIAGRAMBLOCK DIAGRAM
INVERSINVERSE E
MODELMODEL
Inverse Modeling
• In this we construct an inverse model that produces the vector x in response to the vector d.
• This can be given by the eqution :
Where
denote inverse of .Again with the use of stated examples neural network approximation of is constructed.
dfx 1
1f f
1f
• Here is used as input and as desired response.
• is the error signal between and produced in response to .
• This error is used to adjust the free parameters of the network to minimize the squared difference between the outputs of the unknown system and neural network in a statistical sense and computed over entire training samples.
Inverse Modeling
id ix
ie ix iyid