programming in matlab - burapha universityureerat/321641/...programming in matlab 2.2...
TRANSCRIPT
![Page 1: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/1.jpg)
Gp.Capt.ThanapantGp.Capt.Thanapant RaicharoenRaicharoen, PhD, PhD
Programming in MATLABProgramming in MATLAB
Chapter 2: Artificial Neural Network Chapter 2: Artificial Neural Network ((PerceptronPerceptron))
![Page 2: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/2.jpg)
2.2 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
OutlineOutline
n Introduction to Artificial Neural Network
n Machine Learning and Learning Rule
n Perceptron Learning Rule
n Perceptron program in MATLAB
n Perceptron MATLAB Toolbox
![Page 3: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/3.jpg)
2.3 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Human vs. ComputerHuman vs. Computer
n SpeedØ Human: Neuron Cell Signal 1000 times/SecondØ Computer: CPU 1 GHz
n (mathematical) ProcessingØ Human: not goodØ Computer: > 1000 MFLOP
n Problem Analysis Ø Human: goodØ Computer: not good
n (pattern) RecognitionØ Human: goodØ Computer: not good
![Page 4: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/4.jpg)
2.4 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Cell body (soma)
Dendrite (Accept Input)
Nucleus
Neuron(Turn the processed inputs into outputs) Axon
Myelin sheath
Synapse
Human BrainHuman Brain
![Page 5: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/5.jpg)
2.5 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
(Artificial) Neural Networks(Artificial) Neural Networks
n What is a Neural Network?
Ø Biologically motivated approach to machine learning
Ø Similarity with biological network
n Human brainØ interconnected network of 1011
neurones (100,000,000,000)Ø each connected to 104 others
Ø surprisingly complex decisions, surprisingly quickly (10-1s)
Ø robust & fault tolerant, flexible
Ø deals with fuzzy, noisy, or inconsistent information
![Page 6: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/6.jpg)
2.6 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
A biological neurone showing its cell body and connections
A non-linear model for a neurone
(Artificial) Neural Networks(Artificial) Neural Networks
![Page 7: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/7.jpg)
2.7 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Biological and Artificial Neural NetworkBiological and Artificial Neural Network
n Biological NN
Ø Cell Body
Ø Dendrite
Ø Soma
Ø Axon
n Artificial NN
Ø Neurons
Ø Weights or interconnections
Ø Net Input
Ø Output
![Page 8: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/8.jpg)
2.8 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
1
( )N
i ij j ij
y g w x µ=
= −∑wi1
yi
x4
wi2
wi4
wi3
x1
x2
x3
For each node, the output is given by
wij = Connection weight of branch (i,j)xj = Input data from node j in the input layerµi = Bias or Threshold value of node i in the output layerg = Activation or Transfer function
Inpu
t lay
er
Out
put l
ayer
A Single Layer A Single Layer PerceptronPerceptron NetworkNetwork
![Page 9: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/9.jpg)
2.9 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Input nodesHidden nodes
Output nodes
Connections Weight
∑=
++++=
jj
ji
mmiiiii
xwf
xwxwxwxwfy
)(
)( 33
22
11 L
Output of each node (mathematical equation)
Wij = weight of each connectionXi = input from the other nodes
(Artificial) (Artificial) Neural Network Neural Network
![Page 10: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/10.jpg)
2.10 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Mathematic Foundation for Neural NetworkMathematic Foundation for Neural Network
n Linear Algebra is the core of the mathematics required for understanding neural network
Ø Vectors and Vector Spaces
Ø Inner Product, Norm and Orthogonality
Ø Matrices, Linear Transformations
Ø Eigenvalues and Eigenvectors
Ø Etc.
n Calculus
Ø Differentiation and Integration
n Probability
n Set Theory
![Page 11: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/11.jpg)
2.11 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Machine LearningMachine Learning
n Machine learning is programming computers to optimize a performance criterion using example dataor past experience.
n We need learning in cases where we cannot directly write a computer program to solve a given problem.
n Learning is used when:Ø Human expertise does not exist (navigating on Mars),
Ø Humans are unable to explain their expertise (speech recognition)
Ø Solution changes in time (routing on a computer network)
Ø Solution needs to be adapted to particular cases (user biometrics)
![Page 12: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/12.jpg)
2.12 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Three Types of Learning Three Types of Learning
n Supervised LearningØ Learning (Training) with teacher (Supervisor)Ø There exist input (training) data and right output (response)
target. n Unsupervised LearningØ Learning (Training) without teacher (Supervisor)Ø There exist input (training) data without right output
(response) target. n Reinforcement LearningØ Learning (Training) assumed with teacher (Supervisor)Ø There exist input (training) data and output (response)
target (but do not know right or not).
![Page 13: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/13.jpg)
2.13 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neural Network ModelNeural Network Model
n Supervised Learning Ø PerceptronØ Multi-Layer Perceptron (MLP)
- Feed Forward Network- Back Propagation
Ø Radius Bias Function (RBF)Ø Support Vector Machine (SVM)Ø k-Nearest Neighbour (k-NN)Ø etc.
n Unsupervised Learning Ø ClusteringØ Self Organisation Map (SOM)Ø Principal Component Analysis (PCA)Ø etc.
n Reinforcement Learning
![Page 14: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/14.jpg)
2.14 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neural Network ApplicationsNeural Network Applications
n Pattern Recognitionn Process identification & controln Time series predictionn Forecasting/Market Prediction: finance and bankingn Data Mining and Intelligent Data Analysisn Medicine: analysis of electrocardiogram data, RNA &
DNA sequencing, drug development without animal testing
n Control: process, roboticsn Manufacturing: quality control, fault diagnosisn etc.
![Page 15: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/15.jpg)
2.15 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neuron Model (One Neuron) by MATLABNeuron Model (One Neuron) by MATLAB
n p1, p2 , …, pR = inputs (vector)n w1, w2 ,..., wR = synaptic weightsn b = threshold (Bias)n f = activation (transfer) function: n a = output: a = f(Wp+b) = f(n)
=
![Page 16: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/16.jpg)
2.16 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
activation (transfer) functionsactivation (transfer) functions
a = +1 if n >= 0
a = 0 if n<0
a = n
![Page 17: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/17.jpg)
2.17 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neuron Model in MATLAB (Single/OneNeuron Model in MATLAB (Single/One--Input)Input)
n p1 – inputs (patterns) = 0n w1- synaptic weights = -1n b - threshold (Bias) = 1n f - activation (transfer) function: = purelinen a - output: = 1
![Page 18: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/18.jpg)
2.18 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neuron Model in MATLAB (TwoNeuron Model in MATLAB (Two--Input)Input)
% ex_One_Neural.mp = [1; -1]; % R = 2W = [2 0];b = 1;n = W*p + b;a = hardlim(n);
n p1 , p2, - inputs (patterns) = (1,-1)n w1 , w2 - synaptic weights = (2,0)n b - threshold (Bias) = 1n f - activation (transfer) function:
= Hard Limitn a - output: = 1
![Page 19: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/19.jpg)
2.19 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neuron Model in MATLAB (TwoNeuron Model in MATLAB (Two--Input)Input)
% ex_One_Neural.mp = [1; -1]; % R = 2W = [2 0];b = 1;n = W*p + b;a2 = sigmoid(n);
n p1 , p2, - inputs (patterns) = (1,-1)n w1 , w2 - synaptic weights = (2,0)n b - threshold (Bias) = 1n f - activation (transfer) function:
= Sigmoidn a - output: = 0.9526
![Page 20: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/20.jpg)
2.20 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Neuron Model (S Neuron)Neuron Model (S Neuron)
=
% ex_S_Neural.mp = [1; -1]; W = [2 0; 1 2; 1 3]; b = [1; 2; 3];n = W*p + b;a = hardlim(n);a2 = sigmoid(n);
![Page 21: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/21.jpg)
2.21 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Multiple layer of NeuronMultiple layer of Neuron
=
% ex_Multilayer_Neural2.m% Input vector % R = 2 (Column)p = [1; -1]; % Weight & bias for Layer1, 2, 3W1 = [2 0; 1 2; 1 3]; % Layer1: S=3 Nodes b1 = [1; 2; 3];
W2 = [2 0 1; 1 2 1]; % Layer2: S=2 Nodes b2 = [0; 0];W3 = [2 0]; % Layer3: S=1 Node b3 = [1];a3 = sigmoid(W3*(sigmoid(W2*(sigmoid(W1*p+b1))'+b2))'+b3)
![Page 22: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/22.jpg)
2.22 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Programming in MATLAB ExerciseProgramming in MATLAB Exercise
n Exercise:
1. Write MATLAB to solve the question 8 in Exercise 1.
2. Write MATLAB to solve the question 3 in Exercise 2.
![Page 23: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/23.jpg)
2.23 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
How can we use a mathematical function to classify ?How can we use a mathematical function to classify ?
q Consider a simple problem: How can we classify fat and thin boys?
Decision
line
FatFatThinThin
Weight (kg.)
Height(cm.)
40 80
140
180
![Page 24: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/24.jpg)
2.24 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
qWe used 2 inputs, weight (x1) and height (x2), to classify “thin” and “fat”.
x1 (kg.)
x2(cm.)
40 80
140
180Decis
ion lin
e
FatFatThinThin
Line x 2
- x 1 - 100
=0
Area where x2 - x1 - 100 < 0
Area where x2 - x1 - 100 > 0
Weight-height space
ü We can usea line to classify Data.
How can we use a mathematical function to classify ?How can we use a mathematical function to classify ?
![Page 25: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/25.jpg)
2.25 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
q We can write the decision function for classifying“thin” and “fat” as follows:
2 1
2 1
1 (thin) if - -100 00 (fat) if - -100 0
x xy
x x≥
= <
Advantage: Universal linear classifierProblem: For a particular problem, how can we choosesuitable weights w and bias µ of the function ?
or1 1 2 2
1 2
( )
( 100)
y g w x w x
g x x
µ= + −= − + −
1 if z 0( )
0 otherwise.g z
≥=
where
How can we use a mathematical function to classify ?How can we use a mathematical function to classify ?
![Page 26: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/26.jpg)
2.26 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Fat Children Training SamplesFat Children Training Samples
No. Height (Cm.) Weight (Kg.) Fat/Not Fat (+1/-1)
1. 100 20 -1
2. 100 26 +1
3. 100 30 +1
4. 100 32 +1
5. 102 21 -1
6. 105 22 -1
7. 107 32 +1
8. 110 35 +1
9. 111 25 -1
10. 114 24 -1
11. 116 36 +1
12. 118 27 -1
![Page 27: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/27.jpg)
2.27 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Fat Children Training SamplesFat Children Training Samples
?
![Page 28: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/28.jpg)
2.28 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Output/Target
1. 0 0 0
2. 0 1 0
3. 1 0 0
4. 1 1 1
![Page 29: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/29.jpg)
2.29 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Supervised Learning: Perceptron Networks
q Rosenblatt and his colleague proposed the Perceptronmodelin 1962 by inspiration of neural model from McCulloch-Pitts
in 1943.
wi1wi2wi3
bi
Σ yi
x1
x2
x3
McCulloch-Pitts model
∑ −=
−++=
jijij
iiiii
bxwf
bxwxwxwfy
)(
)( 332211
![Page 30: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/30.jpg)
2.30 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
wi1ywi2
wi3
x1
x2
x3
Input Networkoutput
Σ-
Error
Desired output
+
Adjust weights
Learning Algorithm: Training Perceptron Networks
![Page 31: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/31.jpg)
2.31 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
PerceptronPerceptron Learning RuleLearning Rule
![Page 32: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/32.jpg)
2.32 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
PerceptronPerceptron Learning RuleLearning Rule
function [ W_new, b_new ] = perceptron(P,t,W,b)% Perceptron.m (P,t,W,b);% P = Input vector, t = target, W = Weight, b = bias% Use Hard limit as Activation Function a = hardlim(W*P'+b);% e = error and Perceptron Rule as followe = t-a;W_new = W + e.*P;b_new = b + e;end
![Page 33: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/33.jpg)
2.33 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
PerceptronPerceptron Training AlgorithmTraining Algorithm
n Step 1: Initial weights and bias (can be zero or any).
n Step 2: For each training data (input and target) to be classified do Steps 3-4.
n Step 3: Calculate the response of output unit.
n Step 4: The weights and bias are updated by using this learning rule.
n Step 5: Test for stopping condition. may be the number of loops/iterations (epoc).
![Page 34: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/34.jpg)
2.34 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
x1
x2
(0,0)
(0,1)
(1,0)
y = 0
(1,1) y =1
Incorrect Line Lfor Function AND
How a single layer perceptron works (cont.)
The Slope and Position of the line L : w1x1+w2x2 – b = 0depend on parameters w1, w2, and b. Therefore we have to adjust these parameters in order to get a suitable line L.
x1 x2 y
0 0 0
0 1 0
1 0 0
1 1 1
Function AND
Correct line L forFunction AND
![Page 35: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/35.jpg)
2.35 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
How a single layer perceptron works (cont.)
% Run_AND_Perceptron.mP = [0 0 1 1; 0 1 0 1]; % AND FunctionT = [0 0 0 1];plotpv(P,T,[-1, 2, -1, 2]); % plot data% initial weight vector and biasW = [1 1]; b = 1;plotpc(W,b); % plot lineepoc = 1; % number of epocfor j=1:epoc
for i=1:size(P,2)p = P(:,i);t = T(i);[W b] = perceptron(p',t,W,b);plotpc(W,b);
endend % test data test = [0 0]'test = [0 0];output = hardlim(W*test'+b)
qExample: AND Function
![Page 36: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/36.jpg)
2.36 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Output (n) Actual (a) W1 W2 b
1 1 1
1. 0 0 0
2. 0 1 0
3. 1 0 0
4. 1 1 1
P = [P1 P2]T
= [0 0]T
0 =0 =
= 1= 1
= 1
![Page 37: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/37.jpg)
2.37 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Output (n) Actual (a) W1 W2 b
1 1 1
1. 0 0 0 1 1 1 1 0
2. 0 1 0
3. 1 0 0
4. 1 1 1
P = [P1 P2]T
= [0 1]T
0 =1 =
= 1= 1
= 0
![Page 38: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/38.jpg)
2.38 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Output (n) Actual (a) W1 W2 b
1 1 1
1. 0 0 0 0 1 1 1 0
2. 0 1 0 1 1 1 0 -1
3. 1 0 0
4. 1 1 1
P = [P1 P2]T
= [1 0]T
1 =0 =
= 1= 0
= -1
![Page 39: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/39.jpg)
2.39 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Output (n) Actual (a) W1 W2 b
1 1 1
1. 0 0 0 0 1 1 1 0
2. 0 1 0 1 1 1 0 -1
3. 1 0 0 0 1 0 0 -2
4. 1 1 1
P = [P1 P2]T
= [1 1]T
1 =0 =
= 0= 0
= -2
![Page 40: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/40.jpg)
2.40 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Output (n) Actual (a) W1 W2 b
1 1 1
1. 0 0 0 0 1 1 1 0
2. 0 1 0 1 1 1 0 -1
3. 1 0 0 0 1 0 0 -2
4. 1 1 1 -2 0 1 1 -1
= 1= 1
= -1
![Page 41: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/41.jpg)
2.41 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Actual Output W1 W2 b
1 1 1
1. 0 0 0 1 1 1 0
2. 0 1 0 1 1 0 -1
3. 1 0 0 1 0 0 -2
4. 1 1 1 0 1 1 -1
No. P1 P2 Target Actual Output W1 W2 b
1 1 -1
1. 0 0 0 1 1 1 -1
2. 0 1 0 1 1 0 -2
3. 1 0 0 1 1 0 -2
4. 1 1 1 0 2 1 -1
1. epoch
2. epoch
![Page 42: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/42.jpg)
2.42 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND FunctionAND Function
No. P1 P2 Target Actual Output W1 W2 b
2 1 -1
1. 0 0 0 1 2 1 -1
2. 0 1 0 1 2 0 -2
3. 1 0 0 1 1 0 -3
4. 1 1 1 0 2 1 -2
![Page 43: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/43.jpg)
2.43 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
x1 x2 y
0 0 0
0 1 0
1 0 0
1 1 1
Function AND
w1 = 0.5w2 = 2.5θ (b) = 1.0α = 0.2
Initial Values
-0.2 0 0.2 0.4 0.6 0.8 1 1.2-0.2
0
0.2
0.4
0.6
0.8
1
1.2
epoc: 4
epoc: 8epoc: 12
epoc: 16
Initial
Learning Algorithm: Training Perceptron Networks (cont.)
![Page 44: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/44.jpg)
2.44 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Perceptron Learning Example: AND Gate Bias: Let X0 = 1 , Alpha = 0.5Alpha = 0.5
X0 X1 X2 1.0 * W0 X1* W1 X2* W2 Total target actual Alpha * (t-o) W0 W1 W20.1 0.1 0.1
1 1 0 0 0.10 0.00 0.00 0.10 0 1 -0.50 -0.40 0.10 0.101 0 1 -0.40 0.00 0.10 -0.30 0 0 0.00 -0.40 0.10 0.101 1 0 -0.40 0.10 0.00 -0.30 0 0 0.00 -0.40 0.10 0.101 1 1 -0.40 0.10 0.10 -0.20 1 0 0.50 0.10 0.60 0.60
2 1 0 0 0.10 0.00 0.00 0.10 0 1 -0.50 -0.40 0.60 0.601 0 1 -0.40 0.00 0.60 0.20 0 1 -0.50 -0.90 0.60 0.101 1 0 -0.90 0.60 0.00 -0.30 0 0 0.00 -0.90 0.60 0.101 1 1 -0.90 0.60 0.10 -0.20 1 0 0.50 -0.40 1.10 0.60
3 1 0 0 -0.40 0.00 0.00 -0.40 0 0 0.00 -0.40 1.10 0.601 0 1 -0.40 0.00 0.60 0.20 0 1 -0.50 -0.90 1.10 0.101 1 0 -0.90 1.10 0.00 0.20 0 1 -0.50 -1.40 0.60 0.101 1 1 -1.40 0.60 0.10 -0.70 1 0 0.50 -0.90 1.10 0.60
OutputInput
*i iX W∑
*( )t oα −Perceptron Learning Rule depends on initial values
![Page 45: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/45.jpg)
2.45 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
4 1 0 0 -0.90 0.00 0.00 -0.90 0 0 0.00 -0.90 1.10 0.601 0 1 -0.90 0.00 0.60 -0.30 0 0 0.00 -0.90 1.10 0.601 1 0 -0.90 1.10 0.00 0.20 0 1 -0.50 -1.40 0.60 0.601 1 1 -1.40 0.60 0.60 -0.20 1 0 0.50 -0.90 1.10 1.10
5 1 0 0 -0.90 0.00 0.00 -0.90 0 0 0.00 -0.90 1.10 1.101 0 1 -0.90 0.00 1.10 0.20 0 1 -0.50 -1.40 1.10 0.601 1 0 -1.40 1.10 0.00 -0.30 0 0 0.00 -1.40 1.10 0.601 1 1 -1.40 1.10 0.60 0.30 1 1 0.00 -1.40 1.10 0.60
6 1 0 0 -1.40 0.00 0.00 -1.40 0 0 0.00 -1.40 1.10 0.601 0 1 -1.40 0.00 0.60 -0.80 0 0 0.00 -1.40 1.10 0.601 1 0 -1.40 1.10 0.00 -0.30 0 0 0.00 -1.40 1.10 0.601 1 1 -1.40 1.10 0.60 0.30 1 1 0.00 -1.40 1.10 0.60
7 1 0 0 -1.40 0.00 0.00 -1.40 0 0 0.00 -1.40 1.10 0.601 0 1 -1.40 0.00 0.60 -0.80 0 0 0.00 -1.40 1.10 0.601 1 0 -1.40 1.10 0.00 -0.30 0 0 0.00 -1.40 1.10 0.601 1 1 -1.40 1.10 0.60 0.30 1 1 0.00 -1.40 1.10 0.60
*( )'t o
isn t changeα −
Alpha = 0.5X0 X1 X2 1.0 * W0 X1* W1 X2* W2 Total target actual Alpha * (t-o) W0 W1 W2
OutputInput
Perceptron Learning Rule depends on initial values
![Page 46: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/46.jpg)
2.46 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Supervised LearningSupervised Learning
• Perceptron are learning by adjusting weights !
• Feeding Input and Output (target) - One by one (Online Learning) Perceptron- All in one (Batch Learning)
![Page 47: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/47.jpg)
2.47 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
PerceptronPerceptron –– One Neuron and one layer ModelOne Neuron and one layer Model
n p1, ..., pR – input patternsn w1, ..., wR - synaptic weightsn w0 (b) - threshold (Bias)n f(n) - activation function: f(n) = hardlim(n)n a - output: a = f(n)
![Page 48: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/48.jpg)
2.48 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
PerceptronPerceptron –– As decision boundaryAs decision boundary
Linear equation:
y = ax+b, wherea is a slop.
ax – y + b = 0
is equal to
w1 x1 + w2 x2 + b = 0
or
Wp + b = 0
![Page 49: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/49.jpg)
2.49 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
PerceptronPerceptron by using MATLAB toolboxby using MATLAB toolbox
NEWP Create a perceptron.
Syntax
net = newpnet = newp(pr,s,tf,lf)
Description
Perceptrons are used to solve simple (i.e. linearlyseparable) classification problems.
PR - Rx2 matrix of min and max values for R input elements.S - Number of neurons.TF - Transfer function, default = 'hardlim'.LF - Learning function, default = 'learnp'.
Returns a new perceptron.
![Page 50: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/50.jpg)
2.50 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
AND Function by using MATLAB toolboxAND Function by using MATLAB toolbox
P = [0 0 1 1; 0 1 0 1]; % And Input patterns T = [0 0 0 1]; % Target
net = newp([0 1; 0 1],1); %This code creates a perceptron layer with one 2-element % input (ranges [0 1] and [0 1]) and one neuron.% Initial weight [0 0] and bias 0
net.trainParam.epochs = 2; % train (learn) for a maximum of 2 epochsnet = train(net,P,T);
net.IW{1} = [1 1] % display weight vectornet.b{1} = 1 % display bias
a = sim(net,P); % simulate the network's output again (test pattern)a = [0 1 1 1] % Result from test pattern
![Page 51: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/51.jpg)
2.51 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Higher Dimension Feature Space
In case of 3 dimensions input data, decision function is hyper plane
0332211 xwxwxw =−++ θ
Class B
Decision planeX1
X3
X2
Class A
Equation of 3 dimensions
![Page 52: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/52.jpg)
2.52 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
33--D AND Function by using MATLAB toolboxD AND Function by using MATLAB toolbox
% 3D perceptronnet = newp([0 1; 0 1; 0 1],1);P = [0 0 0; 0 0 1; 0 1 0; 0 1 1; 1 0 0; 1 0 1; 1 1 0; 1 1 1]'; % AND FunctionT = [0; 0; 0; 0; 0; 0; 0; 1]';% initial weight and biasnet.IW{1} = [1 1 1]; net.b{1} = 0;% train or learningnet.trainParam.epochs = 1;net = train(net,P,T);plotpv(P,T); plotpc(net.IW{1},net.b{1});
![Page 53: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/53.jpg)
2.53 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
OR Function by using MATLABOR Function by using MATLAB
P = [0 0 1 1; 0 1 0 1]; % OR Input patterns T = [0 1 1 1]; % Target
net = newp([0 1; 0 1],1); %This code creates a perceptron layer with one 2-element % input (ranges [0 1] and [0 1]) and one neuron.% Initial weight [0 0] and bias 0
net.trainParam.epochs = 2; % train (learn) for a maximum of 2 epochsnet = train(net,P,T);
net.IW{1} = [1 1] % display weight vectornet.b{1} = 1 % display bias
a = sim(net,P); % simulate the network's output again (test pattern)a = [? ? ? ?] % Result from test pattern
![Page 54: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/54.jpg)
2.54 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
FAT Children by using MATLAB toolboxFAT Children by using MATLAB toolbox
% FAT Children train by perceptronnet = newp([90 130; 10 50],1);P = [100 20; 100 26; 100 30; 100 32; 102 21; 105 22; 107 32; 110 35; 111 25;
114 24; 116 36; 118 27] '; % Fat ChildrenT = [0; 1; 1; 1; 0; 0; 1; 1; 0; 0; 1; 0] '; % 1 = fat, 0 = not fatnet.trainParam.epochs = 40;net = train(net,P,T);plotpv(P,T);plotpc(net.IW{1},net.b{1});
% FAT Children test a new data (*)NP = [120 32]';hold on;plot(NP(1),NP(2),'*r');Y = sim(net,NP)
![Page 55: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/55.jpg)
2.55 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Programming in MATLAB ExerciseProgramming in MATLAB Exercise
n Exercise:
1. Write MATLAB to solve the question 3 (OR Function)in Exercise 3.
2. Write MATLAB to solve the question 4 (FAT Child Problem)in Exercise 3.
![Page 56: Programming in MATLAB - Burapha Universityureerat/321641/...Programming in MATLAB 2.2 Gp.Capt.Thanapant Raicharoen, Ph.D. Outline nIntroduction to Artificial Neural Network nMachine](https://reader031.vdocuments.us/reader031/viewer/2022012323/5ad6a2737f8b9aff228e8e46/html5/thumbnails/56.jpg)
2.56 Gp.Capt.Thanapant Raicharoen, Ph.D.Programming in MATLAB
Limitation of Limitation of PerceptronPerceptron (XOR Function)(XOR Function)
No. P1 P2 Output/Target
1. 0 0 0
2. 0 1 1
3. 1 0 1
4. 1 1 0
?