ie 585 introduction to neural networks. 2 modeling continuum unarticulated wisdom articulated...
TRANSCRIPT
2
Modeling Continuum
UnarticulatedWisdom
ArticulatedQualitativeModels
Theoretic(First Principles)Models
EmpiricalCategoricalModels
EmpiricalContinuousModels
Low Cost /High Error
High Cost /Low Error
3
Rise of Empirical Models
• Sensoring - lots of data• Fast computing• Computing available on site• More complicated systems - do not
adhere to simple models• Easy to use software
4
Typical Empirical Models
• linear regression• splines• nearest neighbor clustering• neural networks
5
What is a Neural Net?
• An NN is a network of many simple processors (“units, neurons”), each possibly having a small amount of local memory. The units are connected by communication channels (“connections”) which usually carry numeric data, encoded by any of various means. The units operate only on their local data and on the inputs they receive via the connections.
Usenet newsgroup comp.ai.neural-nets
6
What is a Neural Net?
• An NN is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:
1. Knowledge is acquired by the network through a learning process.
2. Interneuron connection strengths known as synaptic weights are used to store the knowledge.
Haykin (1994)
7
Objectives of Neural Nets
• High Computing Speed
• Large Memory Capacity
• Adaptive Learning
• Fault Tolerance
8
Neural Network Predictive Models - Advantages
• Can accommodate non-linear relationships with interactions among variables
• Generalize well even for noisy and imprecise data
• No assumption of analytical function or theoretic relation needed
• User friendly software available• Computationally very fast, once built
9
Neural Network Predictive Models - Disadvantages
• Strongly data dependent• No statistical interpretation of significance or
confidence• Difficult to build and validate properly - too
many choices, too little general guidance, misleading validation results
10
How Do Neural Networks Work?
• Inspired by the biological brain• Consist of small, but numerous, parallel
elements - weighted connections (synapses) and summing nodes (neurons)
• “Learn” relationships through repeated calculations called “training”
• Remain fixed after training to be applied to new data
13
Elements of Neural Networksw1
w2
wn
x2
xn
yy = f ( wixi )i...
Simple SummingNode (Neuron)
x1
1
wi
xi
y
0
0.5 Non-linearTransferFunction
14
Typical Neural Network
HiddenLayer
OutputLayer
InputLayer
Error Feedback
Weighted Synapses During Training
NeuralNetworkOutput
I
N
P
U
T
S
15
Terminology• Neurons / nodes / units / cells / processing elements
(PEs)• Transfer / activation function• Connections / links / synapses• Weights / bias (fixed input of 1)• Feedforward / feedback• Input / output vectors / patterns• Self organizing (unsupervised) / supervised• Training / testing data sets
16
Biological vs Artificial Neural Networks
• Biological neurons are all excitatory (positive) or inhibitory (negative) - ANN neurons can be mixed
• Biological neurons operate asynchronously - ANN neurons usually synchronize by layer
• Biological neurons transmit signals at varying rates but ANN use a single rate
17
Biological vs Artificial Neural Networks
• There are many specialized biological neurons - ANN neurons tend to be generic
• Biological neurons work through chemical / electrical transmission (“wet ware”)
• Biological neurons are much slower but there are many, many more of them (~ 1011 neurons with 104 synapses per neuron!)
19
Common Neural Net Applications
• Pattern classification / recall– medical– defense– manufacturing quality– machine vision / postal– speech recognition– security detection– noise removal
20
Common Neural Net Applications
• Clustering / compression– data mining– signal processing– space exploration applications– speech recognition
21
Common Neural Net Applications
• Prediction / simulation– financial / stock market– music composition– utility usage– fault / degradation detection– sunspots
22
Common Neural Net Applications
• Control - real time / on line– robots– vehicles– manufacturing
• Control - off line– batch manufacturing– process optimization
23
Common Neural Net Applications
• Optimization– traveling salesman– routing– scheduling– facility location