Presented by Scott Lichtor

An Introduction to Neural Networks

Motivation I found for Neural NetworksPavlov’s dogSimple->ComplexLearning

Overview• Basics of the Nervous System

– Neurons– Synapses– Action Potentials

• Neural Networks– Abstract Neurons– More Complicated Neurons

– Learning– Supervised– Unsupervised– Reinforcement

• Conclusion

Basics of the Nervous SystemThe nervous system coordinates the

actions of an animalBody parts send messages to the brainBrain sends messages to body partsThe basic unit of the nervous system is the

neuron

NeuronsReceive messages at the dendritesMessage is sent quickly down the axon using

electrical impulsesWhat happens when the signal reaches the end

of the axon?

Image taken from img460.imageshack.us

SynapsesChemical Synapses

SlowStrongCan be transmitted over long distances

Image taken from http://www.airlinesafety.com/editorials

SynapsesElectrical Synapses

Very fastFade quickly

Image taken from wikipedia.org

Action PotentialsAction potentials are shocks to a particular

neuronThe shock travels along the affected

neuronThen, the action potential is transmitted

from the affected neuron to the neurons connected to it

The shock is transmitted to its destination in the same fashion

Abstract NeuronsSo biological neurons can be used to send

modified messages from place to placeCan be used to accomplish very complex

tasks using relatively simple partsCan neurons represent other things/be

used for other objectives?

Abstract NeuronsNeurons can represent neuron-like thingsInputs -> Processes -> Outputs

Image taken from http://3.bp.blogspot.com/

Abstract NeuronsCan “train” the neurons

Neurons fire (output 1) under certain patterns

Don’t fire (output 0) under other patternsFiring rule: if an outcome doesn’t fit in either

pattern, it fires if it has more in common with the first set, and doesn’t fire if it has more in common with the second set.

If there’s a tie, the neuron may fire, or it may not

Abstract Neurons Example

A neuron takes three inputs (X1, X2, X3)The neuron is trained to output 1 if the inputs are 111

or 101Trained to output 0 if the inputs are 000 or 001Before firing rule:

After firing rule:

X1 0 0 0 0 1 1 1 1

X2 0 0 1 1 0 0 1 1

X3 0 1 0 1 0 1 0 1

Out 0 0 0/1 0/1 0/1 1 0/1 1

X1 0 0 0 0 1 1 1 1

X2 0 0 1 1 0 0 1 1

X3 0 1 0 1 0 1 0 1

Out 0 0 0 0/1 0/1 1 1 1

Abstract NeuronsThe abstract neuron model can be used for

pattern recognitionExample: determine whether a ‘T’ or ‘H’ is

displayedCan we model more complicated processes

with neurons?

More Complicated NeuronsMcCulloch and Pitts modelDifference from previous model: inputs are

weighted.Add weighted inputs together: if the sum is

greater than a threshold, then the neuron fires

Image taken from http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html

More Complicated NeuronsMathematically: neuron fires if

X1W1 + X2W2 + X3W3 + ... > T

Examples

AND Gate XOR Gate

Image taken from http://www.heatonresearch.com

More Complicated NeuronsNew model is very adaptable/powerfulInput weights and threshold can be

changed so the neuron responds differently/more accurately to a situation

Pavlov’s dogVarious algorithms adapt neurons and

neural networks to situationsDelta rule (feed-forward networks)Back-error projection (feedback networks)

LearningFor the network to adapt, it must learn.There are three types of learning used with

neural networks:Supervised learningUnsupervised learningReinforcement learning

Supervised LearningIn supervised learning, the system learns using

test data given from an external teacherThe test data tells the system what outputs

result from certain inputsThe system tries to match the response of the

test data, i.e. minimize the error between the neural network outputs and the test data outputs given the same inputs

Image taken from http://www.learnartificialneuralnetworks.com

Unsupervised LearningIn unsupervised learning, the network is

given no output dataInstead, the network is given just input

dataThe goal of the network, then, is to group

the input dataExample: mortgage requests

The network is given credit ratings, size of mortgage, interest rate, etc.

The network groups the data; probably into accept and deny

Reinforcement LearningNetwork performs actions on the input dataThe environment grades the network (good

or bad)The network makes adjustments

accordinglyMiddle ground between supervised and

unsupervised learning

ConclusionThe learning aspect of neural networks

makes their applications astoundingFor computers, one has to know how to

solve a particular problemNeural networks can solve problems that

one doesn’t know how to solve

ConclusionJust some of the uses: sales forecasting,

stock market prediction, customer research, modeling and diagnosing the cardiovascular system, “Instant Physician”, interpretation of multi-meaning Chinese words, facial recognition, etc. etc. etc.

Something I found interesting: the interconnectedness of different subjects

Sourceshttp://www.doc.ic.ac.uk/~nd/surprise_96/

journal/vol4/cs11/report.htmlhttp://www.learnartificialneuralnetworks.com/http://www.ryerson.ca/~dgrimsha/courses/

cps721/unsupervised.htmlhttp://www.willamette.edu/~gorr/classes/

cs449/intro.htmlhttp://www.statsoft.com/textbook/

stneunet.htmlhttp://www.wikipedia.org