the perceptron: a probabilistic model for information …1-to-1 mapping between sensory stimulus and...

The The PerceptronPerceptron: A Probabilistic Model : A Probabilistic Model for Information Storage and for Information Storage and Organization in the brainOrganization in the brain(F. Rosenblatt)(F. Rosenblatt)

Artificial Intelligence2005-21534

Heo, Min-Oh

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

OutlineOutline

IntroductionProbabilistic model on biological Perceptron♦ Predominant phase♦ Postdominant phase

Two helpful approach♦ Bivalent systems♦ Temporal organizations

Conclusion♦ 3 major points

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Two Main QuestionsTwo Main Questions

Q1 : In what form is information stored, or remembered?

Q2 : How does information contained in storage, or in memory, influence recognition and behavior?

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Answers: 2 positions (1)Answers: 2 positions (1)

The 1st position ♦ coded representations or images

1-to-1 mapping between sensory stimulus and the stored pattern

♦ Able to discover exactly what an organism remembers by reconstructing the original sensory patterns

♦ Recognition : matching or systematic comparison of the contents of storage with incoming sensory patterns

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Answers: 2 positions (2)Answers: 2 positions (2)

The 2nd position♦ The images of stimuli may never be recorded at all.♦ There is never any simple mapping of the stimulus into memory.

♦ The information is contained in connections or associations.

♦ Recognition : Automatically activating the response without requiring any separate process for their recognition or identification.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

What is the writer going to do?What is the writer going to do?

The need for a suitable language for the mathematical analysis of events.

Formulate the current model of perceptron in terms of probability theory.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Assumptions for perceptron(1)Assumptions for perceptron(1)

5 Assumptions ♦ 1. At birth, the construction of the most important networks is

largely random, subject to a minimum number of genetic constraints.

♦ 2. The probability is likely to change, due to some relatively long-lasting changes in the neurons.

♦ 3. Exposure to a large sample of stimuli,Similar thing will tend to form pathways to the same sets of responding cells.Dissimilar thing will tend to develop connections to different sets of responding cells.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Assumptions for perceptron(2)Assumptions for perceptron(2)

♦ 4. The application of positive or negative reinforcement may facilitate or hinder whatever formation of connections is currently in progress.

♦ 5. Similarity depends on physical organization of the perceivingsystem.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

ModellingModelling

Connection♦ Excitatory ♦ Inhibitory

Level♦ S-Points : stimuli impinge on a retina of sensory units♦ A-Units : impulse are transmitted to a set of association cells♦ R-Units : the Response cells which respond as the A-units.

Two phase♦ Predominant Phase♦ Postdominant Phase

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Models of systemsModels of systems

3 types of systems.♦ Alpha system : an active cell simply gains an increment of value

for every impulse, and hold this gain indefinitely.

♦ Beta system : the increments being apportioned among the cells of the source-set in proportion to their activity.

♦ Gamma system : active cells gain in value, so that the total value of source-set is always constant.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Predominant Phase(1)Predominant Phase(1)

Probability : the Expected proportion of A-units activated by a stimulus of a given size

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Predominant Phase(2)Predominant Phase(2)

Conditional Probability : ♦ an A-unit which responds

to a given stimulus will also respond to another given stimulus

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Predominant Phase(3) : stabilityPredominant Phase(3) : stability

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Predominant Phase(4) : learnablePredominant Phase(4) : learnable

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

PostdominantPostdominant Phase(1)Phase(1)

Mathematical Model for learning♦ Reinforcement learning

Two Probabilities♦ Pr : the perceptron will show a bias towards the correct response

in preference to any given alternative response.♦ Pg : the Probability of correct generalization.

The probability that the correct response will be preferred over all alternatives is designated Pr or Pg.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

PostdominantPostdominant Phase(2) Phase(2) : able to find probability Pr, Pg to unity.: able to find probability Pr, Pg to unity.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Two helpful approachTwo helpful approach

Bivalent systems♦ Two types of reinforcement are possible

( positive and negative )

Temporal organizations♦ If the values of the A-units are allowed to decay at a rate

proportional to their magnitude, The perceptron becomes capable of “spontaneous” concept formation.

♦ Spontaneously recognize the difference between the two classes.

(C) 2005, SNU Biointelligence Lab, http://bi.snu.ac.kr/

ConclusionConclusion

3 major points to behaviorism♦ Parsimony

characteristics can clearly be stated.Have potentially measurable physical correlate.

♦ VerifiabilityWe can be considerably more confident of its validity and of itsgenerality than in the case of a theory which must be hand-tailored to meet each situation

♦ Explanatory power and generality