LIN Jie, JIN Xiao-gang, YANG Jian-gang(Institute of Artificial Intelligence, Zhejiang
University, Hangzhou 310027, China) Date of publication: Mar, 2004
Presented by: Bhuban M Seth, Joydip DattaUnder the guidance of: Prof. Dr. Pushpak Bhattacharyya
MotivationUltimate goal of Artificial Neural Net is to
imitate a human brain.But human brain is too complex to
understand.Question: What is a consciousness and How it
is generated in brain? Is there any hierarchical organization in the
brain?How can we incorporate these newfound
insights of human brain into an ANN?
Understanding the brain(Different Approaches)Taylor (1994): Relational MindRakovic (1997): hierarchically organized
and interconnected paradigm for information processing inside the brain.
Vitiello (2003): Quantum ModelRennie et. al. (2002): Evoked potential
Where all these things leads to?Cognitive processes are carried out at different
levels in the brain. Higher levels may be reduced to lower
levels.Thus, higher levels of complex brain
functions require a number of neural modules to cooperate together.
Example: We see a rose, smell the fragrance and remember some memory… this way a conscious state of mind emerges in a thinking process.
Level 1: Physical Mnemonic LayerPhysical Mnemonic Layers (PML) capture input
from external senses and produce a feature vector (patterns) from them. Many modular PMLs run in parallel.
There may be two kinds of external inputs:Arousal Inputs: Reach only up to recognition Layer
– Do not take part in Associative RecognitionAware Inputs: Reaches Abstract Thinking Layer and
may take part in Associative RecognitionThe feature vectors are input to recognition layer
Level 2: Recognition LayerIt is a searching tree composed of layered
storage neurons.It receives a pattern from the PML.
Level 3: The Global WorkspaceIt belongs to the Abstract Thinking Layer.It describes the state of Consciousness.It can project the abstract information it has
and mobilize different parts of the brain.This global availability of information define
the conscious state of mind.
Recognition LayerDivided into levelsEach level consists of number of knowledge
clustersInput is the pattern formed by Physical
Mnemonic Layer (PML)This pattern is compared with stored
patterns at all levels
Recognition LayerIf the pattern is similar to some existing
patterns it will be recognized.
Else, the pattern will be saved (New neurons will be created)
Similarity is measured by resonant coefficient
Definition:It is the feature vector of a pattern.
Inherent frequency of a neuron group k that
memorizes a knowledge pattern can be described by
the weights from one neuron in the group to other
members, as K=[wl, w2 ..... wi .... ].
Similar PatternsSimilarity of two patterns A, B are determined by
their Resonant Coefficient R(A,B). The resonant coefficient is a kind of delta
similarity relation satisfying the following properties:Reflexive: R(A, A)=1Symmetric: R(A,B) = R(B,A)And
1 - | R(A,C) – R(B,C) | >= R(A,B) --(Upper bound) R(A,B) >= max(0, R(A,C)+R(B,C)-1 ) --(Lower bound)
ExampleSuppose a series of four-dimension patterns
Pi (i=0,1,2 .... ,9) formed by PML models enter RL. Say, Pi is the binary format of i as
P3=[0,0,1,1], P5=[0,1,0,1], P1=[0,0,0,1]. We can define resonant coefficient R(Pi, Pj)
asR(Pi,Pj) = 1 – (XOR(Pi and Pj)/ 4)
Then R(P0,P0)=1,R(P0,P1)=0.75,R(P0,P2)=0.75 , R(P0,P3)=0.5 and so on.
Resonant SpaceIt is a representation of pattern showing
similarity between them.Definition: It is a space of patterns to which
any other pattern can be compared to evaluate resonant coefficient.
A pattern P is represented in resonant space by a single point, whose projection on an axis represents the resonant coefficient between the pattern corresponding to the axis and the pattern P.
Resonant Space(contd…)
The resonant space formed by patterns P0 and P5
Cntd…Consider a resonant space Rn with n patterns
Pi and the resonant coefficient R(Pi, Pj) between any two patterns Pi and Pj .
From the resonant space formed by n patterns , a pattern Pm may be represented on Rn as:
where is the unit vector along Pi axis.
Threshold in Recognition LayerDefinition:
The thresholds exist in RL corresponding to different levels (numbered from zero to TOP): to>tL>tL+l>tTOP, patterns are clustered at those levels. For example, at level L, two patterns ~ belong to the same cluster if and only if tL>f(u,v)>tL+I. There also exists a highest threshold tmax and two patterns are recognized to be the same if f(u,v)>tmax.
Abstract Thinking LayerIt can associatively compare (and recognize)
different types of inputs. It can broadcast it’s contents to the nervous
system as a whole allowing different modules to interact.
E.g. The ATL cat take input from the auditory and the vision subsystem and while associatively recognizing the inputs it can mobilize the olfactory subsystem.
Abstract Thinking Layer
Abstract Thinking LayerThe ATL is an Bi-directional Backpropagation
network (BBP).A1 and A2 are both input to of the BBP.The computation is interleaved: only one-way
learning is going on at a particular interval.The structure (no of neurons in different layers of
the BBP) of the ATL may vary depending on the inputs.
A subset of the neurons are excited at a time while rest of them are inhibited. This in general represents the consciousness.
Consciousness in ATLDynamic workspace
states are self sustained and follow one another in a continuous stream, without external help
Consciousness generation requires a stable activation loop.
The system enters a stable state V* (attractor) when there no more change in the state possible:V* = V(t+∆t) = V(t), ∆t > 0
Time span threshold in GWEstablishment of stable state requires a
minimal duration.There is a temporal span of successive
workspace states.If patterns from several subsystems appear in
the ATL longer than some Time span threshold then a conscious state emerges.
Otherwise they can not establish a self sustained activation loop – They are called sub-consciousness.
ConclusionDifferent levels exists in consciousness
generation process.Partial recognition layer threshold helps to
form clusters within RL unconsciously.Strong pattern that persists for more than a
time span threshold can accomplish associative recognition resulting in consciousness.
Background StudyWikipedia articles on: Brain, Human Brain, Cerebral
Cortex, Hippocampus etc (different parts of brain), Neuron, Action Potential, Depolarizing, Hyperpolarizing, Inhibited Neurons, Excited Neurons, Axon Hillock, Back-propagation, Neural Back-propagation, Resting potential, Layered perceptron, MLP, Electrical Inductance, Electrical Resonance etc.
Hierarchical Learning in Neural Network: http://www.cs.iastate.edu/~baojie/acad/current/hnn/hnn.htm
A Bi-Directional Multilayer PerceptronM. JEDRA, A. EL OUARDIGHI, A. ESSAID and M. LIMOURILaboratoire Conception & Systèmes, Faculté des Sciences, Avenue Ibn Batouta, B.P. 1014, Rabat10 000, Morocco, e-mail: [email protected]