A Mechanism for Learning, Attention A Mechanism for Learning, Attention Switching, and ConsciousnessSwitching, and Consciousness

Janusz StarzykSchool of Electrical Engineering and Computer Science, Ohio University, USA

http://people.ohio.edu/starzykj

October 20, 2010.

AttentionBiological perspectiveEmergence of consciousnessFunctional requirementsComputational model of consciousnessAttention switchingMental saccadesImplicationsSummary

OutlineOutline

Photo: https://www.adbusters.org/magazine/87/philosophy-zero-point.html

How a single thought emerges

in your brain?What motivates you to learn or

do anything?

Big QuestionsBig Questions

Photo: http://tsvetankapetrova.wordpress.com/2009/06/30/5-setbacks-that-stop-you-big-time/

How can you switch your attention from one activity to

another?What is necessary for cognition, intelligence, and

consciousness? These are but few questions important to philosophers,

cognitive neuroscientists, psychologists, artificial intelligence

researchers, etc.

Can computational models be

provided that demonstrate some of

these phenomena?Can we make a practical use

of them in autonomous machines

working in real time in natural

environments? This talk will address some of

these questions.

Big QuestionsBig Questions

http://www.geneang.com/Presence_Healing,_LLC/Neuroscience_of_Consciousness.html

5

AttentionAttention

The term attention is used when there is a clear voluntary act. We ask people to pay attention and they can chose to do so or not.

Voluntary attention is involved in preparing and applying goal directed selection for stimuli and responses.

6

AttentionAttention

Attention selects information for cognitive process Selection is driven by perceptions, emotions, motivations and is under

executive control. Without flexible, voluntary attention, we would not be able to change

behavior or deal with unexpected emergencies or opportunities. Without stimulus-driven attention we would not be able to respond

quickly to significant external events. Thus we need both voluntary and automatic attention.

7

Brain basis of attentionBrain basis of attention

Attention can be based on internal goals (finding a friend in the crowd) or external environment (alarm, bright colors)

William James wrote that attention helps to: Perceive Conceive Distinguish Remember Shorten reaction time

Attention to a location improves the accuracy and speed of detecting target at this location.

8

Brain basis of attentionBrain basis of attention

Maintaining attention against distraction requires a significant effort; E.g. trying to study when your roommate plays a loud music

Thus mental effort comes from struggle between voluntary (goal driven) and automatic attention.

9

Brain basis of consciousnessBrain basis of consciousness Conscious cognition is close to attention, but not the same. You can tell people – please pay attention but not - please be conscious. You may be aware (conscious) of reading this text but you may be not

aware of the touch of your chair, gravitational forces, background conversation, your feelings for a friend, or your major life goals.

Consciousness is not just a passive experience of sensory inputs, but an active involvement and perception.

“Self "-related phenomena such as preference, self-recognition, reflection, and planning are central to an understanding of consciousness.

10

Differences between conscious and

unconscious phenomenaConscious Unconscious 1. Explicit cognition Implicit cognition 2. Immediate memory Longer term memory 3. Novel, informative, and Routine, predictable,

significant events and nonsignificant events 4. Attended information Unattended information 5. Focal contents Fringe contents (e.g., familiarity) 6. Declarative memory Procedural memory

(facts, etc.) (skills, etc.) 7. Effortful tasks Spontaneous/automatic

tasks 8. Remembering (recall) Knowing (recognition) 9. Available memories Unavailable memories

ConsciousnessConsciousness

11

Differences between conscious and

unconscious phenomenaConscious Unconscious 10. Strategic control Automatic control 11. Grammatical strings Implicit underlying

grammars 12. Rehearsed items in Unrehearsed items in

Working Memory Working Memory 13. Wakefulness and Deep sleep, coma, sedation

dreams (cortical arousal) (cortical slow waves) 14. Explicit inferences Automatic inferences 15. Episodic memory Semantic memory

(autobiographical) (conceptual knowledge) 16. Intentional learning Incidental learning 17. Normal vision Blindsight (cortical blindness)

ConsciousnessConsciousness

Evolution and consciousnessEvolution and consciousness – appearance and evolution of consciousness– appearance and evolution of consciousness

Living Being Evolutionary traitsAnalogous feasibility in machines

Human

Beings

Fully developed cross-modal representation

Sensory capabilities: auditory, taste, touch, vision, etc.

Pre-frontal cortex: planning, thought, motivation

Impossible at

present

Hedgehog

(earliest

mammals)

Cross-modal representation

Sensory capabilities: auditory, touch, vision (less

developed), etc.

Small frontal cortex

Impossible at

present

Birds

Primitive cross-modal representation

Sensory capabilities: auditory, touch, vision, olfactory.

Primitive associative memory

Associative

memories

Photos: http://images.google.com/

Living Being Evolutionary traitsAnalogous feasibility in machines

Reptiles*Olfactory system

Primitive vision

Computer vision (emerging)

Hagfish (early vertebrate)

Primitive olfactory system

Primitive nervous system

Artificial neural networks

Lower level animals

(hydra, sponge, etc.)

Sensory motor units

Point to point nervous system

Mechanical or electronic control systems

* inconclusive\consciousness in transition

Photos: http://images.google.com/

Evolution and consciousnessEvolution and consciousness – absence of consciousness– absence of consciousness

Emergence of ConsciousnessEmergence of Consciousness

Week Human Fetus brain development

6 Cortical cells come at the correct position

20 Cortical region is insulated with myelin sheath

25 Development of local connections between neurons

30 Fetus’ brain generates electrical wave patterns

Photos: http://daymix.com/Fetus-Brain-Development/http://www.humanillnesses.com/Behavioral-Health-A-Br/The-Brain-and-Nervous-System.html?Comments[do]=mod&Comments[id]=1

Brain is self-organizing and sparse

Human Brain at Birth 6 Years Old 14 Years Old

Rethinking the Brain, Families and Work Institute, Rima Shore, 1997.

Emergence of ConsciousnessEmergence of Consciousness

Thompson, R. A., & Nelson, C. A. (2001). Developmental science and the media: Early brain development. American Psychologist, 56(1), 5-15.

Synaptic Density over the LifespanSynaptic Density over the Lifespan

Conclusion : Consciousness emerges gradually

17

1. Planning, setting goals and initiating actions2. Monitoring outcomes and adapting to errors3. Mental effort in pursuing difficult goals4. Having motivations5. Initiating speech and visual imagery6. Recognizing other’s people’s goals7. Engaging in social cooperation and competition8. Feeling and regulating emotions 9. Storing and updating working memory10. Active thinking11. Enabling conscious experiences12. Sustained attention in the face of distraction13. Switching attention14. Decision making and changing strategies15. Planning and sequencing actions16. Unifying the syntax and meaning of language17. Resolving competition between plans

csiwebcomics.com

Frontal lobe functionsFrontal lobe functions

Nobody has a slightest idea of how anything material can be conscious – Jerry Alan Fodor prof. of philosophy and cognitive science at Rutgers

The quality or state of being aware especially of something within oneself 

- Merriam Webster Dictionary

…our subjective experience or conscious state involving awareness, attention, and self reference

– prof. Jeanette Norden neuroscientist in Vanderbilt . Consciousness is a dynamic process and it changes with

development of brain. Further, at macro-level there is no consciousness centre and at micro-level there are no committed neurons or genes dedicated to consciousness – prof. Susan Greenfield neuroscientist director of Royal Institution GB

Description of ConsciousnessDescription of Consciousness

1.Conscious system is aware of past and present and is capable of critical analysis;

2.is aware of the environment in which it resides;

3.has a perception of its internal states

4.is capable to predict and explain current and past events;

5.is capable of autonomous construction of future actions;

6.can utilize past actions in the formulation of future plans:

7.is able to locate itself in its relationship to other entities;

8.can generate an internal representation of itself and its environment

9.is capable of autonomously and selectively directing its attention to address current important situations.

Conscious System RequirementsConscious System Requirements

20

Neural model for consciousnessNeural model for consciousness A neural net architecture

for attention and visual consciousness. Visual information flows

from V1 to areas V2-V4, and finally IT where objects are detected.

Each area has its inhibitory neurons to sharpen differences at that level. Posterior parietal neurons

(PP) bias visual neurons that detect the object in that spatial location.

Prefrontal neurons in area 46 are involved in voluntary attentional selection.

Attention and conscious flows.

Proposed approach to Proposed approach to machine consciousnessmachine consciousnessDefine consciousness in functional terms

Identify minimum functional requirements for consciousness

Identify functional blocks, their roles, their inter-relationships

Propose a computational model of a conscious machine

Photo: http://www.theglobalintelligencer.com/aug2007/fringe

Definition of Machine ConsciousnessDefinition of Machine Consciousness

Consciousness is attention driven cognitive perception

motivations, thoughts, plans and action monitoring.

A machine is conscious IFF besides ability to perceive, act,

learn and remember, it has a central executive mechanism

that controls all the processes (conscious or subconscious)

of the machine;

Photo: www.spectrum.ieee.org/.../biorobot11f-thumb.jpg

Consciousness requires– Intelligence (ability)– Awareness (state)

Not necessary alive

How to model consciousness?

Consciousness:Consciousness:

Intelligence

Central executive

Attention and attention switching

Mental saccades

Cognitive perception

Cognitive action control

Photo: http://eduspaces.net/csessums/weblog/11712.html

Consciousness: Consciousness: functional requirementsfunctional requirements

Computational Model of Machine ConsciousnessComputational Model of Machine Consciousness

Semantic memory

Sensory processors

Data encoders/ decoders

Sensory units

Motor skills

Motor processors

Data encoders/ decoders

Motor units

Emotions, rewards, and sub-cortical processing

Attention switching

Action monitoring

Motivation and goal processor

Planning and thinking

Episodic memory

Queuing and organization of episodes

Episodic Memory & Learning

Central Executive

Sensory-motor

Inspiration: human brainInspiration: human brainPhoto (brain): http://www.scholarpedia.org/article/Neuronal_correlates_of_consciousness

26

Sensory and Motor HierarchiesSensory and Motor Hierarchies Sensory and motor

systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies.

Sensory- Motor BlockSensory- Motor BlockSemantic memory

Sensory processors

Data encoders/ decoders

Sensory units

Motor skills

Motor processors

Data encoders/ decoders

Motor units

Emotions, rewards, and sub-cortical processing

Sensory-motor

sensory processors integrated with semantic memorymotor processors integrated with motor skillssub-cortical processors integrated with emotions and rewards

Central ExecutiveCentral ExecutivePlatform for the emergence, control, and manifestation of consciousnessControls its conscious and subconscious processesIs driven by

attention switching learning mechanism creation and selection of

motivations and goals

ahsmail.uwaterloo.ca/kin356/cexec/cexec.htm

Attention switching

Action monitoring

Motivation and goal processor

Planning and thinking

Central Executive

Taskso cognitive perceptiono attentiono attention switchingo motivationo goal creation and selectiono thoughtso planningo learning, etc.

Central ExecutiveCentral Executive

Interacts with other units for o performing its tasks o gathering data o giving directions to other units

No clearly identified decision centerDecisions are influenced by

o competing signals representing motivations, pains, desires, plans, and interrupt signals

• need not be cognitive or consciously realizedo competition can be interrupted by attention switching signal

Attention switching

Action monitoring

Motivation and goal processor

Planning and thinking

Central Executive

Central ExecutiveCentral Executive

Attention is a selective process of cognitive

perception, action and other cognitive experiences like

thoughts, action planning,

expectations, dreams

Attention switching is needed to have a cognitive experience

leads to sequences of cognitive experiences

Comic: http://lonewolflibrarian.wordpress.com/2009/08/05/attention-and-distraction-what-are-you-paying-attention-to-08-05-09/

Attention Switching !!!Attention Switching !!!

Attention Switching !!!Attention Switching !!!Dynamic process resulting from competition between

• representations related to motivations

• sensory inputs

• internal thoughts including spurious signals (like noise).

blog.gigoo.org/.../

Attention Switching !!!Attention Switching !!!

Thus, while paying attention is a conscious experience,

switching attention does not have to be.

May be a result of :

•deliberate cognitive experience (and thus fully conscious signal)

• subconscious process (stimulated by internal or external signals)

Mental SaccadesMental Saccades

This in turn activates memory traces in the global workspace area

that will be used for mental searches (mental saccades).

saccade

John

Input image

Episodic and associative memory network

his wife his house

his dogfriends

business

Spotlight on John

Frontal cortex

Mental saccade

wife house

dogfriends

business

Memory traces in frontal cortex

saccade

John

Input image

Episodic and associative memory network

his wife his house

his dogfriends

business

Spotlight on John

Frontal cortex

Mental saccade

wife house

dogfriends

business

Memory traces in frontal cortex

Selected part of the image resulting from an eye saccade.

Perceived input activates object recognition and associated areas of semantic and episodic memory.

Mental saccades in a conscious machineMental saccades in a conscious machine

Attention spotlight

Mental saccades

Perceptual saccadesChanging perception

Changing environment

Associative memory

Advancement of a goal?

Yes

No

Plan action?

NoYes

Continue search? No

Yes

Action control

Action?

Yes

No

Changing motivation

Learning

Loop 1

Loop 3

Loop 4

Loop 5

Loop 2

Attention spotlight

Mental saccades

Perceptual saccadesChanging perception

Changing environment

Associative memory

Advancement of a goal?

Advancement of a goal?

Yes

No

Plan action?Plan action?

NoYes

Continue search?Continue search? No

Yes

Action control

Action?Action?

Yes

No

Changing motivation

Learning

Loop 1

Loop 3

Loop 4

Loop 5

Loop 2

Computational Model: SummaryComputational Model: SummarySelf-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines.A central executive controls conscious and subconscious processes driven by its attention switching mechanism.Attention switching is a dynamic process resulting from competition between representations, sensory inputs and internal thoughtsMental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring

Photo: http://www.prlog.org/10313829-homeless-man-earns-250000-after-viewing-prosperity-consciousness-video-subliminal-mind-training.html

Computational Model: ImplicationsComputational Model: ImplicationsMotivations for actions are physically distributed

o competing signals are generated in various parts of machine’s mind

Before a winner is selected, machine does not interpret the meaning of the competing signals Cognitive processing is predominantly sequential

o winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner

Top down activation for perception, planning, internal thought or motor functions

o results in conscious experience• decision of what is observed and where is it • planning how to respond

o a train of such experiences constitutes consciousness

ConclusionsConclusions1.Consciousness is computational2.Intelligent machines can be conscious

Sounds like science fiction?Sounds like science fiction?

If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong.

But if it doesn’t seem like science fiction, it’s definitely wrong.

From presentation by Feresight Institute

Questions ??Questions ??

Photo: http://bajan.wordpress.com/2010/03/03/dont-blame-life-blame-the-way-how-you-live-it/

ReferencesReferences J. A. Fodor, "The big idea: can there be science of the mind," Times

Literary Supplement, pp. 5-7, July 1992. J. Norden, Understanding the brain, Video lecture series. M. Velmans, "Where experiences are: Dualist, physicalist, enactive and

reflexive accounts of phenomenal consciousness," Phenomenology and the Cognitive Sciences, vol. 6, pp. 547-563, 2007

A. Sloman, "Developing concept of consciousness," Behavioral and Brain Sciences, vol. 14 (4), pp. 694-695, Dec 1991.

W. H. Calvin and G. A. Ojemann, Conversation with Neil's brain: the neural nature of thought and language: Addison-Wesley, 1994.

J. Hawkins and S. Blakeslee, On intelligence. New York: Henry Holt & Company, LLC., 2004.

S. Greenfield, The private life of the brain. New York: John Wiley & Sons, Inc., 2000.

Nisargadatta, I am that. Bombay: Chetana Publishing, 1973. D. C. Dennett, Consciousness Explained, Penguin Press,1993. D. M. Rosenthal, The nature of Mind, Oxford University Press, 1991. B. J. Baars “A cognitive theory of consciousness,” Cambridge

University Press, 1998.Photo: http://s121.photobucket.com/albums/o209/TiTekty/?action=view&current=hist_sci_image1.jpg

Embodied IntelligenceEmbodied Intelligence

– Mechanism: biological, mechanical or virtual agent

with embodied sensors and actuators– EI acts on environment and perceives its actions– Environment hostility is persistent and stimulates EI to act– Hostility: direct aggression, pain, scarce resources, etc– EI learns so it must have associative self-organizing memory– Knowledge is acquired by EI

Definition Embodied Intelligence (EI) is a

mechanism that learns how to minimize hostility of its environment

Embodiment

Actuators

Sensors

Intelligence core

channel

channel

Embodiment

Sensors

Intelligence core

Environment

channel

channelActuators

Embodiment

Actuators

Sensors

Intelligence core

channel

channel

Embodiment

Sensors

Intelligence core

Environment

channel

channelActuators

Embodiment of a MindEmbodiment of a Mind

Embodiment is a part of the environment that EI controls to interact with the rest of the environment

It contains intelligence core and sensory motor interfaces under its control

Necessary for development of intelligence

Not necessarily constant or in the form of a physical body

Boundary transforms modifying brain’s self-determination

Brain learns own body’s dynamic Self-awareness is a result of

identification with own embodiment Embodiment can be extended by

using tools and machines Successful operation is a function

of correct perception of environment and own embodiment

Embodiment of a MindEmbodiment of a Mind

Motivated LearningMotivated Learning

Various pains and external signals compete for attention. Attention switching results from competition. Cognitive perception is aided by winner of competition.

Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent. Machine creates abstract goals based on the

primitive pain signals. It receives internal rewards for satisfying its goals

(both primitive and abstract). ML applies to EI working in a hostile environment.

Reinforcement LearningReinforcement Learning Motivated Learning Motivated Learning Single value function Measurable rewards

Can be optimized

Predictable Objectives set by

designer Maximizes the reward

Potentially unstable

Learning effort increases with complexity

Always active

Multiple value functions One for each goal

Internal rewards Cannot be optimized

Unpredictable Sets its own objectives Solves minimax problem

Always stable

Learns better in complex environment than RL

Acts when needed

http://www.bradfordvts.co.uk/images/goal.jpg