visual attention - international lecture serie
TRANSCRIPT
![Page 1: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/1.jpg)
Visual AttentionInternational Lecture Serie
Nicolas P. Rougier(ニコラルジエ)
INRIANational Institute for Research
in Computer Science and Control
国立情報学研究所National Institute of Informatics
Tokyo, December 2, 2010
1 / 37
![Page 2: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/2.jpg)
Outlook
..1 We see only what we look at
..2 What is Attention ?
..3 Visual Attention
..4 Theories of Visual Attention
..5 Behavioral perspectives
2 / 37
![Page 3: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/3.jpg)
We see only what we look at(Maurice Merleau Ponty, 1961)
3 / 37
![Page 4: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/4.jpg)
Slow change blindness(O'Regan, 2001)
(http://nivea.psycho.univ-paris5.fr/sol_Mil_cinepack.avi)
From J.Kevin O’Regan (http://nivea.psycho.univ-paris5.fr/)4 / 37
![Page 5: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/5.jpg)
Slow change blindness(O'Regan, 2001)
First frame Last frame
5 / 37
![Page 6: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/6.jpg)
How blind are you ?
(http://www.dothetest.co.uk/whodunnit.html)
Transport for London campaign to make drivers aware how easy it is to misscyclists on the road and make cyclists understand how difficult they are to see.
6 / 37
![Page 7: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/7.jpg)
Eyes and retina
• Images on retina are formedupside-down
• There is a blind spot on theretina where optic nervespasses throught it
• Retina receptors are notuniformly distributed over thesurface of the retina
• Eye is always moving evenwhen fixing a point(micro-tremors)
Coprygith (c) 2007 Ignacio Icke
7 / 37
![Page 8: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/8.jpg)
The case of Stephen Wilshire(Sacks, 1995)
”They were images and showed us some of the immensely complex neutralprocesses that are needed to make a visual and graphic image.”
(Sacks, 1995)
Copyright (c) 2005 Stephen Wiltshire
8 / 37
![Page 9: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/9.jpg)
What is Attention ?
9 / 37
![Page 10: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/10.jpg)
What is Attention ?
Everyone knows what attention is. It is the possession by the mind, in clear andvivid form, of one out of what seem several simultaneously possible objects ortrains of thought. Focalization, concentration, of consciousness are of itsessence. It implies withdrawal from some things in order to deal effectively withothers, and is a condition which has a real opposite in the confused, dazed,scatterbrained state which in French is called distraction, and Zerstreutheit inGerman.
W. James, 1890
10 / 37
![Page 11: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/11.jpg)
Everyday attention effects
Cocktail party effectDivided auditory attention allows you to listen to a conversation while mostlyignoring others.
Pop-out effectBottom-up visual processing direct your attention to salient stimuli (loudsound, moving/growing object in the visual field, intense heat, etc.)
Blindness effectSelective attention allows you to recruit processing onto specific aspectsmaking you virtually blind to other aspects.
11 / 37
![Page 12: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/12.jpg)
Early experiments
Problem of air traffic controllers: hearing intermixed voices of pilots made thetask quite difficult. Cherry conducted experiments where people have toseparate sentences presented to each ear.
Dichotic listening experiment (Cherry, 1953)• Left ear: sentence A• Right ear: sentence B
Subjects able to report one sentence and alsmost nothing about the other.→ early selection theory (Broadbent, 1958)
Mixed dichotic listening experiments (Gray & Wedderburn, 1960)• Left ear: cat-4-mouse• Right ear: 3-eats-5
Subjects report ”cat eats mouse” and ”3 4 5”→ late selection theory (Deutsch & Deutsch, 1963)
12 / 37
![Page 13: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/13.jpg)
Description
Clinical Description (Sohlberg & Mateer, 1989)
Focused To respond discretely to a specific stimuli.Sustained To maintain a consistent behavioral responseSelective To maintain attention in the face of distractors
Alternating To shift focus of attentionDivided To respond simultaneously to multiple tasks
Cognitive DescriptionMotor movements preparation, priming, etc.
Sensory auditory, visual, proprioception, etc.Overt motor response (explicit)
Covert cognitive response (implicit)Top-down goal driven, bias, etc.
Bottom-Up stimulus driven, pop-out, etc.
13 / 37
![Page 14: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/14.jpg)
Visual Attention
14 / 37
![Page 15: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/15.jpg)
Cortical connectivity(Felleman and Van Essens, 1991)
Model of cortical connectivity• 32 cortical areas• 10 hierarchical levels
Copyright (c) 1991 Felleman and Van Essens
15 / 37
![Page 16: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/16.jpg)
Main visual pathways
The dorsal pathway• V1 → V2 → MT → posterior
parietal cortex• Where or How pathway• Motion and representation of
object locations
The frontal pathway• Executive control• Temporal organization of
behavior• Visual Awareness
The ventral pathway• V1 → V2 → V4 → inferior
temporal cortex• What pathway• Form and object representation
16 / 37
![Page 17: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/17.jpg)
Visual pathways(Itti & Koch, 2001)
Eye movement
Memory andcognition
Motor andother systems
Visual information
Visual cortex
Posteriorparietal cortex
Prefrontalcortex
Superiorcolliculus
Inferotemporalcortex
17 / 37
![Page 18: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/18.jpg)
Spotlight metaphor
Behavioral levelAttention is the capacity to select a relevant region of the sensory space
• Topological region of the sensory space → spatial attention• Featural region of the sensory space → feature oriented attention• Object as such → object oriented attention
18 / 37
![Page 19: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/19.jpg)
Exogeneous and endogeneous factors
Exogeneous visual attention (Desimone & Duncan, 1995)Visual attention is driven by physical properties of stimuli
• color• orientation
• movement• curvature
Endogeneous visual attention (Yarbus, 1967)
Visual attention is biased by a prioriknowledge and goals.
19 / 37
![Page 20: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/20.jpg)
Faciitation and suppression
Facilitation & suppression, non spatial attributes• Influence of novelty in LIP• Influence of relevant attributes
• IT: complex objects (e.g. faces)• V4: simple attributes (e.g. color, orientation)• MT: movements (e.g. speed, direction)
Facilitation & suppression, spatial attributes• ”Directing spatial attention to a stimulus increases its effective contrast”
(Reynolds et al., 1999)• Inhibition of return (Posner et al., 1980; Klein, 2000)
20 / 37
![Page 21: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/21.jpg)
Visual Search
Where is Waldo ?
21 / 37
![Page 22: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/22.jpg)
Parallel Search
It’s pretty easy to find the X among the L’s.
22 / 37
![Page 23: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/23.jpg)
Parallele Search
Still easy
23 / 37
![Page 24: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/24.jpg)
Parallel Search
Still easy
24 / 37
![Page 25: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/25.jpg)
Sequential Search
Much harder...
25 / 37
![Page 26: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/26.jpg)
Sequential Search
26 / 37
![Page 27: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/27.jpg)
Theories of Visual Attention
27 / 37
![Page 28: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/28.jpg)
Theories of Visual Attention
Features Integration Theory (Treismann & Gelade, 1980)
1 2 3 4 5Number of items
300
400
500
600
700
800
900
React
ion t
ime (
ms)
Serial search
Parallel search
Several primary visual features are processed and represented with separatefeature maps that are later integrated into a saliency map.
• Parallel search → Pre-attentive attention• Sequential search → Attentive attention
28 / 37
![Page 29: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/29.jpg)
Theories of Visual Attention
Automatic vs Controlled (Stroop, 1935)Say color of words out loud as quickly as possible
yellow red green blue red green(黄色) (赤) (緑) (青) (赤) (緑)
yellow blue red green blue yellow(黄色) (青) (赤) (緑) (青) (黄色)
Reading is quite automatic and color naming requires control (suppression) ofreading, thus it is slower.
29 / 37
![Page 30: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/30.jpg)
Theories of Visual Attention
Inhibition of return (IOR, Posner, 1980)Fixation frame
Uncued Cued
Cue
Time
30 / 37
![Page 31: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/31.jpg)
Theories of Visual Attention
Inhibition Of Return (IOR, Posner, 1980)”IOR operates to decrease the likelihood that a previsously inspected item inthe visual scene will be reinspected” (Klein, 2008)
• Valid for mobile targets (Tipper et al., 1991)→ updated via perception
• Up to five indices (Pylyshin, 2004)• Valid only when spatial working memory is available→ implied memorization of previously attended targets
• Appears after a time dependent of task difficulty→ Neurons dynamic does not drive IOR
31 / 37
![Page 32: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/32.jpg)
Theories of Visual AttentionSaliency Maps (Itti & Koch, 2001)
”Saliency map is a topographically arranged map that represents visual saliencyof a corresponding visual scene.” (Niebur, 2007)
Localization• Frontal Eye Field (FEF) ?• LGN (Lateral Geniculate Nucleus) ?• Superior Colliculus ?• Distributed ?
32 / 37
![Page 33: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/33.jpg)
Theories of Visual AttentionPremotor Theory of Attention (Rizzolati, 1987)
No need to postulate for two distinct control mechanisms• One dedicated circuit for action• One dedicated circuit for attention
The premotor theory of attention postulates that attention may derive fromweaker activation of same frontal-parietal circuits. 33 / 37
![Page 34: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/34.jpg)
Behavioral perspectives
Modeling perspectives• Functional separation between What and Where pathways• Non spatialy guided attention to facilitate processing of attributes• Spatialy guided attention to facilitate processing of a spatial location• Spatial attention to be deployed sequentially• IOR to avoid attending a previously attended location
Computational perspectives• To represent attention focus from saliency map→ To re-use spatal attentional model
• To memorize already attended locations• To allow updating through perception→ Dynamic spatial working memory
• To dynamically inhibit point of attention• To temporally control inhibition effect
34 / 37
![Page 35: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/35.jpg)
Behavioral perspectives
Perception in action (Gibson, 1979)• Without perception action would be unguided• Without action perception would serve no purpose
Sensori-motor account on vision (O’Regan & Noë, 2001)• Refute the hypothesis of an internal representation of the world• The outside world serves as its own, external representation• To master the laws of sensorimotor contingency
Deictic codes for the embodiment of cognition (Ballard et al., 1997)• System of implicit reference (called deictic) to bind objects to cognitive
programs• External frame of reference centered at the fixation point
35 / 37
![Page 36: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/36.jpg)
Conclusion
”On ne voit que ce qu’on regarde” (Merleau-Ponty, 1961)
Visual perception and attention are quite different from our unified visualexperience and implies a lot of different and complex processings:
• Parallel/Serial• Attentive/Pre-attentive
• Automatic/Controlled• Conscious/Unconscious
The challenge for computational neuroscientist is thus to handle thiscomplexity within a unified model in order to understand attention and makesthe link to cognition.
36 / 37
![Page 37: Visual Attention - International Lecture Serie](https://reader034.vdocuments.us/reader034/viewer/2022050512/627224ec403d840e22045111/html5/thumbnails/37.jpg)
Bibliography
Some references• Posner, M.I., Rafal, R.D., Choate, L.S. & Vaughan, J. (1985). Inhibition
of return: Neural basis and function. Cognitive Neuropsychology, 2,211–228.
• Rizzolatti, G., Riggio, L., Dascola, I. & Umiltà C. (1987). Reorientingattention across the horizontal and vertical meridians: evidence in favor ofa premotor theory of attention, Neuropsychologia 25: 31-40.
• Ballard, D., Hayhoe, M., Pook K. & Rao, R. (1997). Deictic codes for thembodiment of cognition, Behavioral and Brain Science 20:4.
• O’Regan, K. & Noë, A. (2001). A sensorimotor account of vision andvisual consciousness, Behavioral and Brain Sciences 24, 939—10031.
• Itti, L. & Koch, C. (2001). Computational modelling of visual attention.Nature Reviews Neuroscience, 2, 1-10.
37 / 37