fast readout of object identity from macaque inferior tempora cortex
DESCRIPTION
Fast Readout of Object Identity from Macaque Inferior Tempora Cortex. Chou P. Hung, Gabriel Kreiman, Tomaso Poggio, James J.DiCarlo McGovern Institute for Brain Research, Brain and Cognitive Sciences, MIT. Object Recognition is difficult: trade-off between selectivity and invariance. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/1.jpg)
Fast Readout of Object Identity from Macaque Inferior Tempora Cortex
Chou P. Hung, Gabriel Kreiman,Tomaso Poggio, James J.DiCarlo
McGovern Institute for Brain Research,Brain and Cognitive Sciences, MIT
![Page 2: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/2.jpg)
Object Recognition is difficult:trade-off between selectivity and invariance Selectivity
Many different images can correspond to the same type of object
Invariance Similar activation patterns can correspond
to different objects
![Page 3: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/3.jpg)
The end station of the ventral stream in visual cortex is IT
![Page 4: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/4.jpg)
Can we readout what the monkey is seeing?
![Page 5: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/5.jpg)
Single electrode recordings Anterior inferior temporal cortex: highest visual area
in the ventral “what” pathway Spiking activity in AIT shows selectivity for complex
shapes
![Page 6: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/6.jpg)
![Page 7: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/7.jpg)
![Page 8: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/8.jpg)
Can we “read-out” the subject’s object percept from IT? number of sites for reliable, real-time
performance temporal properties (onset + integration scale)
of object information neural code for different tasks invariance to object position, size, pose,
illumination, clutter recognition: ‘classification’ vs. ‘identification’? spatial scale of object information (single unit, multi-
unit, LFP) stability of these neuronal codes? improvement with experience? …
![Page 9: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/9.jpg)
77 objects, 8 classes
![Page 10: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/10.jpg)
Recording at each recording site during passive viewing
77 visual objects 10 presentation repetitions per object presentation order randomized and counter-
balanced
![Page 11: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/11.jpg)
One-versus-all classification g classes (g=8): G1, …, Gg (toys, monkey
faces, vehicles, etc.) For each class i, build a binary classifier fi
(toys vs. rest, monkey faces vs. rest, etc.) sj labeled examples (j=1,…,n), For each example j, compute the output of
each classifier (e.g. pi=sj. fi ) Take prediction that maximizes pi
One-versus-all is not worse than other methods (Rifkin et al, 2003)
ij Gs
![Page 12: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/12.jpg)
Comparison of different statistical classifiers
![Page 13: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/13.jpg)
Decoding the population response
Categorization 8 groups
![Page 14: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/14.jpg)
Pattern of mistakes made by the classifier
![Page 15: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/15.jpg)
Very rapid read-out of object information
![Page 16: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/16.jpg)
Categorization and Identification
![Page 17: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/17.jpg)
IT representation is invariant to changes in position and size
![Page 18: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/18.jpg)
IT representation is invariant to changes in position and size
![Page 19: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/19.jpg)
IT representation is invariant to changes in position and size
![Page 20: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/20.jpg)
Neural code in IT: time resolution
![Page 21: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/21.jpg)
Neural code in IT: latency and integration time
![Page 22: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/22.jpg)
Reading out another type of object info: scale and location
![Page 23: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/23.jpg)
How are different kinds of information coded?
![Page 24: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/24.jpg)
Reading out another type of object info: stimulus onset
![Page 25: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/25.jpg)
Specific wiring significantly improves classifier performance
![Page 26: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/26.jpg)
Extrapolation to novel pictures within the same categories
![Page 27: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/27.jpg)
Strong overlap between the best neurons for categorization and identification
![Page 28: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/28.jpg)
The SNR for categorization and identification are positively correlated
![Page 29: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816357550346895dd40b29/html5/thumbnails/29.jpg)
Invariance to scale and position