the machine insidecs.brown.edu/people/mjblack/talks/inaugfinal.pdf · 2001. 10. 15. · curt...
TRANSCRIPT
![Page 1: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/1.jpg)
THE MACHINE INSIDE
“VOYAGES OF DISCOVERY”
ELIE BIENENSTOCK, MICHAEL J. BLACK,
MIJAIL SERRUYA, & DAVID SHEINBERG
BROWN UNIVERSITY
Applied Mathematics Computer Science Neuroscience
![Page 2: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/2.jpg)
Why study the brain?
![Page 3: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/3.jpg)
BRAINS IN ACTION
![Page 4: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/4.jpg)
FACE RECOGNITION
M2VTS Multimodal Face Database
![Page 5: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/5.jpg)
English:
French:
? ? ?
TRANSLATION
Les places américaines sont également attendues en baisse.
The American places are also awaited in fall.
“The translation you are seeing wasproduced automatically by state-of-the-art technology without the intervention ofhuman translators”
![Page 6: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/6.jpg)
XEROX ALTO 1973
Dell workstation, 2001
![Page 7: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/7.jpg)
BRAIN-LIKE COMPUTERS
Image: Jonathan Oliver
![Page 8: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/8.jpg)
BRAIN-COMPUTER INTERFACES
“If I could find … a code which translates the relationbetween the reading of the encephalograph and the mentalimage …the brain could communicate with me.”
Curt Siodmak, 1942
Brain
“Mad” scientist Nancy Reagan
![Page 9: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/9.jpg)
NEURAL PROSTHETICS
Sensation
Action
![Page 10: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/10.jpg)
NEURAL PROSTHETICS
Sensation
Action
![Page 11: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/11.jpg)
Why is studying the brain difficult?
![Page 12: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/12.jpg)
3D TOPOLOGY IN PHYSICS
interactions between constituentsof physical systems take place in
3-D topology
University of Glasgow
gallium arsenide
![Page 13: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/13.jpg)
3D TOPOLOGY IN BIOLOGY
A. Pohorille and C. Henze, NASA Advanced Supercomputing Facility
• interactions takeplace in 3-D topology• simulation onsupercomputer takesmonths
four stages of foldingof poly-leucine into analpha-helix andinsertion into amembrane
![Page 14: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/14.jpg)
nD TOPOLOGY OF CORTEX
n > 3
![Page 15: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/15.jpg)
EXPLOSION OF CORTEX
![Page 16: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/16.jpg)
VISUALIZING THE PLAYERS
Single cells ofthe nervous system
NEURON
![Page 17: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/17.jpg)
BRAIN VERSUS COMPUTER
100,000,000Transistors
Computational Elements
100,000,000,000Neurons
Speed (operations/second/element)
30-300 1.5 * 109
![Page 18: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/18.jpg)
MOORE’S LAW
Intel
![Page 19: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/19.jpg)
MASSIVE CONNECTIVITY
SYNAPSES
![Page 20: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/20.jpg)
LONG DISTANCE CALLS
Conturo et al., 1999
![Page 21: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/21.jpg)
IF EACH NEURON WERE A PERSON…
Cortex would be:
• a world of about 1011 people,
• each person communicating through a direct high-speed private line with about 103 friends,
• about half of one’s friends living in the same city,
• but many living in other continents …
![Page 22: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/22.jpg)
DEGREES OF SYNAPTIC SEPARATION
Drawings: James Yang
1D graph random graph
“small-world” graph
(Duncan Watts, Steven Strogatz)
![Page 23: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/23.jpg)
How do you study the brain?
![Page 24: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/24.jpg)
![Page 25: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/25.jpg)
![Page 26: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/26.jpg)
![Page 27: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/27.jpg)
THE EEG
![Page 28: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/28.jpg)
SINGLE UNIT ACTIVITY
1/10 mm
2/1000’s second
Spikes
![Page 29: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/29.jpg)
CELL ENSEMBLES
![Page 30: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/30.jpg)
PATTERNS IN MUSIC
![Page 31: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/31.jpg)
PATTERNS IN NEURAL ACTIVITY
![Page 32: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/32.jpg)
LANGUAGE OF THE BRAIN
Language of the brain. Language of the computer.
Interpretation
“Translation”
![Page 33: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/33.jpg)
AMBIGUOUS SIGNALS
![Page 34: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/34.jpg)
INFERENCE
Ambiguous measurements
?Inference
“Prior” knowledgeabout how brains work.
“Prior” knowledgeabout the environment
History of measurements andinterpretations
![Page 35: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/35.jpg)
THE MATHEMATICS OF MIND
))|()|(()|( 1 iggpgfpp vivv vvfg ηκ =∏∏=
tt Axy =
))))(),((|(log( ttrgfp tt θΣ−
∏=v
vvv gpgfpp )|()|()|( gfg κ
222
3
)(
2)(
ggp
∆+=∆
σπσ
−−=2
2
2
)(exp
2
1)|(
σσπgf
gfpG
gfP eg
fgfp −=
!
1)|(
![Page 36: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/36.jpg)
Brain Science Today
![Page 37: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/37.jpg)
ALGORITHMS FOR VISION
pixel
pixel
pixelpixel
pixelpixelpixel pixel
pixel
pixelpixel
pixel
pixel
blob
blob blob blob
blob
blobblob
line
line line
line
E
![Page 38: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/38.jpg)
DECODING NEURAL MESSAGES
10ms
Performance
Eye Position
Reaction Time
BEHAVIOR
NEURAL
SIGNAL
![Page 39: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/39.jpg)
NEURAL REPRESENTATIONS
![Page 40: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/40.jpg)
![Page 41: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/41.jpg)
![Page 42: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/42.jpg)
ACTIVE VISUAL PROCESSING
![Page 43: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/43.jpg)
NEURONS AND BEHAVIOR
![Page 44: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/44.jpg)
MODELING NEURAL FUNCTION
Simultaneously recordhand position, velocity,and neural activity inmotor cortex.
![Page 45: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/45.jpg)
MODELING NEURAL FUNCTION
direction
speed0
3
0
3
time…
direction (radians)
spee
d(c
m/s
)
Cell 3
Spik
etr
ains
![Page 46: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/46.jpg)
NEURAL ACTIVITY
−π πdirection (radians)
0
12
spee
d(c
m/s
)
Cell 3 Cell 16 Cell 19
Is there some “true” underlying response function?
![Page 47: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/47.jpg)
MODELING NEURAL ACTIVITY
angle (radians)
Cell 3 Cell 16 Cell 19
![Page 48: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/48.jpg)
INFERENCE FROM ACTIVITY
speed
?direction
…
t
Inference
PredictionNeural Data
![Page 49: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/49.jpg)
INFERENCE FROM ACTIVITY
speed
?direction
…
t
“prior” knowledge
PredictionNeural Data
![Page 50: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/50.jpg)
A NEURAL PROSTHETIC
EEG
single andmulti-neuronactivity
Voluntarycontrolsignal
mathematicalalgorithm
Computer cursorand
keyboard entryRobotic arm
Stimulation of Muscles,Spinal Cord, and Brain
![Page 51: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/51.jpg)
NEURAL-PROSTHETIC LIMBS
![Page 52: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/52.jpg)
Our Goals
![Page 53: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/53.jpg)
MUSIC OF THE BRAIN
Notes
Chords
Composition rules
![Page 54: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/54.jpg)
UNCOVERING THE MECHANISM
![Page 55: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/55.jpg)
THE MAN WHO MISTOOK HIS
COMPUTER FOR A HAND
Hybrid, brain-computer, systems.
new physical pathways forinteracting with the world
the computer learns about thebrain while the brain is constantlychanging
Metaphor:
from “desktop” to “body part”.
![Page 56: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/56.jpg)
OUR BODIES OURSELVES?
Probotics/Jim Judkis
Service robots under neural control.
Sensation and action at a distance.
Stimulating the brain.
Ethics, liminality, fear,and the “uncanny”.
Honda robot
![Page 57: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/57.jpg)
FUTURE
Music
Mechanism
Machine
![Page 58: THE MACHINE INSIDEcs.brown.edu/people/mjblack/Talks/inaugFinal.pdf · 2001. 10. 15. · Curt Siodmak, 1942 Brain “Mad” scientist Nancy Reagan. NEURAL PROSTHETICS Sensation Action](https://reader036.vdocuments.us/reader036/viewer/2022071421/611acca4adbd09489a189f39/html5/thumbnails/58.jpg)
THANKS
J. P. Donoghue, NeuroscienceN. Hatsopoulos, NeuroscienceW. Patterson, EngineeringA. Nurmikko, EngineeringG. Friehs, Brown Medical SchoolS. Geman, Division of Applied MathematicsY. Gao, Division of Applied MathematicsM. Fellows, NeuroscienceL. Paninski, NYU, Center for Neural ScienceN.K. Logothetis, Max Planck Institute, Tuebingen