lecture 21 cochlea to brain, source localization tues
TRANSCRIPT
![Page 1: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/1.jpg)
1
COMP 546
Lecture 21
Cochlea to brain,Source Localization
Tues. April 3, 2018
![Page 2: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/2.jpg)
Ear
2
auditory canal
pinna
cochlea
outer middle inner
![Page 3: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/3.jpg)
3
Eye
β’ Lens
β’ Retina
β’ Photoreceptors
(light -> chemical)
β’ Ganglion cells (spikes)
β’ Optic nerve
Ear
β’ ?
β’ ?
β’ ?
β’ ?
β’ ?
![Page 4: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/4.jpg)
4
Eye
β’ Lens
β’ Retina
β’ Photoreceptors
(light -> chemical)
β’ Ganglion cells (spikes)
β’ Optic nerve
Ear
β’ Outer ear
β’ Cochlea
β’ hair cells
(mechanical -> chemical)
β’ Ganglion cells (spikes)
β’ VestibuloCochlear nerve
![Page 5: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/5.jpg)
Basilar Membrane
5
BM fibres have bandpass frequency mechanical responses.
20,000 Hz20 Hz
![Page 6: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/6.jpg)
Basilar Membrane: Place code (βtonotopicβ)
6
Nerve cells (hair + ganglion) are distributed along the BM. They have similar bandpass frequency response functions.
20,000 Hz20 Hz
![Page 7: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/7.jpg)
7
0 1000 2000 3000 4000 β¦. 22,000
Bandpass responses (more details next lecture)
![Page 8: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/8.jpg)
Neural coding of sound in cochlea
β’ Basilar membrane responds by vibrating with sound.
β’ Hair cells at each BM location release neurotransmitter that signal BM amplitude at that location
β’ Ganglion cells respond to neurotransmitter signals by spiking
8
![Page 9: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/9.jpg)
9
Louder sound within frequency band
β greater amplitude of BM vibration at that location
β greater release of neurotransmitter by hair cell
β greater probability of spike at each peak of filtered wave
![Page 10: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/10.jpg)
10
Hair cell neurotransmitter release can signal exact timingof BM amplitude peaks for frequencies up to ~2 kHz.
For higher frequencies, hair cells encode only the envelope of BM vibrations.
t
![Page 11: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/11.jpg)
Timing of ganglion cell spikes:for frequencies up to 2 KHz (βphase lockingβ)
11
Hair cells release more neurotransmitter at BM amplitude peaks.
Ganglion cells respond to neurotransmitter peaks by spiking.
This allows exact timing of BM vibrations to be encoded by spikes.
BM vibration
Spikes
![Page 12: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/12.jpg)
12
3,000 hair cellsin each cochlea(left and right)
30,00 ganglion cells in each cochlea
cochlear nerve(to brain)
Ganglion cells cannot spike faster than 500 times per second.So we need many ganglion cells for each hair cell.
![Page 13: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/13.jpg)
βVolleyβ code
13
Different ganglion cells at same spatial position on BM
![Page 14: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/14.jpg)
14
From cochlea to brain stem
cochlea β cochlear nucleus β lateral and medial superior olive (LSO, MSO)β¦ β auditory cortex
BRAIN STEM
![Page 15: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/15.jpg)
Tonotopic maps
15
cochlear nucleus (CN)
cochlea
lateral superior
olive (LSO)
medial superior
olive (MSO)
auditory nerve
high π low π
![Page 16: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/16.jpg)
Binaural Hearing
16
CN
LSO MSO
high π low π
CN
MSO LSO
low π high π
![Page 17: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/17.jpg)
Binaural Hearing
17
CN
LSO MSO
high πlow π
CN
MSO LSO
low πhigh π
MSO combines low frequency signals.
![Page 18: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/18.jpg)
Binaural Hearing
18
CN
LSO MSO
high π
CN
MSO LSO
high π
LSO combines high frequency signals.
![Page 19: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/19.jpg)
Levels of Analysis
19
- what is the task ? what problem is being solved?
- brain areas and pathways
- neural coding
- neural mechanisms
high
low
![Page 20: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/20.jpg)
20
For high frequency bands, β’ the head casts a shadowβ’ the timing of the peaks cannot be accurately coded
by the spikes (only the rate of spikes is informative)
For low frequency bands,β’ the head casts a weak shadow onlyβ’ the timing of the peaks can be encoded by spikes
![Page 21: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/21.jpg)
Duplex theory of binaural hearing(Rayleigh, 1907)
β’ level differences computed for higher frequencies (ILD -- interaural level differences)
β’ timing differences computed for lower frequencies (ITD - interaural timing differences)
21
![Page 22: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/22.jpg)
22
CN
LSO MSO
high π
CN
MSO LSO
high π
Level differences (high frequencies)
β β+ +
Excitatory input comes from the ear on the same side. Inhibitory input comes from ear on the opposite side.
![Page 23: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/23.jpg)
23
CN
LSO MSO
high πlow π
CN
MSO LSO
low πhigh π
Timing differences (low frequencies)
Sum excitatory input from both sides. Reminiscent of binocular complex cells in V1 ?
++
++
![Page 24: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/24.jpg)
Jeffress Model (1948) for timing differences
24
http://auditoryneuroscience.com/topics/jeffress-model-animation
E D C B A from right earfrom left ear
A
B
C
D
E
![Page 25: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/25.jpg)
Spike Timing precision required for Jeffress Model ?
25
from right earfrom left ear
distance = 1
10ππππππππ‘πππ
speed of spike = 10 πππ‘πππ π πππππβ1
βΉ β time =πππ π‘ππππ
π ππππ=
1
100ππππππ πππππ
See Exercises 19 Q2c
E D C B A
![Page 26: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/26.jpg)
26
Jeffress model remains controversial. It is not known exactly how βcoincidence detectionβ occurs in MSO.
π
Coincidence detection for each low
frequency band
A B C D E
A B C D E
A B C D E
A B C D E
![Page 27: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/27.jpg)
NaΓ―ve Computational Model of Source Localization(Recall lecture 20)
27
πΌπ (π‘) = πΌ πΌπ(π‘ β π) + π(π‘)
shadow model error
delay
Find the πΌ and π that minimize
where π < 0.5 ππ .
π‘=1
π
{ πΌπ (π‘) β πΌ πΌπ(π‘ β π) }2
![Page 28: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/28.jpg)
28
10 πππ10
π‘=1π πΌπ (π‘)2
π‘=1π πΌπ (π‘)2
π‘
πΌπ (π‘) πΌπ(π‘ β π) .
Timing difference: find the π that maximizes
Level difference:
![Page 29: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/29.jpg)
29
π‘
πΌππππ‘π(π‘) πΌπππβπ‘
π(π‘ β π) .
For each low frequency band π, find the π that maximizes
(or use summation model similar to binocular cells or Jeffress model)
An estimated value of delay π in frequency band j is consistent with various possible source directions ( π, ΞΈ ).
Similar to cone of confusion, but more general because of frequency dependence
![Page 30: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/30.jpg)
30
πΌπΏπ·π = 10 πππ10
π‘=1π πΌππππ‘
π (π‘)2
π‘=1π πΌπππβπ‘
π (π‘)2
For each high frequency band π, compute interaural level difference (ILD) :
What does each πΌπΏπ·π tell us ?
![Page 31: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/31.jpg)
31
πΌππππ‘π
π‘; π, π = ππ π‘ β βππππ‘(π‘; π, π) β πΌπ ππ π‘; π, π
πΌπππβπ‘π
π‘; π, π = ππ π‘ β βπππβπ‘(π‘; π, π) β πΌπ ππ π‘; π, π
Recall head related impulse response function (HRIR) from last lecture..
If the source direction is (q, f), and ππ π‘ is the filter for band π.
thenβ¦
![Page 32: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/32.jpg)
32
πΌ ππππ‘π
π; π, π = ππ π βππππ‘(π; π, π) πΌπ ππ π; π, π
πΌ πππβπ‘π
π; π, π = ππ π βπππβπ‘(π; π, π) πΌπ ππ π; π, π
Take the Fourier transform and apply convolution theorem :
![Page 33: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/33.jpg)
33
πΌ ππππ‘π
π; π, π = ππ π βππππ‘(π; π, π) πΌπ ππ π; π, π
πΌ πππβπ‘π
π; π, π = ππ π βπππβπ‘(π; π, π) πΌπ ππ π; π, π
Take the Fourier transform and apply convolution theorem :
If there is just one source direction (π, π), then for each frequency π within band π βΆ
πΌ πππβπ‘π
π
πΌ ππππ‘π
π βππππ‘(π; π, π)
βπππβπ‘(π; π, π)β
![Page 34: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/34.jpg)
34
One can show using Parsevalβs theorem of Fourier transforms that if βππππ‘(π; π, π) and βπππβπ‘(π; π, π) are approximately constant
within band π, then:
π‘=1π πΌππππ‘
π (π‘)2
π‘=1π πΌπππβπ‘
π (π‘)2
| βππππ‘π
( π, π) |2
| βπππβπ‘π
( π, π) |2β
![Page 35: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/35.jpg)
35
π‘=1π πΌππππ‘
π (π‘)2
π‘=1π πΌπππβπ‘
π (π‘)2
| βππππ‘π
( π, π) |2
| βπππβπ‘π
( π, π) |2β
The ear can measure thisβ¦ and can infer source directions ( π, π) that are consistent with it.
One can show using Parsevalβs theorem of Fourier transforms that if βππππ‘(π; π, π) and βπππβπ‘(π; π, π) are approximately constant
within band π, then:
![Page 36: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/36.jpg)
36
https://auditoryneuroscience.com/topics/acoustic-cues-sound-location
Each iso-contour in each frequency band is consistent with a measured level difference (dB).
Interaural Level Difference (dB) as a function of (π, π) for two fixed Ο.
700 Hz 11,000 Hz
![Page 37: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/37.jpg)
Monaural spectral cues(Spatial localization with one ear?)
37
πΌπ π‘; π, π = ππ π‘ β β(π‘; π, π) β πΌπ ππ π‘; π, π
πΌπ π; π, π = ππ π βπ(π; π, π) πΌπ ππ π; π, π
If the source is noise, then all frequencies make the same contribution on average.
Pattern of peaks and notches across bands will be due to HRTF, not to the source.
![Page 38: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/38.jpg)
38
HRTF from last lecture
e.g. medial plane
Azimuth π = 0
βPinnal notchβ frequency varies with elevation of source e.g. in the medial plane.
πΌπ π; π, π = ππ π βπ(π; π, π) πΌπ ππ π; π, π
00
![Page 39: Lecture 21 Cochlea to brain, Source Localization Tues](https://reader030.vdocuments.us/reader030/viewer/2022020705/61fb76b7827725064617c7d3/html5/thumbnails/39.jpg)
Levels of Analysis
39
- what is the task ? what problem is being solved?Source localization using level and timingdifferences within frequency channels.
- brain areas and pathways(cochlea to CN to MSO and LSO in the brainstem)
- neural coding(gave sketch only)
- neural mechanisms
high
low