neural noise and neural signals - spontaneous activity and...
TRANSCRIPT
![Page 1: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/1.jpg)
Neural noise and neural signals - spontaneous activity and signal transmission
in models of single nerve cells
Benjamin Lindner
Theory of Complex Systems and Neurophysics
Institut für PhysikHumboldt-Universität Berlin
1
![Page 2: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/2.jpg)
Information theory
2
![Page 3: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/3.jpg)
time
Information theory of neural spiking
3
![Page 4: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/4.jpg)
• How do we quantify information?• What is the max info a spike train can carry?• How much info does the spike train carry
about the sensory signal?
time
Information theory of neural spiking
3
![Page 5: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/5.jpg)
• How do we quantify information?
Information theory of neural spiking
4
![Page 6: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/6.jpg)
• How do we quantify information?
Information theory of neural spiking
Shannon entropy (in bits)
mean information (reduction of uncertainty) we obtain by measuring the state of a discrete
system with probabilities
4
![Page 7: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/7.jpg)
Last time: Information theory of neural spiking
• What is the max info a spike train can carry?
Specifically: a stationary spike train with rate
0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
6t
N bins
5
![Page 8: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/8.jpg)
Last time: Information theory of neural spiking
• What is the max info a spike train can carry?
Specifically: a stationary spike train with rate
Information rate of a Poisson process
0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
6t
N bins
5
![Page 9: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/9.jpg)
Last time: Information theory of neural spiking
• General Insight: Maximal entropy depends on constraints?
Fixed finite range:
Fixed mean, semi-infinite range:
Fixed mean and variance, infinite range:
uniform
exponential
Gauss
6
![Page 10: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/10.jpg)
• Last time: How much info does the spike train carry about the sensory signal?
noisy neuron output spike traininput signal
1.Compute full entropy of the output2.For frozen stimulus calculate noise entropy unrelated to the stimulus3.Take the difference Mutual information
7
![Page 11: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/11.jpg)
time window = 0.2s
time
The direct method of determining mutual informationfor a spiking neuron
8
![Page 12: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/12.jpg)
S = !
!
Sequenzen
pilog2[pi]
(Claude Shannon,1948)
Entropy of bit sequences
sequences
Time
9
![Page 13: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/13.jpg)
Variabilitywithout
reference to the signal
N = !
!
"
Sequenzen
p̃ilog2[p̃i]
#
stimulus
Entropy with frozen signal
sequences
Time
10
![Page 14: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/14.jpg)
Mutual information rate
Mutual information
11
![Page 15: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/15.jpg)
H1 Neuron in the visual system of the blow fly (Strong et al. Phys. Rev. Lett. 1998)
The direct method for determining the info rate
12
![Page 16: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/16.jpg)
H1 Neuron in the visual system of the blow fly (Strong et al. Phys. Rev. Lett. 1998)
The direct method for determining the info rate
12
![Page 17: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/17.jpg)
Finite-size corrections to the entropy
H1 Neuron in the visual system of the blow fly (Strong et al. Phys. Rev. Lett. 1998)
The direct method for determining the info rate
12
![Page 18: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/18.jpg)
Finite-size corrections to the entropy
H1 Neuron in the visual system of the blow fly (Strong et al. Phys. Rev. Lett. 1998)
The direct method for determining the info rate
12
![Page 19: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/19.jpg)
Finite-size corrections to the entropy
H1 Neuron in the visual system of the blow fly (Strong et al. Phys. Rev. Lett. 1998)
The direct method for determining the info rate
Finite-window corrections to the entropy rate
13
![Page 20: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/20.jpg)
Finite-size corrections to the entropy
H1 Neuron in the visual system of the blow fly (Strong et al. Phys. Rev. Lett. 1998)
The direct method for determining the info rate
Finite-window corrections to the entropy rate
13
![Page 21: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/21.jpg)
Information rates of real neurons
Borst & Theunissen Nat. Neurosci. (1999)
14
![Page 22: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/22.jpg)
Information rates of real neurons
Borst & Theunissen Nat. Neurosci. (1999)
14
![Page 23: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/23.jpg)
Information rates of real neurons
Borst & Theunissen Nat. Neurosci. (1999)
14
![Page 24: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/24.jpg)
Last time: The Gaussian channel
signal
noise
Assumptions
•statistically independent•Gaussian with zero mean
and
15
![Page 25: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/25.jpg)
The dynamic Gaussian channel
signal
noise
16
![Page 26: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/26.jpg)
The dynamic Gaussian channel
signal
noise
Assumptions
•statistically independent•Gaussian with zero mean•power spectra
and
16
![Page 27: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/27.jpg)
The dynamic Gaussian channel
signal
noise
Assumptions
•statistically independent•Gaussian with zero mean•power spectra
and
16
![Page 28: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/28.jpg)
The dynamic Gaussian channel (more general)
signal
noise
17
![Page 29: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/29.jpg)
The dynamic Gaussian channel (more general)
signal
noise
Assumptions
•statistically independent•Gaussian with zero mean•power spectra
and
17
![Page 30: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/30.jpg)
The dynamic Gaussian channel (more general)
signal
noise
Assumptions
•statistically independent•Gaussian with zero mean•power spectra
and
17
![Page 31: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/31.jpg)
18
neuron
stimulus
A lower bound on the mutual information
spike train
![Page 32: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/32.jpg)
19
linearsignal
reconstructionneuron
stimulus
Data processing inequality:
spike train
A lower bound on the mutual information
estimatedstimulus
![Page 33: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/33.jpg)
19
linearsignal
reconstructionneuron
stimulus
Data processing inequality:
spike train
A lower bound on the mutual information
estimatedstimulus
Lower bound
![Page 34: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/34.jpg)
C(f) =|Sx,s(f)|2
Sx,x(f)Ss,s(f)
Spectral Coherence function Cross-spectrum (stimulus-spike train)
Stimulus power spectrumSpike train power spectrum
A lower bound on the mutual information
Lower bound
![Page 35: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/35.jpg)
Comparison of direct & lower-bound methods
Aldworth et al. PLoS Comp. Biol. (2011)
21
![Page 36: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/36.jpg)
Using the coherence function to discuss frequency-dependent information transmission
Middleton et al. J. Neurophysiol. (2009) Chacron et al. Nature (2003)
P-units Pyramidal cells
Weakly electric fish
22
![Page 37: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/37.jpg)
Using the coherence function to discuss frequency-dependent information transmission
Paddle fish
Neiman & Russell Chaos (2011)
23
![Page 38: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/38.jpg)
Using the coherence function to discuss frequency-dependent information transmission
Monkey, vestibular system
24
![Page 39: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/39.jpg)
Bullfrog(
Using the coherence function to discuss frequency-dependent information transmission
25
![Page 40: Neural noise and neural signals - spontaneous activity and ...people.physik.hu-berlin.de/...3_4_infotheory_2015.pdf · - Rieke et al. Spikes: Exploring the neural code MIT Press (1996)](https://reader033.vdocuments.us/reader033/viewer/2022050218/5f640c29e4ddb15a4d0c7c23/html5/thumbnails/40.jpg)
Summary: information theory of spike trains
-information can be quantified by entropies and differences between them (mutual information)
-mutual information can be determined directly (from lots of data!) or can be estimated from below (lower bound)
-information rates are often not far from their theoretical limit; they are higher for natural stimuli
References
- Dayan & Abbott Theoretical Neuroscience MIT Press (2001)
- Gerstner & Kistler Spiking Neuron Models Cambridge University Press (2002)
- Rieke et al. Spikes: Exploring the neural code MIT Press (1996)
- Pierce An Introduction to Information Theory Dover (1980)
26