Do retinal ganglion cells project natural scenes totheir principal subspace and whiten them?

Reza Abbasi-Asl∗†, Cengiz Pehlevan†, Bin Yu∗‡, and Dmitri Chklovskii†∗Department of Electrical Engineering and Computer Science, University of California, Berkeley, CA.

† Center for Computational Biology, Flatiron Institute, New York, NY.‡ Department of Statistics, University of California, Berkeley, CA.

Abstract—Several theories of early sensory processing suggestthat it whitens sensory stimuli. Here, we test three key predictionsof the whitening theory using recordings from 152 ganglioncells in salamander retina responding to natural movies. Weconfirm the previous finding that firing rates of ganglion cells areless correlated compared to natural scenes, although significantcorrelations remain. We show that while the power spectrum ofganglion cells decays less steeply than that of natural scenes, it isnot completely flattened. Finally, we find evidence that only thetop principal components of the visual stimulus are transmitted.

I. INTRODUCTION

Efficient coding theories have been extremely influential inunderstanding early sensory processing in general [1], [2] and,visual processing in the retina in particular [3], [4], [5], [6], [7],[8], [9]. Many efficient coding theories predict that the retinashould whiten sensory stimuli [10]. For instance, whitening isoptimal from the predictive coding theory perspective, whichpostulates that only the unpredicted part of a signal shouldbe transmitted. An optimal linear predictive coding filter isa whitening filter, and when applied to the retina this theorypredicts that the receptive fields of retinal ganglion cells whitenvisual stimuli [3], [4]. Redundancy reduction theory proposesthat redundant information transmission at the retinal outputshould be minimized by decorrelated firing of ganglion cells[5]. Information theory, too, favors whitening. A linear filterthat maximizes mutual information between noisy inputs andnoisy outputs is a whitening filter [7], [8], [11]. Finally,whitening was proposed by another normative theory [12],whose starting point was efficient dimensionality reductionthrough similarity matching [13].

Here, using electrophysiological recordings from 152 gan-glion cells in salamander retina responding to natural movies,we test three key predictions of the whitening theory. 1) Reti-nal ganglion cells should be uncorrelated. We confirm previoustests of this prediction finding that retinal ganglion cells areindeed decorrelated compared to strong spatial correlationsof natural scenes [14], but significant redundancy remains[15]. 2) Output power spectrum should be flat. Whereas weobserve flattening of the ouput compared to the input, it is notcompletely flat. 3) If the number of channels are reduced fromthe input to the output, the top input principal components aretransmitted.

We start by detailing the dataset we used for this work.Then, we present our results, which test the three predictions.

Our results show that perfect whitening is not the whole story,consistent with findings of [15], [14].

II. EXPERIMENTAL DATASET AND NOTATION

The dataset was collected by Michael Berry’s laboratoryat Princeton University. It consists of recorded responses from152 larval tiger salamander retinal ganglion cells to a 7-minutenatural movie stimulus. The stimulus consists of gray scalemovie of leaves and branches blowing in the wind (figure 1-A) and is projected onto the array from a CRT monitor at60 Hz. The frame size of the stimulus is 512 × 512. Detailsof recording procedure, including the spike-sorting algorithm,are described in [16].

In this paper, X and Y denote the stimulus and the firingrate matrices respectively. X is a n × T matrix, where n isthe total number of pixels in each frame (reshaped to form avector) and T is the number of time samples. In our setting, forcomputational purposes, stimulus frames are down-sampled ton = 128 × 128 = 16, 384 and the 7-minute movie shown at60 Hz corresponds to T = 25, 200. Moreover, Y is the k×Tfiring rate matrix, where k = 152 is the number of ganglioncells that data is recorded from. The firing rates are reportedbased on spike counts in sliding 80 ms time windows. BothX and Y are centered in time.

III. RESULTS

A. Do retinal ganglion cells decorrelate natural scenes?

We start by testing whether output of ganglion cells areindeed decorrelated. This question was to a large extentanswered by previous studies [15], [14], and here we confirmtheir findings.

Our goal is to study the correlation between firing ratesof ganglion cells as a function of distance between theirreceptive field center, therefore, it is necessary to first estimatethe receptive field of each cell. We use regularized spike-triggered average also known as regularized reverse correlationtechnique [17] to estimate the spatial receptive field of eachcell. In this technique, firing rate matrix is assumed to bea linear function of stimulus matrix, Y = RX. Here, R isthe k × n receptive field matrix for ganglion cells. R canbe estimated using ridge regression which has the followingclosed form:

R = YXT(XXT + αI)−1.

arX

iv:1

612.

0348

3v1

[q-

bio.

NC

] 1

1 D

ec 2

016

A B

C

Samples,mulusframe Spa,alRecep,vefield

OFFCell ONCell

1

-1

0

0

0.4

0.8

2 6 10Distance(Pixels)

Correla,

onCoe

fficien

t

2 4 6 8 10 12

Distance in pixels-0.2

0

0.2

0.4

0.6

0.8

1

Cor

rela

tion

Coe

ffici

ent

On-OffOn-Off - Fitted CurveOffOff - Fitted CurveOnOn - Fitted CurveStimulus

2 4 6 8 10 12 14 16 18

Distance in pixels-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Cor

rela

tion

Coe

ffici

ent

On - Off pairsOn - On pairsOff - Off pairsStimulus

5 10 15 20 25 30

5

10

15

20

25

30

0<t<16.7 ms 2

0.5 1 1.5

5

10

15

20

25

30-1

-0.5

0

0.5

1

5 10 15 20 25 30

5

10

15

20

25

30

-16.7<t<0 ms 34

0.5 1 1.5

5

10

15

20

25

30-1

-0.5

0

0.5

1

5 10 15 20 25 30

5

10

15

20

25

30

-16.7<t<0 ms 34

0.5 1 1.5

5

10

15

20

25

30-1

-0.5

0

0.5

1

Fig. 1. Output of ganglion cells are decorrelated compared to natural scenes.A. A sample frame from stimulus. B. Estimated receptive field for an On andan Off ganglion cell. C. Pearson correlation coefficient of firing rates for allganglion cell-type pairs as a function of the distance between their receptivefield centers. The curves corresponds to median correlation coefficient in adistance window of 1 pixel. Similar correlation coefficient and curve areshown for stimulus.

Here, α is regularization parameter which we chose with 10-fold cross validation and I is identity matrix. Examples of theestimated On and Off receptive fields are shown in figure 1-B.The center of receptive field for each cell is simply identifiedas the location with maximum (for On cells) or minimum (forOff cells) value of estimated receptive field.

Figure 1-C shows the Pearson correlation coefficient (CC)of firing rates for all ganglion cell-type pairs as a functionof the distance between their receptive field centers. This isdepicted for On-On, Off-Off, and On-Off pairs. To show thetrend of each group, the median correlation coefficient in adistance window of 1 pixel is also shown. We compare thiscorrelation with the spatial correlation in the stimulus. Con-sistent with Pitkow and Meister’s results [14], firing rates ofganglion cells are less correlated than the pixels in naturalisticstimulus. However, significant correlations remain, questioningthe validity of the whitening theory (see also [15]).

B. Is the power spectrum of ganglion cells flattened?

Next we test whether the power spectrum of ganglion cellsis flattened. In Figure 2-A, we show the eigenvalues of theganglion cell firing rate covariance matrix, normalized by the

highest eigenvalue. We also show similar plot for the visualstimuli covariance matrix for comparison. We observe a decayin the power spectrum of the firing rates.

To compare the rate of power spectrum decay, we plot thesorted eigenvalues of the firing rate covariance matrix againsttop 152 eigenvalues of natural stimuli covariance matrix,Figure 2-B, and observe that a line can be fitted to this plotwith slope 1.09, suggesting again that the decay is similar (Aperfectly equalized output spectrum would lead to a slope of0). This analysis assumed that in the scatter plot of Figure2-B, the correct correspondence between stimulus and firing-rate covariance eigenvalues is in order of their magnitude. Itis possible that due to noise, we misestimate the rank-order ofeigenvalues. We explore this possibility in more detail below.

C. Do retinal ganglion cells project natural scenes to theirprincipal subspace?

Finally we test whether retinal ganglion cell outputs repre-sent a projection of natural scenes to their principal subspace.To be mathematically precise, we want to test if Y = ODPX,where P is a k × n matrix with k < n, rows of P are theprincipal eigenvectors of the stimulus covariance matrix, D isa diagonal, nonnegative, k×k matrix, which could be rescalingpower in each component for e.g. whitening, and O is anarbitrary k-dimensional orthogonal matrix. Since we do notknow OD, we do not have a direct prediction for what Yshould be.

However, a prediction can be made for the right-singularvectors of Y. Suppose a singular-value decomposition (SVD)for X = UXΛXVT

X , where singular values are ordered in de-creasing order. Then, Y = ODPUXΛXVT

X . But, rows of Pare the first k columns of UX , and hence ΛY := DPUXΛX

is a k×N diagonal matrix. Then, Y = RΛY VTX is an SVD,

and we can claim that the top k right singular vectors of Xand Y (columns of VX ) should match.

Of course the match will not be perfect due to noise, whichmay effect our analysis in two ways. First, our estimate ofthe ordering of singular vectors will be imperfect. Second, thesingular vector estimates will be imperfect.

Therefore, we came up with the following procedure. Wecalculate the SVD of X = UXΛXVT

X and Y = UY ΛY VTY

from data. We match right singular vectors of X and Y,columns of VX and VY , using CC as a similarity measurebetween these vectors. We compute CC for all possible pairsof input-output singular vectors and rank each pair based onthis similarity measure.

To address the delay of neural response to stimulus, CC iscomputed for several time shifts between stimuli and firingrates and the peak value is considered as the similarity. Fig.3-A (red curve) shows the CC between the pair of singularvectors with the highest peak value. There is a considerablecorrelation between the two vectors, when compared to peakCC values for randomly permuted data (see also below).

Finally we use a greedy sorting procedure to identify toppairs. We select the pair with the highest rank as the firstpick. This pair corresponds to certain index of input and

A

-8

-4

0

0

Logofeigen

value

1 0 1Eigenvalueindex(Normalized)

S=mulus Ganglioncellfiringrate

0

0.5

1

Eigenvalue

ofganglioncell

firingratecovariancematrix

Eigenvalueofs=muluscovariancematrix

B

0 0.2 0.4 0.6 0.8 1x

0

0.2

0.4

0.6

0.8

1

y

Singular valuesFitted Curve

0 0.5 1Eigenvalueindex(Normalized)

0 0.2 0.4 0.6 0.8 1x

-8

-6

-4

-2

0

y

EigenvalueFitted line

0 0.2 0.4 0.6 0.8 1x

-8

-6

-4

-2

0

y

EigenvalueFitted line

-8

-4

0

0 0.2 0.4 0.6 0.8 1x

-8

-6

-4

-2

0

y

EigenvalueFitted line

Fig. 2. Power spectrum of ganglion cells are not flattened. A. Eigenvalues of stimulus and ganglion cells firing rate covariance matrices. Eigenvalues arenormalized by the largest eigenvalue and are shown in log scale. The index is also normalized by the largest value. A line can be fitted on each plot withslope -4.7 and -5.3, respectively. B. Scatter plot of top 152 eigenvalue pairs. A line can be fitted to this plot with slope 1.09.

A

Correla(

onCoe

fficien

t

Shi1in(mesamples

B C

-30 0 30-0.2

0

0.4

Indexofsortedsingularvectorforoutput Indexofsortedsingularvectorforoutput

Inde

xofso

rted

singularvector

forinp

ut

Inde

xofso

rted

singularvector

forinp

ut

0 0

0.35 0.351

150 50

1

150 50

D

Highestcorrela(oncoefficientbetweeninputandoutputsingularvectors

0

20

010 20 30 40 50

10

20

30

40

50 0

0.05

0.1

0.15

0.2

0.25

10 20 30 40 50

10

20

30

40

50 0

0.05

0.1

0.15

0.2

0.25

0.20.1

0.2

Freq

uency

10

0.30 0.05 0.1 0.15 0.2 0.25 0.3 0.350

5

10

15

20

25

0 50 100-0.2

-0.1

0

0.1

0.2

0.3

0.4

31pairs

0 0.05 0.1 0.15 0.2 0.25 0.3 0.350

5

10

15

20

25

0 0.05 0.1 0.15 0.2 0.25 0.3 0.350

5

10

15

20

25

Original dataRandomly permuted dataFitted GaussianThreshold line

Fig. 3. Input-output singular vector pair identification. A. Red curve shows correlation coefficient between the input-output right singular vector pair with thehighest peak value among other pairs. Other curves show correlation coefficients between right singular vectors for 100 repeats of randomly time-permuteddata (see text for more details). In this plot, the output singular vector is shifted by -30 to +30 time samples. B. Heatmap of peak correlation coefficientsfor all of the possible input-output pairs. The singular vectors of input and output belonging to the top 50 matches are ordered differently to maximize thematched pair correlation coefficients. For details of the sorting procedure see text. C. Same as B for randomly permuted data. D. Red: Histogram of correlationcoefficients for 152 identified pairs from original data. Blue: Histogram of highest peak correlation coefficient for 100 randomly permuted trials. A Gaussianfunction can be fitted on this histogram. The threshold is set to be three standard deviation from mean of these 100 points. This leads to the selection of 31pairs with significant peak correlation coefficient.

output singular vectors. We then remove all the other pairscoming from these indexes and repeat a similar process forthe remaining pairs. Fig. 3-B shows the heatmap of similaritymeasures for sorted pairs. The diagonal structure implies thatthere is an approximate one-to-one mapping between the topsorted singular vectors.

To test the significance of our matching between singularvalues, we provide a reference by breaking the temporalrelationship between stimuli and firing rates and redoingthe similarity analysis. We randomly permute columns in Y(neural response matrix) while we keep X (stimulus matrix)the same. This dataset is called the randomly permuted datain this paper. We redo the similarity analysis for 100 trialsof this random permutation. Fig. 3-A shows 100 CC curveswith the highest peak for each of these randomly permuteddata. The red curve which corresponds to the original datahas higher peak CC compared to other curves. Fig. 3-C showsthe heatmap of peak CC values for one repeat of randomlypermuted data. Comparing this figure to fig. 3-B, we observethat correlations in the original data are higher compared to

the randomly permuted case.To identify pairs with significantly higher peak CC in

original data, it is necessary to find a threshold CC basedon randomly permuted data. Figure 3-D shows the process tofind this threshold. We first take the highest peak CC from100 randomly permuted trials. Then, the threshold is set tobe three standard deviations from mean of these 100 points.This leads to the selection of 31 pairs with significant peakCC. Out of these 31 input-output pairs, 12 was among the top31 input singular values and 20 was among the top 31 outputsingular values.

To summarize our findings in this section up until now,1) we found that there is a significant correlation betweensome pairs of stimulus and firing rate right singular vectors,2) we can use these correlations to come up with a one-to-one mapping between the singular vectors, and 3) the one-to-one mapping we obtained suggests that the top principalcomponents of the input is represented at the output, consistentwith whitening theory.

After identifying the top input-output pairs of singular vec-

A

Inputeigenvalue

Outpu

teigen

value

B

Indexofinputrightsingularvector

Prob

ability

1

0.5

00 10.5

0.8

0.4

00 50 100 1500 5 10 15 20 25 30

0

20

40

60

80

Original dataRandomly permuted data

C

Indexofoutputrightsingularvector

Prob

ability

0.8

0.4

00 50 100 1500 5 10 15 20 25 30

0

20

40

60

80

Original dataRandomly permuted data

0 0.2 0.4 0.6 0.8 1x

0

0.2

0.4

0.6

0.8

1

yPairs with significant CCFitted line

Fig. 4. A. Scatterplot of top selected eigenvalues and fitted linear function. The slope of the fitted line is 0.08. B. Probability of selecting each right singularvector by the selection process (see text for more details) for stimulus matrix. Singular vectors are ordered by values of the corresponding singular values(high-to-low). C. Same as B for ganglion cell firing rate matrix.

tors, which also gives a mapping between covariance matrixeigenvalues, we go back to the question we asked in theprevious section: does the power spectrum of retinal ganglioncells decay less steeply that the power spectrum of naturalscenes? In Fig. 4-A, we show the scatterplot of the normalizedcovariance eigenvalues with our new matching and the fittedlinear function. This should be compared to Figure 2-B. Wefound that the slope is lower in this case, 0.08, suggesting aflattening effect at the output.

To test the validity of our results, we perform furtheranalyses. We estimate the probability of a singular vector beingselected by our selection process (described above). First, wedivide out the data into 20 non-overlapping segments andrepeat the pair identification process for these subsets of data.The probability of selecting each singular value is estimatedby dividing the total number of selection for that singularvalue by 20. To provide a reference, we randomly permutecolumns in each of the neural response subset matrices whilekeeping stimulus subset matrix the same and re-estimate theprobabilities. Figure 4-B and 4-C show the probability ofselection for top 150 right singular vectors of the stimulus andfiring rate matrices, respectively. We find that the top singularvectors, both for stimuli and firing rates, have a significantlyhigher than chance level probability of selection. These figuressuggest that top right singular vectors of the stimulus matrixare highly expected to have a significant CC with the topright singular values of the ganglion cell firing rate matrix.This observation is consistent with the theory that ganglioncell activity represents a projection of visual stimulus onto itsprincipal subspace.

IV. CONCLUSION

In this paper, we put three key predictions of whiteningtheory to test. We found that 1) retinal ganglion cells outputsare not perfectly uncorrelated (also see[15], [14]), 2) thepower spectrum of ganglion cell firing is not completelyflat, but rather slopes downwards and 3) there are significantcorrelations between top right singular vectors of visual stimuliand retinal ganglion cell firing rates, suggesting that retinaloutput transmits dynamics in the principal subspace of visualscenes. Overall, our findings suggest that whitening alone

cannot fully explain response properties of retinal ganglioncells.

ACKNOWLEDGMENT

The authors would like to thank Michael Berry, JasonPrentice and Mark Ioffe for providing the dataset and relatedinstructions. Bin Yu acknowledges partial research supportfrom National Science Foundation (NSF) Grant CDS/E-MSS1228246, and the Center for Science of Information, an NSFScience and Technology Center, under Grant Agreement CCF-0939370.

REFERENCES

[1] F. Attneave, “Some informational aspects of visual perception.” Psycho-logical review, vol. 61, no. 3, p. 183, 1954.

[2] H. B. Barlow, “Possible principles underlying the transformations ofsensory messages,” in Sensory Communication, W. Rosenblith, Ed.Cambridge, MA: MIT press, 1961, pp. 217–234.

[3] M. V. Srinivasan, S. B. Laughlin, and A. Dubs, “Predictive coding: afresh view of inhibition in the retina,” Proceedings of the Royal Societyof London B: Biological Sciences, vol. 216, no. 1205, pp. 427–459,1982.

[4] Y. Huang and R. P. Rao, “Predictive coding,” Wiley InterdisciplinaryReviews: Cognitive Science, vol. 2, no. 5, pp. 580–593, 2011.

[5] J. J. Atick and A. N. Redlich, “What does the retina know about naturalscenes?” Neural computation, vol. 4, no. 2, pp. 196–210, 1992.

[6] J. J. Atick, “Could information theory provide an ecological theory ofsensory processing?” Network: Computation in neural systems, vol. 3,no. 2, pp. 213–251, 1992.

[7] J. Van Hateren, “A theory of maximizing sensory information,” Biolog-ical cybernetics, vol. 68, no. 1, pp. 23–29, 1992.

[8] M. Haft and J. Van Hemmen, “Theory and implementation of infomaxfilters for the retina,” Network: Computation in Neural Systems, vol. 9,no. 1, pp. 39–71, 1998.

[9] E. Doi, J. L. Gauthier, G. D. Field, J. Shlens, A. Sher, M. Greschner,T. A. Machado, L. H. Jepson, K. Mathieson, D. E. Gunning et al.,“Efficient coding of spatial information in the primate retina,” TheJournal of Neuroscience, vol. 32, no. 46, pp. 16 256–16 264, 2012.

[10] D. J. Graham, D. M. Chandler, and D. J. Field, “Can the theory of“whitening” explain the center-surround properties of retinal ganglioncell receptive fields?” Vision research, vol. 46, no. 18, pp. 2901–2913,2006.

[11] R. Linsker, “Self-organization in a perceptual network,” Computer,vol. 21, no. 3, pp. 105–117, 1988.

[12] C. Pehlevan and D. Chklovskii, “A normative theory of adaptivedimensionality reduction in neural networks,” in Advances in NeuralInformation Processing Systems, 2015, pp. 2269–2277.

[13] C. Pehlevan, T. Hu, and D. B. Chklovskii, “A hebbian/anti-hebbianneural network for linear subspace learning: A derivation from mul-tidimensional scaling of streaming data,” Neural computation, vol. 27,pp. 1461–1495, 2015.

[14] X. Pitkow and M. Meister, “Decorrelation and efficient coding by retinalganglion cells,” Nature neuroscience, vol. 15, no. 4, pp. 628–635, 2012.

[15] J. L. Puchalla, E. Schneidman, R. A. Harris, and M. J. Berry, “Redun-dancy in the population code of the retina,” Neuron, vol. 46, no. 3, pp.493–504, 2005.

[16] J. S. Prentice, O. Marre, M. L. Ioffe, A. R. Loback, G. Tkacik, andM. J. Berry, II, “Error-robust modes of the retinal population code,”PLOS Computational Biology, vol. 12, no. 11, pp. 1–34, 11 2016.

[17] E. Chichilnisky, “A simple white noise analysis of neuronal lightresponses,” Network: Computation in Neural Systems, vol. 12, no. 2,pp. 199–213, 2001.