The Statistical Properties of Large Scale

Structure

Alexander SzalayDepartment of Physics and Astronomy

The Johns Hopkins University

Outline

• Lecture 1: – Basic cosmological background

– Growth of fluctuations

– Parameters and observables

• Lecture 2: – Statistical concepts and definitions

– Practical approaches

– Statistical estimators

• Lecture 3: – Applications to the SDSS

– Angular correlations with photometric redshifts

– Real-space power spectrum

Lecture #2

• Statistical concepts and definitions– Correlation function

– Power spectrum

– Smoothing kernels

– Window functions

• Statistical estimators

Basic Statistical Tools

• Correlation functions

– N-point and Nth-order

– Defined in real space

– Easy to compute, direct physical meaning

– Easy to generalize to higher order

• Power spectrum

– Fourier space equivalent of correlation functions

– Directly related to linear theory

– Origins in the Big Bang

– Connects the CMB physics to redshift surveys

• Most common are the 2nd order functions:

– Variance 82

– Two-point correlation function (r)

– Power spectrum P(k)

32

21 321 21

The Galaxy Correlation Function

• First measured by Totsuji and Kihara, then Peebles etal

• Mostly angular correlations in the beginning

• Later more and more redshift space

• Power law is a good approximation

• Correlation length r0=5.4 h-1 Mpc

• Exponent is around =1.8

• Corresponding angular correlations

0

)(r

rr

1

0

)(rw

The Overdensity

• We can observe galaxy counts n(x), and compare to the expected counts

• Overdensity

• Fourier transform

• Inverse

• Wave number

1)(

)( n

n xx

)(

2

1)( 3

3 kkx kx

ied

)()( 3 xxk kx ied

2

k

n

)(xn

The Power Spectrum

• Consider the ensemble average

• Change the origin by R

• Translational invariance

• Power spectrum

• Rotational invariance

)()( 2*

1 kk

)()()(~

)(~

2*

1)(

2*

121 kkkk Rkk ie

2

121)3(3

2*

1 )()()2()()( kkkkk

)()( kPP k

2)()( kk P

Correlation Function

• Defined through the ensemble average

• Can be expressed through the Fourier transform

• Using the translational invariance in Fourier space

• The correlation function only depends on the distance

)()(),( 2121 xxxx

)()()2(

1),( 2

*1

)(2

31

3621

2211 kkkkxx xkxk

iedd

)()(),( 21)(3

2121 xxkxx xxk kPed i

)()()2(

1),( 121

)3()(2

31

3321

2211 kPedd i kkkkxx xkxk

P(k) vs (r)

• The Rayleigh expansion of a plane wave gives

• Using the rotational invariance of P(k)

• The power per logarithmic interval

• The power spectrum and correlation function form a Fourier transform pair

)()(4

1)()( 0

2221 kPkrjkdkr

xx

l

llli Pkrjlie )ˆˆ()()12( rkkr

)()(ln4

1)( 3

02kPkkrjkdr

Filtering the Density

• Effect of a smoothing kernel K(x), where

• Convolution theorem

• Filtered power spectrum

• Filtered correlation function

1)(3 xx Kd

)'()'(')( 3 xxxxx KdKs

)()()( kkk Ks

222)()()()()( kkkkk KPKPs

2302

)()()(ln4

1)( kKkPkkrjkdrs

Variance

• At r=0 separation we get the variance:

• Usual kernel is a ‘top-hat’ with an R=8h-1 Mpc radius

• The usual normalization of the power spectrum is using this window

232

2 )()(ln4

1kKkPkkd RR

kr

krjkK

Rr

RrrK RR

)()(

if,0

if,1)( 1

2

83

2

28 )()(ln

4

1kKkPkkd

Selection Window

• We always have an anisotropic selection window, both over the sky and along the redshift direction

• The observed overdensity is

• Using the convolution theorem

outside

insideW

if,0

if,0)(r

)()()( xxx Ww

)'()'(')( 3 kkkkk Wdw

The Effect of Window Shape

• The lines in the spectrum are at least as broad as the window – the PSF of measuring the power spectrum!

• The shape of the window in Fourier space is the conjugate to the shape in real space

• The larger the survey volume, the sharper the k-space window => survey design

)()()()()()( ''''*'''''*'''''3''3'* kkkkkkkkkk WWdd wwww

)()()()2()()( '''*''''''33'* kkkkkkkk WWPdww

1-Point Probability Distribution

• Overdensity is a superposition from Fourier space

• Each depends on a large number of modes

• Variance (usually filtered at some scale R)

• Central limit theorem: Gaussian distribution

)(

2

1)( 3

3 kkx kx

ied

2

2

2)(

2

2

e

P

22

N-point Probability Distribution

• Many ‘soft’ pixels, smoothed with a kernel

• Raw dataset x

• Parameter vector

• Joint probability distribution is a multivariate Gaussian

• M is the mean, C is the correlation matrix

• C depends on the parameters

)()(

2

1exp)2(),( 12/12/ mxCmxCΘx Tnf

xm jijiij mmxxC

Fisher Information Matrix

• Measures the sensitivity of the probability distribution to the parameters

• Kramer-Rao theorem:one cannot measure a parameter more accurately than

jiij

f

ln2

F

iiF

1

Higher Order Correlations

• One can define higher order correlation functions

• Irreducible correlations represented by connected graphs

• For Gaussian fields only 2-point, all other =0

• Peebles conjecture: only tree graphs are present

• Hierarchical expansion

• Important on small scales

321

4321

terms) 1-N(... nijkijNN Q

Correlation Estimators

• Expectation value

• Rewritten with the density as

• Often also written as

• Probability of finding objects in excess of random

• The two estimators above are NOT EQUIVALENT

)()(),( 2121 xxxx

21

221112

1121

2112

RR

DD

Edge Effects

• The objects close to the edge are different

• The estimator has an excessvariance (Ripley)

• If one is using the firstestimator, these cancelin first order (Landy and Szalay 1996)

21

2211

21

221112

))((

RR

RDRD

RR

RRDRDD

212

Discrete Counts

• We can measure discrete galaxy counts

• The expected density is a known <n>, fractional

• The overdensity is

• If we define cells with counts Ni

)()( rrr Dn

n

nn )(r

i

iii N

NN )(r

Power Spectrum

• Naïve estimator for a discrete density field is

• FKP (Feldman, Kaiser and Peacock) estimatorThe Fourier space equivalent to LS

n

i neN

f krk1

)(ˆ

Ne

Ne

NfkP

nn

i

nn

i nnnn111

)(ˆ)(ˆ'

)(2

',

)(2

''

rrkrrkk

)()()(ˆ krk kr wefn

in

n

)()(1

)()(

kPrn

rn

r

Wish List

• Almost lossless for the parameters of interest

• Easy to compute, and test hypotheses(uncorrelated errors)

• Be computationally feasible

• Be able to include systematic effects

The Karhunen-Loeve Transform

• Subdivide survey volume into thousands of cells

• Compute correlation matrix of galaxy counts among cells from fiducial P(k)+ noise model

• Diagonalize matrix– Eigenvalues

– Eigenmodes

• Expand data over KL basis

• Iterate over parameter values:– Compute new correlation matrix

– Invert, then compute log likelihood

Vogeley and Szalay (1996)

Eigenmodes

Eigenmodes

• Optimal weighting of cells to extract signal represented by the mode

• Eigenvalues measure S/N

• Eigenmodes orthogonal

• In k-space their shape is close to window function

• Orthogonality = repulsion

• Dense packing of k-space => filling a ‘Fermi sphere’

Truncated expansion

• Use less than all the modes: truncation

• Best representation in the rms sense

• Optimal subspace filtering, throw away modes which contain only noise

NMbfM

iii

1

,ˆ

N

Miiff

1

2)ˆ(

Correlation Matrix

• The mean correlation between cells

• Uses a fiducial power spectrum

• Iterate during the analysis

)()(),( 2121)(

23

13 rWrWrrrdrd ji

sij

Whitening Transform

• Remove expected count ni

• ‘Whitened’ counts

• Can be extended to other types of noise=> systematic effects

• Diagonalization: overdensity eigenmodes

• Truncation optimizes the overdensity

i

iii n

ngd

ji

ij

i

ijijij nnn

R

Truncation

• Truncate at 30 Mpc/h – avoid most non-linear effects

– keep decent number of modes