estimating the likelihood of statistical models of natural image patches daniel zoran icnc – the...
TRANSCRIPT
Estimating the Likelihood of Statistical Models
of Natural Image PatchesDaniel Zoran
ICNC – The Hebrew University of Jerusalem
Advisor: Yair Weiss
CifAR NCAP Summer School
Natural Image Statistics
• Natural scenes and images exhibit very distinctive statistics
• A lot of research has been made in this field since the 1950s
• Important in image processing, computer vision, computational neuroscience and more…
Natural Image Statistics Properties
• The space of all possible images is huge– For a 256 gray levels, NxN sized matrix, there are
possible images– Natural Images occupy a tiny fraction of this space
• Some statistical properties of natural images:– Translation invariance– Power law spectrum – – Scale invariance– Non-Gaussianity of marginal statistics - (more on
that later)
2
256N
2( ~A
Pw
Estimating the Likelihood of different statistical models• During the years, a lot of models for natural
image distributions have been proposed• It is hard to test the validity of such models,
especially when comparing one model to the other
• A step towards this – estimating the (log) likelihood of a given model and comparing the results with other models
Estimating the Likelihood of different models• Variable sized patches were extracted from
natural images• Different models assumed• A training set was used to estimate various
parameters of the model• Likelihood was calculated over a test set• 5000 patches in each set• Source images are mostly JPEGs from a Panasonic
digital camera, portraying outdoor scenes• Also tested on standard images (Lena, Boat and
Barbara – PNG format)
The models – 1D Gaussian• A 1D Gaussian distribution for every pixel
– Mean and Variance estimated directly from the sample
– The likelihood of an image x under this model is:
– Where:
• This model captures nothing about natural images
2222
2
1( | )
2
i i
i
x
i i
L e
x
1 pi i
p
xP
22 1
1p
i i ip
xP
Results – 1D Gaussian
Patch SizeTest Set 1 (JPG)Test Set 2
(JPG)Test Set 3 (PNG)Test Set 4
(Noise)
10x1020.3-19.618.4-26.9
12x1229.4-28.026.2-38.8
14x1441.4-38.435.8-54.5
16x1652.1-44.647.7-70.5
18x1868.3-54.760.5-89.1
20x2084.5-6772.9-106.4
The models – Multidimensional Gaussian with PCA• Using the covariance matrix, rotate the images in
the image space towards directions of maximum variance (PCA)
• A Multidimensional Gaussian distribution for the components:
• Where the covariance matrix is estimated from the training set:
• This captures the Power-Law spectrum property
/2 1/2
1( | 0 ) exp( )
2NL
T -1x Σ x Σ x
Σ
Txx
Results – Multidimensional Gaussian
Patch SizeTest Set 1 (JPG)Test Set 2
(JPG)Test Set 3 (PNG)Test Set 4
(Noise)
10x1043840046526946
12x1263157769640345
14x1488679294557808
16x1611771039128579020
18x18150213341661104777
20x20191617022007135830
The models – Gaussian Mixture Model with PCA• Using the same rotation scheme (PCA), now
assume a Gaussian Mixture Model for the marginal filter response distributions
• Under this model:
• Where W’s rows are the eigenvectors of the covariance matrix
• The GMM parameters were found using EM• This captures both the Power-Law spectrum and
the sparseness properties
2 2
1
( | 0 , ) ( | 0, )K
k i kki
L c N y
x σ c y = Wx
Results – GMM with PCA
Patch SizeTest Set 1 (JPG)Test Set 2
(JPG)Test Set 3 (PNG)Test Set 4
(Noise)
10x10200210181-4852
12x12285295244-8138
14x14376400312
16x16472511404
18x18595646488
20x20698767557
The models – Generalized Gaussian with PCA• Finally, instead of using a GMM, we now use a
Generalized Gaussian• This has the advantage of having less
parameters, while still capturing Sparseness:
• Parameters were obtained directly from the training set
( | 0 , ) ( , ) exp( )ii i
i i
L A
Wxx σ α
Results – Generalized Gaussian with PCA
Patch SizeTest Set 1 (JPG)Test Set 2
(JPG)Test Set 3 (PNG)Test Set 4
(Noise)
10x10208212191-1894
12x12291305273-3099
14x14386411356-5242
16x16497529456-7875
18x18615663543-11966
20x20739796668-16783
Mean Log Likelihood - 12x12 patches
-500
-300
-100
100
300
500
700
900
Noise Natural Patches
1D Gaussian
PCA-MDGaussian
PCA-GMM
PCA-GG
Mean Log Likelihood - 18x18 patches
-500
0
500
1000
1500
Noise Natural Patches
1D Gaussian
PCA-MDGaussian
PCA-GMM
PCA-GG
The GG shape parameter
• During the analysis of the data we have encountered a strange phenomena
• Marginal distributions get wider as we go measure higher frequency filter responses
• This is not due to increase in variance (which drops as we go to high frequencies)
• We modeled this using the shape parameter obtained from the samples
Shape parameter for test set 1
Shape parameter for test set 2
Shape parameter for test set 3
Shape parameter for test set 4 - PNG
Shape parameter for noise test set
Conclusion
• This is (very) early work, still in progress• A lot of things left to do:
– Try more models and filter (ICA is in progress)– Actually compare the different models– Try to make some sense out of the shape of the
distributions– Look into higher order dependencies and
correlations
• A lot more…
Thank you!
Questions?