![Page 1: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/1.jpg)
11/11/02 IDR Workshop
Dealing With Location Uncertainty in Images
Hasan F. Ates Princeton University
11/11/02
![Page 2: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/2.jpg)
11/11/02 IDR Workshop
Outline:• Introduction:
– Wavelet transform : a sparse representation for images?– Dependencies of wavelet coefficients around edges
• Contour + profile approximation– How to exploit edge-directed smoothness of wavelet
coefficients?
• Envelope + phase representation– How to achieve linear phase for wavelet coefficients?
• Applications– Image coding using profile space approach– Image interpolation
• Conclusion and future work
![Page 3: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/3.jpg)
11/11/02 IDR Workshop
Sparseness of wavelet transform:
- Coefs. classified as significant or insignificant.
- Location: clusters of insignificant coefs. coded efficiently to reduce bit rate spent on classification map.
- Sparseness: Each significant coefficient contributes to total bit rate.
![Page 4: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/4.jpg)
11/11/02 IDR Workshop
Dependencies of wavelet coefficients:• Interband dependencies:
– Zerotrees: for insignificant coefs.Rigid structure.
• Intraband dependencies:– Local variance estimation (EQ)– Adaptive nonlinear prediction
(RMS).
•Edge based adaptive modeling:
Edge-directed smoothness needs to be exploited.
Problems: 1) Sensitive to mistakes in edge location estimates
2) Side information?
![Page 5: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/5.jpg)
11/11/02 IDR Workshop
Shuffling Experiment:
Wavelet coefficients shuffled in one subband Edge structure lost in reconstruction. Same coding performance.
![Page 6: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/6.jpg)
11/11/02 IDR Workshop
Contour+Profile Representation:
• Cone of influence for :
(wavelet and point spread function).
• Piecewise constant approximation
• Wavelet values for approximation
• parametric description for contour:
![Page 7: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/7.jpg)
11/11/02 IDR Workshop
Contour+profile (cont’d) :
• Wavelet values along the contour direction:
• Wavelet profile
• Contour + profile model for wavelet values:
![Page 8: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/8.jpg)
11/11/02 IDR Workshop
Modeling wavelet coefficients:
- Same 1-D profile at each row, sampled at different locations.
- Find the profile and the contour that fits the data best (MAP estimation).
![Page 9: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/9.jpg)
11/11/02 IDR Workshop
Estimating contour and profile:
• Discrete wavelet coefficients:
step size: (wavelet transform), (redundant)• MAP estimation:
– AWG noise
– Bandlimited profile, high-pass filtered contour; ( )
![Page 10: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/10.jpg)
11/11/02 IDR Workshop
Contour optimization:
• Absolute edge locations by imposing antisymmetric profile.
• Gradient-descent iterative optimization• Initial estimate crucial:
– Wavelet local maxima connected by smooth contours
– Edge locations updated by estimating zero crossings, and point of antisymmetry for the wavelet coefficients.
– COI region depends on wavelet filter length, and other contours in the vicinity.
– If initial contour and profile estimate captures less than 50% of energy within COI, then discard.
![Page 11: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/11.jpg)
11/11/02 IDR Workshop
Contour+profile (cont’d) :
• Modeling assumptions:– Small contour curvature.
Effects of filtering in vertical direction ignored. Justified for locally linear edges.
– Isolated edge: no other high-freq. patterns within COI Not applicable for texture and multiple neighboring edges.Edge estimates at coarser scales less reliable than
estimates at finer scales. – AWGN model for residuals– Bandlimited profile: a more localized description is better
bandwidth: if too big, noise components enter if too small, fails to capture
variations
![Page 12: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/12.jpg)
11/11/02 IDR Workshop
Contour Estimation Using Previous Subband
• Estimate contour from previous subband- Profile estimate defined by orthogonal projection
to a subspace (profile space) in current subband.
HP2coarser scale
HP1finer scale
(Redundant)
![Page 13: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/13.jpg)
11/11/02 IDR Workshop
• Phase Location
Fourier phase:
1. Aliasing: non-integer displacements don’t give linear phase shifts
(if not bandlimited).
2. FFTGlobal transform;
no localized phase
information.
Searching for smooth and localized phase:
![Page 14: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/14.jpg)
11/11/02 IDR Workshop
Phase descriptions:
3. Superposition of compact isolated signals Fourier phase doesn’t reflect locations.
4. Signal smoothness not captured by Fourier phase.
The need for a smooth phase description
that isn’t affected by aliasing and that provides
location information of compact signals.
![Page 15: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/15.jpg)
11/11/02 IDR Workshop
Phase descriptions (cont’d):
• The signal given in terms of two smooth (preferably bandlimited) components:
Corresponding phase description:
• Design problem: choice of F (linear, nonlinear),
properties of m1(t), m2(t).
![Page 16: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/16.jpg)
11/11/02 IDR Workshop
Envelope+Phase Description:
• Signal described as an amplitude and phase modulated sinusoid.
![Page 17: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/17.jpg)
11/11/02 IDR Workshop
Envelope+Phase (cont’d):
• m1, m2 have smaller (effective) bandwidth
Effects of aliasing reduced.• Modulation freq: choose W to minimize m2(t).
Linear phase:
Special case (FT symmetric w.r.t. W).• Location uncertainty linear shifts in phase
![Page 18: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/18.jpg)
11/11/02 IDR Workshop
Envelope+Phase (cont’d):
• Compact m1, m2 env.+phase descriptions are additive for compact isolated signals.
• h[k], m1[k], m2[k] have equal lengths
# of dimensions doubled (redundant).
fewer degrees of freedom allowed for m1, m2
linear phase approximation less successful.
![Page 19: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/19.jpg)
11/11/02 IDR Workshop
Envelope+Phase in 2-D:
• Wavelet coefs. for jump discontinuity compact; aliased due to downsampling in wavelet transform.
• Ideal edge in 2-D: wavelet coefs. in vertical subband; effects of vertical low-pass filter ignored.
Coefficients at each row k given by a common envelope+ phase representation and 1-D contour function
![Page 20: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/20.jpg)
11/11/02 IDR Workshop
Envelope+Phase in 2-D:
• m1, m2 shaped by point spread function and wavelet filter.
• Design wavelet filters to minimize m2 (achieve linear phase).
![Page 21: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/21.jpg)
11/11/02 IDR Workshop
Constructing envelope representation:
• Method of relaxation: achieves smooth m1, m2.– Start with (# dimensions) k=1,– Choose m1, m2 that is closest to the projection of
previous estimates onto a smooth subspace of dim k,
– Increase k and repeat until m1, m2 converges to smooth functions.
![Page 22: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/22.jpg)
11/11/02 IDR Workshop
Filterbanks for phase descriptions:
• Redundant description: not suitable for coding.
• m1, m2 sampled at half-rate filterbanks:
![Page 23: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/23.jpg)
11/11/02 IDR Workshop
Image Coding:
![Page 24: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/24.jpg)
11/11/02 IDR Workshop
Profile space approximation:
•Edge structure captured
•Over 90% energy in a single basis vector.
![Page 25: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/25.jpg)
11/11/02 IDR Workshop
Image Interpolation:
• Image pixels sampled from same edge profile.• Wavelet filtering in horizontal direction.
![Page 26: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/26.jpg)
11/11/02 IDR Workshop
Model-based Interpolation:
• High-frequency enhancement based on profile model.• Baseline interpolation : bilinear.
• Interpolation given by the least square solution of:
such that
![Page 27: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/27.jpg)
11/11/02 IDR Workshop
• Reliability of model measured with percentage error
• Use the model when
•For multiple edges, texture, etc., no improvement.
• Diagonal edges: weighted average of vertical and horizontal models used.
Interpolation (cont’d):
![Page 28: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/28.jpg)
11/11/02 IDR Workshop
Interpolation (cont’d):• Smooth edge contours.
• Up to 6dB improvement
![Page 29: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/29.jpg)
11/11/02 IDR Workshop
Conclusion:• Stability: Edge-directed adaptation too restrictive.
More general measures of similarity among neighboring wavelet coefficients.
Edge contour: can be defined as a probability distribution on the possible set of locations
Modeling phase based on m1, m2
Edge profiles at different directions
Contour estimation: Simpler, computationally less expensive strategies.
![Page 30: 11/11/02 IDR Workshop Dealing With Location Uncertainty in Images Hasan F. Ates Princeton University 11/11/02](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649c7b5503460f9492e85a/html5/thumbnails/30.jpg)
11/11/02 IDR Workshop
Conclusion:
• Extending the model to multiple edges, texture, etc. Related to the model being too restrictive.More localized descriptions need to be used.