11/11/02 idr workshop dealing with location uncertainty in images hasan f. ates princeton university...

11/11/02 IDR Workshop

Dealing With Location Uncertainty in Images

Hasan F. Ates Princeton University

11/11/02


Outline:• Introduction:

– Wavelet transform : a sparse representation for images?– Dependencies of wavelet coefficients around edges

• Contour + profile approximation– How to exploit edge-directed smoothness of wavelet

coefficients?

• Envelope + phase representation– How to achieve linear phase for wavelet coefficients?

• Applications– Image coding using profile space approach– Image interpolation

• Conclusion and future work


Sparseness of wavelet transform:

- Coefs. classified as significant or insignificant.

- Location: clusters of insignificant coefs. coded efficiently to reduce bit rate spent on classification map.

- Sparseness: Each significant coefficient contributes to total bit rate.


Dependencies of wavelet coefficients:• Interband dependencies:

– Zerotrees: for insignificant coefs.Rigid structure.

• Intraband dependencies:– Local variance estimation (EQ)– Adaptive nonlinear prediction

(RMS).

•Edge based adaptive modeling:

Edge-directed smoothness needs to be exploited.

Problems: 1) Sensitive to mistakes in edge location estimates

2) Side information?


Shuffling Experiment:

Wavelet coefficients shuffled in one subband Edge structure lost in reconstruction. Same coding performance.


Contour+Profile Representation:

• Cone of influence for :

(wavelet and point spread function).

• Piecewise constant approximation

• Wavelet values for approximation

• parametric description for contour:


Contour+profile (cont’d) :

• Wavelet values along the contour direction:

• Wavelet profile

• Contour + profile model for wavelet values:


Modeling wavelet coefficients:

- Same 1-D profile at each row, sampled at different locations.

- Find the profile and the contour that fits the data best (MAP estimation).


Estimating contour and profile:

• Discrete wavelet coefficients:

step size: (wavelet transform), (redundant)• MAP estimation:

– AWG noise

– Bandlimited profile, high-pass filtered contour; ( )


Contour optimization:

• Absolute edge locations by imposing antisymmetric profile.

• Gradient-descent iterative optimization• Initial estimate crucial:

– Wavelet local maxima connected by smooth contours

– Edge locations updated by estimating zero crossings, and point of antisymmetry for the wavelet coefficients.

– COI region depends on wavelet filter length, and other contours in the vicinity.

– If initial contour and profile estimate captures less than 50% of energy within COI, then discard.


Contour+profile (cont’d) :

• Modeling assumptions:– Small contour curvature.

Effects of filtering in vertical direction ignored. Justified for locally linear edges.

– Isolated edge: no other high-freq. patterns within COI Not applicable for texture and multiple neighboring edges.Edge estimates at coarser scales less reliable than

estimates at finer scales. – AWGN model for residuals– Bandlimited profile: a more localized description is better

bandwidth: if too big, noise components enter if too small, fails to capture

variations


Contour Estimation Using Previous Subband

• Estimate contour from previous subband- Profile estimate defined by orthogonal projection

to a subspace (profile space) in current subband.

HP2coarser scale

HP1finer scale

(Redundant)


• Phase Location

Fourier phase:

1. Aliasing: non-integer displacements don’t give linear phase shifts

(if not bandlimited).

2. FFTGlobal transform;

no localized phase

information.

Searching for smooth and localized phase:


Phase descriptions:

3. Superposition of compact isolated signals Fourier phase doesn’t reflect locations.

4. Signal smoothness not captured by Fourier phase.

The need for a smooth phase description

that isn’t affected by aliasing and that provides

location information of compact signals.


Phase descriptions (cont’d):

• The signal given in terms of two smooth (preferably bandlimited) components:

Corresponding phase description:

• Design problem: choice of F (linear, nonlinear),

properties of m1(t), m2(t).


Envelope+Phase Description:

• Signal described as an amplitude and phase modulated sinusoid.


Envelope+Phase (cont’d):

• m1, m2 have smaller (effective) bandwidth

Effects of aliasing reduced.• Modulation freq: choose W to minimize m2(t).

Linear phase:

Special case (FT symmetric w.r.t. W).• Location uncertainty linear shifts in phase


Envelope+Phase (cont’d):

• Compact m1, m2 env.+phase descriptions are additive for compact isolated signals.

• h[k], m1[k], m2[k] have equal lengths

# of dimensions doubled (redundant).

fewer degrees of freedom allowed for m1, m2

linear phase approximation less successful.


Envelope+Phase in 2-D:

• Wavelet coefs. for jump discontinuity compact; aliased due to downsampling in wavelet transform.

• Ideal edge in 2-D: wavelet coefs. in vertical subband; effects of vertical low-pass filter ignored.

Coefficients at each row k given by a common envelope+ phase representation and 1-D contour function


Envelope+Phase in 2-D:

• m1, m2 shaped by point spread function and wavelet filter.

• Design wavelet filters to minimize m2 (achieve linear phase).


Constructing envelope representation:

• Method of relaxation: achieves smooth m1, m2.– Start with (# dimensions) k=1,– Choose m1, m2 that is closest to the projection of

previous estimates onto a smooth subspace of dim k,

– Increase k and repeat until m1, m2 converges to smooth functions.


Filterbanks for phase descriptions:

• Redundant description: not suitable for coding.

• m1, m2 sampled at half-rate filterbanks:


Image Coding:


Profile space approximation:

•Edge structure captured

•Over 90% energy in a single basis vector.


Image Interpolation:

• Image pixels sampled from same edge profile.• Wavelet filtering in horizontal direction.


Model-based Interpolation:

• High-frequency enhancement based on profile model.• Baseline interpolation : bilinear.

• Interpolation given by the least square solution of:

such that


• Reliability of model measured with percentage error

• Use the model when

•For multiple edges, texture, etc., no improvement.

• Diagonal edges: weighted average of vertical and horizontal models used.

Interpolation (cont’d):


Interpolation (cont’d):• Smooth edge contours.

• Up to 6dB improvement


Conclusion:• Stability: Edge-directed adaptation too restrictive.

More general measures of similarity among neighboring wavelet coefficients.

Edge contour: can be defined as a probability distribution on the possible set of locations

Modeling phase based on m1, m2

Edge profiles at different directions

Contour estimation: Simpler, computationally less expensive strategies.


Conclusion:

• Extending the model to multiple edges, texture, etc. Related to the model being too restrictive.More localized descriptions need to be used.

11/11/02 idr workshop dealing with location uncertainty in images hasan f. ates princeton university...

Documents

discrete wavelet coefficients

filtered contour slide

profile estimate

idr workshop outline

initial contour

contour direction

antisymmetric profile

wavelet local maxima