Edge-Directed Image Interpolation

Edge-Directed Image Interpolation

Nickolaus Mueller, Yue Lu, and Minh N. DoNickolaus Mueller, Yue Lu, and Minh N. Do

“In theory, there is no difference between theory and practice; In practice, there is.”-Chuck Reid

I. Description of ProblemA. ExamplesB. One-Dimensional SignalsC. Two-Dimensional Images

II. State of the ArtA. Description of MethodsB. Results

III. Wavelet AlgorithmsA. Regularity Preserving Image InterpolationB. Proposed Method using Contourlets

Outline of the TalkOutline of the Talk

Basic Image InterpolationBasic Image Interpolation

n Given a low-resolution image, increase resolution by a factor of 2 or larger

Description of ProblemDescription of Problem

n Problem: Basic interpolation techniques cause “jagged” or “blurred” edges

n Goal: Reduce artifacts using edge information

n Simple image model: continuous, smooth objects piecewise continuous, smooth edges

Examples of Edge Artifacts

Examples of Edge Artifacts

Original

Bilinear Bilinear Bicubic

Original Original

One-Dimensional Problem

One-Dimensional Problem

Images: A More Difficult Task

Images: A More Difficult Task

n 2-D Edges - Magnitude and directional component

n Edges have “Geometric Regularity”

n Challenge: Estimate orientation so that edges are both sharp and free from artifacts.

State of the Art MethodsState of the Art Methodsn Sub-pixel Edge Localization

n Kris Jensen and Dimitris Anastassiou, 1995

n New Edge Directed Interpolationn Xin Li and Michael T. Orchard, 2001

n Canny Edge Based Interpolationn Hongjian Shi and Rabab Ward, 2002

n Data-Dependent Triangulationn Dan Su and Phillip Willis, 2004

n Edge-Guided Interpolationn Lei Zhang and Xiaolin Wu, 2006

Sub-pixel Edge Localization

Sub-pixel Edge Localization

n Explicitly calculate edges in 3 x 3 window of image

n Ideal step edge assumption

n Calculating the parameters:

n Develop continuous space theory - projections onto an orthonormal basis

n Use discrete approximations to inner products.

A

B

New Edge-Directed Interpolation

New Edge-Directed Interpolation

n Classical Wiener theory to develop MMSE weighting scheme for interpolation

n Estimate high resolution covariances from low resolution image.

n y is the data vector, C is a matrix used to estimate the high resolution covariance matrix

Dark Pixels: Low Resolution LatticeRed Pixel: Pixel to be Interpolated in

Step 1Green Pixels: Pixels Interpolated in

Step 2

Canny Edge Based Expansion

Canny Edge Based Expansion

n First, expand image using bilinear or bicubic interpolation

n Run Canny edge detector on expanded image

n Determine if magnitude of gradient is larger vertically or horizontally at each edge pixel

n Modify pixels on either side of edge in vertical or horizontal direction

Data-Dependent Triangulation

Data-Dependent Triangulation

n For each set of four low resolution pixels, estimate edge as dividing pixels into two triangles

n Create an image mesh which stores the direction of each edge

n Use linear interpolation within triangles

Image Mesh

Edge Guided Image Interpolation

Edge Guided Image Interpolation

n More general triangulation technique

n Use directional variances to produce weighting scheme

n Perform interpolation using both triangles, fuse with weighting scheme

Comparison of MethodsComparison of Methods

Original Bilinear Sub-pixel Edge Loc.

NEDI Canny Edge Based DDT

Comparison of MethodsComparison of Methods

Original Bilinear Sub-pixel Edge Loc.

NEDI Canny Edge Based DDT

Comparison of MethodsComparison of Methods

Original Bilinear Sub-pixel Edge Loc.

NEDI Canny Edge Based DDT

Factor of Four InterpolationFactor of Four Interpolation

Original Bilinear

NEDI Canny Edge Based DDT

Algorithm ComparisonAlgorithm Comparison

Lena Gaussian Disc

Bilinear 32.42 39.58

SEL 33.09 46.04

NEDI 37.37 42.76

Canny 37.29 40.37

DDT 37.42 41.68

Edge Guided 37.37 41.68

PSNR Lena Gaussian Disc

Bilinear 0.287 0.314

SEL 2.047 3.026

NEDI 42.5 36.1

Canny 1.386 1.299

DDT 0.945 0.982

Edge Guided

1.124 1.230

Speed in Seconds

Regularity Preserving Image Interpolation

Regularity Preserving Image Interpolation

n High similarity between different wavelet scales in regions of low regularity

n Convergence of series of features across scales for edge detection

n Goal: Synthesize a new sub-band by extrapolating from rate of decay of features across known sub-bands

n Apply algorithm separably along rows and columns

Regularity Preserving Image Interpolation

Regularity Preserving Image Interpolation

Take Home MessageTake Home Message

n Higher cost methods can result in significant improvement

n Still room for improvement using low-cost algorithms

n Current wavelet techniques still have room for improvement

n Proposed Method: Edge-Directed Interpolation using Multiscale Geometric Representations

n Questions?