1

ECE/OPTI 531 – Image Processing Lab for Remote Sensing Fall 2005

Using Hierarchical WarpUsing Hierarchical WarpStereo for TopographyStereo for Topography

Dr. Daniel FilibertiDr. Daniel Filiberti

Fall 2005HWS Topography 2

Introduction

• Topography from Stereo– Given a set of stereoscopic imagery, two perspective

views of a three-dimensional object, we can determineelevation differences in the terrain using stereotriangulation.

– Stereoscopic parallax or disparity is the difference inposition of an imaged ground feature from one phototo the next overlapping photo.

• Parallax differences, the change in disparity dueto a change in relief of the terrain being imaged,are used to determine relative elevations togenerate a DEM

2

Fall 2005HWS Topography 3

Imaging Geometry

• Overlapping pair ofvertical aerialphotographs

• Parallax equation

hA= H !

B "f

pa

#h =H ! h

1

p2

#p

Fall 2005HWS Topography 4

Numerical Example

• H = 10,000 ft, B = 8 mi, f = 4 in, h = 30 ft

• This is a 1.41 pixel disparity difference on 1 footimagery

• Note that the air base, B, determines thesensitivity to changes in elevation

p =f !B

H "h=0.333 # 42,240

10, 000 " 30= 1.41

3

Fall 2005HWS Topography 5

Reference Photo (Cuprite, NV)

Fall 2005HWS Topography 6

Overlap Extraction

4

Fall 2005HWS Topography 7

Stereo Region

Reference Target

Fall 2005HWS Topography 8

Stereo Region (Cont.)

Reference Target

5

Fall 2005HWS Topography 9

Finding Disparity

• Types of Stereo Algorithms– Computer vision algorithms extract and match features

such as edges, contours, and shapes to find a coarsedisparity map on a non-uniform grid

– Correlation-based algorithms match a local areaaround a point (target template) into a larger searcharea to find a disparity at every point, producing adense and uniform grid of samples

Fall 2005HWS Topography 10

HWS Algorithm

• Hierarchical Warp Stereo (Quam, 1984)– Correlation-based approach– Uses multiresolution image pyramid to match from

coarse to fine spatial resolution– Disparities propagate as estimates to higher

resolutions, reducing the necessary search area(correlation window) size

6

Fall 2005HWS Topography 11

HWS Processing

• Matching results indisparity image which isexpanded and used as aninitial estimate at nextlevel

• Further processing mustusually be done to convertpixel disparities to a digitalelevation map

Fall 2005HWS Topography 12

0

1

2

3

Image Pyramid

• Level 0 is full spatial resolution image• Resolution decreases as level increases

– Reduction can be done in spatial or Fourier domain– Scale factor is typically 0.5

7

Fall 2005HWS Topography 13

Pyramid Construction

• Multiresolution Hierarchical Representation– Image pyramid generated by

– Where REDUCE() performs a downsample and filteroperation on the previous level

– Using a simple weighting function is common,

P/(i, j ) = W(m,n)P

l !1n= !N

N

"m= !N

N

" (2i +m,2j + n)

P/= REDUCE(P

l!1)

Fall 2005HWS Topography 14

Weight Selection

• Properties of a good generating kernel (Burt,1983)– Separable, W(m,n) = w(m)w(n)– Normalized, weights sum to one– Symmetric, w(m) = w(-m)– Equal contribution, all nodes at a given level must

contribute the same total weight to nodes at next level• Use a 5x5 Burt kernel

– Let W(0) = a, W(1) = W(-1) = b, and W(2) = W(-2) = c– Equal contribution requires a + 2c = 2b– Constraints satisfied when

• W(0) = a• W(1) = W(-1) = 1/4• W(2) = W(-2) = 1/4 – a/2

8

Fall 2005HWS Topography 15

Characteristic Functions

Fall 2005HWS Topography 16

• W1 in two dimensions0.0025 0.0125 0.0200 0.0125 0.00250.0125 0.0625 0.1000 0.0625 0.01250.0200 0.1000 0.1600 0.1000 0.02000.0125 0.0625 0.1000 0.0625 0.01250.0025 0.0125 0.0200 0.0125 0.0025

Equivalent Weighting Functions (a=0.4)

9

Fall 2005HWS Topography 17

Area Matching

• Define a match score operator,

– Improved performance in matching has been shownwhen the ACF is weighted by a Gaussian functionfavoring low disparity changes, or a smooth disparitysurface

– The ACF is a normalized, Gaussian weighted cross-correlation

C(i, j ) = ref (m,n) !tgt(m " i ,n " j )n#

m#

ACF(i, j ) =C(i, j )

ref 2 (m,n)n#

m#[ ]

1/ 2

Fall 2005HWS Topography 18

Match Location

• Best match point is found by subpixelapproximation– Fit a parabola to the ACF peak and its nearest

neighbors– Problem: this approximation generates ripple artifacts

when coupled with the image quantization• Match Confidence

– Issues• Disparity out of range• Multiple ACF peaks

– Anomaly Detection• After finding the ACF peak, estimate the distance between

the peak and center of mass of the ACF• Match is considered valid if the distance meets a threshold

10

Fall 2005HWS Topography 19

HWS Example

• Apply to Cuprite, NV stereo pair– HWS Parameters

• Correlation Window Size of 13x13 pixels• Search area of 17x17 pixels• Maximum disparity of +-8 pixels per pyramid level

– Anomaly detection used to mark holes– Holes filled using interpolation by bisection (cubic

spline)

Fall 2005HWS Topography 20

HWS Example (Cont.)

• Hand digitized contours are interpolated fortruth

11

Fall 2005HWS Topography 21

HWS Results

Truth HWS

Fall 2005HWS Topography 22

HWS Results (Cont.)

12

Fall 2005HWS Topography 23

Application: Shaded Relief

• Assumptions– Lambertian surface– Nadir view– No atmospheric scattering– No path radiance

Lsensor

= A!cosi +C

Fall 2005HWS Topography 24

Shaded Relief Equations

• Find cos(i) using surface gradient– E, N components from DEM– Rotatation for component along direction of solar

irradiance

• Take the normalized dot product of the twogradient vectors,

p!=f (x

i +1,y

i) " f (x

i "1,y

i)

2d

q!=f (x

i,y

i +1) " f (x

i,y

i "1)

2d

ps= !sin"

scot(

#

2! $

s)

qs= !cos"

scot(

#

2!$

s)

cosi =1+ p

!p

s+q

!q

s

1+ p!

2

+ q!

2 1+ ps

2

+ qs

2

13

Fall 2005HWS Topography 25

Shaded Relief

Aerial Photo,193°az, 34° zn

Aribitrary,140°az, 66° zn

Fall 2005HWS Topography 26

Application: HFM Fusion

14

Fall 2005HWS Topography 27

HFM Results

Fall 2005HWS Topography 28

Further Reading

• Text R.A. Schowengerdt, “Remote sensing, modelsand methods for image processing”, 2nd ed.– 3.9.7 Topographic Distortion– 6.5.1 Image Resolution Pyramids– 8.4.2 High-Resolution DEM and Hierarchical Warp Stereo– 8.5 Multi-image Fusion