automated image registration using morphological region of interest feature extraction antonio plaza...

Applied Information SciencesAutomated Image Registration Using Morphological Region of

Interest Feature Extraction

Antonio PlazaUniversity of Extremadura. Caceres, Spain

Jacqueline Le MoigneNASA Goddard Space Flight Center, USA

Nathan NetanyahuBar-Ilan University, Israel & University of Maryland, USA

Automatic Multiple Source

Integration

Prediction Models

Satellite, Aircraft and Field Data

Improved Data Sets

Validation & Verification

Feedback

Design of FutureIntelligent

Sensor Webs

Earth Science Data Integration

Applied Information Sciences

MultiTemp 2005 Jacqueline Le Moigne, 3

What is Image Registration ?

• Navigation or Model-Based Systematic Correction– Orbital, Attitude, Platform/Sensor Geometric Relationship,

Sensor Characteristics, Earth Model, ...

• Image Registration or Feature-Based Precision Correction– Navigation within a Few Pixels Accuracy– Image Registration Using Selected Features (or Control Points)

to Refine Geo-Location Accuracy

• 2 Approaches:(1) Image Registration as a Post-Processing (Taken here)(2) Navigation and Image Registration in a Closed Loop



Image Registration Challenges

• Multi-Resolution / Mono- or Multi-Instrument• Multi-temporal data• Various spatial resolutions• Various spectral resolutions

• Sub-Pixel Accuracy• 1 pixel misregistration=> 50% error in NDVI computation

• Accuracy Assessment• Synthetic data• "Ground Truth" (manual registration?)• Use down-sampled high-resolution data• Consistency ("circular" registrations) studies



Image to Image Registration

Incoming Data Image Characteristics(Features) Extraction

• Multi-Temporal Image Correlation• Landmarking• Coregistration

FeatureMatching

Compute Transform



Image to Map Registration

Input Data

Map

Masking andFeature Extraction

Feature Matching

Compute Transform



Multi-Sensor Image RegistrationETM/IKONOS Mosaic of Coastal VA Data

IKONOSETM+



Image Registration Components

0 Pre-Processing• Cloud Detection, Region of Interest Masking, ...

1 Feature Extraction (“Control Points”)• Edges, Regions, Contours, Wavelet Coefficients, ...

2 Feature Matching• Spatial Transformation (a-priori knowledge)• Search Strategy (Global vs Local, Multi-

Resolution, ...)• Choice of Similarity Metrics (Correlation, Optimization

Method, Hausdorff Distance, ...)

3 Resampling, Indexing or Fusion



Image Registration Subsystem Based on a Chip Database

LandmarkChip

Database

UTM of 4 Scene Corners Known from Systematic Correction

Correct UTM of

4 Chip Corners

Input Scene(1) Find Chips that Correspond to the Incoming Scene(2) For Each Chip, Extract Window from Scene, Using UTM of: - 4 Approx Scene Corners - 4 Correct Chip Corners(3) Register Each (Chip,Window) Pair and Record Pairs of Registered Chip Corners(4) Compute Global Registration from Multiple Local Ones(5) Compute Correct UTM of 4 Scene Corners of Input Scene



Image Registration Subsystem Based on Automatic Chip Extraction

UTM of 4 Scene Corners Known from Systematic Correction

Input Scene

(1) Extract Reference Chips and Corresponding Input Windows Using

Mathematical Morphology

(2) Register Each (Chip,Window)

Pair and Record Pairs of Registered Chip Corners (refinement step)

(3) Compute Global Registration

from Multiple Local Ones

(4) Compute Correct UTM of 4 Scene Corners of Input Scene

Reference Scene

Advantages:• Eliminates Need for Chip Database• Cloud Detection Can Easily be Included in Process• Process Any Size Images• Initial Registration Closer to Final Registration => Reduces Computation Time and Increases Accuracy.



Step 1: Chip-Window Extraction Using

Mathematical MorphologyMathematical Morphology (MM) Concept: • Nonlinear spatial-based technique that provides a framework.

• Relies on a partial ordering relation between image pixels.

• In greyscale imagery, such relation is given by the digital value of image pixels

Structuring element

Dilation

3x3 structuring elementdefines neighborhood aroundpixel P

Erosion

Max Min

P

Original image

Dilation

3x3 structuring element defines neighborhood around pixel P

Erosion

Max Min

P

Original image

Original image

Erosion

K K

Dilation

(4-pixel radius Disk SE)

Greyscale MM Basic Operations:



Step 1 (Cont.)

Structuring element

Binary Erosion

Structuring element

Structuring element



Step 1 (Cont.)

Structuring element

Binary Dilation

Structuring element

Structuring element



K

Greyscale Morphology: Combined Operationse.g., Erosion + Dilation = Opening

Step 1 (Cont.)




Mathematical Morphology• Scale-Orientation Morphological Profiles (SOMP): From Openings

and Closings with SEs=Line Segments of Different Orientations – SOMP = Feature Vector D(x,y) at each Pixel (various scales & orientations) – Entropy of D(x,y) = H(D(x,y))

• Algorithm:a. Compute D(x,y) for each (x,y) in reference sceneb. Extract reference chip centered around (x’,y’) with Max[H(D(x’,y’))], e.g.

256x256c. Compute D(X,Y) for each (X,Y) in search area input scene centered (e.g.,

1000x1000) around location (x’,y’)d. Compute RMSE(D(X,Y),D(x’,x’)) for all (X,Y) in search areae. Extract input window centered around (X’,Y’) with Min(RMSE)f. Return to step 2. until predefined number of chips is extracted




Mathematical MorphologyResults(Landsat-7/ETM+ Data - Central VA)

10 Chips Extracted from Reference Scene (Oct. 7, 1999)

10 Windows Extracted from Input Scene (Nov. 8, 1999)



Step 2: Chip-Window RefinedRegistration Using Robust Feature Matching

Reference Chip

Input Window

WaveletDecomposition

WaveletDecomposition

Robust Feature Matching (RFM)

UsingHausdorff Distance

MaximaExtraction

MaximaExtraction

Choice ofBest

Transformation

At EachLevel ofDecomposition{

• Overcomplete Wavelet-type Decomposition: Simoncelli Steerable Pyramid• “Maxima” Extraction: Top 5% of Histogram



Step 2: Robust Feature Matching Using Hausdorff Distance

• Search Transformation Space through Hierarchical Spatial Subdivisions

• Perform Monte Carlo Sampling of Control Points

• Compute Robust Similarity Measure- k-th smallest squared distance to nearest neighbors, i.e., partial

Hausdorff DistancePartial Hausdorff Distance:

Hk(A, B) = Kth a in A minb in B dist (a,b)

(1≤ k ≤ |A|; Kth is the kth smallest element of set; dist(a,b): Euclidean distance)

• Prune Search Space by "Range" Similarity Estimates

• Iterate and Refine on each Level of Wavelet Decomposition



• From each Local Registration, Window-Chip:• Corrected Locations of Four corners of Each Window• i.e.: for each chip-window i, pair correspondences:

– (UL_i_X1,UL_i_Y1) to (UL_i_X2,UL_i_Y2)– (UR_i_X1,UR_i_Y1) to (UR_i_X2,UR_i_Y2)– (LL_i_X1,LL_i_Y1) to (LL_i_X2,LL_i_Y2)– (LR_i_X1,LR_i_Y1) to (LR_i_X2,LR_i_Y2)

• Use of a Least Mean Square (LMS) Procedure to Compute Global Image Transformation (in pixels)

• If n chips, 4n points used for the LMS

=> Step 4: Use Global Transformation to Compute new UTM Coordinates for each of the 4 Corners of the Incoming Scene

Step 3: Compute Global Registrationfrom All Local Registrations



Table I Registration Results Obtained for Ten Chip-Window Pairs

From Two Landsat Scenes in Central Virginia

Chip-window

Rotation (deg)

Initial Shift x

Initial Shift y Adjusted Shift x

Adjusted Shift y

(a) 0.0 12.0 4.0 14.0 1.0

(b) 0.0 -1.0 0.0 13.0 1.0

(c) 0.0 2.0 -2.0 13.0 1.0

(d) 0.0 0.0 0.0 13.0 1.0

(e) 0.0 -1.0 3.0 13.0 1.0

(f) 0.0 59.0 1.0 14.0 1.0

(g) 0.0 -3.0 -3.0 13.0 1.0

(h) 0.0 0.0 0.0 13.0 2.0

(i) 0.0 36.0 4.0 14.0 2.0

(j) 0.0 2.0 -2.0 13.0 1.0

Results of Global RegistrationOn Landsat-7 VA Test Data



Conclusions

• Fully Automated System for Registration of Multi-Temporal Landsat Scenes of Any Size, Using Mathematical Morphology and Robust Feature Matching Techniques

• MM Chip-Window Extractor Can be Used with Any Other Registration Method

• Eliminates Need of Database

• Provides Close Initial Match => Follow-up Computations Faster and More Accurate

• Further Experimentation On-Going

automated image registration using morphological region of interest feature extraction antonio plaza...

Documents

image registration subsystem

image registration components

automated image registration

compute global registration

chip corners input scene

correct chip corners

usa slide

approx scene corners