Image Fusion

Prepared by:

Rushabh P Jhaveri (15)

Introduction

Developments in the field of sensing technology Multi-sensor systems in many applications such

as remote sensing, medical imaging, military, etc. Result is increase of data available Can we reduce increasing volume of information

simultaneously extracting all useful information?

Basics of Image Fusion

Aim of image fusion is to reduce the amount of data retain important features and create new image that is more suitable for the

purposes of human/machine perception or for further processing tasks.

Single sensor image fusion system

Sequence of images are taken by a sensor Then they are fused in a image It has some limitations due to capability of sensor

Multi-sensor image fusion system

Images are taken by more than one sensor Then they are fused in a image It overcomes limitations of single sensor system

Fusion Categories

Multi-view fusion Images are taken from different viewpoints to

make 3D view

Multi-modal fusion

Multi-focus fusion

Multi-modal Fusion

N

M

R

SPECT

Fused image

Multi-focus fusion

Fused image

System level consideration

System level consideration

Three key non-fusion processes: Image registration Image pre-processing Image post processing

Continued.

Post-processing stage depends on the type of display, fusion system is being used and the personal preference of a human operator.

Pre-processing makes images best suited for fusion algorithm.

Image registration is the process of aligning images so that their details overlap accurately.

Image registration

Fields of view, resolutions, lens distortions and frame rates cannot be expected to match.

In all application fundamental problem is same; to find mapping between the pixels (x, y) in one image and the pixels (u, v) in another.

Straightforward geometric translation or rotation is the simplest technique.

Affine, polynomial and projective transformations are more advanced global approaches.

Methodology

Feature detection Algorithm should be able to detect the same features

Feature matching Correspondence between the features detected in the

sensed image and those detected in the reference image is established

Transform model estimation Type and parameters of the mapping functions are

chosen Image resampling and transformation

The sensed image is transformed

Example

How to register these two images?

The user specifies and pairs points.

Methods of Image fusion

Classification

Spatial domain fusion Weighted pixel averaging Brovey method Principal component analysis (PCA) Intensity-Hue-Saturation (IHS)

Transform domain fusion Lapacian pyramid Curvelet transform Discrete wavelet transform (DWT)

Weighted pixel averaging

Simplest image fusion technique F (x, y) = WA * A (x, y) + WB * B (x, y)

Where, WA ,WB are scalars

It has an advantage of suppressing any noise present in the source imagery.

It also suppresses salient image features, inevitably producing a low contrast fused image with a ‘washed-out’ appearance.

Pyramidal Method

Produce sharp, high-contrast images that are clearly more appealing and have greater information content than simpler ratio-based schemes.

Image pyramid is essentially a data structure consisting of a series of low-pass or band-pass copies of an image, each representing pattern information of a different scale.

Flow of pyramidal method

G1

G1 to GN

Ek

source image G 0convolution withgaussian kernal k

sub-sampling repeating until GN

duplicating each row & column of image G k+1

convolving with kLk=Gk - E krepeat until L N-1

Discrete Wavelet Transform method

It represents any arbitrary function x (t) as a superposition of a set of such wavelets or basis functions -mother wavelet by dilation or contractions (scaling) and translation (shifts)

Advantages of DWT in Image Fusion

Well suited to manage the different image resolutions. Allows the image decomposition in different kinds of

coefficients. Coefficients coming from different images can be

appropriately combined to obtain new coefficients. Final fused image is achieved through the IDWT, where

the information in the merged coefficients is also preserved.

2 level DWT of each image Low frequency sub-band is chosen based on the combined

edge information in the corresponding high frequency sub-bands.

Mean and standard deviation over 3 × 3 windows are used as activity measurement to find the edge information.

Final fused image is obtained by applying the inverse DWT on the fused wavelet coefficients.

Algorithm

Results

MMW Visible Fused

IR Visible Fused

Applications of Image Fusion

Medical image fusion

Helps physicians to extract the features from multi-modal images

Two types-structural (MRI, CT) & functional (PET, SPECT)

MRI-T2 PET Fused

Remote sensing

Remote sensing systems measure and record data about a scene.

Powerful tools for the monitoring of the Earth surface and atmosphere

Different types of images are taken by different sensors but multi-spectral and multi-polarization images are most important because they increase the separation between the segments.

So what is the requirement of image fusion in remote sensing?

Objectives of image fusion in remote sensing

Improve the spatial resolution. Improve the geometric precision. Enhanced the capabilities of features display. Improve classification accuracy. Enhance the capability of the change detection . Replace or repair the defect of image data. Enhance the visual interpretation.

Example

PAN (1 m) Color (4 m) Fused

Conclusion

Thank you