image fusion algorithm for multi-focus single sensor...

ARAB ACADEMY FOR SCIENCE, TECHNOLOGY AND MARITIME TRANSPORT

(AASTMT)

College of Computing & Information Technology Department of Computer Science

IMAGE FUSION ALGORITHM FOR MULTI-FOCUS SINGLE SENSOR IMAGES BASED ON COMPLEXITY

MEASUREMENTS

By

Dina Yehia Abd EI-Haleem E)-Gammal

A dissertation submitted to AASTMT in partial fulfillment of the requirements for the award of the degree of

MASTER of SCIENCE in

Computer Science

Supervisors

Prof. Dr. Mostafa Abd EI-Aziem Mostafa.

Prof of Computer Science Arab Academy for Science and Technology &

Maritime Transport

June - 2011

A. Prof. Dr. Gouda Ismail Salama.

Assoc. Prof of Computer Science Military Technical Colleague

~~~ .,....~-

Arab Academy for Science, Technology & Maritime Transport

~ ..>#- u...)1 4Sji::..A ':;\.J ~llA~ ~ ~ .li u...)1 0:3.\ ~ ii.l).,ll ~I ii.lWI w~ .Ji ~ ~.J ~WI ~411 .. I) ~ u...)1 0:3.\ W.J-Aa wl.J ' 4Spl ~ 4.J.l 4S1 ~ J.".......n

. WWI ~llA~ ~I-.. Ijil ii.).J~ :.!.....411

~, ~, ~ ~ li:!~ : f""''J I ............................... ....................... '!! :::~,J,).e4. ~... : ~ jill ........................... ~.:~! ... /.'].I .. ~ ...... :~.)\..jJ1

DECLARATION I certify that all the material in this thesis that is not my own work has been identified, and that no material is included for which a degree has previously been conferred on me.

The contents of this thesis reflect my own personal views, and are not necessarily endorsed by the University.

(Name) Dina Yehia Abd EI-Haleem EI-Gammal (Signature) ..... a Len ala" ~ ............................................... . (Date) .................................... 1\.L~.'.~all ........ ............... .

.~~&l .• .;-aytl

cairo· MisrEiGedida blanch ~""~'."""~'" Alexandria· Main Campus

P.O.Box 6IFI Latal<io aI: (+96341) 210045 (JX:(+96341) 45.."977

Aswon-Sadat Rood- PO.Box 11 As'#/<r1 Tel: (+2097) 2332845/2332843

Fox:( +2(97) 2332842

23 Doctor Sobi<y 51 P.O.Box 2033· ElhOrrio Tel: (+202) 37481593/33365491 EI Mashir Ismail St.-behind Sheraton Bldg

Fox:(+202) 33365492 Tel (+202) 22685616 / 22685615 Fox:( +2(2) 22685892

P.o.Box 1029 - Miami Miami Tel (+203) 5565429 / 5481163

Fox:( +203) 5487786/5506042 Abukir Tel: (+203) 5622366/5622388

Fox:( +2(3) 5610950

www.oostmt.org

Arab Academy for SCience. Technology & Maritime Transport

DECLARATION

We clarify that we have read the present work and that in our opinion it is fully adequate in scope and quality as dissertation towards the partial fulfillment of the Master Degree requirements in

Specialization :computer Science

College of Computing and Information Technology (AASTMT)

Date June - 2011

Thesis Title

"Image Fusion Algorithm for Multi-Focus Single-Sensor Images Based on Complexity Measurements"

Submitted By

Dina Yehia Abd EI-Haleem EI-Gammal

Supervisor (s):

Name: Prof. Dr. Mostafa Abd EI-Aiem Mostafa

Position: Vice Dean of college of Computing & Information Technology for Teaching and Students,

Arab Academy for Science and Technolo'''lL---

Signature:·········· .. ····oIl·~;;.oi~ ...

Name: Assoc. Prof. Dr. Gouda Ismail Salama.

Position: Associative Professor of Computer Science, Military Technical Colleague.

Signature: .....• ~ ........•...

Examiners: -

Name: Prof. Dr. Reem Mohamed Reda Bahgat

Position: Dean of Faculty Compute and Information Technology, Cairo University.

Signature:··· .. ·~~········~··4······· Name: Prof. Dr. Ismail Amr Ismail

Position: Dean of Faculty Compute and Information Technology, Misr International University.

Signature: .....•.•.. ~ .••• ~ ....

p O,Box 869 LatoklO Tet: (+96341) 210045 Fax:( +96341) 453977

AsY.,'or-Saoo+ ROOd- P 0 Bex : 1 As ..... o:-; Tel (+2097) 23328451 2332843

Fox:( +2097) 2332842

23 Joe;)' $cOl(', S'" Tel: (+202) 37481593/33365491

Fox (+202) 33365492

www,aostmt.org

i~ .. \,~U~~_;;..,p-AWJI Cairo· M;sr EI Gedida branch

POBox 2033 - Elhomo EI Mosh,r Ismail St -/Jehina Sl'.eratcn Bldg

Tel (+202) 22685616 1 22685615 Fox,( +202) 22685892

~'yI"""""I-&....:u~'Yl Alexandria - Main campus

POBox 1029 - Miami Miami Tel' (+203) 556542915481:63

Fox (+203) 5487786/5506042 Abukir Tel (+203) 5622366/5622388

Fox:( +203) 5610950

ACKNOWLEDGEMENT

This thesis would have remained a dream had it not been for Allah who has

made me being well. I want to grant this dream to the 25th January martyrs, who paid

their lives for our country Egypt, irrigating its lands with their blood to allow us reap

the fruits of freedom, equality and social justice.

It is with immense gratitude that I acknowledge the support and help of Prof.

Dr. Mostafa Abd EI-Aziem Mostafa. I consider it an honor to work with him. This

thesis would not have been possible had it not been for the guidance and patience of my

advisor Assoc. Prof. Dr. Gouda Ismail. Without his insights, feedbacks, and

encouragement, this thesis would not have been a success. I share the credit of my work

with Eng. Mohammed Seyam, assistant lecturer in Infonnation Systems department,

Faculty of Computers and Infonnation, Mansoura University.

I am indebted to my parents, my family, my friends and my classmates for their

emotional support. They have shown the greatest care and patience which I truly

appreciate.

Abstract

Photographers have been developed to be small and equipped with a focus.

However, they suffer from limited lens capabilities. Objects, in multi-focus images for

the same scene, lie in different depths. Capturing such images will make some of these

objects focused while the others are out of focus or blurred. Acquiring images that

contain objects at different distances in focus requires advanced hardware capabilities.

lIDs makes people take many images of the same scene in different focuses to get all

objects in focus. lIDs leads to the terminology Image Fusion, which extracts

perceptibly clearer images from a set of distorted images of the same scene. The fused

image contains common features and important information in all input images without

noise.

lIDs thesis compares six algorithms, which are Simple Average, spatial pixel

base algorithm, Discrete Wavelet Transform with four different fusion rules, multi scale

decomposition pixel-based algorithms, and Spatial Frequency, region-based algorithm.

Utilizing two measurements, which are the mutual information (MI) and the mean and

standard deviation (SO), the Spatial Frequency image fusion algorithm is the superior

of the six compared algorithms. The differences between these algorithms in the MI are

up to about 3.6%, almost the same, 0.44%, 1.03% and 0.45, respectively, and in the SO

up to about 13.07%, almost the same, almost the same, 0.28% and 0.14%, respectively.

This thesis introduces a novel algorithm for multi-focus single-sensor image

fusion based on complexity measurements. The main idea is to convert each N-bit input

image into N binary images and then divide these binary images into a set of blocks.

Then, using one complexity measurement, weights are given for each block in each

binary image of each input image. Finally, the fmal fused image is acquired according

to these weights. The proposed algorithm utilizes five complexity measurements, which

are: Length of Black and White Border, Run-Length Irregularity, Border Noisiness,

Transition Density and 4-Connectivity.

11

Utilizing the same two quality measurements, which are Mutual Information

(Ml) and Mean and Standard Deviation (SD), the results show that the proposed

algorithm using Length of Black and White Border and the 4-Connectivity complexity

measurements are almost the same or better than the Spatial Frequency image fusion

algorithm. These algorithms are better than the Border Noisiness in the Ml and SD up

to about 0.74% and 2.95%, respectively. The differences between the Border Noisiness,

Transition Density and the Run Length Irregularity in the MI up to about 0.91 % and

0.31 %, respectively, and in the SD up to about 1.24% and 0.57%, respectively.

iii

Table of Contents

Title Page , . , ,

List of Figures ....................................................................................................... vii I-

List of Tables ....... ..... ......... ....... .... .. ....... .................. ...... ....... .... ... .... .. .. .......... .......... x

Lists ...................................................................................................................... xiii

CHAPTER 1: INTRODUCTION

1.1 Introduction ................................................................................................. 2

1.2 Motivation .. ........... .. ..... ... ....... .... .. .. .... ... .... ................. ........... ..... ...... ... ........ 3

1.3 Problem Formulation For Image Fusion ..................................................... 4

1.4 Proposed Solution ......... .. ........ .... .... .. .. ...... ........ .... ........ ...... .. ...................... 5

~. ,, 1.5 'Thesis Organizati0ll. .................................................................................... 5

CHAPTER 2: IMAGE FUSION: AN OVERVIEW

2.1 Introduction ................................................................................................. 8

2 .2 Image Fusion ... ....... ...... ........................ .... .. .. .... ........................... ....... ......... 8

I ' 2.3 Pixel-Based bnage Fusion ......................................................................... 11

2.3.1 Spatial Image Fusion Algorithms .......................... ...... ...... .. ........... 11

2.3 .1.1 Simple Average .................................................................. 11

2.3 .1.2 Simple Greatest Pixel .. ............ ................ ...... .... .. ............... 12

2.3.1.3 Computationally Efficient Pixel-level Image Fusion , ---i

(CEMIF) ......................................................................................... 13

2.3.2 Multiscale Decomposition Algorithms ........ .. ....... .... .. ...... .... .. ........ 14 .-

2.3.2.1 Pyramid-Based Image Fusion ............................................. 15

2.3.2.2 Wavelets for Image Fusion .... ........ .. ............. .......... ... .... ..... 16 -

2.4 Region-Based Image Fusion ................................... .................................. 24

2.4.1 Simple Block Replacement.. ................. .. .......... .... ......... .. .. ....... .. .... 25 .-

2.4.2 Focal Connectivity ............................. ................. ........ .. .................. 26 .-2.4.3 Spatial Frequency ..... ... .. .... ........ .. ... ..... .. ............. ........ .. ................ .. 26

IV

F~!';'::'~ ~:i&. _ ~

:~t-j'?~-' :~: L ;"y~ ~,. .'~ • ':!':.<'~;:"'" '-<'t',>.. -.,

l':mattc? Fusion Algorithm .................. , .. __ :~ .>.{

2.4.3.2 Two Levels Fusion Region Based Multi-Focus Algorithm 29

~ •.. r~~~ .~ •. '" ,,,:-'.4.

::'7~~~on Fusion Algorithm of Multi-Focus Images .... 29 ,"

2.4.5 Image Fusion Using eural etworks ........................................... 30

~.i~;::~ 2.4.6 ~9i~g ............................................................................... 31 J".c.::..~ ,..-. ..

2.5 Hybrid Approach for Image Fusion .... ... ......... ........ .... .............................. 32

,.'. -2.6 bnage Registnition .................•.................................................................. 33

2.7 Image Fusion Evaluation and Quality Measurement ..................... ........... 35

.......... 0" .....: ... •••••••••• ~.;;.~,-:=-•••••••.•.••....••......•.•..•....••••.••••••••••.•...• 39

r.~:f""" ~.;-. ...... . .

'-' '~. T

CHAPT ER 3: IMPLEME TATIO OF IMAGE FUSIO ALGORITHMS

,~:: :.,,3.1 Introduction ..•.......•.................................................................................... 42

3.2 Experimental Results Using Real Images ..... .. .... .............. .......... .. ............ 43

3.4 Summary ................................................................ .......................... ......... 86

CHAPTER 4: PROPOSED IMAGE FU SION ALGORITHM BASED ON COMPLEXITYMEASUREMENTS

I~' ~:A 1 l,J;I:trO.4.uctiQ.D. .. ~ ...............•••...••.....•............................................................... 91

4.2 The Used Complexity Measurements ..................... .................... ...... ........ 92

4.2.1 Length of Black and White Border ........................................ ......... 92

4 .2.2 Run-Length Irregularity ....... .... .... ...... ......... .................................... 93

1~~i~(C-~- "~~ --'::--~-:""/'

~s ............................................................................. 94

4.2.4 Transition Density .... .. ... .... ....................... ..... .............. ................... 95

4.2.5 4-Connectivity ................................................................................ 96

4.3 The Proposed Algorithm ............ ...... ...................... .... ............ .... .. ... .... ...... 97

' .. 4.3.1 Introduction to the proposed algorithm .... .. .................................... 97

4.3.2 urnerical example .. ................................................... .. ........... ....... 99

1:io;.r~~.:<.5 4.1. ~ent#l ~t Using Real Testing Images ......................... 101 ~+-q'" - --&?"

4.4 Summary ............. ..... ....... .. .... ....... ......... ..... ... .. .. ... ................ ... .... ... .. ... .... 148

-][ ' I-t:A'Y 1:'J!j1l 5: CONCLUSION AND RECOMMENDATION FOR FUTURE l~1o',,~ WORK 1 52

v

5.1 Experimental Analysis .. ..... ... ..... ..... ...... ............ ... ... ... ... ... ... .... .. .... ... ....... 153

~.~~~ tV'OI~;,; ........................................................ ..

REFERENCES ... ... .. ... .... ............... ..... .. ... ... ....... .. ... .. ... ... .... .. .. .. .. .... .. ... .. ...... ... .. 156

VI

List of Figures

A Functional Block Diagram for the Proposed Solution .................. 6

Figure 2.1: Information Fusion Hierarchy .. .... .. .. ... ....................... .... ....... ... .. .... 10

Figure 2.2: Pixel-Based Image Fusion Hierarchy ............................................. 11

Figure 2.3: CEMIF Block Diagram ..... ... ... .. ............. .... .. ......... ......... ... ............. 13

Figure 2.4: Laplacian Pyramid Block Diagram ................................................ 16

Figure 2.5: Discrete Wavelet Transform Image Fusion Block Diagram .......... 18

Figure 2.6: DWT levels vs. DWPT levels ......................................................... 20

Figure 2.7: DWT Decomposition ... .. ...... .. ..... .. ....... ... ............. .................. .. .. ..... 21

Figure 2.8: DWPT Decomposition ...................... ............................................. 22

Figure 2.9: Block Diagram of the DWF for one image .... ..... .. ......................... 23

Figure 2.10: DWT vs. DT-CWT ......................................................................... 24

Figure 2.11: Region-Based Image Fusion Hierarchy ....... ........ .. ..... .... ....... .... ..... 25

Figure 2.12: The Basic SF algorithm Block Diagram ......................................... 28

Figure 2.13: Partition Fusion Algorithm of Multi-Focus Images .............................................................................. 31

~:;--~~~-::--::;O;-<;-:'~--=--;-:,....----;;-.....-;W:;-;-a-v-el;-e~t based Image Fusion Block

I----=-----~m:.~~~=~ .. = .. ~ ... ~ ........................................................................ 33

Figure 3.1: Disk Image Fusion Results ................................. ... .................... .... .45

Figure 3.2: The mouse image results ................. ................... ........................... .46

Figure 3.3: The Mickey image results ................... ...................... .. ........... .. ...... .48

Figure 3.4: The mobiles image results .............................................................. 50

Figure 3.5: The flash-USB image results .......................................................... 51

Figure 3.6: The circles image results ................................................................ 53

Figure 3.7: The stapler image results ............ .. ......... ............ ...... ....... .. .... ...... .... 54

J9~e 3.8: The CD-R image results ................................................................. 56

Figure 3.9: The remote-moderator image results ..... ... ...... .. ............... ...... ... .... .. 58

V ll

1·~ .. 3.IO: l'be stapler2 image results .............................................................. 59

Figure 3.11: The outdoor image results .... ......... ....... ... ..... ... .. ....... ... ... ...... .......... 61

I·'.ti .L:;Q. ...... 1!.li.e-:gfasB'eS ~e results •.............................................................. 62

-'>:r.""'"

Figure 3.13: The chargers image results .. .. ...... ................................................. .. 64

Pigure 3.14: The Walt Disney image results ....................................................... 66

Figure 3.15: The hearts image results ................ .. ..... ............ .... ................ ....... ... 67

KigQre 3.16: The W-aIt Disny2 image results ...................................................... 69

Figure 3.17: The blocks image results ................. ... .. ....... ... ... ..................... ........ 70

;.,M. 3.18: The<-flashmemories image results .................................................. 72

~

Figure 3.19: The medals image results . ......... ........... ... .............. ..... ... ................. 74

~.

3.20: The medals2 image results ............................................................. 76 -Figure 3.21: The charger image results ......................................... .... .. ...... .... ..... . 77

Figure 3.22: The key-gun image results .............................................................. 79

Figure 3.23: The bottles image results ... ................................................ ............. 80

1,.Fi...gp.re 3.24: The cards image results .................................................................. 82

Figure 3.25: The mobiles2 image results ... ................................ .... ........... ... ....... 84 .-

Figure 3.26: The 7up image results ..................................................................... 85

Figure 3.27: TheMI for the six implemented algorithms for the 25 image sets . 88 -

lli~3.28: The SD for the six implemented algorithms for the 25 image sets.89

Figure 4.1: urnerical Examples of Complexity Measurements ..... .......... .. .. ... 93 -

Figure 4.2: Binary Block Labeling for Blocks in Figure 4.1. ........................... 97

Figure 4.3: Input Images of the urnerical Examples. ... ....... .. .... .... ........... ... ... 99 -

Figure 4.4: The CGC of the First Input Image Shown in Figure 4.3 .... ...... .... 100

Figure 4.5 : The CGC of the Second Input Image Shown in Figure 4.3 .. ........ 100

~re-4.6. "<c'

Weight of.the Blocks for the Images Shown in Figure 4.3 .......... 101

Figure 4.7: The Final Fused Image ..... .... ................................ ............ ..... .. ..... 101 -

Figure 4.8: The disk image results ................................................. ................. 103

Vlll

Figure 4.9: The mouse image results . ......... ..................... ... ... ............. ........ .... 104

.10: The Mickey image results ............................................................. 1 06 .~"-'

Figure 4.11: The mobiles image results . .. ......... .... .. ... ............ ........ ..... .... ..... .... . 108

Pigure 4.12: The flash-USB image results ........................................................ 11 0

Figure 4.13: The circles image results . .......... ... ........................................ .... ... . 112

Bigare 4.14: The stapler image results .............................................................. 113

Figure 4.15: The CD-R image results ...... ..... ............ .. .. ....... .. ..... ... ... ..... ......... .. 115

Figure 4.16: The remote-moderator image results ............................................ 117

Figure 4.17: The stapler2 image results . .. ... .. ... ....... ... .... ... ... .. ... ....... ... ...... ........ 119 ,-

Figure 4.18: The outdoor image results ............................................................ 120

Figure 4.19: The glasses image results . ... .. ........ ......... .. .... ...... .... ... ......... .. .. ...... 122

Figure 4.20: The chargers image results ........................................................... 124

Figure 4.21: The Walt Disney image results ........ .. ................... ..... ..... ......... .. ... 126

Figure 4.22: The hearts image results ............................................................... 127

Figure 4.23: The Walt Disney2 image results ... ........................... ............ ..... .... 129

I 'lligUre 4.24: The blocks image results ..................... ......................................... 131

Figure 4.25: The flash memories image results ... .... .. ... ...... ... .......... .... .. .. ...... .. . 133 .~

Figure 4.26: The medals image results ............................................................. 134

Figure 4.27: The medals2 image results. .... .... ... .. ... ............. .... .... ..... .... .... ...... .. 136

Figure 4.28: The charger image results ...................... ............ .. ......................... 138

Figure 4.29: The key-gun image results ................ .. ........................ .. .............. .. 140

Figure 4.30: The bottles image results .............................................................. 141

Figure 4.31: The cards image results ..... ... .. .. .... .... .... .. ... ...... ... ... .. .... ....... .. ........ 143

Figure 4.32: The mobiles2 image results .......................................................... 145

Figure 4.33: The 7up image results ...................... ...... .. ..... ... .. .. .. .. ... ....... ........... 147

Figure 4.34: The MI Values of the Spatial Frequency Algorithm and the Proposed Algorithm of the 25 Test~g Im~es. .................. ..................... .. ... 150 .-

Figure 4.35: The SD Values of the Spatial Frequency Algorithm and the Proposed Algorithm of the 25 Testing Images. ................. .. .. .. .... ............ ... .. 151

IX

List of Tables

Dil;k Image Results ......................................................................... 44

Table 3.2: The Mouse Image Results ........ ..... .... ....... ...... .... .. .. .......... ..... .. ... ..... 45 r-----------~------Table 3.3: The Mickey Image Results ................. .... .. ...... .......................... ....... 47

Table 3.4: The Mobiles Image Results ............................................................. 49

Table 3.5: The Flash USB Image Results ........................................................ 49

Table 3.6: The Circles Image Results .............. .. ................. .......... ... ...... ........ .. 52

Table 3.7: The Stapler Image Results ............................................. ................. 55

Table 3.8: The CD-R Image Results ............... .. .......... ..................................... 55

Table 3.9: The Remote-Moderator Image Results ........................................... 57

Table 3.10: The Stapler2 Image Results ............................... ............ ................. 57

Table 3.11: The Outdoor Image Results ............................................................ 60 -----\

Table 3.12: The Glasses Image Results .............. ..... ............... ...... ...................... 63 --...--,1

Table 3.13: The Chargers Image Results ................. .... ...... ....... ......................... 63

Table 3.14: The Walt D isney Image Results .. .. .................................. ........ ........ 65 ---I

Table 3.15: The Hearts Image Results ............................................................... 65

Table 3.16: The Walt Disney2 Image Results .................................................... 68 ----I

Table 3.17: The Blocks Image Results ............................................................... 71

Table 3.18: The Flash Memories Image Results ....... .. ........ .. .......................... .. . 71 ----I

Table 3.19: The Medals Image Results .............................................................. 73 ------1

Table 3.20: The Medals2 Image Results ............................................................ 75

Table 3.21: The Charger Image Results ............................................................. 75 --I

Table 3.22: The Key-Gun Image Results .......................................................... . 78 -----I

Table 3.23: The Bottles Image Results .............................................................. 81

Table 3.24: The Cards Image Results ................................................................ 81

x

tw~~ The NifObileS2: 'll:blIge Results ........................................................... 83

Table 3.26: The 7Up Image Results . .... ...... .............. .... ... ... .... ... .. .. ... ..... ... ... ..... .. 83

~~:'.n,--~:- :r-·z,:", . . ' _.,- ..;.,~.". ~,- . .............................................. ................ 102

Table 4.2: The Mouse Image Results ... ..... .... ......... ... .. ..... ........... ...... ............. 105

Table 4.4: The Mobiles Image Results ........ ... .. .... ...... ..... .. ....... .... .... .... ... .. ... .. 107 ~""_,~;-;"J. ',. ,d -- " __ .... .:0';'

.• ~, "':< •. ,. ,.,.., '~."""'f'-............................................. 109

Table 4.6: The Circles Image Results . .... ........... ..... .. ..................................... 111

Table 4.8: The CD-R Image Results . .. .............. .. .. .... ............ .. .. .. .. ...... ... .. ...... 114

able 4..9: The Remo~Moderator Image Results ......................................... 116

Table 4.10: The Stapler2 Image Results . .. .... .. .. .. ...... .... ..... .. ... .... ........... .. .. ...... 118

r~ ..... ~ ... .t. :t...... The OUtdoor1mage Results .......................................................... 121

Table 4.12: The Glasses Image Results .............. .. ............ .. .............................. 121

Ir~ ~': .. ;aL_ The Chargers-Im.ag~Results . .................. .. .................................... 123

Table 4.14: The Walt Disney Image Results .............................. .. .. .. .......... .... .. 125

..... ~. , ~ ..... 4.15. 'Jibe Heatts Jrnage Results ............................................................. 128

Table 4.16: The Walt Disney2 Image Results .. ..... .... .. ..................................... 128

~Je 4.17' The Blocks Image Results ............................................................. 130

Table 4.18: The Flash Memories Image Results .............. .... ............... .. ........... 132

~ 4.19: The Medals Image Results ............................................................ 135

Table 4.20: The Medals2 Image Results . .. .............................. ...... .. ...... .... .. ..... 135

. rr"& ... , ...... 21! The Charger Image Results ........................................................... 137

Table 4.22: The Key-Gun Image Results ........ ........ .. .. ..... .... .... .. ...... .... .... .... .... 139

~_~~~ The J3o.tflj$Jpu~~ ;Rtsults ............................................................ 142

Table 4.24: The Cards Image Results ... ........ .. .............. ...... .......... .. ........... .. .... 142

'liable 4.25: The Mobiles2 Image Results .......................................... ............... 144

Xl

I Table 4.26: The 7Up Image Results ..... ............................................................ 146

xu

Lists

ComIJUtafij:>millyr~mt:~litPixeI-Level Image Fusion Pseudo Code.14 ,......----Laplacian Pyramid Pseudo Code ............ .... ........... .... .. ....... ....... ..... 17

~~~~~~~.~t~rmltsA)tQ:i;.liD,al~e Fusion Pseudo Code ............. 18

DWPT with Directive Contrast Image Fusion Pseudo Code ......... 21

S.imple Block ReplacementPseudo Code ...................................... 25

Focal Connectivity Image Fusion Pseudo Code ........................ .... 26

.. 31

List 2.12: Image Blinding Image Fusion Pseudo Code ..... .......... ................... 32

List 2.13: Hybrid Approach for Image Fusion Pseudo Code ......................... 34

List 2.14: Image Registration Main Steps ................ ........ ..... ............. ...... ...... 34

B~ Block Labeling ................................................................... 96

List 4.2: Pseudo Code for the Proposed Algorithm ................... ... .... .... ... ..... 98

Xlll

Publication

The work in this thesis was published in these papers:

1. Dina EI-Gammal, Gouda Ismail, Ibrahim F. Imam, 2010, "Comparing Fusion

Algorithms Using Single-Sensor Multi-Focus Images", Journal of AI Azhar

University Engineering Sector, Dec 2010.

2. Dina EI-Gammal, Gouda Ismail, Mostafa Abd EI-Aziem, 2011, "A Novel Image

Fusion Algorithm for Multi-Focus Single-Sensor Image Fusion Based on Image

Complexity Measurements", Egyptian Computer Science Journal, May 201l.

References:

[1] Z. Wang, D. Ziou, C. Armenakis, D. Li, and Q. Li. "A comparative Analysis of Image Fusion Methods." IEEE Trans. Geosci. Remote Sens., 2005, pp. 1391-1402.

[2] H. Bostrom, Sten F. Andler, Marcus Brohede, Ronnie Johansson, Alexander Karlsson, Joeri van Laere, Lars Niklasson, Maria Nilsson, Anne Persson, Tom Ziemke. Technical report, Topic: "On the definition of information fusion as a field of research." University of Skovde, School of Humanities and Informatics, 2007.

[3] Shuo.Li Hsu, Peng.Wei Gau, I-Lin Wu, and Jyh-Horng Jeng. "Region-Based Image Fusion with Artificial Neural Network." World Academy of Science, Engineering and Technology, 2009, pp. 156 - 159.

[4] Mrityunjay Kumar and Sarat Dass. "A total variation-based algorithm for pixel-level image fusion," in IEEE Transactions on Image Processing, 2009, pp.2137-2143.

[5] G. Corsini, M. Diani, A. Masini and M. Cavallini. "Enhancement of Sight Effectiveness by Dual Infrared System: Evaluation of Image Fusion Strategies," in ICT A, 2005, pp. 376-381.

[6] Anjali Malviya, S. G. Bhirud. "Image Fusion of Digital Images." International Journal of Recent Trends in Engineering, vol. 2, No.3, Nov. 2009, pp. 146 - 148.

[7] H. Hariharan, A. Koschan, and M. Abidi. "Extending Depth of Field by Intrinsic Mode Image Fusion," in Proc. IEEE 19th International Conference on Pattern Recognition ICPR, 2008, pp. 1-4.

[8] Jan Flusser, Filip ~Sroubek, and Barbara Zitov'a. Tutorial proposal, Topic: "Fusion in Image Processing." Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic, 2008.

[9] Sam Kirby. Summer Ventures in Science and Mathematics, Topic: "Image Fusion and Distributed Intelligence in Multi-Sensor Networks." Appalachian State University, Jul. 21, 2007.

[10] Dong, J.; Zhuang, D.; Huang, Y.; Fu, J. "Advances in Multi-Sensor Data Fusion: Algorithms and Applications." Sensors, pp. 7771-7784, Sept. 2009.

156

[11] Tijani Delleji, Mourad Zribi, and Ahmed Ben Hamida. "On the EM Algorithm and Bootstrap Approach Combination for Improving Satellite Image Fusion." International Journal of Signal Processing, vol. 4, No.1, pp. 85-94, Apr. 2007.

[12] N. Memarsadeghi, J. Le Moigne, and D. Mount. "Image fusion Using Cokriging," in Geosci. and Remote Sens. Symp. IGARSS, IEEE Int. Conf. , 2006,pp.2518-2521.

[13] J. Boehm. "Multi-image fusion for occlusion-free facade texturing." International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XXXV, part 5, pp.867-872, 2004.

[14] M. Ghantous, S. Ghosh, and M. Bayoumi. "A Gradient-Based Hybrid Image Fusion Scheme Using Object Extraction," in Proceedings of IEEE International Conference on Image Processing, 2008, pp. 1300-1303.

[15] J. Lanir, Stanley R. Rotman. "Comparing Multispectral Image Fusion Methods For A Target Detection Task." Optical Engineering, vol. 46, pp. 066402-1 - 066402-8, Jun. 2007.

[16] Tanish Zaveri, Mukesh Zaveri. "A Novel Two Step Region Based Multifocus Image Fusion Method." International Journal of Computer and Electrical Engineering, vol. 2, No.1, pp. 1793-8163, Feb., 2010.

[17] Susmitha Vekkot, and Pancham Shukla. "A Novel Architecture for Wavelet based Image Fusion." World Academy of Science, Engineering and Technology, vol. 57, 2009.

[18] E. F. Canga. "Image fusion." MSc thesis, University of Bath, UK, 2002.

[19] Z. Zhang and R. Blum. "A Categorization of Multiscale-DecompositionBased Image Fusion Schemes With A Performance Study For A Digital Camera Application," in Proceedings of the IEEE, 1999, pp. 1315-1328.

[20] Harishwaran Hariharan, Andreas Koschan, Mongi A. Abidi. "Multifocus Image Fusion by Establishing Focal Connectivity," in ICIP (3), 2007, pp. 321-324.

[21] S. Nikolov, P.R. Hill, D.R. Bull, C.N. Canagarajah. "Wavelets for Image Fusion," In A. Petrosian and F. Meyer, editors, Wavelets in Signal and Image Analysis, from Theory to Practice. Kluwer Academic Publishers, 2001.

[22] Hui Li, B. S. Manjunath and Sanji K. Mita. "Multi-Sensor Image Fusion Using the Wavelet Transform." IEEE, 1994, pp. 51-55.

157

[23] R. Balasubrarnanian, Gaurav Bhatnagar. "A new Contrast based Image Fusion Using Wavelet Packets," in the procedding of IEEE Conference on Applications of Intelligent Systems (AIS 2008), 2008, pp. 141-145.

[24] Gaurav Bhatnagar and Balasubramanian Raman. "A New Image Fusion Algorithm Based on Directive Contrast," in Electronic Letter on Computer Vision and Image Analysis, Vol. 8, No.2, pp. 18-38,2009.

[25] Paul R. Hill, David R. Bull, Cedric Nishan Canagarajah. "Image Fusion Using a New Framework for Complex Wavelet Transforms," in ICIP (2), 2005, pp. 1338-1341.

[26] Shutao Li, James T. Kwok and Yaonan Wang. "Combination of Images with Diverse Focuses Using the Spatial Frequency," in Information Fusion, 2001, pp. 169 - 176.

[27] Jun Kong, Kaiyuan Zheng, Jingbo Zhang, Xue Feng. "Multi-Focus Image Fusion Using Spatial Frequency and Genetic Algorithm." International Journal of Computer Science and Network Security, VOL.8 No.2, pp. 220-224, Feb. 2008.

[28] Dheeraj Agrawal, Dr.Al-Dahoud Ali, Dr.J.Singhai. "A Modified Partition Fusion Algorithm of Multifocus Images for Improved Image Quality." UbiCC Journal, vol. 4 No.3, pp. 658-663, 2009.

[29] Li Shutao.Kwok J T.Wang Yaonan. "Multifocus Image Fusion Using Artificial Neural Network," in Pattern Recognition Letters, 2002, pp. 985-987.

[30] Chao Wang, and Zhongfu Yeo "Perceptual Contrast-Based Image Fusion: a Variation Approach." Acta Automatica Sinica, vol. 33, No.2, pp.132-137, 2007.

[31] Goshtasby, A.. "Fusion of Multifocus Images to Maximize Image Information." Proceedings of the SPIE, vol. 6229, pp. 21-30, 2006.

[32] M. Ghantous, S. Ghosh, and M. A. Bayoumi. "A Multi-modal Automatic Image Registration Algorithm based on Complex Wavelets," in Proceedings of IEEE Int'l Conference on Image Processing (lCIP '09), 2009, pp. 173-176.

[33] G. Piella and H. Heijmans. "A New Quality Metric for Image Fusion," in Proceedings of the IEEE International Conference on Image Processing, 2003, pp. 173-176.

158

[34] M. Hossny, S. Nahavandi, and D. Creighton. "A Quadtree Driven Image Fusion Quality Assessment," in 5th IEEE International Conference on Industrial Informatics, 2007, pp. 419-424.

[35] Cvejic, N., Loza, A., Bull, D. and Canagarajah, N .. "A Similarity Metric for Assessment of Image Fusion Algorithms." International Journal of Signal Processing, vol. 2, No.3, pp.178-182, 2005.

[36] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. "Image Quality Assessment: From Error Visibility to Structural Similarity," in IEEE Trans. Image Processing, 2004, pp. 600- 612.

[37] Richard Dosselmann and Xue Dong Yang. Technical Report, Topic: "A Formal Assessment of the Structural Similarity Index." University of Regina, Saskatchewan, Canada, Sep. 2008.

[38] E. Kawaguchi and R. O. Eason. "Principle and Applications of BPCS-Steganography." Proceedings of SPIE: Multimedia Systems and Applications, vol. 3528, pp. 464-472, Nov. 1998.

[39] Yeshwanth Srinivasan. "High Capacity Data Hiding System Using Bpcs Steganography." MSc. Thesis, Texas Tech University, 2003.

[40] H. Hirohia. "A Data Embedding Method Using BPCS Principle with New Complexity Measures," in Proc. of Pacific Rim Workshop on Digital Steganography,2002, pp. 30-47.

[41] Ali Fahmy, Mohamed Shaarawy, Kamal EI-Hadad, Gouda Salama and Khaled Hassanain. "Image Steganography Based on Complexity Measures," in 4th International Conference on Electrical Engineering ICEENG, 2004, pp. 302 - 310.

[42] Deniz Aktlirk. "Visual Inspection of Pharmaceutical Color Tablets." Master Thesis, The Graduate School Of Natural And Applied Sciences Of Middle East Technical University, 2006.

159

,~4.!.fo '%13.07 ~I'p ,-:?.J4a-J1 ul~)'1 -' ~ji.J1 ~ -' H.~.~.Jil~

.~.Jil~ ,0/00.14 -' %0.28 ,~'-:Y.fo

A biil,J1 -' .J~I 1,.).a:i..J1 .J..,.......JI ~.l..l ~~ ~j.JI~ ~~yl l,iA ~.l.ij

Image -11 0"'Yil ~~.;al ..)±lk..JI ~ ~ ~I -' I,-*\'I ~ ~I~H

4 bijlA 1,.)..,......- J.S J..:l~ ~ ~j.J\~ ~~\'I I,..;-Sil\ ~ -' .Complexity

oiA ~ ~ lJ-A -' (binary images) ~\.:UII .J..,.......JIlJ-A .)~ ~} (input image)

J.5.l ~lti I,.J"""'- J.S ~ ~? J.S tJJij ~ ~~ ~ -' .~I-»\ .)} ¥\.:UI\ .J...,..-ll

-' .Image Complexity..J1 0"'4i\ ~\;"j .",JI ..)±l'-...JI ~i ~I~H 4...b..).Q I,.J"""'

(~I ~j.JI~1 ~~ -' .0U-,\' 1 o~ W-' ~~I 0.J...,..-lI0;.fo ~ "~i

:~ -' ,~ ..)±lk..a ~

Length of Black and White Border, Run-Length Irregularity, Border

Noisiness, Transition Density and 4-Connectivity.

O~\..jS 0"'4i\ lJ:l.A,.;"j ".JI 0.u~1 ~ ~I~H ,~1....:ill1 ~ lJ:1-:U ~-'

-' ~ji.J1 -' (Mutual Information) U.)'-:U-JI WL....a..,LuJ1 ~ -' ,w~j.JI.YJ1

(..;...:iiJ1 ~j.JI..,....=JI 0\ '(Mean and Standard Deviation) ,-:?.J~I ul~)'1

ts!lS:l ~lJ.j Uk.. ~ 4-Connectivity -,Length of Black and White Border ~I~H

~.l...J ~LS...JI .).) ~I ~ j.JI~ ~\~ ~ J-, . .., .... 1\ r...:i ~I ~\...j,jjl

o:U .J .4...l)..i.JI 4.:i....JI w~j.JI.YJI ~i ~ ,-:?~I .J '(Spatial FrequencY).J...,..-l1

ul ~)'I .J ~.b'-:U-JI w~..,LuJI ~ Border Noisiness -11 LJA J,...a.!i w~ j.J 1 y:J1

Border ~ l5.J~1 .J .~Jil~ '%2.95 .J 0100.74 ~I~ ,-:?)~I.h...,ji.JI.J

4_b'-:U-JI wt-~I ~ Run Length Irregularity .J Transition Density 'Noisiness

~I~ ,-:?.J4-J1 ul~)'I.J~ji.J1 ~ ,~jiJ~ ,0.31 -' %0.91 ~I'p

.~.Jil~ ,0.57 .J %1.24

. ·W \ . ~<I ~, ~. ~" .<-:1 - --:11 ~. i ~ -L-: (.)-4 c..F ~ J ."..J.)-H o~ J o~ U....,......... ~~ ,,~ '-J..J~

~~YI ~ ,~I ud:J ..J>.JI o~~1 ..J~\ ~ .o~J~1 wL...~1 W~~J

WYloiA' . L _.. . _II oiA 1': .bt..:9:il1 'Ulu ~ ~l...Ji. ~ ~ ~ lJ-4 ~ ~ ..J~ ~ '-F. J . . '-r?

.blUl) ,.db'JJ .,,~ Ji ~IJ yP ~ -..?-'il ~\ u~ ~~~IJ

wl~ o--*i rl~) ~) ~ A.-:uY. ~~i ~ ~~i ~ (.j~ ~I ..J~I

~LLL .1:.'. -'1 ... ~~ I - -'1' .l.J~I..b:ut L:JI L - ·liA A.....ai9:i...G W~~\ , .~ ~..J~ lJ-4 ~ l.>'l ~ J . ~,

~~ elb '04 j~~) ..s~i ~ .C~~ ~~YI ~ )~'1 ~ A..;..JY.

~ A.A~I ..J~1 LJ.A ~~ 0A ~IJ OJJ-'--<"' U"'\';"'''J (.jill J ..Jy..::JI

~ ~t.A wl.A~ J A.S~ ~~ ~ ~~I o j.,.....-1 I ..s~ J .~I

<I...J .. ~ • J ·1.:.~·1I .. ~·,IA..l;..~1 ·-11 . ~~ UJ ~ U""""" ..J~

Simple) ~I ~ji..JI ~ J 'w4-aj)~ ~ ~ ~L....JI o~ U).:g:;

spatial pixel-based ) ~I ~ .La» 4 ~~ rj..Jl~ ..J--A J '(Average

(Discrete Wavelet Transform) ~ • bCi,,JI w~yJI J.:!~ ('" j..Jl~ J '(algorithm

oJ~1 wU1..hi\1 ~ w4-aj..JI~ LJ.A.Yo J ,~ ~J .lCI..,g C') ('"I~~

~J~I J '(multiscale decomposition pixel-based algorithm) ~I ~ o)..,oaJ- 411

region-based ) ~u..JI ~ ~ rj..JI-'p J-A J '(Spatial Frequency) ~lS.JI

4w4-aj..JI.>=J1 ;;~US l.>'lYil ('"~ ~I ~1."...J1 0A ~) ('"I~~ J .(algorithm

(.j..J~1 wl~'11 J ~ji..J1 J (Mutual Information) 4 .. .b~1 w~~1 W J

~JJ ~lS.JI JJ~I ('"j..JI-'p Ui ~w.J1 ~ji '(Mean and Standard Deviation)

~ w4-a j..JI.>=JI oiA ~ ~J..;ill . A.,j ).:i.J\ A..:l....JI w~ j..JI.>=JI ~i .Yo ..Jy..::JI

,0/00.45 J %1.03 ,%0.44 ,~~..fo '%3.6 ~I~ A. ... b\+l.JI w~~1

,':'(3b A:ademv ie,' SCienCE, Tecnnology & Maritime Transport

'I ' u-"'""1

" oJla...l1 \L.l1 I~u ',. - 'I ' ~ ~I ' '. ,'\ oJJll.JI ,., -'I ~ . I ' " ~ f' .. J~ ~ J ~ ..;..y- 1:,,- f'JJ ~


) ~.,Jl1 ~ J:c.t.........) • .:l~ ,.:l,i :{'-"'JI

~.fi..-lI4il1 ~I- ~I.:JI r~.lc.L...... ~u..i :~yl

: u.,..s.:....J1 ol.1.....~1

~ l....<:..J.l.A:..o r=.J ,-",i :{'-"'JI

• .JA~I"""'~ - ":"'L.~I-, ..:..,t,w1..:J1 ~.l:!= :~"ll

0::'-V/<F'--t-)~.,Jl1 J!ct.........) J.>= J:c.t.........) ,.:l,i:{,-,,'JI ~ J.lli .J-=A ~~ - '::'L.."k-JI J '::'4o,.1.:J1 ~ ~ :~ yl

'-i~1 J.lllIJ ~.;l~IJ~.;.--la.1l ~yJl ~Jl...S~1 ~L..;l-ll ~.;l~ J ~t;.....b.!1 ~

~WI r..,k ole\........ ~u.....i ~fi..-JI ~I ~I

J\$.\

u\y!.!

...,.,.....WI r..,k .:.u.....i I.,?~I ~I.J ~)ys:;.lI.J r..,h1! ~y.ll ~..l\sYI

(

(

~'i"b A,aje-n.y ier SCience, Technology & Mantime Transport

'1 . i.J~

"oJia..ll \Lli I~u ·'·-'I·~~I ····'Io~I ···-'Ift..4..I • I' " ~ f" .• .)...-:- 1.1""""":' ..J .»t" .»-- 1:,- f".).) ..Po


) ~jill

~

~ (/ )~jill

~~<F"~)~jill

4...~ J.!c.l.......j o.lF' . .l.i :~':il A...!ft....o.ll ~I ~I - ~bll f'Yr- ~L..... ~u...i :~)I

:~\ oj:jL...~\

~ \....;".J .lA:...o~.J . .l.i :~':il 'Oy.lill ~4-- wt.."k-JI J w~bll4..;lS ~ :~."JI

J!c.L......,j J...>= J.!c.L......,j . .l.i:~':il ~ ".lll ~ 4-\.;. - wL..,h-Ji " u4--bll ~ ~ :~)I

($~I J..i.ll1.J ~..,l~I.J /".J--lall ~.;all ~JLS':i1 '::"L.....fl-.ll ~..,l~ .J .::..~bJl 4.#

..,.....WI ('..,Ie .lc.1........o jli....i

~fi-.ll ~I ~I

- WI ~

..,.....WI ('..,1c jli....i

L:?~I Jilll J 4;..)'pI J ('.,WJ ~~I ~.l\S'J1

DIS 77295 C2 006.42 GA-IM

IMAGE FUSION ALGORITHM FeR MULTI-FOCUS SINGLE SENSOR IM.A,GES BASED ON COMPLEXITY

1111111 77295

image fusion algorithm for multi-focus single sensor...

Documents