image fusion algorithms for hdr imaging changemdified

7/30/2019 Image Fusion Algorithms for Hdr Imaging Changemdified

1/51

IMAGE FUSION

ALGORITHMS FORHDR IMAGING

DINESH R

ATHUL SMANU ELDHOJITHIN KARUNAKARANARJUN M

1


2/51

What is HDR?

High Dynamic Range

Can store large variation ofintensity in an HDR image.

Photographs capturedusing ordinary cameras areSDR images.

Low range of intensityvariation can be depicted.

SDRStandard DynamicRange

2


3/51

Why HDR?

Clouds are visible.

But the rest of the part isdark.

3


4/51

Why HDR?

Cloud is washed out.

So an SDR image cantinclude all the details in ascene.

Human eyes can perceive10-3 to 10+5 cd/m2.

An SDR images dynamicrange is about 102.

4


5/51

How to make an HDR image?

SDRImage1

SDR Image1

SDR Image2

SDR Imagem

.

.

.

.

.

.

.

CombineHDR

Image

5


6/51

How to display HDR image

Normal devices cannot display luminance with high dynamicrange.

So an HDR image has to be remapped to an SDR image.

This is called tone mapping.

Tone mapping should ensure that the maximum details in the HDRimage is made available in the resultant SDR image.

6


7/51

How to display HDR image

Scene SDR Image1

SDR Image2

SDR

Image m

.

.

.

.

.

.

.

Combine

HDR

Image

Finalviewed

image

ToneMapping

Exposure1

Exposure2

Exposurem

7


8/51

How to combine?

Using Camera response function

Gradient method

Probabilistic Exposure method

8


9/51

Using Camera Response

Function

Scene SDR Image 1

Exposure time 1

Luminance value * Exposure Time = Exposure

SDR Image 1

Pixel value in SDR Image is afunction of exposure at that pixel.

The function is called the CameraResponse Function (CRF).

CRF is obtained from SDR imagesat different exposures.

9


10/51

Mimicked HDR

So another idea is togenerate an SDR imagedirectly from the input

images.

This SDR image is constructedin such a way that themaximum information iscontained it.

Also called mimicked HDRimage.

10


11/51

Probabilistic Exposure Fusion [1]

Two stages: 1. Scene detail extraction

2. SDR image synthesis

SceneDetailExtraction

VisibleContrast

VisibleGradient

11


12/51


13/51

Finding Visible Contrast

Consider a small square sized area within the image ofsize (m)*(m) with pixel (i,j) as its centre.

Find the mean of the luminance of the pixels in thatsquare.

Contrast of pixel (i,j) is the absolute difference betweenthe luminance of the pixel (i,j) and the mean of theluminance of the pixels in the square.

To have a better human like perception we multiply the

contrast with Weber coefficients.

If the resultant contrast is greater than the predefinedthreshold then the visible contrast of the pixel (i,j) isupdated with this value. Else it is considered to be zero.

13


14/51

Visible Contrast

Visible contrast for eachpixel.

Consider a window withpixel (i,j) as centre.

Visible difference betweenmagnitude of pixel andaverage luminance of

window.

14


15/51

Finding visible contrast

v(i,j)=T(c(IH,mean(i,j))IH(i,j)) c(IH,mean(i,j))IH(i,j)

v(i,j) : Visible contrast at pixel (i,j)

T(x)=1, x>=0.2 and T(x)=0, x


16/51

Finding luminance uI(i,j)

We have to find the visible contrast for the samepixel in all the images.

Find the maximum visible contrast for the pixel (i,j).

Update uI(i,j) with the luminance of the pixel thatgives the maximum visible contrast.

Also update the chrominance of our resultant

image with the chrominance of the pixels givingmaximum visible contrast.

16


17/51

Visible Contrast

17


18/51

Image with uI as the

luminance

Using uI(i,j) as the luminance we can synthesize anSDR image.

Image will have luminance which give maximumvisible contrast.

But there will be halos in the image due to gradientinconsistency.

18


19/51

19


20/51

Image 1 Image 2 Image 3

Sample images

20


21/51

SDR image synthesized from uI(i,j)

21


22/51

Visible Gradient

For all pixels (i,j) in the window

Gradients in x & y direction

VMAX = max(x_grad,y_grad)

If VMAX > Threshold

VISIBLE_GRAD = VISIBLE_GRAD +VMAX

VISIBLE_GRAD is the visible gradient for(i,j).

22


23/51

Visible Gradient

Fig. Visible Gradient Extraction [1]

23


24/51

Scene Gradient

For all pixels (i,j) in the window

Find VISIBLE_GRAD for all images.

Find the maximum value. Gradient of the corresponding pixel isscene gradient.

24


25/51

SDR Image Synthesis

For a given scene IH, the corresponding SDR image ISshould preserve the visible contrasts presented in IH

and suppress the gradient reversal between IH andIS.

IS should be selected in such a way that it maximizesp(IS/ IH).

25


26/51

SDR Image Synthesis

Fig 2: Probability model [1]

26


27/51

Probability model

27


28/51

Image Synthesis

28


29/51

SDR Image Synthesis

Fig. Luminance evaluation [1]

Simplifying the above equation we have

IS(x+1,y)+IS(x-1,y)+IS(x,y+1)+IS(x,y-1)-(4+C2/S

2)IS(x,y)

= Gx(x,y) - Gx(x-1,y) + Gy(x,y) - Gy(x,y-1) -

C2/S2uI(x,y)

29


30/51

SDR Image Synthesis

Solving the equation for all pixels (x,y) in the imagewe can generate the luminance values of the

resultant SDR image.

30


31/51

Image 1 Image 2 Image 3

Sample images

31


32/51

Output images

Window size=10 Window size=10 Window size=10

(c/s)2=1 (c/s)

2=0.008 (c/s)2=0.08

32

Output images


33/51

Output images

Window size=36 Window size=36 Window size=6

(c/s)2=0.08 (c/s)

2=0.008 (c/s)2=0.08

33


34/51

Gradient + Gaussian

smoothing [2] Divide the images into smaller

segments

Find the magnitude of gradient of

each pixel in the segment in x and ydirections.

Find the maximum of the x gradientand y gradient for each pixel in thesegment.

Find the value for all pixels in thesegment and add up. This would bethe amount of detail MD in thesegment of the particular inputimage.

34


35/51

Detail Extraction

Similarly find MD for thesame segment in theother input images.

One with themaximum detail isconsidered for theresultant imageprocessing.

Repeat this for all the

segments.

The image obtainedjust using this will havesharp transitions onthe borders of thesegments.

35


36/51

Gaussian Smoothing

So to avoid the sharptransition a 2 DGaussian curve can beused.

The 2 D Gaussian isused to smoothen theluminance of the pixelsin a segment.

Previously obtainedluminance value atpixel (i,j) is multipliedwith the Gaussianfunction value at thepixel to obtain theluminance of theresultant image.

36


37/51

Using Photomatix PRO [3] Using Gradient method [2]

37


38/51

FUTURE WORK

We used gradient method and probabilisticexposure fusion technique to fuse SDR images to

obtain HDR image. One of the chief limitation of all these fusion

technique is its requirement that the camera andscene be still between different exposures

If there are movements between different

exposures the HDR output image would be blurredwith double images and sometimes in worst casemultiple images.

38


39/51

So two main problem with fusion techniques are

Misalignment: camera motion that results inmisalignment of images that cause the combinedHDR image look blurry.

Ghosting: moving objects in scene while capturingimages ,will appear in different locations of thecombined HDR image causing double images

39


40/51

40


41/51

41


42/51

IMAGE REGISTRATION

Its practically impossible to not have motionbetween different exposures.

Image registration for mitigating effects of ghostingdue to misalignment and moving object in scene

So far no algorithms can completely undo theeffects of ghosting though MTB (Median ThresholdBitmap by Ward) method is an efficient solution

We first going to implement MTB method and laterhoping to come up with our own better algorithms.

42


43/51

Ghost mitigation techniques involves two process.

Ghost detection

Ghost removal

43


44/51

Ghost detection techniques: based on motiondetection in exposure sequence.

We can identify two types of motion in dynamicscene

Moving objects in static background.eg movingcars and people

Moving back ground with static or dynamic objectseg; windblown leaves or waves

44


45/51

METHODS FOR MOTION

DETECTION

Variance based ghost detection

Entropy based ghost detection

Prediction based ghost detection

Pixel order relation

Multilevel based ghost detection

Bitmap based ghost detection

45


46/51

GHOST REMOVAL TECHNIQUES

Keeping a single occurrence of moving object:

In this method we are going to apply multiple

exposure fusion in ghost free region while selectingsingle reference exposure in ghost affected area

Removing all moving objects: It would be desirableto have no moving objects in the final HDR.

Here the simple approach is to discard exposures

that are affected by ghosting in each location.

46


47/51

47


48/51

48


49/51

Depending on users interest moving object couldbe either completely removed or kept at a fixed

position High contrast movements (moving object with

different back ground)could be correctly detected

Low contrast movements are difficult detect andremove.

49


50/51

We are hoping to come up with our owntechniques for image fusion and image registration by

next semesterWe will code these algorithms in c language and

produce a real product for HDR image production

50


51/51

References

[1] Mingli Song, Dacheng Tao, Chun Chen,

Jiajun Bu, JieboLuo and Chengqi Zhang,

Probabilistic exposure fusion", in IEEETransactions on Image Processing, vol. 21,

no. 1, pp. 341-357, January 2012.

[2] Andras Rovid, Annamaria R, Varkonyi-Koczy ,Takeshi Hashimoto, Szilveszter Balogh, YoshifumiShimodaira, Gradient Based Synthesized

Multiple Exposure Time HDR Image, inInstrumentation and Measurement TechnologyConference Proceedings, pp. 1-6, 2007

[3] www.hdrsoft.com

51

image fusion algorithms for hdr imaging changemdified

Documents