hdr image construction from multi-exposed stereo ldr images
DESCRIPTION
HDR Image Construction from Multi-exposed Stereo LDR Images. Ning Sun, Hassan Mansour, Rabab Ward Proceedings of 2010 IEEE 17th International Conference on Image Processing September 26-29, 2010, Hong Kong. Andy { [email protected] }. Algorithm description. - PowerPoint PPT PresentationTRANSCRIPT
Ning Sun, Hassan Mansour, Rabab WardProceedings of 2010 IEEE 17th International Conference on Image Processing
September 26-29, 2010, Hong Kong
HDR Image Construction from Multi-exposed Stereo LDR Images
Andy {[email protected]}
2Intelligent Systems
Lab.
Algorithm descriptionTwo LDR images
with different exposures
Initial disparity map
Camera response function
Radiance maps of LDR
images
Refined disparity
mapHDR image
Main concept:
1. Multi-exposed stereo images are captured using identical cameras placed adjacent to each other on a horizontal line.
2. Stereo matching is then used to find a disparity map that matches each pixel in one image to the corresponding pixel in another image.
3. A subset of the matched pixels is used to generate the camera response function which in turn is used to generate the scene radiance map for each view with an expanded dynamic range.
4. The disparity map is refined by performing a second stereo matching stage using the radiance maps
3Intelligent Systems
Lab.
Imaging models
RI l eRI r
Pp n
nrn
n
nlnn pIcepIccJ
Gamma-correction model Polynomial camera response
Imaging models are used to determine the scene radiance from the measured pixel data
nn cJc minarg
Left image Right image
Scene radiance
Correction factor
Exposure ration between images
Exposure ration between images
Left image Right image
Scene radiance
4Intelligent Systems
Lab.
Computing the disparity map
NfEfEf SdFf,minarg*
Best disparity map
Dissimilarity term
Set of feasible disparities
Smoothing term
p
pp
ppd fpNCCfDfE ,1
p pNq
qps VqpNfE ,,,
Pixel dissimilarity Disparity smoothness
Used for initial disparity estimation
5Intelligent Systems
Lab.
Pixel dissimilarity
22 ~~
~~
,prrll
pWqprlrl
p
fpIwpIw
fqIqIwwfpNCC
pW - Search window centered on p
pf - displacement tw - Bilateral weight
2
2
2
2
2''
2exp
sd
pItItptw
Spatial smoothing Intensity smoothing
ReII logloglog' I’ - intensity in log space defined as:
0.146.2 rs
6Intelligent Systems
Lab.
Pixel dissimilarity
pWt
pWt
pWt
jpWt
ll tw
RtwR
tw
ItwII
loglog~
pWt
pWt
pWt
pWtr tw
RtwR
tw
RetwReI
loglog
loglogloglog~
7Intelligent Systems
Lab.
Disparity smoothness
max
2
, ,min, VffffV qpqpqp
p pNq
qps VqpNfE ,,,
2
2
2
2
2
2
2
2
2222exp,
r
bb
r
aa
r
LL
s
qIpIqIpIqIpIqpqp
NfEfEf SdFf,minarg*
Initial disparity and camera response
1. Minimize using graph cut algorithm
2. Compute polynomial coefficients for camera response function
0.164.2 rs
8Intelligent Systems
Lab.
Error correction
NfEfEf SdFf,minarg*
Minimize energy function one more time with different dissimilarity function
For valid pixels
R~Convert images to radiance space (results should be same for both images)
p
ppd fDfE
otherviseK
ffiffD
initialpp
pp,
,0
For erroneous pixels
rlppprlpp RRpWfCfpRpRfD ~,~,,~~
Hamming distance between pixels p and p+fp after applying
Census transform
9Intelligent Systems
Lab.
Input LDR images
10Intelligent Systems
Lab.
Disparity maps
Reference disparity map Initial disparity estimation Final map
11Intelligent Systems
Lab.
HDR images
12Intelligent Systems
Lab.
Experimental results
Image name Exposure Ratio RMSE Error Error pixels (%)
Statue 416
0.99430.9976
8.238.82
Dolls 416
0.84540.8591
4.775.58
Clothes 416
1.54591.1556
7.438.15
Baby 416
1.4321.4642
9.4210.13
13Intelligent Systems
Lab.
ConclusionsDisparity map computation algorithm is proposed
Proposed method is able to compute disparity between differently exposed images
Can deal with saturated regions in the image
Can be used for capturing motion scenes with different exposures
Disadvantages
- High computational costs
- Generated images are slightly blurred
- No rotation is considered
14Intelligent Systems
Lab.
Ideal image formation system
eLfI
Image brightness
Sensor response
Camera exposureCamera response function
Response = Gray-level
Irrad
ianc
e
L
I
BgBfL 1
Reverse camera response function
42
cos4
hdRE
From optics
Image radiance
Scene radianceFocal length
Aperture
Angle from ray to optical axis
EtL
Radiometric responseShutter speed
or
RkeL
Where
tde4
242 cos
1h
k
N
c
nn
n
Ic0
15Intelligent Systems
Lab.
Response function examples
Response functions of a few popular cameras provided by their manufacturers
I
L
16Intelligent Systems
Lab.
Graph-cut algorithm
1. Start with an arbitrary labeling f2. Set success := 03. For each label 2 L
3.1. Find f* = arg min E(f’) among f’ within one α-expansion of f
3.2. If E(f*) < E(f), set f := f* and success := 14. If success = 1 goto 25. Return f
17Intelligent Systems
Lab.
Census transform
If (CurrentPixelIntensity<CentrePixelIntensity) boolean bit=0else boolean bit=1
Input image 3x3 transform 5x5 transform