IEEE 2015 Conference on Computer Vision and Pattern
Recognition
DASC: Dense Adaptive Self-Correlation for Multi-modal and Multi-spectral Correspondence Seungryong Kim, Dongbo Min, Bumsub Ham, Seungchul Ryu, Minh N. Do, and Kwanghoon Sohn
Download the code at http://seungryong.github.io/DASC/
Introduction DASC Descriptor Experimental Results
Conclusion
Motivation • In multi-modal and multi-spectral image, conventional
descriptors often fail to estimate correspondence
Goal • To establish dense correspondence for those images
NIR
Visible
No-flash
Flash
Low exp.
High exp.
Blur
Sharp
Randomized Receptive Field Pooling • Unlike center-biased max pooling in LSS descriptor, the DASC descriptor incorporates
randomized receptive field pooling
Sampling Pattern Learning • Exploit supports vector machine (SVM)
with linear kernel • Features: • Energy function for SVM
Small Support Window Similarity • Adaptive self-correlation measure
Efficient Computation • Asymmetric weights • Reference-biased sampling pairs
• Approximated adaptive self-correlation
Middlebury Stereo Benchmark
Multi-modal and Multi-spectral Image Pairs
Background
Img
1
Img
2
color, gradient, structural similarity
RGB Image NIR Image
Matching Cost A Matching Cost B Matching Cost C
Challenge on Multi-modal Images • Nonlinear photometric deformation, e.g., gradient
reverses and intensity order variations
LSS descriptor DASC descriptor
Middlebury multi-modal SINTEL
1 2 2 2, , ,exp ( ) / 2m l m l m l rr d d
2( ) || || max(0,1 ( ))m mmv v y r
• Intuitions Center-biased pooling
sensitive to noise Randomness of BRIEF
LSS, bin ( )max ( , )
ii l j ld i j
LSS descriptor
DASC descriptor
, , , , ,( , ), ,i l i l i l i l i l id s t s t
, ,,
2 2, ,
( )( )
( , ){ ( )} { ( )}
s s t t s s t ts t
s s s s t t t ts t
f f
s tf f
, ,,
2 2, , ,
,
( )( )
( , )( ) ( )
i i i i j i ji j
i i i i i i j i ji i j
f f
i jf f
, 's s
, ,( ) ( , - ), i l i ltj i si i
• Robust estimation
( , ) max(exp( (1 ( , )) / ), )s t s t • The correlation is normalized
with norm of all
( , )s t
l
Img1 Img2
RSNCC
BRIEF
DAISY
LSS
DASC
RGB-NIR flash-no flash different exposure blur-sharp
• Robust novel descriptor called the DASC for multi-modal correspondence • Leverages adaptive self-correlation and randomized receptive pooling • Efficient computation with fast edge-aware filters
Limitation of Conventional Descriptors • Image gradient (SIFT) or intensity comparison (BRIEF)
cannot capture coherent matching evidence
Img1 Img2 ANCC BRIEF
SIFT LSS DASC Ground Truth Illumination Exposure
