2019-04-10

Scale Invariant Feature Transform

Han Sol Kang

ISL Lab Seminar

: SIFT

2019-04-10

Contents

2

Fundamental theory

SIFT

Introduction

Example

Summary

2019-04-10

Introduction

3

illumination illumination + Scale

illumination + Scale + Rotation illumination + Scale Rotation + Affine

2019-04-10

Introduction

4

David G Lowe

[2] Lowe, David G. "Distinctive image features from scale-invariant keypoints." International journal of computer vision 60.2 (2004): 91-110.

[1] Lowe, David G. "Object recognition from local scale-invariant features." Computer vision, 1999. The proceedings of the seventh IEEE international conference on. Vol. 2. Ieee, 1999.

A senior research scientist at Google (Seattle) in the Machine Intelligence Group.

99: Object recognition from local scale-invariant features [1]

04: Distinctive Image Features from Scale-Invariant Keypoints [2]

Autostich

: Atuomated paranoma creation

SIFT

: Matching with local invariant features

Augmented reality in natural scenes

[Overview of Research Projects]

2019-04-10

Introduction

5

Scale-Space Extrema Detection

Accurate KeypointLocalization

Orientation Assignment

KeypointDescription

Search over multiple scales and Image locations.

Select keypoints based on a measure of stability.

Compute best orientation(s) for each keypoint region.

Use local image gradients at selected scale and rotation to describe each keypoint region.

2019-04-10

Fundamental theory

6

DOG (edge)

Scale-space axioms

2019-04-10

Fundamental theory

7

LOG (blob)

2019-04-10

Fundamental theory

8

Normalized LOG

Normalization : LOG

2019-04-10

Fundamental theory

9

DOG & LOG

GkyxGkyxG 22)1(),,(),,(

GG 2

Heat Diffusion Equation

k

yxGkyxGG ),,(),,(

)()1()1( 22 GaussianofLaplacianNormalizedNLOGkGkDOG

1:

2:

DOG:

2019-04-10

Fundamental theory

10

Gaussian Pyramid

-

-

-

Difference of Gaussian(DOG)Gaussian

Convolution with

Gaussian

2

2

22

-4

-

-

-Downsample

Scale

(next

octave)

…

Scale

(1st octave)

2019-04-10

SIFT

11

Detection of Scale-Space Extrema

Extrema : maxima & minima

2019-04-10

SIFT

12

Detection of Scale-Space Extrema

Scale

),(*),,(),,( yxIyxGyxL

222 2/)(

22

1),,(

yxeyxG

),,(),,(

),(*)),,(),,((),,(

yxLkyxL

yxIyxGkyxGyxD

GkyxGkyxG 2)1(),,(),,(

sk /12:ratioscaling

s:interval

3s:ImageGaussianofnumberthe

2019-04-10

13

aerial photographs, industrial images)

SIFT

Detection of Scale-Space Extrema

2019-04-10

14

xx

xxx

2

2T

2

1)x(

DDDD

T

xxx

DD2

12

-ˆ

xx

x ˆ2

1)ˆ(

TDDD

))(( Tx,y,σxxxx

2

2

0)x('

DDD

T

xx

x

TDD2

2

xxx

TDD2

12

SIFT

Accurate Keypoint Localization (low contrast)

xxx

xx

ˆˆ

2

1ˆ

ˆ)ˆ(

TT

T DDDD

xx

xx

ˆˆ2

1ˆ

ˆ

TT DDD

xx

ˆˆ2

1

TDD

)5.0ˆ( xif

)03.0)ˆ(( xDif

Taylor Expansion

2019-04-10

15

SIFT

Accurate Keypoint Localization (edge)

yyxx DD)Tr(H 2)()Det( xyyyxx DDDH

r

r

r

r 2

2

222 )1()()(

)Det(

)(Tr

H

H

r

r 22 )1(

)Det(

)(Tr

H

H)10( r

yyxy

xyxx

DD

DDH Hessian Matrix

)(

2019-04-10

16

SIFT

Accurate Keypoint Localization

(a) 233x189 pixel original image

(d) 536 keypoints location

(threshold on ratio of principal curvatures)

(c) 729 keypoints location (threshold on minimum contrast)

(b) 832 keypoints location

2019-04-10

17

SIFT

Orientation Assignment

22 ))1,()1,(()),1(),1((),( yxLyxLyxLyxLyxm

))),1(),1(/())1,()1,(((tan),( 1 yxLyxLyxLyxLyx

2019-04-10

18

SIFT

Orientation Assignment

Histogram : Using 36bins

2019-04-10

19

SIFT

The Local Images Descriptor

illumination : normalization vector

(Feature vector < 0.2)

2019-04-10

20

SIFT

The Local Images Descriptor

r : the number of orientations

n : the width

The size of the resulting

descriptor vector is 2rn

2019-04-10

21

SIFT

Keypoint Matching

Object model

(train image)Test image

DB

offline online

1) Nearest-neighbor search

2) Cluster identification by Hough transform

voting

3) Model verification by linear least squares

4) Outlier detection

: Euclidean distance, K-D tree, BBF(Best-Bin-First)

2019-04-10

22

SIFT

Keypoint Matching

2019-04-10

23

Example

Recognition

2019-04-10

24

Example

Recognition

Q & A

2019-04-10 26

y

x

t

t

y

x

mm

mm

v

u

43

21

v

u

t

t

m

m

m

m

yx

yx

y

x

4

3

2

1

...

...

1000

0100

bAx

bAA][AxT1T