# Intro to Sift

Topics: Gaussian function, Maxima and minima, Scale space Pages: 18 (1504 words) Published: December 30, 2012
Yu Liu 4.17.2012

Why? What? How?

USCT, Dept. EEIS, Yu Liu

3

1. Image Matching in a common and important problem in computer vision. 2. Application in: ◦ ◦ ◦ ◦ ◦ Object or scene recognition 3D reconstruction Stereo correspondence Motion tracking Image Searching

USCT, Dept. EEIS, Yu Liu

4

3. Traditional method: simple corner detectors is not stable when you have images of different scales and rotations. 4. We need a method can solve:
◦ ◦ ◦ ◦ ◦ ◦ Different Scale Rotation Different Illumination Changed Viewpoint Afﬁne Distortion Addition of noise USCT, Dept. EEIS, Yu Liu 5

SITF is a method for extracting distinctive invariant features, providing robust matching across a substantial range of afﬁne distortion, change in 3D viewpoint, addition of noise, and change in illumination. Raised by David G. Lowe in Distinctive Image Features from Scale-Invariant Keypoints in 2004. David G. Lowe Computer Science Department 2366 Main Mall University of British Columbia Vancouver, B.C., V6T 1Z4, Canada E-mail: lowe@cs.ubc.ca

USCT, Dept. EEIS, Yu Liu

6

Main idea: Extract a set of features, which are invariant to image scaling and rotation, change in illumination and 3D viewpoint. 1. Create Scale space:
◦ Create DoG Pyramid of Images.
Octave 5 Octave 4 Octave 3

8

4

2

Octave 2

Octave 1

USCT, Dept. EEIS, Yu Liu

7

2. Detect Keypoint:
◦ ◦ ◦ ◦ (1) (2) (3) (4) Scale-space extrema detection Keypoint localization Select optimal keypoints Orientation assignment of Keypoint D  2 D 1 X  ( 2) X X
 T

Tr  H   r  1  Det  H  r
2

2

USCT, Dept. EEIS, Yu Liu

8

3. Keypoint descriptor: features
◦ ◦ ◦ ◦ Sample region- Around keypoint Rotation by angle of keypoint’s orientation Orientation histogtram entry- descriptor: 4*4*8 Normalization x

x
y

y

3 oct 3 oct 3 oct 3 oct

x

y

USCT, Dept. EEIS, Yu Liu

9

Application of SIFT on image matching:

USCT, Dept. EEIS, Yu Liu

10

1. 2. 3.

Create Scale Space Keypoint Detection Keypoint Descriptor

USCT, Dept. EEIS, Yu Liu

11

USCT, Dept. EEIS, Yu Liu

12

Definition: a series of representation of image in multi-scale thought scale transform.

 

Multi-scales: multi-sigma, not size of image! Scale transform: Gaussian ◦ Gaussian is the only scale space kernel; ◦ LoG is the only kernel with scale invariance;

Object observed in certain scales;

USCT, Dept. EEIS, Yu Liu

13

 

Nature of Image; Necessary in scale invariant features: image matching must be in the same scale!

USCT, Dept. EEIS, Yu Liu

14

Convolute image with Gaussian kernel with varying sigmas, then make desampling. Octave 5 Octave 4 Octave 3

8

4

2

Octave 2

Octave 1

USCT, Dept. EEIS, Yu Liu

15

Gaussian pyramid with o octaves, and each octave has S intervals:  ( s)   0 2
s S
8

Octave 5 Octave 4 Octave 3

 

4 σ: scale; σ0: initial scale; 2 s: coordinate of intervals S: number of intervals in one octave;  …

Octave 2

Octave 1

USCT, Dept. EEIS, Yu Liu

16

 
scale

Scale is only decided by sigma, but not relevant to size of image. Then why sampling? Reduce calculation. When sampling? When sigma is doubled. σ0 1 1 σ0*2^(1/3) 1 2 σ0*2^(2/3) 2*σ0 1 3 2 1 σ0*2^(4/3) σ0*2^(5/3) 2 2 2 3

Octave Interval

USCT, Dept. EEIS, Yu Liu

17

Signal spectrum filtered with Gaussian (σ0) Signal spectrum filtered with Gaussian (2σ0)

1 3σ0

1 6σ0

Signal spectrum filtered with Gaussian (2 σ0) and desampling. USCT, Dept. EEIS, Yu Liu 18

One property of Gaussian filter:

◦ Situation 1: Image blurred with Gaussian kurnel with sigma= σ1, then blurred with Gaussian kernel with sigma= σ2; ◦ Situation 2: Image blurred with Gaussian kernel with sigma= σ12 + σ22 ;...