ai final presentation- an introduction to feature tracking
TRANSCRIPT
![Page 1: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/1.jpg)
An Introduction to Feature Tracking
N. D.
![Page 2: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/2.jpg)
Outline
➔ Introduction: What is feature tracking?
➔ Features
➔ Feature Tracking Algorithm
➔ Alternatives
➔ Applications
➔ Q & A
![Page 3: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/3.jpg)
Introduction: Motivation
How does object tracking work?
How do you determine the
similarity between images?
![Page 4: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/4.jpg)
Introduction: Preview
You have two images. How does the first map onto the second?
![Page 5: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/5.jpg)
Introduction: Feature Tracking
From Wikipedia: Motion estimation is...
“Process of determining motion vectors that describe the transformation
from one 2D image to another, usually from adjacent frames in a video
sequence. ”
![Page 6: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/6.jpg)
Features: Definition
Official Definition:
“a n-tuple or vector whose components are functions of the initial
measurement pattern variables or some subset of the initial
measurement pattern variables.”
Human-Speak Definition:
A location on an image where you track change.
![Page 7: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/7.jpg)
Features
(A location on an image where you track change
==
Any location) ?
![Page 8: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/8.jpg)
Features
(A location on an image where you track change
==
Any location) ?
true
![Page 9: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/9.jpg)
Features
What is a good feature?
![Page 10: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/10.jpg)
Features: An Example
(1)
(3)
(5)
(2)
(4)
(6)
![Page 11: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/11.jpg)
Features: Computed Answer
![Page 12: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/12.jpg)
Features: Computed Answer
![Page 13: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/13.jpg)
Features: The Good, the Bad, the Ugly
A good feature should be locally unique and invariant to geometric
transformations.
Good Bad Ugly
![Page 14: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/14.jpg)
Features: Local Uniqueness
To determine local uniqueness, look at sub-parts of the feature.
Good
Bad
Ugly
. . .
. . .
. . .
![Page 15: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/15.jpg)
Tracking: Algorithm Overview
Inputs: (a: Image, b: Image)
Step 1: vectora = Extract Features ( a ) vectorb = Extract Features ( b )
Step 2: matches = Compute Difference ( vectora , vectorb ) Step 3: result = Eliminate False Positives ( matches )
![Page 16: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/16.jpg)
Tracking: Shoes Example
Inputs:
![Page 17: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/17.jpg)
Tracking: Shoes Example
Output: Inputs:
![Page 18: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/18.jpg)
Tracking: Shoes Example --- Step 1
Step 1: Feature Extraction using the Harris Operator
![Page 19: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/19.jpg)
Tracking: Shoes Example --- Corner Detection
The Harris Operator is used in corner detection.
![Page 20: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/20.jpg)
Tracking: Shoes Example --- Harris Points
Harris Operator output: bright points = locally unique candidates
![Page 21: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/21.jpg)
Tracking: Shoes Example --- Choosing Points
Choose the brightest points that are at least delta distance apart.
![Page 22: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/22.jpg)
Tracking: Shoes Example --- Choosing Points
Same result, on the colored images:
![Page 23: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/23.jpg)
Tracking: Shoes Example --- Harris Equation
where ➔ W is a weights kernel (Gaussian kernel; smoothing) ➔ Ix is the horizontal Sobel operator (horizontal edge detection) ➔ Iy is the vertical Sobel operator (vertical edge detection)
![Page 24: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/24.jpg)
Tracking: Shoes Example --- Sobel Operator
Sobelx and Sobely applied to Image a
[ Horizontal edge detection ] [ Vertical edge detection ]
![Page 25: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/25.jpg)
Tracking: Shoes Example --- Step 2
Step 2: Feature Matching using the Sum of Squared Differences (alternative: Normalized Cross Correlation) to measure error and find
similarity between images.
![Page 26: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/26.jpg)
Tracking: Shoes Example --- SSD
Low error measurements between features means a potential match.
![Page 27: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/27.jpg)
Tracking: Shoes Example --- SSD Equation
where
➔ E(u, v) is the error measurement at (u, v) compared to (x, y) ➔ W(x, y) is a weights kernel (Gaussian kernel; smoothing) ➔ I(x, y) is the Sobel operator (edge detection)
![Page 28: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/28.jpg)
Tracking: Shoes Example --- Step 3
Step 3: Eliminate False Positives using the Random Sample Consensus (RANSAC) method
![Page 29: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/29.jpg)
Tracking: Shoes Example --- RANSAC
“An iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers. ”
![Page 30: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/30.jpg)
Tracking: Flowers Example
Example #2:
![Page 31: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/31.jpg)
Tracking: Museum Example
Example #3:
![Page 32: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/32.jpg)
(A Few) Alternatives
➔ Pixel-by-pixel computation (horribly inefficient)
➔ Relational matching
◆ Bag-of-Words
◆ Both generative or discriminative models
➔ Particle Filter
➔ Multi-Scale Oriented Patches
➔ Eigenfaces / Space of Faces
![Page 33: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/33.jpg)
Applications: Feature Extraction & Tracking
➔ Film ◆ Video tracking ◆ Replacement sets or characters (ex: Gollum) ◆ Motion tracking markers
➔ Biometrics
◆ Facial recognition (Eigenfaces) ◆ Xbox Kinect (?)
![Page 34: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/34.jpg)
Questions ? Ask : Clap
(You may clap.)
![Page 35: AI Final Presentation- An Introduction to Feature Tracking](https://reader033.vdocuments.us/reader033/viewer/2022051020/55cf8573550346484b8e2823/html5/thumbnails/35.jpg)
Sources Gollum Motion Estimation - Wiki Ransac - Wiki Car with Tracking Markers Motion Tracking Markers Joffrey Clapping GIF Glossary of Computer Vision Terms Skeleton code for reading in/writing out images from ColorFront Flowers Pictures from ColorFront Computer Vision: Algorithms and Applications by Richard Szeliski