robust moving object detection & categorization using self-improving classifiers
DESCRIPTION
Robust Moving Object Detection & Categorization using self-improving classifiers . Omar Javed, Saad Ali & Mubarak Shah. Moving Object Detection & Categorization. Goal Detect moving objects in images and classify them into categories, e.g., humans or vehicles. Motivation - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/1.jpg)
Robust Moving Object Detection & Categorization
using self-improving classifiers
Omar Javed, Saad Ali & Mubarak Shah
![Page 2: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/2.jpg)
Moving Object Detection & Categorization
Goal Detect moving objects in images and
classify them into categories, e.g., humans or vehicles.
Motivation Most monitoring and video
understanding systems require knowledge of, location and type of objects in the scene.
![Page 3: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/3.jpg)
Object Classification:Major Approaches
Supervised Classifiers Adaboost (Viola & Jones), Naive Bayes
(Schniederman et al.), SVMs (Papageorgiou & Poggio)
Limitations Requirement of large number of training
examples, 1000000 negative examples for face detection (Zhang et al.). More than 10000 examples used by (Viola & Jones)
Fixed parameters after training. After deployment, parameters are not tunable to best performance in a particular scenario.
![Page 4: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/4.jpg)
Object Classification:Major Approaches
Semi-Supervised Classifiers Co-training (Levin et al.) Limitations:
Requirement for collection of large amount of training data, though no need for labels.
Offline training, i.e., Fixed parameters in the testing phase.
![Page 5: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/5.jpg)
Properties of an “Ideal” Object Detection System
Learns both background and object models online with no prior training.
Adapts quickly to changing background and object properties
![Page 6: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/6.jpg)
Overview of the Proposed Approach
In a single boosted framework, Obtain regions of Interest (ROI) from a background
subtraction approach. Obtain motion and appearance features from the
ROI. Use separate views (motion and appearance
features) of the data for online co-training, i.e., If one set of features confidently predicts a label of an
object, then use this label to online update the base classifiers and the boosting parameters.
Use combined view (both features) for classification decisions.
![Page 7: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/7.jpg)
Properties of the Proposed Object Detection Method
Background model is learned online. Object models are learned offline with a small
number of training examples. The object classifier parameters are
continuously updated online using co-training to improve detection rates.
![Page 8: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/8.jpg)
Proposed Object Detection Method
Co-Training Decision (if confident prediction by one set)
ROIs
Background
Appearance FeatureExtraction
Background
Updated weak learners
Background Models Foreground Models
Updated parameters
Classification Output
Color Classifier
Base Classifiers(Appearance)
Motion FeatureExtraction
Edge Classifier Base Classifiers
(Motion)
Boosted Classifier
Updated Boosted Parameters
![Page 9: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/9.jpg)
Background Detection First level
Per-pixel Mixture of Gaussian color models
Second Level Gradient magnitude and gradient direction
models Gradient boundary check Feedback to first level
Current Image from video
Output of first level Output of second level
![Page 10: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/10.jpg)
Features for Object Classification
Base classifiers learned from global PCA coefficients of appearance and motion templates of Image regions.
Appearance subspace learned by performing PCA separately on a small set of labeled ‘d’ dimensional gradient magnitude images of people and vehicles.
![Page 11: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/11.jpg)
Features for Object Classification
The people and vehicle appearance subspaces are represented by d x m1 and d x m2 projection matrices (S1 and S2) respectively.
m1 and m2 are chosen such that the eigenvectors account for 99% of variance in the respective subspaces.
![Page 12: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/12.jpg)
Features for Object Classification
Appearance features for base learners are obtained by projecting each training example ‘r’ in the two subspaces
1 1 1[ ,..., ] Tmv v r S
1 1 1 2 2[ ,..., ] Tm m mv v r S
![Page 13: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/13.jpg)
Features for Object Classification
Row 1: Top 3 eigenvectors for person appearance subspace. Row 2: Vehicle appearance subspace
![Page 14: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/14.jpg)
Features for Object Classification
To obtain motion features, person and vehicle motion subspaces (matrices S3 and S4)are constructed from m3 and m4 dimensional person and vehicle examples respectively.
Optical flow is obtained using the method by Lucas and Kanade.
Motion features for base learners are obtained by projecting each training motion example ‘o’ in the two subspaces
1 2 1 1 2 3 3[ ,..., ] Tm m m m mv v o S
1 2 3 1 1 2 3 4 4[ ,..., ] Tm m m m m m mv v o S
![Page 15: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/15.jpg)
Base Classifiers We use the Bayes Classifier as the base
classifier. Let c1, c2 and c3 represent the person,
vehicle and background classes. Each feature vector component vq ,where
q ranges from 1,.., m1+m2+m3+m4 , is used to learn the pdf for each class.
The pdf is represented by a smoothed 1D histogram.
![Page 16: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/16.jpg)
Base Classifiers The classification decision by the qth
base classifier is taken as ci,
( | ) ( | )i q j qP c v P c v i j If
![Page 17: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/17.jpg)
Adaboost Boosting is a method for combining
many base classifiers to come up with a more accurate ‘strong’ classifier.
We use the Adaboost.M1 (Freund and Schapire) to learn the strong classifier, from the initial training data and the base classifiers.
![Page 18: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/18.jpg)
The online co-training Framework
In general co-training requires at least two classifiers trained on independent features for labeling of data. Examples confidently labeled by one classifier are used to train the other.
In our case, individual base classifiers either represent motion or appearance features.
To determine confidence thresholds for each base classifier, we use a validation data set.
![Page 19: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/19.jpg)
The online co-training Framework
For class ci and jth base classifier the confidence threshold, is set to be the highest probability achieved by a negative example, i.e.,
All examples in the validation set with probability higher than the threshold are correctly classified.
, ibasej cT
![Page 20: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/20.jpg)
During the test phase, If more than 20% of the appearance based or motion based classifiers predict the label of an example with the probability higher than the validation threshold, then the example is selected for online update.
Online update is only necessary if the boosted classifier decision has a small or negative margin.
Margin thresholds are also computed from the validation set.
The online co-training Framework
i
adacT
![Page 21: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/21.jpg)
The online co-training Framework
Once an example has been labeled by the co-training mechanism, an online boosting algorithm is used to update the base classifiers and the boosting coefficients.
![Page 22: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/22.jpg)
Online Co-training Algorithm
![Page 23: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/23.jpg)
Experiments Initial Training
50 examples of each class All examples scaled to 30x30
vector Validation Set
20 images per class Testing on three sequences
![Page 24: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/24.jpg)
Experiments Results on Sequence1.
![Page 25: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/25.jpg)
Experiments Results on Sequence1. Performance over time Performance over
number of co-trained examples
![Page 26: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/26.jpg)
Experiments Results on Sequence 2.
![Page 27: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/27.jpg)
Experiments Results on Sequence 2.
Performance over time Performance over number of co-trained
examples
![Page 28: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/28.jpg)
Experiments Results on Sequence 3.
![Page 29: Robust Moving Object Detection & Categorization using self-improving classifiers](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681602c550346895dcf3ba9/html5/thumbnails/29.jpg)
Experiments Results on Sequence 3.
Performance over time Performance over number of co-trained
examples