recognition template matching o chamfer matching statistical pattern recognition (spr) robust...

RecognitionTemplate Matching

o Chamfer MatchingStatistical Pattern Recognition (SPR)Robust object recognition using a cascade of Haar classifiers (Haar)Performance

RecognitionBased on A Practical Introduction to Computer Vision with

OpenCV by Kenneth Dawson-Howe © Wiley & Sons Inc. 2014 Slide 1

Template Matching - Topics Applications Matching criteria Use of Fourier space Use of chamfering Control strategies



Template – Applications for template matching Searching

Locating objects



Template – Applications for template matching Recognition



Template – Applications for template matching Visual inspection

Golden template matching

MatchingStereo VisionTracking



Template – Matching AlgorithmBasic Algorithm

Inputs – Image & ObjectFor every possible position of the object in the image

Evaluate a match criterionSearch for local maxima of the match criterion above some threshold

Problems‘Every possible position’?‘Match criterion’ ‘Local maxima above some threshold’



Template – Matching criteria



Template – Matching criteria example

Simple exampleMatching criteriaBoundaries…

Real example Uses normalised cross correlation.



Template Matching in OpenCV



Mat matching_space;

matching_space.create(

search_image.cols–template_image.cols+1,

search_image.rows–template_image.rows+1, CV_32FC1 );

matchTemplate( search_image, template_image,

matching_space, CV_TM_CCORR_NORMED );

// Other measures: CV_TM_CCORR, CV_TM_SQDIFF,

// CV_TM_SQDIFF_NORMED

Template – Finding Local Maxima in OpenCVLocal maxima – Dilate + Look for unchanged values + ThresholdLocal minima – Erode + Look for unchanged values + Threshold



Mat dilated, thresholded_matching_space, local_maxima,

thresholded_8bit;

dilate( matching_space, dilated, Mat());

compare( matching_space, dilated, local_maxima, CMP_EQ );

threshold( matching_space, thresholded_matching_space,

threshold, 255, THRESH_BINARY );

thresholded_matching_space.convertTo( thresholded_8bit,

CV_8U );

bitwise_and( local_maxima,thresholded_8bit,local_maxima );

Template – Control Strategies for MatchingGoal: Localise close copies

Size, orientationGeometric distortion

Use an image hierarchyLow resolution firstLimit higher resolution search

Search higher probability locations firstKnown / learnt likelihoodFrom lower resolution



Template – Chamfer matchingTemplate matching requires very close matchesObjects often appear very slightly different

OrientationNoiseSampling

We want a more flexible approach



Template – ChamferingCompute chamfered image for a binary edge image.

For every pointIf (edge point)

Set c(i, j) = 0Else Set c(i, j) = ∞

For j = min to maxFor i = min to max

c(i, j) = minqAL [distance( (i, j), q ) + f( q )]For j = max to min

For i = max to minc(i, j) = minqBR [distance( (i, j), q ) + f( q )]



Canny( gray_image, edge_image, 100, 200, 3);

threshold(edge_image,edge_image,127,255,THRESH_BINARY_INV);

distanceTransform(edge_image,chamfer_image,CV_DIST_L2,3);

Template – Chamfer matching



Template – Chamfering for matchingCompare with boundary template

Sum of chamfer values for boundary pointsLow values best



Chamfer matching is a simple routine to write (provided in the text);

SPR – Statistical Pattern RecognitionProbability ReviewFeaturesClassification



SPR – Probabitity Review – BasicsP(A) = limn N(A) / nProbability of two events A and B:

Independent: P(AB) = P(A)P(B)

Dependent:P(AB) = P(A|B)P(B)

Conditional Probability P(A|B)

Typical problem:Given some evidence x from an unknown object.What class Wi is the object?Training

A-priori probability – p(x | Wi)Relative probability – p(Wi)

A-posteriori probability – p(Wi | x)



SPR – Probability Density FunctionsGiven a class (Wi) and an event (x)

Determine the probability of any particular value occurring…



SPR – Features: Area Count the points

Location dependent?

Determine from n polygon vertices



vector<vector<Point>> contours;

vector<Vec4i> hierarchy;

findContours( binary_image, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_NONE);

for (int contour=0; (contour>=0); contour=hierarchy[contour][0])

{

double area = contourArea(contours[contour]) +

contours[contour].size()/2 + 1;

for (int hole=hierarchy[contour][2]; (hole>=0); hole=hierarchy[hole][0])

area -= ( contourArea(contours[hole]) – contours[hole].size()/2 + 1 );

}

CANNOT BE the ratio of the length and width of the minimum bounding rectangle

Ratio of region area divided by the square of it’s thickness

Erosion

SPR – Features: Elongatedness



SPR – Features: Minimum Bounding Rectangle Turn rectangle through discrete steps

Only one quadrant

Metrics:Length to Width ratio

Length / WidthRectangularity

Area / (Length * Width)Convex Hull to Minimum Bounding Rectangle area ratio

Area inside convex hull / (Length * Width)



RotatedRect min_bounding_rectangle =

minAreaRect(contours[contour_number]);

SPR – Features: Convex hull Smallest convex region

Algorithm1. Start with

Any point on convex hullPrevious vector direction

2. Search all other boundary pointsFind point with least angle to previous vector

3. Switch to new point and vector4. Go to 2 unless new point = start point.



SPR – Features: Convex hull



vector<vector<Point>> hulls(contours.size());

for (int contour=0; (contour<contours.size()); contour++)

{

convexHull(contours[contour], hulls[contour]);

}

SPR – Features: Concavities and Holes Identify concavities from the convex hullIdentify holes in the binary shape



SPR – Features: Concavities and Holes



vector<vector<int>> hull_indices(contours.size());

vector<vector<Vec4i>> convexity_defects(contours.size());

for (int contour=0; (contour<contours.size()); contour++)

{

convexHull( contours[contour], hull_indices[contour] );

convexityDefects( contours[contour], hull_indices[contour],

convexity_defects[contour]);

}

SPR – Features: Perimeter Length and CircularityPerimeter length

Approximately the number of boundary elements

Should really take into account directionCircularity = Perimeter length / (4*π*Area)



contours[contour].size()

SPR – Features: Moments & Moment invariantsMoments measure the distribution of shapeCentral moments:

Scale invariant moments:



SPR – Features: Moment invariantsMoment invariants / Hu moments:



SPR – Features: Moments and Moment invariants



Moments contour_moments;

double hu_moments[7];

contour_moments = moments( contours[contour] );

HuMoments( contour_moments, hu_moments );

SPR – Classification – Introduction Object recognition

Classes (w1, w2, … wR)Classifier

Input Pattern / features (x1, x2, … xn)Feature space

Choosing the features (Example)Clusters in feature space

SeparabilityHyper-surfacesLinear separabilityInseperable classes

ClassifiersMinimum Distance ClassifierLinear ClassifierProbabilistic Classifier



SPR – Minimum distance classifier

Each class represented by an exemplarFor an unknown object

Determine the distance to the exemplarsPick the class with the smallest distance

Unknown class?Distance must be less than some threshold

AdvantagesTrainingComputational



SPR – Linear classifierLinear discrimination functions

gr(x) = qro + qr1x1 + … + qrnxn

Decision rulewr = d(x)gr(x) gs(x)

Unknown classgr(x) > threshold



SPR – Probabilistic classifierOptimal classifierBased on Probability Density Functions

Remove normalisation from Bayes rule…

Resultant Discrimination function:

Mean Loss Function:

Advantages / DisadvantagesAccuracyExtensive training



SPR – Classifier learningTraining set

Must be representativeMust be inductive

Training set sizeTraining set provides the unknown statistical information Size will typically have to be increased several times

Sample learning strategiesSupervised:

Probability density estimation – estimating p(x|wr) & P(wr) Training set includes class specification for every instance

Unsupervised:Cluster AnalysisLook for similarities in feature space



SPR: Real example



Haar – Robust Object Detection using a cascade of classifiers.

FeaturesEfficient calculation

TrainingWeak & Strong Classifiers

AdaboostRecognition



Haar – In OpenCV.



CascadeClassifier cascade;

if( !cascade.load(

"haarcascades/haarcascade_frontalface_alt.xml" )

{

vector<Rect> faces;

equalizeHist( gray_image, gray_image );

cascade.detectMultiScale( gray_image, faces, 1.1, 2,

CV_HAAR_SCALE_IMAGE, Size(30, 30) );

}

Haar – OverviewTraining using a number of positive and negative samples

Uses simple (Haar like) featuresEfficient calculation…

Selects a large number of these features during training to create strong classifiers

Links a number of strong classifiers into a cascade for recognitionEfficiency…

Can work at different scales



Haar – FeaturesFeatures determined as the difference of the sums of a number of rectangular regions

Place the mask in a specific locationand at a specific scaleThen subtract the sum of the ‘white pixels’from the sum of the ‘black pixels’

Why does this work?



Haar – Efficient Calculation – Integral ImageIntegral Image:

Every point ii(i,j) = i’=0..i j’=0..j image(i’,j’)Sum of points in rectangle D:

sum(D) = ii(p) + ii(s) – ii(q) – ii(r)

Features can be computed at any scale for the same cost



An integral function is provided which computes the integral image both normally and at 45o

Haar – TrainingNumber of possible features.

the variety of feature typesallowed variations in size and position

Training mustIdentify the best features to use at each stageTo do this positive and negative samples are needed…



Haar – Weak & Strong ClassifiersWeak Classifier

pjfeaturej(x) < pjθj Tune threshold (θj)

Strong ClassifiersCombine a number of weak classifiers…E.g. 100% true positives with only 40% false positives using only two face features…



Haar – Strong Classifiers – AdaboostGiven n example images x1..xn together with classifications y1..yn

where yi = 0, 1 for negative and positive examples respectively.

Initialise weights w1,i = 1 / (2*(m*(1- yi)+l*yi)) where m and l are the number of negative and positive examples respectively.

For t=1,…,T

1. Normalize the weights (i.e. for all i): wt,i = wt,i / (j=1..n wt,j)

2. For each feature, j, train a weak classifier hj(x) and evaluate the error taking into account the weights: j = i wt,i | hj(xi) – yi |

3. Select the classifier, hj(x), with the lowest j , save as ct(x) with error Et

4. Update the weights (i.e. for all i): wt+1,i = wt,i t(1- ei )

where ei = | ct(xi) – yi | and t = Et / (1–Et)

The final strong classifier is: h(x)= 1 if t=1..T αtct(x) ½ t=1..T αt ,

0 otherwise where αt = log 1/t



Haar – Classifier cascadeObject recognition is possible with a single strong classifier

To improve detection rates AND to reduce computation time:A cascade of strong classifiers can be usedEach stage either accepts or rejectsOnly those accepted pass to the next stage

Efficient computation…

Strong classifiers trained using AdaBoostOn the remaining set of data



Haar – RecognitionFace recognition

38 stages6000+ features4916 positive samples9544 negative samplesScale independence



PerformanceTwo aspects

Computation time

Success and failure ratesWhat is the correct answer?

We need ground truthHow do we assess success?

We need metrics.



double before_tick_count=static_cast<double>(getTickCount());

// Put methods to be timed here…

double after_tick_count=static_cast<double>(getTickCount());

double duration_in_ms=1000.0*(after_tick_count-

before_tick_count) / getTickFrequency();

Performance – Ground TruthHas to be manually computedVery difficult to get agreement…



Performance – MetricsCompute

TPTNFPFN

and then compute…



Performance – TuningWe can tune performance by altering parameters

For example altering the threshold used for accepting a match in our template matching example



Performance – Precision-Recall CurvesAlternative (common) visualisationSelect the parameters which result in the point on the PR curve which is closest to P=1.0, R=1.0



recognition template matching o chamfer matching statistical pattern recognition (spr) robust...

Documents

template matching recognition

space matching

mat matching

opencv recognition

normed slide

computer vision

template applications

kenneth dawsonhowe wiley