openCV based image processed attendence system

1.abhiroop ghatak2. rohit verma3. sunny Jain4. vivek mittal

Under the guidance

of

Sangeetha jaganathan

openCV based image proopenCV based image processed attendance systecessed attendance systemm

1. rohit verma1. rohit verma2. sunny Jain2. sunny Jain3. vivek mittal3. vivek mittal4. abhiroop ghatak4. abhiroop ghatak

Problem Statement

: Given an image, to identify it as a face and/or extract face images from it.

• To retrieve the similar images (based on a heuristic) from the given database of face images.

System steps

Intro. to openCV

• (OPEN Source Computer Vision) A library of real time computer vision routines from Intel. First released in 2000, Open CV code is used in applications such as object, face and gesture recognition, lip reading and motion tracking.

• Example applications of the OpenCV library are Human-Computer Interaction (HCI); Object Identification, Segmentation and Recognition; Face Recognition; Gesture Recognition; Motion Tracking, Motion Understanding; and Mobile Robotics.

www.viisage.com

Access Control

ApplicationsApplications

www.visionics.com

Features

• Image and video I/O – file and camera based input, image/video file output

• Image data manipulation – allocation, release, copying, setting, conversion

• Matrix and vector manipulation, and linear algebra routines – products, solvers, eigenvalues, – Various dynamic data structures – lists, queues, sets, trees, graphs

• Basic image processing – filtering, edge detection, corner detection, sampling

and interpolation, color conversion, morphological operations, histograms, image pyramids

Modules• OpenCV Functionality

-more than 350 algorithms

• Cxcore– Data structures and linear algebra support.

• cv – Main OpenCV functions.

• Cvaux– Auxiliary (experimental) OpenCV functions.

• Highgui– GUI functions.

Why is Managing OpenCV Objects Important?

• Video, 30 frames per second

• Each frame is an image

• Images are arrays of pixels

• A 640x480 image is 307,200 pixels

• These must be represented in memory

• How much memory does your machine have?

How Does OpenCV Represent Images?

• IplImage: Structure from Intel Image Processing Library In addition to representing the image data IplImage holds utilizes

a subset of IPL data useful in IP/CV:

• IPL(Intel Image Processing Library) Image class.

• IplImage is subclass of CvMat. IplImage support ROI(region of interest) and COI(color of interest). Most of CvMat method support ROI, and some of CvMat method support COI.

• nSize : Size of Image• nChannels : Number of Image Channels (1-4)• width, height• ROI : Region of Interest

Handling IplImage

•cvCreateImage( CvSize size, int depth, int channels );•header = cvCreateImageHeader(size,depth,channels); cvCreateData(header);•cvCloneImage( const IplImage* image ); •cvLoadImage( const char* filename, int iscolor=1

);

Creating an IplImage

•The first two functions are useful for creating a blank image of the specified parameters. A possible use is in functions that require a pointer to a result.•The clone function performs an exact copy of the IplImage* parameter.•The load function loads an image from a file.

•cvReleaseImage( IplImage** image );•cvReleaseData( *image ); cvReleaseImageHeader( image );

Destroying an IplImage

• cvReleaseImage will work for the 3 single step creation functions•The cvReleaseData/cvReleaseImageHeader combination is used when there is separate data and header information

When allocating IplImage in a

loop be sure to deallocate in the

loop as well

Utilizing IplImageThe IplImage structure makes it

possible to target specific regions of an object for processing.

•cvSetImageROI( IplImage* image, CvRect rect ); •cvResetImageROI( IplImage* image ); •cvGetImageROI( const IplImage*

image );

Setting the Region of Interest (ROI)

Setting the ROI of the image allows the user to select a rectangular region of the image to work with. This is useful after localizing objects for extraction and further processing. While the region is set the rest of the image will be ignored. Meaning any operation directed on the image will act on only the region (including cvShowImage).

•cvSetImageCOI( IplImage* image, int coi ); •cvGetImageCOI( const IplImage* image );

Setting the Channel of Interest (COI)

Setting the channel of the image allows the user to work with a particular layer of the image. i.e. The ‘R’ layer of an RGB image or the ‘V’ layer in the HSV format. NOTE: Not all OpenCV functions support this.

Other Static Array Types• OpenCV uses the CvMat* as its general purpose

matrix structure. It is managed in an equivalent style to IplImage*

»CvMat• cvCreateMat( int rows, int cols, int type ); • mat = cvCreateMatHeader( rows, cols, type );• cvCreateData( mat ); • cvCloneMat( const CvMat* mat );

• cvReleaseMat( CvMat** mat );

• OpenCV also has built in functions for mult-dimensional arrays (CvMatND) and sparse arrays (CvSparseMat).

Detecting a specified object class within a image.

• Object detection has many applications in computer based vision.– Object tracking, object recognition, scene

surveillance.

• The focus of this project was to implement object detection, and to detect objects of the class face.

Face detection is an important component of a video information retrieval system.

This research focuses on the upright-frontal face detection problem.

• System flow chart:

Resize imageImage Contrast stretching

Face Detection Faces

Non-faces

Transform to integral images

Paul Viola and Michael Jones technique

• Devised a technique that was both robust and very quick. – 15 times quicker than any technique at the time of release.

• A detection algorithm that could be operated in real-time. – 95% accuracy at around 17fps.

• Understanding is the primary goal.

It is a good technique!

The Viola-Jones Technique(contd..)

• Feature extraction and feature evaluation.– Rectangular features are used, with a new image

representation their calculation is very fast.

• Classifier training and feature selection using a method called AdaBoost.– A long and exhaustive training process.

• A degenerate decision tree of classifiers is formed.– The key to the techniques speed.

Features• Four basic types.

– They are easy to calculate.– The white areas are subtracted from the black

ones.– A special representation of the sample called the

integral image makes feature extraction faster.

Feature Extraction

• Features are extracted from sub windows of an sample image.– The base size for a sub window is 24 by 24 pixels.

• In a 24 pixel by 24 pixel sub window there are 180,000 possible features to be calculated.

• What is the end result of feature extraction?– A lot of data!

• This is called over fitting and the amount of data must be reduced.

• Overfitting can be compensated to an extent by logical elimination.

Face Detection AlgorithmFace Detection Algorithm

Face Localization

Lighting Compensation

Skin Color Detection

Color Space Transformation

Variance-based Segmentation

Connected Component &Grouping

Face Boundary Detection

Verifying/ WeightingEyes-Mouth Triangles

Eye/ Mouth Detection

Facial Feature Detection

Input Image

Output Image

Face Recognition

Problem Statement Identify a person’s face image from face database.

Applications Human-Computer interface,

Static matching of photographs,

Video surveillance,

Biometric security,

Image and film processing.

Challenges

Variations in pose Head positions, frontal view, profile view and head tilt, facial expressions

•Illumination Changes Light direction and intensitychanges, cluttered background, low quality images

•Camera Parameters Resolution, color balance etc.

Occlusion Glasses, facial hair and makeup

Inter-class SimilarityInter-class Similarity• Different persons may have very similar appearance

Twins Father and son

www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americas/2000/us_elections

Intra-class VariabilityIntra-class Variability• Faces with intra-subject variations in pose,

illumination, expression, accessories, color, occlusions, and brightness

Face Recognition Face Recognition Using EigenfacesUsing Eigenfaces

Face Images are projected into a feature Face Images are projected into a feature space (“space (“Face SpaceFace Space”) that ”) that best encodes the best encodes the variationvariation among known face images. among known face images.

The face space is defined by the The face space is defined by the ““eigenfaceseigenfaces”, which are the ”, which are the eigenvectorseigenvectors of of the set of faces.the set of faces.

Eigen Space and Eigen FacesEigen Space and Eigen Faces

Principal Component Analysis (PCA)

For a set M of N-dimensional vectors {x1, x2…xM}, PCA

finds the eigenvalues and eigenvectors of the covariance matrix of the vectors

TM

iii xx

MC

1

1 - the average of the image vectors

an image as 1d vector

kkk uu uk - Eigenvectorsk - Eigenvalues

Keep only k eigenvectors, corresponding to the k largest eigenvalues.

Method of Eigenfaces

•Apply PCA on the training dataset

•Project the Gallery set images to the reduced dimensional eigenspace.

• For each test set image:

•Project the image to the reduced dimensional eigenspace.

•Measure similarity by calculating the distance between the projection coefficients of two datasets

•The face is recognized if the closest gallery image belongs to same person in test set

References..

• “Eigenfaces for Recognition”, Turk, M. and Pentland A., (1991)Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71-86.

• . http://cnx.org/content/m12531/latest/#• file://localhost/C:/Program%20Files/

OpenCV/docs/index.htm

Questions?

• How was my talk?

• Can anything be explained better?

» Thank you