open cv tutorial - university of washington
TRANSCRIPT
![Page 1: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/1.jpg)
Open CV Tutorial
Ubiquitous Computing Lecture
February 15th, 2012
![Page 2: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/2.jpg)
OpenCV installation
![Page 3: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/3.jpg)
Setup – Windows (1)
• Download OpenCV 2.1 http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.1/
![Page 4: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/4.jpg)
Setup – Windows (2)
• Installation
Add OpenCV DLLs into system path
![Page 5: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/5.jpg)
Setup – Windows (3)
• Include the header files and static libraries
![Page 6: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/6.jpg)
Setup – Linux
• Install OpenCV (say yes to any dependencies): $> sudo apt-get install libcv4 libcv-dev libhighgui4 libhighgui-dev libcvaux4 libcvaux-dev
• To build: $> gcc -lcv -lhighgui –lcvaux YourFile.cpp
![Page 7: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/7.jpg)
Links
• Main wiki http://opencv.willowgarage.com/wiki/
• On-line library http://opencv.willowgarage.com/documentation/c/
• Tutorial http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html
![Page 8: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/8.jpg)
Getting Images from the Webcam
Two Methods: • Event Driven (or interrupt driven)
• New Frame == interrupt • Adv: constant FPS, multi-threaded processing • Dis: overhead, must be real-time
• Polling (what we will use, the norm)
• Grab frame when ready • Adv: only process what we can in real time • Dis: might grab same frame, variable FPS
![Page 9: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/9.jpg)
Getting Images from the Webcam
#include <stdio.h> #include "opencv/cv.h" #include "opencv/highgui.h" #include "opencv/cxcore.h" int main(int argc, char* argv[]) {
CvCapture *capture = 0; //The camera IplImage* frame = 0; //The images you bring out of the camera capture = cvCaptureFromCAM( 0 ); //Open the camera if (!capture ){ printf("Could not connect to camera\n" ); return 1; } cvNamedWindow( "demowin1", CV_WINDOW_AUTOSIZE ); //Create output window
while(key != 27 /*escape key to quit*/) { //Query for the next frame frame = cvQueryFrame( capture ); if( !frame ) break; //show the raw image in the first window cvShowImage( "demowin1", frame ); //Get the last key that's been pressed for input key = cvWaitKey( 1 ); } //Shutdown cvReleaseCapture( &capture ); // WARNING: DO NOT release the "frame" variable cvDestroyWindow( "demowin1" ); return 0;
}
![Page 10: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/10.jpg)
Saving a video file
int main(int argc, char* argv[]) {
CvCapture *capture = 0; //The camera IplImage* frame = 0; //The images you bring out of the camera capture = cvCaptureFromCAM( 0 ); //Open the camera if (!capture ){ printf("Could not connect to camera\n" ); return 1; } CvVideoWriter *writer; int isColor = 1; int fps = 25; // or 30 writer = cvCreateVideoWriter("output.avi", CV_FOURCC_DEFAULT, fps, cvSize(640, 480)); while(key != 27 /*escape key to quit*/) { //Query for the next frame frame = cvQueryFrame( capture ); if( !frame ) break; cvWriteFrame(writer, frame); //Just like showing an image //Get the last key that's been pressed for input key = cvWaitKey( 1 ); } cvReleaseVideoWriter(&writer); //Release the video file cvReleaseCapture( &capture ); // WARNING: DO NOT release the "frame" variable return 0;
}
![Page 11: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/11.jpg)
Accessing a pixel
{
int i = frame->height/2; int j = frame->width/2; //slow way CvScalar bgr = cvGet2D(frame, i, j); printf("B:%d, G:%d, R:%d\n", (int)bgr.val[0], (int)bgr.val[1], (int)bgr.val[2]); //set the value cvSet2D(frame, i, j, CV_RGB(0,255,0)); cvShowImage( "demowin2", frame );
}
//fast way (access the array directly) printf("B:%d, G:%d, R:%d\n", ((uchar *)(frame->imageData + i*frame->widthStep))[j*frame->nChannels + 0], // B ((uchar *)(frame->imageData + i*frame->widthStep))[j*frame->nChannels + 1], // G ((uchar *)(frame->imageData + i*frame->widthStep))[j*frame->nChannels + 2]);// R
![Page 12: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/12.jpg)
Transforming color images
• Grayscale
• 0.3*R+0.59*G+0.11*B, luminance
• HSV • What we perceive as color, rather than “sense” as color (well… sort of)
• Hue: the color value • Saturation: the richness of the color relative to brightness • Value: the gray level intensity
IplImage* grayscaleImg = cvCreateImage(cvSize(640,480), 8/*depth*/, 1/*channels*/); {
cvCvtColor(frame, grayscaleImg, CV_BGR2GRAY); cvShowImage( "demowin2", grayscaleImg );
}
IplImage* singleChannelImg = cvCreateImage(cvSize(640,480), 8/*depth*/, 1/*channels*/); IplImage* hsvImg = cvCreateImage(cvSize(640,480), 8/*depth*/, 3/*channels*/); {
cvCvtColor(frame, hsvImg, CV_BGR2HSV); cvSplit(hsvImg, singleChannelImg, NULL, NULL, NULL); // get the "Hue" component cvShowImage( "demowin3", singleChannelImg );
}
![Page 13: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/13.jpg)
Reducing image size
For reducing image size quickly (e.g., to increase frame rate) • Pyramid
• nearest neighbor (no interpolation) • integer multiple down-sampling (throws rows and columns) • Anti-aliasing built in
IplImage* smallerImg = cvCreateImage( cvSize(640/2,480/2), 8/*depth*/, 3/*channels*/); cvPyrDown( frame, smallerImg );
![Page 14: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/14.jpg)
Image filtering
For example low pass filtering
CV_BLUR_NO_SCALE all ones (has ringing) CV_BLUR all ones, sum to one (has ringing) CV_GAUSSIAN a gaussian smoothing kernel (no ringing if large enough) CV_MEDIAN choose the median of the block (non-linear) CV_BILATERAL “perceptual closeness” (texture/color) of the pixels
cvSmooth(frame, frame, CV_GAUSSIAN, 15, 15); //size=15x15
![Page 15: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/15.jpg)
Edge Detection
The Canny edge detector
//Requires a single-channel image: cvCvtColor(frame, grayscaleImg, CV_BGR2GRAY); //Reduce the image to the edges detected: cvCanny(grayscaleImg, edgesImg, hyst1, hyst2, sobelSize, l2needed?);
1. Image input
2. Smoothing
3. Gradient
4. Local
Maxima
5. Hysteresis
Thresholding
From magnitude and direction of
Gx,y
Gx,y hyst1 = image hyst2 = along lines
![Page 16: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/16.jpg)
Hough Circles
//Requires a single-channel image: cvCvtColor(frame, grayscaleImg, CV_BGR2GRAY); //Want to smooth it to get less noise cvSmooth(grayscaleImg, grayscaleImg, CV_GAUSSIAN, 7, 9 ); //Detect the circles in the image CvSeq* circles = cvHoughCircles(grayscaleImg, storage, CV_HOUGH_GRADIENT, downsampling, minCenterDist, CannyThresh, AccumThresh minRadius, maxRadius );
• In General, Hough transform consists of 1. Edge detection 2. Accumulation in parameter space For each detected point, draw shape with different parameter Look for local maxima For a circle space is (x,y,R)
R=5 R=10
![Page 17: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/17.jpg)
Morphology Primer
• Pick a structuring element • Cross • Disk • Rectangle
• Pick an operation
• Dilation (Max, “OR”) • Erosion (Min, “AND”) • Opening (E+D) • Closing (D+E)
![Page 18: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/18.jpg)
Torso Tracking
Ke-Yu Chen, 2012/02/15
![Page 19: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/19.jpg)
Project goal
19
• Track the human torso in real time from a single general-purpose camera
![Page 20: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/20.jpg)
Motivation
20
• High-tech camera is not available
• Only have 2D sequences
![Page 21: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/21.jpg)
Algorithm
21
1. Background Model
Construction
2. Foreground
Object Extraction
Video Source
3. Upper body
Identification
4. Distance
Transform
5. Silhouette extraction
![Page 22: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/22.jpg)
22
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
1. Background Model Construction
0~50
… …
51~60
Background model
![Page 23: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/23.jpg)
23
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
2. Foreground Object Extraction 1. Background
Model Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
- =
Fragmented pixels
Shadow
![Page 24: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/24.jpg)
24
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
2-1. Fill the fragmented pixles
Closing by Reconstruction [5]
![Page 25: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/25.jpg)
25
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
2-2. Shadow detection [4]
R
G
B
Shadow pixel
![Page 26: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/26.jpg)
26
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
3. Upper body identification 1. Background
Model Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
Th * HEIGHT
Th = 50%
Th = 60%
Th = 70%
![Page 27: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/27.jpg)
27
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
4. Distance Transform 1. Background
Model Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
DT [1]
w(x, y) = 1 + ¼ * ( w(x+1, y) + w(x, y+1) + w(x-1, y) + w(x, y -1) )
1 1 1 1 1
1 2 2 2 1
1 2 3 2 1
1 2 2 2 1
1 1 1 1 1
Poisson [3]
![Page 28: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/28.jpg)
28
1. Background Model
Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
5. Silhouette Extraction 1. Background
Model Construction
2. Foreground Object Extraction
Video Source
3. Upper body Identification
4. Distance Transform
5. Silhouette extraction
Threshold = 0.55
![Page 29: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/29.jpg)
Smooth the result
29
• Improve shaking results
t t-1 t-2 t-3
0.4 0.6
+ = Final Rectangular
of the torso
![Page 31: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/31.jpg)
Reference
31
1. Distance transform: http://en.wikipedia.org/wiki/Distance_transform
2. Siddiqi, K.; Member, S. & Kimia, B. B. Parts of Visual Form: Computational Aspects IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17, 239-251
3. Gorelick, L.; Galun, M.; Sharon, E.; Basri, R.; Society, I. C.; Society, I. C. & Br, A. Shape representation and classification using the poisson equation In In Proc. of CVPR’04, 2004, 61-67
4. Horprasert, T.; Harwood, D. & Davis, L. S. A statistical approach for real-time robust background subtraction and shadow detection 1999, 1-1
5. Closing by Reconstruction: http://artis.imag.fr/Members/Adrien.Bousseau/morphology/morphomath.pdf
![Page 32: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/32.jpg)
Future work
32
• Dynamic threshold for shadow detection
• Track multiple objects
• Program to get ground truth (4 corners of the torso)
• Adaptive morphological closing operation
![Page 33: Open CV Tutorial - University of Washington](https://reader035.vdocuments.us/reader035/viewer/2022071600/613d1215736caf36b758f777/html5/thumbnails/33.jpg)
Questions?
33