opencv introduction
TRANSCRIPT
(Open Source Computer Vision)
Outline
● Overview and practical issues.
● A selection of OpenCV functionality:– Image enhancement– Object classification and tracking– Face detection and recognition
● Conclusion and further resources.
Overview: Capabilities
Overview: License
● BSD Licensed (free and open source)● May be used in commercial software.● No requirement to publish the source!● Must acknowledge OpenCV was used in the
documentation by including its copyright notice.
Note: There is a C#/.NET wrapper for OpenCV called “Emgu CV” that may be commercially licensed.
Overview: Patents
● Note: A couple of algorithms (SIFT and SURF) that are implemented are patented.– You can't accidentally use them because they are in
a separate module called “nonfree”.
Overview: Users
● Stitching street-view images together,● Detecting intrusions in surveillance video in Israel● Detection of swimming pool drowning accidents in
Europe
Overview: Environment
Overview: EnvironmentPrimary API
is C++
LeveragesARM NEON
Overview: Installation
● Ubuntu VM:– sudo apt-get install libopencv-dev
● Windows:– Download latest version from http://opencv.org/
For Python:● Also install Python from http://www.python.org/● Install numpy module● Copy the “cv2” module from OpenCV to C:\Python27\Lib\site-packages
Overview: Hello WorldMakefile
CC=g++CFLAGS+=-std=c++0x `pkg-config opencv --cflags`LDFLAGS+=`pkg-config opencv --libs`
PROG=helloOBJS=$(PROG).o
.PHONY: all clean$(PROG): $(OBJS)
$(CC) -o $(PROG).out $(OBJS) $(LDFLAGS)
%.o: %.cpp$(CC) -c $(CFLAGS) $<
all: $(PROG)
clean:rm -f $(OBJS) $(PROG)
hello.cpp
#include <opencv2/core/core.hpp>#include <opencv2/highgui/highgui.hpp>#include <iostream>
int main(){ cv::Mat image = cv::imread("lena.bmp"); if (image.empty()) { std::cerr << "Could not load image"; return 1; }
cv::namedWindow("Image"); cv::imshow("Image", image); cv::waitKey(); return 0;}
hello.cpp
#include <opencv2/core/core.hpp>#include <opencv2/highgui/highgui.hpp>#include <iostream>
int main(){ cv::Mat image = cv::imread("lena.bmp"); if (image.empty()) { std::cerr << "Could not load image"; return 1; }
cv::namedWindow("Image"); cv::imshow("Image", image); cv::waitKey(); return 0;}
Overview: Hello WorldMakefile
CC=g++CFLAGS+=-std=c++0x `pkg-config opencv --cflags`LDFLAGS+=`pkg-config opencv --libs`
PROG=helloOBJS=$(PROG).o
.PHONY: all clean$(PROG): $(OBJS)
$(CC) -o $(PROG).out $(OBJS) $(LDFLAGS)
%.o: %.cpp$(CC) -c $(CFLAGS) $<
all: $(PROG)
clean:rm -f $(OBJS) $(PROG)
Overview: Hello Worldhello.cpp
#include <opencv2/core/core.hpp>#include <opencv2/highgui/highgui.hpp>#include <iostream>
int main(){ cv::Mat image = cv::imread("lena.bmp"); if (image.empty()) { std::cerr << "Could not load image"; return 1; }
cv::namedWindow("Image"); cv::imshow("Image", image); cv::waitKey(); return 0;}
Overview: Hello Worldhello.cpp
#include <opencv2/core/core.hpp>#include <opencv2/imgproc/imgproc.hpp>#include <opencv2/highgui/highgui.hpp>#include <iostream>
int main(){ cv::Mat image = cv::imread("lena.bmp"); if (image.empty()) { std::cerr << "Could not load image"; return 1; } cv::blur(image, image, cv::Size(10, 10));
cv::namedWindow("Image"); cv::imshow("Image", image); cv::waitKey(); return 0;}
Add a filter to blurthe image before
displaying it.
hello.cpp
#include <opencv2/core/core.hpp>#include <opencv2/imgproc/imgproc.hpp>#include <opencv2/highgui/highgui.hpp>#include <iostream>
int main(){ cv::Mat image = cv::imread("lena.bmp"); if (image.empty()) { std::cerr << "Could not load image"; return 1; } cv::blur(image, image, cv::Size(10, 10));
cv::namedWindow("Image"); cv::imshow("Image", image); cv::waitKey(); return 0;}
Overview: Hello World
Python: Display an image file
import cv2
image = cv2.imread("lena.bmp");if image.empty(): print "Could not load image" exit(1)
cv2.namedWindow("Image")cv2.imshow("Image", image)cv2.waitKey()
Similar structureand naming as C++
version meansPython is good for
prototyping.
Video from IP camera w/ RTSP!#include <opencv/cxcore.h> #include <opencv/highgui.h>
int main(int argc, char* argv[]){
cv::Ptr<CvCapture> capture = cvCaptureFromFile("rtsp://admin:[email protected]/video");
cv::namedWindow("Frame");for (;;){
cv::Mat frame = cvQueryFrame(capture);cv::imshow("Frame", frame);
if (cv::waitKey(1) >= 0)break;
}return 0;
}
Network comm.,RTSP protocol, etc.is all handled for youso all you have to do
is process eachframe as an image(a cv::Mat object).
A Selection of Functionality
● Image enhancement– Noise reduction, local contrast enhancement
● Object classification and tracking– Track the paths that objects take in a scene
– Differentiating between cars and trucks
● Face detection and recognition– Identify faces seen in images or video.
Image Enhancement
Many many algorithms. Here are a few:● Deconvolution – used to reduce focus blur or
motion blur where the motion is known.● Unsharp masking – increases sharpness and
local contrast (like WDR)● Histogram equalization – stretches contrast
and somewhat corrects for over- or under-exposure.
Image Enhancement: Demo!● Deconvolution – Reducing motion blur below
where the motion is known.
Image Enhancement: Demo!● Deconvolution – Can also be used for poor
camera focus, but the parameters of the blur must be estimated in advance.
Image Enhancement: Demo!● Deconvolution – Can also be used for poor
camera focus, but the parameters of the blur must be estimated in advance.
Generated using OpenCV example: /opencv/samples/python2/deconvolution.py
Image Enhancement
● Histogram equalization: equalizeHist(img, out)
Image Enhancement
● Histogram equalization: equalizeHist(img, out)
Increases therange of intensities
in an image, therebyincreasing contrast.
Object detection and tracking
● Foreground/background segmentation – identify objects moving in a scene.– cv::BackgroundSubtractorMOG2
● Histogram backprojection – identify objects by their colour (even if they're not moving).– cv::calcBackProject()
● Camshift tracking – track objects by their colour.– cv::CamShift
Face Detection and Recognition
Face detection and recognition
● Detection:– Haar cascade – detect faces by identifying
adjacent light and dark regions.– cv::CascadeClassifier
● Recognition:– Eigenfaces classifier – for facial recognition– cv::FaceRecognizer
Face detection: C++cv::CascadeClassifier profileFaceCascade;profileFaceCascade.load("haarcascade_profileface.xml");
std::vector<cv::Rect> faceRects;profileFaceCascade.detectMultiScale(image, faceRects);
cv::Mat foundFacesImage = image.clone();for (std::vector<cv::Rect>::const_iterator rect = faceRects.begin(); rect != faceRects.end(); ++ rect){
cv::rectangle(foundFacesImage, *rect, cv::Scalar(0, 0, 255), 3);}
cv::namedWindow("Faces");cv::imshow("Faces", foundFacesImage);cv::waitKey();
Face detection: C++cv::CascadeClassifier profileFaceCascade;profileFaceCascade.load("haarcascade_profileface.xml");
std::vector<cv::Rect> faceRects;profileFaceCascade.detectMultiScale(image, faceRects);
cv::Mat foundFacesImage = image.clone();for (std::vector<cv::Rect>::const_iterator rect = faceRects.begin(); rect != faceRects.end(); ++ rect){
cv::rectangle(foundFacesImage, *rect, cv::Scalar(0, 0, 255), 3);}
cv::namedWindow("Faces");cv::imshow("Faces", foundFacesImage);cv::waitKey();
OpenCV comes withother classifier XML
files for detecting otherthings (e.g eyes,
glasses, profile faces)
Face detection
● Can be defeated with makeup...
Face detection
● ... or with special glasses containing IR LEDs.
Conclusion
● OpenCV is for image/video processing and computer vision.
● Free and open source (BSD licensed)● Cross-platform and actively developed (also
downloaded over 3 million times)!● This presentation covered just a few of the over
2,000 algorithms available in OpenCV.
More Information
● Official Page: http://opencv.org
● Tutorials: http://docs.opencv.org/doc/tutorials/tutorials.html
● Books: