icvs2013 - opencv for ios
Post on 02-Jun-2018
220 Views
Preview:
TRANSCRIPT
-
8/10/2019 ICVS2013 - Opencv for IOS
1/38
OpenCV for iOS
ICVS 2013
Alexander Shishkov
alexander.shishkov@itseez.com
-
8/10/2019 ICVS2013 - Opencv for IOS
2/38
Agenda
1.
OpenCV for iOS2.
Hello, World! application
3.
OpenCV camera example
-
8/10/2019 ICVS2013 - Opencv for IOS
3/38
Why bother?
Its fun
Its trendy
It can make you rich
Its impressive way to
show some technology
-
8/10/2019 ICVS2013 - Opencv for IOS
4/38
OpenCV for iOS
Started with GSoC 2011 project
Firstly released 2012-07-04 with OpenCV 2.4.2
Thanks to C++ !Objective C interoperabilitynative OpenCV for iOS just works (excludingsome parts of highguimodule):
working with camera reading/writing video files
namedWindow, imshow,
-
8/10/2019 ICVS2013 - Opencv for IOS
5/38
-
8/10/2019 ICVS2013 - Opencv for IOS
6/38
Changelog
2.4.6
fixed bug with incorrect video saving with CvVideoCamera
added rotateVideoflag to the CvVideoCameraclass
added functions for conversion between
UIImageand cv::Mat
-
8/10/2019 ICVS2013 - Opencv for IOS
7/38
iOS specific part
Camera stack
#import
CvAbstractCamera, CvPhotoCamera, CvVideoCamera
Conversion functions #import
UIImageToMat, MatToUIImage
-
8/10/2019 ICVS2013 - Opencv for IOS
8/38
-
8/10/2019 ICVS2013 - Opencv for IOS
9/38
iOS Anatomy
Apps
Media (CoreImage, CoreGraphics, OpenGL ES,
AVFoundation)
Cocoa Touch (UIKit etc.)
Core Services (Databases, iCloud, GDC, )
Core OS (Unix, POSIX, )
-
8/10/2019 ICVS2013 - Opencv for IOS
10/38
OpenCV+iOS Anatomy
Apps
Media (CoreImage, CoreGraphics, OpenGL ES,
AVFoundation)
Cocoa Touch (UIKit etc.)
Core Services (Databases, iCloud, GDC, )
Core OS (Unix, POSIX, )
OpenCV for iOS is just another framework
-
8/10/2019 ICVS2013 - Opencv for IOS
11/38
Framework layout
-
8/10/2019 ICVS2013 - Opencv for IOS
12/38
OpenCV Framework
Prebuilt framework
from opencv.org
From source code
cd ~/
git clone https://github.com/Itseez/opencv.git
git checkout b 2.4 origin/2.4
python opencv/platforms/ios/build_framework.py ios
-
8/10/2019 ICVS2013 - Opencv for IOS
13/38
Agenda
1.
OpenCV for iOS2.
Hello, World! application
3.
OpenCV camera example
-
8/10/2019 ICVS2013 - Opencv for IOS
14/38
-
8/10/2019 ICVS2013 - Opencv for IOS
15/38
iOS Simulator iOS Device
Cost Free* $99 per year
Prerequisites OSX 10.7 + Xcode
4.3.x or later
+ iOS-5 capable
deviceTarget CPU x86 (SSE, no
Neon)ARM (Neon, no
SSE)
Speed fast slower
RAM lots of limited
OpenGL + +
Photo Library, StillPhoto processing
+ +
Camera - +
Accelerometer - +
Debugging,Logging
+ +
Performanceevaluation
+/- +
-
8/10/2019 ICVS2013 - Opencv for IOS
16/38
-
8/10/2019 ICVS2013 - Opencv for IOS
17/38
Hello world
#import
int main(int argc, char *argv[])
{
//printf("\n\nHello, world!\n");
@autoreleasepool {
//NSLog(@"\n\nHello, world!);
NSLog([[NSString stringWithUTF8String:"\n\nHello"]
stringByAppendingString:@", world!"]);
return 0;
}
Its a superset of C
Since OSX 10.7 & iOS 5 memory is automatically managed
#import == #include with automatic guards.
[recipient messageWithArg1:A andArg2:B!
];
-
8/10/2019 ICVS2013 - Opencv for IOS
18/38
Cocoa(Touch) key concept: MVC
View is created visually as aStoryboardelement
For each View component (image view,
button, slider) we add corresponding
IBOutlet-markedproperty in theController class
For each action we add IBAction-marked
method in the Controller
Model-View-Controller(For simple demos Model
is a part of Controller class.) View
Controller
Model
update update
user action notify
-
8/10/2019 ICVS2013 - Opencv for IOS
19/38
Class declaration
#import
@interface ViewController : UIViewController {
UIImage* image;
}
@property (weak, nonatomic) IBOutlet UIImageView *imageView;
@property (weak, nonatomic) IBOutlet UIBarButtonItem *loadButton;
-(IBAction)effectButtonPressed:(id)sender;
@end
@interface @end delimits class declaration
there must be one and only one base class (UIViewController in our sample)
data members are declared inside {}.
@property declares a property, IBOutletqualifier denotes outlets.
No need to declare overloaded methods, only the newly added
Actions have IBActionreturn type
-
8/10/2019 ICVS2013 - Opencv for IOS
20/38
Class implementation#import ViewController.h
@implementation ViewController
@synthesize imageView;
- (void)viewDidLoad {
[super viewDidLoad]; // call the superclass method
NSString* filename = [[NSBundle mainBundle]
pathForResource:@"helloworld" ofType:@"png"];
UIImage *image = [UIImage imageWithContentsOfFile:filename];
if( image != nil ) imageView.image = image;
}
@end
@implementation @end delimits class implementation
@synthesize automatically generates correct code for reading and writing properties
viewDidLoadis the place to put additional initialization. Xcode generates stub for it
Some methods, including viewDidLoad, require to call the original superclass method
To show the image, just assign it to the image property of UIImageViewer
-
8/10/2019 ICVS2013 - Opencv for IOS
21/38
Lets now add OpenCV!
-
8/10/2019 ICVS2013 - Opencv for IOS
22/38
Hello world1.
Download framework from OpenCV website
2.
Add framework to your project
3.
Change project language to Objective C++.
4.
Add #import to the *.pch file.
-
8/10/2019 ICVS2013 - Opencv for IOS
23/38
UIImage !cv::Mat
UIImage* MatToUIImage(const cv::Mat& m) {
}
void UIImageToMat(const UIImage* image, cv::Mat& m) {
}
These are not super-fast functions,
minimize such conversions
#import
-
8/10/2019 ICVS2013 - Opencv for IOS
24/38
Hello world
Add OpenCV code
if( image != nil ) {
cv::Mat m, gray;
UIImageToMat(image, m);
cv::cvtColor(m, gray, CV_RGBA2GRAY);
cv::GaussianBlur(gray, gray, cv::Size(5, 5), 1.2, 1.2);
cv::Canny(gray, gray, 0, 50);
m = cv::Scalar::all(255);
m.setTo(cv::Scalar(0, 128, 255, 255), gray);
imageView.contentMode = UIViewContentModeScaleAspectFit;
imageView.image = MatToUIImage(m);
}
-
8/10/2019 ICVS2013 - Opencv for IOS
25/38
Live demo #1
25
-
8/10/2019 ICVS2013 - Opencv for IOS
26/38
Agenda
1.
OpenCV for iOS2.
Hello, World! application
3.
OpenCV camera example
-
8/10/2019 ICVS2013 - Opencv for IOS
27/38
Adding Camera input
Camera is not supported by Simulator "
Setting up camera, retrieving frames,
displaying them, handling rotations etc.
takes a lot of code. Thats why OpenCV
encapsulates this logic.
Its built using delegate pattern. The
delegate will be ourViewController
.
-
8/10/2019 ICVS2013 - Opencv for IOS
28/38
Camera interfaceUpdate ViewControllerinterface part
#import
#import //or ios.h
using namespace cv;
@interface ViewController : UIViewController
{
UIImage* image;
CvVideoCamera* videoCamera;
}
@property (nonatomic, retain) CvVideoCamera* videoCamera;
Protocols and delegates are not exclusive things in standard Cocoacomponents; well-designed components may introduce custom protocols
and use delegates. See cap_ios.h for details.
ARC works with pointers to your own classes as well (e.g. with
CvVideoCamera*)
-
8/10/2019 ICVS2013 - Opencv for IOS
29/38
Camera interfaceSetup camera in ViewController viewDidLoadmethod
- (void)viewDidLoad
{
[super viewDidLoad];
self.videoCamera = [[CvVideoCamera alloc] initWithParentView:imageView];
self.videoCamera.delegate = self;self.videoCamera.defaultAVCaptureDevicePosition =
AVCaptureDevicePositionFront;
self.videoCamera.defaultAVCaptureSessionPreset =
AVCaptureSessionPreset352x288;
self.videoCamera.defaultAVCaptureVideoOrientation =AVCaptureVideoOrientationPortrait;
self.videoCamera.defaultFPS = 30;
}
CvVideoCameraclass is super-easy to use. Leave the defaultresolution, fps etc. or customize them as needed
Using lowest possible resolution and reasonable frame rate can save a
lot of power and make apps more responsive
-
8/10/2019 ICVS2013 - Opencv for IOS
30/38
Camera interface
Modify button callback
- (IBAction)captureButtonPressed:(id)sender {[self.videoCamera start];
}
- (void)processImage:(Mat&)image;
{
// Do some OpenCV stuff with the image
Mat image_copy;
cvtColor(image, image_copy, CV_BGRA2BGR);
// Invert image
bitwise_not(image_copy, image_copy);
cvtColor(image_copy, image, CV_BGR2BGRA);
}
-
8/10/2019 ICVS2013 - Opencv for IOS
31/38
iOS performance tips
Do expensive initializations once
(e.g. in NSViewControllersviewDidLoad
method)
Use smaller resolution and FPS
Plan your work with memory, avoid redundantmemory allocations and copies (reuse buffers)
Profile your apps using Xcode profiler But avoid premature optimization
-
8/10/2019 ICVS2013 - Opencv for IOS
32/38
iOS performance tips (2)
Use Accelerate.framework
Use OpenGL and CoreGraphics for rendering,
CoreImage for image processing
Use Grand Dispatch Central (implements data-
level and task-level parallelism)
Use NEON SIMD intrinsics & OpenGL ES shaders
-
8/10/2019 ICVS2013 - Opencv for IOS
33/38
Live demo #2
33
-
8/10/2019 ICVS2013 - Opencv for IOS
34/38
Multi-threading
Backends:
Intel Threading Building Blocks (Intel TBB)
Microsoft Concurrency Runtime
OpenMP
Grand Central Dispatch (GCD)
C=
void parallel_for_(const Range& range, const ParallelLoopBody& body);
class LoopBody : public cv::ParallelLoopBody
{
public:
void LoopBody::operator() (const cv::Range& range) const
{}
}
-
8/10/2019 ICVS2013 - Opencv for IOS
35/38
Future plans
Improve documentation and tutorials
Add iOS samples
Publish few applications in App Store Support iOS 7
-
8/10/2019 ICVS2013 - Opencv for IOS
36/38
Resources
http://developer.apple.com(including excellent introductory guides to
Objective-C, Cocoa and iOS programming)
http://docs.opencv.org/master/doc/tutorials/ios
(iOS tutorials for the latest OpenCV version) http://code.opencv.org/svn/gsoc2012/ios/trunk
(our GSoC 2012 iOS project repository)
http://answers.opencv.org
(Q&A forum for OpenCV)
-
8/10/2019 ICVS2013 - Opencv for IOS
37/38
Resources
Coming soon
OpenCV for iOS minibook
Chapters: Getting Started with iOS
Displaying Image from Resources
Linking OpenCV to iOS Project Detecting Faces with Cascade Classifier Printing Postcard
Working with Images in Gallery
Applying Retro Effect Taking Photos From Camera
-
8/10/2019 ICVS2013 - Opencv for IOS
38/38
Thank you!
Any questions? alexander.shishkov@itseez.com
admin@opencv.org
top related