tutorial practice session - khronos group · tutorial practice session import-export extension...

© Copyright Khronos Group 2017 -

Tutorial Practice SessionImport-Export Extension

Thierry Lepley - Cadence


Agenda

•Import-Export Functionality (10 mins)

- Principles & API

•Hands-on

- Use-case : keypoint detector and tracker (5 min)

- Exercise 1 : Export the detector/tracker (15 mins)

- Exercise 2 : Import the detector/tracker (15 mins)


Import/Export

Principles


Standard OpenVX Flow

vxVerifyGraph() Execute

graphs

Create data

objects and

graphs

(Graph validation

& optimization)

Get Data

from camera


Import-Export Extension Flow

Execute

graphs

Get Data

from camera

runtime

Import

Ahead of time

Deployment system

Create data

objects and

graphs

Export

objects/graphBinary

blob

Offline

Development System

(Desktop)

(Graph validation

& optimization)


What can be exported ?

• Data objects

- vx_image, vx_array, vx_pyramid, vx_scalar, …

- data object values optionally exported

• Delay objects

- vx_delay

• Graphs

- Exports anything needed to reproduce the same behavior at import

✓ Relationship between objects will be preserved between export and import

- ROI, delay slots, pyramid levels, …

- Data objects used by graphs


Benefits of Import-Export

• Minimize overheads on the target system

- Faster setup-time

- Simpler OpenVX runtime

- Simpler certification (ISO 26262, etc..)

• Enable more graph optimizations when offline

- No timing/resource constraints

- Example: generate & compile specialized code for graphs


Import/Export:

API


Object Export Options

• VX_IX_USE_NO_EXPORT_VALUES:

The import will create an object with uninitialized value

(Mandatory for graphs)

• VX_IX_USE_EXPORT_VALUES:

The import will initialize an object with the export value

• VX_IX_USE_APPLICATION_CREATE:

The application will provide the object at import time

(Mandatory for images created from handle)


Export API

vx_status vxExportObjectsToMemory (vx_context context,

vx_size numrefs,

const vx_reference *refs, const vx_enum *uses,

const vx_uint8 **ptr_addr, vx_size *length)

vx_status vxReleaseExportedMemory (vx_context context, const vx_uint8 *ptr)

uses[]refs[]

In

Out

Gra

ph

scalar

ExportBinary

blob

*(ptr_addr)

VX_IX_USE_NO_EXPORT_VALUES



In

Out

Graph


Import API

vx_import vxImportObjectsFromMemory (vx_context context,

vx_size numrefs,

vx_reference *refs, const vx_enum *uses,

const vx_uint8 *ptr, vx_size length)

vx_reference vxGetImportReferenceByName(vx_import import, const vx_char name)

vx_status vxReleaseImport (vx_import *import)

uses[]refs[]




NULL

NULL

NULL

Out

Gra

phImportBinary

blob

ptr refs[]

In

Out

Graph

In


Exercise Use Case & Setup


Use Case : Keypoint Detection & Tracking


Color

convert

Channel

Extract

Harris

Corner

Configuration

params

Detection graph

Detection and Tracking Graphs

Color

convert

Gaussian

pyramid

pyrLK optical flow

pyr 0

delay of pyramids

Tracking graph

Channel

Extract

pyr -1

pts 0pts -1

delay ofarray of vx_keypoint_t

Gaussian

pyramid

Configuration

params

Input Image(created from handle)


File Structureimport-export├── CMakeLists.txt├── LICENSE├── README.md├── data│ └── PETS09-S1-L1-View001.avi├── exercise1-export│ ├── CMakeLists.txt│ ├── feature_tracker.cpp│ ├── feature_tracker.hpp│ └── main_feature_tracker.cpp├── exercise2-import│ ├── CMakeLists.txt│ ├── feature_tracker_import.cpp│ ├── feature_tracker_import.hpp│ └── main_feature_tracker.cpp├── solution-exercise1-export│ ├── CMakeLists.txt│ ├── feature_tracker.cpp│ ├── feature_tracker.hpp│ └── main_feature_tracker.cpp├── solution-exercise2-import│ ├── CMakeLists.txt│ ├── feature_tracker_import.cpp│ ├── feature_tracker_import.hpp│ └── main_feature_tracker.cpp├── utils│ └── ...├── openvx│ ├── bin| └── include└── build

└── ...

Exercise 1

Exercise 2

Solution to exercise 1

Solution to exercise 2

Utility (opencv wrapper, timer etc…)

Prebuilt version of the Khronos

OpenVX ‘Sample’ implementation

Where the tutorial is built


Code Editing : QTCreator


Compiling/Checking Exercises

cd ~/import-exportmkdir build && cd buildcmake \

-DCMAKE_BUILD_TYPE=Debug \-DVX_INC_DIR=~/import-export/openvx/include \-DVX_LIB_DIR=~/import-export/openvx/bin \..

make

cd ~/import-export/buildbin/exercise1-export ../data/PETS09-S1-L1-View001.avi

1 - Build

2 - Test


Exercise 1:

Export OpenVX Objects


Detection and Tracking Graphs

delay of pyramids

Tracking graph

delay ofarray of vx_keypoint_t

Input Image(created from handle)

Detection graph


‘exercise1-export’ directory

class FeatureTracker:// Export the detector/tracker to a filevoid export2file(const char *fileName);

// ConstructorFeatureTracker(vx_context context,

const HarrisTrackerParams& params,vx_image input);

// Processingvoid detectKeypoints(void *newFrameHandle);void trackKeypoints(void *newFrameHandle);

feature_tracker.[hpp|cpp] main_feature_tracker.cpp

while (1) {inputVideo.read(cv_src_bgr);cvtColor(cv_src_bgr, cv_src_rgb, COLOR_BGR2RGB);

// Detectionif ( (frame_id % DETECTION_FREQUENCY) == 0) {

tracker->detectKeypoints(src_ptrs);}

// Trackingelse {

tracker->trackKeypoints(src_ptrs);}

}


Exercise 2:

Import OpenVX Objects


‘exercise2-import’ directory

class FeatureTracker:// ConstructorFeatureTracker(vx_context context,

const char *import_file, vx_image input);

// Processingvoid detectKeypoints(void *newFrameHandle);void trackKeypoints(void *newFrameHandle);

feature_tracker_import.[hpp|cpp] main_feature_tracker.cpp

while (1) {inputVideo.read(cv_src_bgr);cvtColor(cv_src_bgr, cv_src_rgb, COLOR_BGR2RGB);

// Detectionif ( (frame_id % DETECTION_FREQUENCY) == 0) {

tracker->detectKeypoints(src_ptrs);}

// Trackingelse {

tracker->trackKeypoints(src_ptrs);}

}

tutorial practice session - khronos group · tutorial practice session import-export extension...

Documents