Download - Introduction to OpenVX
Page 1System Software Lab., NCTU
Introduction to OpenVX
An open, royalty-free standard for cross platform acceleration of computer vision applications.
Presenter: Jia-Rung ChangInstructor: Yi-Ping You
The last modification date : 2016/08/10
Page 2System Software Lab., NCTU
Outline
OpenVX’s role in the Khronos ecosystemOpenVX in realityExperience from SynopsysOpenVX relevant paperConclusion
Page 3System Software Lab., NCTU
Outline
OpenVX’s role in the Khronos ecosystemOpenVX in realityExperience from SynopsysOpenVX relevant paperConclusion
Page 4System Software Lab., NCTU
What is OpenVX?
An open standard , not open source.Do not specify the details of implementation.
For cross platform acceleration of computer vision applications.
Every vendor can do their optimization for their own Soc.
Ex: NVIDIA, AMD and Synopsys,etc.
Currently,only few of vendors release their own implementation.
Page 5System Software Lab., NCTU
Vision Acceleration
Targeted at low-power, real-time applicationsMobile and embedded platforms
Portability across diverse heterogeneous processors
ISPs, Dedicated hardware, DSPs and DSP arrays, GPUs, Multi-core CPUs …
Doesn’t require high-power CPU/GPU ComplexLow-power host can setup and manage frame-rate vision processing pipeline
Page 6System Software Lab., NCTU
Khronos APIs for Vision Processing
ref: http://www.slideshare.net/embeddedvision/1-c01-khronostrevett
Page 7System Software Lab., NCTU
Layered Vision Processing Ecosystem
ref: http://www.slideshare.net/embeddedvision/1-c01-khronostrevett
Page 8System Software Lab., NCTU
OpenVX and OpenCV are Complementary
OpenCV OpenVX
Implementation Community driven open source library
Open standard API designed to be implemented by hardware vendors
Conformance Extensive OpenCV Test Suite butno formal Adopters program
Implementations must pass defined conformance test suite to use trademark
ConsistencyAvailable functions can vary depending on implementation / platform
All core functions must be available in all conformant implementations
Page 9System Software Lab., NCTU
OpenVX Framework Objects Definition
ContextThe object domain for all OpenVX objects.
KernelThe abstract representation of a computer vision function
NodeAn instance of a kernel that will be paired with a
specific set of the parameters.
GraphA set of nodes connected in a directed (only goes one-way) acyclic (does not loop back) fashion.
Page 10System Software Lab., NCTU
OpenVX Framework
ref: https://www.khronos.org/openvx/
Page 11System Software Lab., NCTU
Outline
OpenVX’s role in the Khronos ecosystemOpenVX in realityExperience from SynopsysOpenVX relevant paperConclusion
Page 12System Software Lab., NCTU
OpenVX Status
OpenVX 1.1 Specification released in 2016 at Embedded Vision Summit
OpenVX 1.0 in 2014Expands node functionality AND enhances graph frameworkThe sample implementation v1.1 from Khronos is failed to build.
The sample implementation v1.0.1 from Khronos can build,but can not fully support their own tutorial.Official tutorial uses AMDOVX(v1.0.1) as the default OpenVX libraries.
Page 13System Software Lab., NCTU
NVIDIA VisionWorks
The implementation for OpenVX in VisionWorks is not open source.Available on Ubuntu 14.04(64-bits)
Because it is not an open source project, we skip trying it.
Page 14System Software Lab., NCTU
AMD OpenVX (AMDOVX)
The official OpenVX tutorial use AMDOVX as the default libraries.In the tutorial
AMDOVX has better performance than the “Khronos sample implementation”.
The “Khronos sample implementation” version 1.1 is failed to build on my Ubuntu16.04 platform.
With AMD GPU will get better performance, but not necessarily.
AMD official Linux driver stop updating since 2016.
Page 15System Software Lab., NCTU
Outline
OpenVX’s role in the Khronos ecosystemOpenVX in realityExperience from Synopsys
Focusing on the idea, not the details.ref:Embeded VISION ALLIANCE
OpenVX relevant paperConclusion
Page 16System Software Lab., NCTU
Page 17System Software Lab., NCTU
Page 18System Software Lab., NCTU
Page 19System Software Lab., NCTU
Outline
OpenVX’s role in the Khronos ecosystemOpenVX in realityExperience from SynopsysOpenVX relevant paper
Does not focus on the details,but the possible direction of research for us.
Conclusion
Page 20System Software Lab., NCTU
Graph execution model in OpenVX
Synchronous blocking mode In that vxProcessGraph() will block until the graph has completed
Causing the synchronous processing of a graph.
Asynchronous single-issue-per-reference mode via vxScheduleGraph() and vxWaitGraph().
Schedules a graph for future executionWaits for a specific graph to complete.
Page 21System Software Lab., NCTU
Graph execution model in OpenVX
Synchronous blocking mode In that vxProcessGraph() will block until the graph has completed
Causing the synchronous processing of a graph.
Asynchronous single-issue-per-reference mode via vxScheduleGraph() and vxWaitGraph().
Schedules a graph for future executionWaits for a specific graph to complete.
Page 22System Software Lab., NCTU
Scheduling Methods for OpenVX Programson Heterogeneous Multi-core Systems
Authors: Tzu-Hsiang Lin, Cheng-Yen Lin, and Jenq-KuenLee
Department of Computer Science, National Tsing-Hua University, Taiwan
Source: PDPTA'15 The 21st International Conference on Parallel and Distributed Processing Techniques and Applications
Page 23System Software Lab., NCTU
Keynotes
AssumptionA linear relationship for data size and computation time, data size and data transfer time
The graph execution modelA directed acyclic graph (DAG) that determines the computation process of an OpenVX application
Optimization opportunitiesDispatching graphs to a different target, such as multicore CPUs with C, OpenMP runtime, OpenCLon GPUs, or a dedicated hardware. Memory locality and system throughput.
Page 24System Software Lab., NCTU
Scheduling Algorithm(Node coarsen algorithm)Two phases
Node coarsen algorithm– Calculating the nodes that whether belonging to specific cluster via
thier ranking algorithm,including computation time and data transfer time between host and targets.
– vxVerifyGraph(graph)» Nodes in the same cluster will be forced to dispatch to the same target in the
scheduling phase» We only select the nodes which can reduce data transfer time or largely shorten the
computation time into the same cluster
Node scheduling algorithm– Dispatching the clustered nodes.
» Otherwise, finding the most free/available target compared to the preferred targets for the nodes, and rank both of them.
» According to the rank,dispatch it.– vxProcessGraph(graph)
» The remaining nodes not clustered will be scheduled for the target in the scheduling(this) phase.
Keynotes
Page 25System Software Lab., NCTU
Keynotes
Experiment results
Page 26System Software Lab., NCTU
Supporting Real-Time Computer Vision Workloads using OpenVX on
Multicore+GPU Platforms
Authors: Glenn A. Elliott, Kecheng Yang, and James H. Anderson
Department of Computer Science, University of North Carolina at Chapel Hill
Source: 2015 IEEE Real-Time Systems Symposium
Page 27System Software Lab., NCTU
Keynotes
The OpenVX model imposes three significant implications on real-time scheduling
Having no notion of a repeating (i.e., periodic or sporadic) task, and lacks any framework for real-time analysis.
The key issue is the allowance of “back edges” that can create cycles in a graph.
Does not defining a threading model for graph execution.Requiring a graph to execute end-to-end before it may be executed again.
Significantly hindering the ability to exploit parallelism by “pipelining” portions of a graph’s structure
Page 28System Software Lab., NCTU
Keynotes
Implementation model
Page 29System Software Lab., NCTU
Outline
OpenVX’s role in the Khronos ecosystemOpenVX in realityExperience from SynopsysOpenVX relevant paperConclusion
Page 30System Software Lab., NCTU
Conclusion
The Possible direction of research in OpenVXThe world-first OpenVX benchmarkScheduling AlgorithmOpenVX in embedded/real-time system
Page 31System Software Lab., NCTU
Main References
Khronos websiteMay 2014 Embedded Vision Summit Technical Presentation: KhronosMay 2016 Embedded Vision Summit Technical Presentation: SynopsysOpenVX tutorial(official)AMDOVXkhronos OpenVX sampleOpenVX specification