"lessons learned from bringing mobile and embedded vision products to market," a...
TRANSCRIPT
Copyright © 2016 ARM Ltd 1
Lessons Learned from Bringing Mobile and
Embedded Vision Products to Market Tim Hartley, Product Manager, ARM
May 3, 2016
Copyright © 2016 ARM Ltd 3
Increased processing
power
Improving computer
vision techniques
Opportunities for new and
existing markets
The Vision Opportunity
Copyright © 2016 ARM Ltd 4
Growth Maturity Innovation
Embedded Vision Industry Growth
Embedded
Vision Markets
Platforms &
Tools Mark
et
penetr
ation
Copyright © 2016 ARM Ltd 5
Challenges for Companies Developing
Vision Products
Sensors
Processors & Platforms
Power & Thermal
Software Enablement
Copyright © 2016 ARM Ltd 8
Sensor Summary
Speed
• 90 /120fps necessary for VR & AR
• Above 30 or 60 fps often not supported
3D scanning
• Quality of calibration
• Depth map technology implementation
Accuracy & Range
• Consistent color characteristics
• Accurate focus
2D Scanning
• Quality of textures
• Low light performance
Copyright © 2016 ARM Ltd 10
Choosing the Right Platform: Processors
CPU
• Core count?
• Core groups?
• SIMD?
GPU
• GPGPU?
• OpenCL?
DSP
• Features?
• Flexibility?
• SW library?
FPGA
• SW library?
• OpenCL?
ASIC
• Dev costs
Copyright © 2016 ARM Ltd 11
Processor Choices: Costs & Benefits
Ease/speed of
development
Performance/
watt
Flexibility
Economy
Copyright © 2016 ARM Ltd 12
Choosing the Right Platform: a Case Study
Size and weight
Price
Performance
(Power)
© Raspberry Pi Foundation
© Hardkernel
Odroid XU4
• 8-core CPU
• 6-core GPU
• $74
Raspberry Pi 2
• 4-core CPU
• $35
Copyright © 2016 ARM Ltd 14
Low power Sustained
performance
Burst for
responsiveness
Power & Thermal P
ow
er/
Perf
orm
ance
Time
Maximum Sustainable Power
Trip points
Copyright © 2016 ARM Ltd 17
Over 2,500 algorithms & growing!
But not optimal on low power platforms when targeting…
• Most specialized vision processors
• Heterogeneous platforms
• GPUs
• CPU vector extensions like NEON
Looking forward…
• New developments coming — e.g., better heterogeneous low power support
• Unlikely to provide long term cross-platform optimal performance
Doesn’t Everyone Just use OpenCV?
Copyright © 2016 ARM Ltd 18
• Platform lock-in
• Proprietary vision hardware often means proprietary middleware
• Refactoring software for new platforms is time-consuming
• Feature lock-out
• Does the middleware let you exploit latest developments in vision algorithms?
• Sub-optimal use of processors
• Can the middleware target all available cores?
• How do you target multiple core types?
Vision Software: Key Risks
Copyright © 2016 ARM Ltd 19
GPU DSP CPU GPU DSP CPU
CPU GPU DSP
Application
Middleware
Drivers
Hardware
Traditional middleware Middleware based on nodes and graphs
Middleware Architecture Today vs. Tomorrow
Graph
APIs
Vision
Nodes
Compute
Cores
Copyright © 2016 ARM Ltd 20
OpenVX to the Rescue?
• A new computer vision API from Khronos
• Based on vision nodes and graphs
• Aimed at low power platforms from the outset
• Has potential for:
• Performance portability
• Heterogeneous deployment of vision pipelines
• Initial version defines 40 vision building blocks
• Some omissions makes extensions important
• OpenVX-SC (safety critical) version in development
Copyright © 2016 ARM Ltd 21
Optimise
Build nodes into
dependency
graphs
Load-balance
Merge /
recompile
Merge /
recompile
Merge /
recompile
Runtime
Framework
Application
Vision API
Interface
Graph
Builder
GPU Vision
Nodes
CPU Vision
Nodes
DSP Vision
Nodes
Create
GPU
binaries
Create
DSP
binaries
GPU
DSP
CPU
HW
Repository of CV
algorithms Vision Middleware
Heterogeneous Vision Framework
Compile
Power Performance
Copyright © 2016 ARM Ltd 22
In Summary
Expect and plan for power & thermal problems
Be aware of the risks of different middleware choices
Expect change
Copyright © 2016 ARM Ltd 23
• The trademarks featured in this presentation are registered and/or
unregistered trademarks of ARM Limited (or its subsidiaries) in the EU
and/or elsewhere. All rights reserved. All other marks featured may be
trademarks of their respective owners
• Copyright © 2016 ARM Limited
Thank you!