revision stack overview 2-16-17 for asia...
TRANSCRIPT
© Copyright 2017 Xilinx
EMBARGOED UNTIL 7:00AM EASTERN, MARCH 13, 2017
Stack Overview
© Copyright 2017 Xilinx 1 | P a g e
Overview
The Xilinx reVISION stack includes a broad range of development resources for platform, algorithm and application development. This includes support for the most popular neural networks including AlexNet, GoogLeNet, SqueezeNet, SSD, and FCN. Additionally, the stack provides library elements including pre‐defined and optimized implementations for CNN network layers, required to build custom neural networks (DNN/CNN). The machine learning elements are complemented by a broad set of acceleration‐ready OpenCV functions for computer vision processing. For application level development, Xilinx supports industry‐standard frameworks including Caffe for machine learning and OpenVX for computer vision. The reVISION stack also includes development platforms from Xilinx and third parties, including various types of sensors.
© Copyright 2017 Xilinx 2 | P a g e
COMPUTER VISION
xFOpenCV Library Functions
OpenCV library functions are essential to developing many computer vision applications. Xilinx’s xFAST library for computer vision, based on key OpenCV functions, will allow you to easily compose and accelerate computer vision functions in the FPGA fabric through SDx or HLx environments. In addition, xFAST functions are consistent with OpenCV and are optimized for performance, resource utilization and ease of use. Visit www.xilinx.com/revision for details.
Thousands of functions in OpenCV 3.1 library for Cortex A9 and Cortex A53 OpenCV functions (including the OpenVX subset) available as a library of optimized functions for Xilinx SoCs Complete library user guide with device utilization and performance
The figure below details a typical design flow with reVISION for Computer Vision.
© Copyright 2017 Xilinx 3 | P a g e
MACHINE LEARNING
Frameworks, Networks and Functions
Xilinx’s SoCs are an ideal fit for machine learning achieving up to 6x efficiency over the state‐of‐the‐art embedded GPUs, Xilinx’s reVISION stack removes the traditional FPGA design barrier by allowing you to quickly take a trained network and deploy it on Zynq SoCs and MPSoCs for inference.
Features:
Full software stack for deploying machine learning applications Hardware optimized libraries supporting Conv, ReLU, Pooling, Dilated conv, Deconv, FC, Detector & Classifier, SoftMax layers Caffe inter‐operability allows easy porting from Proto‐Text files for Network definition and trained weights Optimized reference models available for a wide range of network topologies, such as AlexNet, GoogLeNet, SqueezeNet, FCN and SSD Networks can be customized through software running on ARM processor without compiling for FPGA
The figure below details a typical design flow with reVISION for Machine Learning.
© Copyright 2017 Xilinx 4 | P a g e
reVISION Design Examples for Machine Learning and Computer Vision
The reVISION stack includes four initial design examples (with more to come) that are intended to get you up‐and‐running in a very short period of time. These design examples will help you easily see the distinct advantage Xilinx All Programmable SOCs have in high performance embedded vision applications. The following is a brief description of these four design examples.
LK Dense Optical Flow @ 4K60 – Real‐time dense implementation of optical flow, detecting object motion for every single pixel. This example uses non‐iterative, non‐pyramidal
implementation on 4K@60 FPS input coming from a Sony IMX274 sensor via the MIPI interface Stereo Vision
– Real‐time stereo disparity map calculation including remap, rectification and local block matching. It can process dual 1080p30 stereo camera input via USB3
Deep Learning: GoogLeNet – GoogleNet benchmark with INT8 demonstrated using standard ImageNet inputs.
Combination of dense optical flow, stereo vision and deep learning example designs (the three described above) – Using combination of MIPI sensors and USB3 cameras, the design represents a real‐life use case of autonomous vision system by combining dense
optical flow, stereo vision and CNN network into a single design example.
© Copyright 2017 Xilinx 5 | P a g e
SDSoC Environment Overview
The reVISION stack builds on the SDSoC Environment. SDSoC enables algorithm and/or application development in C, C++ and/or OpenCL using the reVISION resources. The SDSoC Environment can also be used to expand the reVISION resources with new acceleration‐ready software libraries. Familiar Embedded C/C++/OpenCL Application Development Experience The SDSoC™ development environment provides a familiar embedded C/C++/OpenCL application development experience including an easy to use Eclipse IDE and a comprehensive design environment for heterogeneous Zynq® All Programmable SoC and MPSoC deployment. Complete with the industry's first C/C++/OpenCL full‐system optimizing compiler, SDSoC delivers system level profiling, automated software acceleration in programmable logic, automated system connectivity generation, and libraries to speed programming. It also enables end user and third party platform developers to rapidly define, integrate, and verify system level solutions and enable their end customers with a customized programming environment.
© Copyright 2017 Xilinx 6 | P a g e
Boards, Kits and Modules
The reVISION stack is targeted to both the Zynq SoC and Zynq MPSoC families. Xilinx and its Alliance Program members produce several boards that can enable your development with the reVISION stack. The following is a list of Xilinx development boards and sensors supported by the stack. Xilinx reVISION Targeted Boards (& BSPs) & Kits
© Copyright 2017 Xilinx 7 | P a g e
Connectivity and Sensor Support
Xilinx’s Zynq platforms provide acceleration of computer vision and machine learning algorithms for fast response, the reconfigurability required for hardware platform developers to rapidly upgrade to the best available type and mix of sensors, and the ability to rapidly upgrade connectivity to new machines and/or the cloud.
Xilinx supports a number of connectivity standards for sensor interfacing as well as video input and output.
Interface Type Provider Description
LVDS Sensor Xilinx Natively supported in Zynq‐7000 and Zynq UltraScale+ devices. Refer to device datasheet for more information.
MIPI‐CSI 2.0 Sensor Xilinx MIPI CSI2 Receiver Subsystem and MIPI CSI 2 Transmitter Subsystems implement the Mobile Industry Processor Interface (MIPI) based Camera Serial Interface (CSI‐2) according to version 1.1 on Xilinx's UltraScale+™ devices allowing users to capture raw images from MIPI CSI2 sensors.
logiSLVS_RX Camera Sub‐LVDS Receiver
Sensor Xylon IP core supporting interfacing of ultra‐high resolution Sony CMOS image sensors to image signal processing pipelines and application processors implemented in Xilinx All Programmable devices
HDMI In/Out Xilinx HDMI TX and RX subsystems. The HDMI Subsystems are designed in compliance with the HDMI Forum version 2.0 of the HDMI specification.
DisplayPort In/Out Xilinx DisplayPort LogiCORE™ and DisplayPort TX and RX subsystems help users implement DisplayPort video interface as defined by VESA DisplayPort v1.2 specification.
UHD‐SDI (up to 12G) In/Out Xilinx UHD Serial Digital Interface (UHD‐SDI)is used for the transport of uncompressed digital video streams up to 4K resolutions over coax cable. The LogiCORE™ IP UHD‐SDI interface provides receiver and transmitter interfaces for the SMPTE SD‐SDI, HD‐SDI, 3G‐SDI, 6G‐SDI and 12G‐SDI standards.
GigE Vision In/Out Sensor to Image
GigE solution consists of one or more FPGA IP cores to design GigE Vision compliant devices mainly for the machine vision market.
© Copyright 2017 Xilinx 8 | P a g e
USB3 Vision In/Out Sensor to Image
U3V solution consists of one or more FPGA IP cores to design U3V compliant camera applications, primarily for the machine vision market.
Copyright 2017 Xilinx, the Xilinx logo, Artix, ISE, Kintex, Spartan, Virtex, Zynq, Vivado, UltraFast and other designated brands included herein are trademarks of Xilinx in the United States and other countries. All other trademarks are the property of their respective owners.