Intel® RealsensE™Depth Camera technologyMachine Vision Technology Forum 2017

Miro Mlejnek

Technical Sales Manager, New Technology Group, Intel® Corporation

2

Legal DisclaimerINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, ORINFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTORWAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.

Code names featured are used internally within Intel to identify products that are in development and not yet publicly announced for release. Customers, licensees and other third parties are not authorized by Intel to use code names in advertising, promotion or marketing of any product or services and any such use of Intel's internal code names is at the sole risk of the user.

*Other names and brands may be claimed as the property of others.

Copyright © 2017, Intel Corporation. All rights reserved.

3

Content• Intel® RealSense™ Technology

Introduce Market Focus D400 Series of Products

• Intel® RealSense™ Depth Cameras Overview and Positioning Product Brief Software Roadmap

• Product Briefs

4

Intel® RealSense™ Technology

Intel® RealSense™ Technology is a leader in bringing human like depth sensing and

intelligence to devices and machines

Stereo Matching

• Triangulation reveals depth:

Internal geometric calibration

Relative orientation calibration

Pixel matching

5

6

WHAT IS DEPTH?Color Image from a Color Camera: Each PIXEL has a Color (Red-Green-Blue) Value

Depth Image from a Depth Camera: Each PIXEL has a Depth (distance from camera) Value

The Depth Map conveys the distance information between points on an object's surface and the camera

7

HOW CAN THIS DATA BE USED?

Visualize & ProcessSense and Capture

SDK Tools for prototyping

Samples Codes & APIs for applications

Design & Validation

Devices and applications that can use this data – Ex. Robots,

3D Scans/ Models, Person/Object tracking,

background segmentation, Collision Avoidance etc.

Integrate into Applications & Products

Raw Depth Map

Intelligence from depth information

Enhanced Depth perception enables smarter devices and applications across multiple target markets

Market Focus

Robotics

Virtual Reality

Mobile Phones

Autonomous Driving

Broad Market

Drones

8

9

Bolster your Prototype with D400 Series Computer Vision

Develop Applications with Ease

Integrate Enhanced Depth Perception into products

Make Smart Devices Smarter with D400 Series

Intel® RealSense™ D400 Series Value Proposition

Intel® RealSense™ Depth Modules D400 Series

Intel® RealSense™ Depth Cameras D400 Series

Intel® RealSense™ Vision Processor

Inte

l® Re

alS

en

se S

DK

2.0

an

d

Visio

n L

ibra

ries fo

r inte

llige

nce

Intel® RealSense™ D400 Family Solutions

10

Recommended for: Academia, Makers, Enthusiasts, System Integrators,

Product integration for <10K units

Recommended for: Customers that need tight integration and are designing

their own board for maximum efficiency

Recommended for: Custom designs and those with experience building

camera modules

Intel® RealSense™ Depth Camera Intel® RealSense™ Depth Module D400 Series with Intel® RealSense™ Vision Processor D4 based card

Intel® RealSense™ Depth Module D400 Series with Intel® RealSense™ Vision Processor D4

Recommended for: Customers that need tight integration, but are not

designing a custom board

Intel® RealSense™ Vision Processor D4

+

11

Introducing the New D400 Series Depth Cameras

Intel® RealSense™ Depth Cameras D415/D435

• The two cameras are supersets of the

functionality of our offered modules

• Higher resolution depth of up to

1280x720

• Frame rates up to 90 FPS

• Improved range (10m+)*

• High accuracy & large Field of View

• Best performance in sunlight &

outdoor environments

Intel® RealSense™Depth Camera D415/D435

* Varies depending on performance accuracy, scene and light conditions

Intro

Our new generation depth cameras brings superior

capabilities – better algorithms, wider range of lighting, better

use of visible light, higher accuracy depth

D400 Series

D400 Series

12

Providing a path to System IntegrationModules Intel® RealSense™ Depth Camera D415

Intel® RealSense™ Depth Camera D435The Intel® RealSense™ Depth Camera D415 and D435 Differences:

Depth Field of View• D415 (H x V x D) 69.4 x 42.5 x 77• D435 (H x V x D) 91.2 x 65.5 x 100.6

Shutter Technology• D415 has rolling shutter• D435 has a global shutter

Pricing• D415 $TBD Retail• D435 $TBD Retail

Application• D415 general all around depth camera• D435 provides wider field of view and better

high speed image capture

Intel® RealSense™

Camera D415(D4 vision Processor + D415 Module)

Intel® RealSense™

Camera D435(D4 Vision Processor + D430 Module + RGB Camera)

Intel® RealSense™

ModuleDescription

D400 D400 Series

Passive, Stereo, Rolling Shutter

D410D400 Series

Active, Stereo, Rolling Shutter

D415 D400 Series

Active, Stereo, Rolling Shutter + RGB

D420D400 Series

Passive, Wide Stereo, Global Shutter

D430 D400 Series

Active, Wide Stereo, Global Shutter

* Photo from FLIR Website

Bundled CamerasSystem Integration

Boxed CameraDeveloper/starter kit

Single Box Camera

USB cable, Mount accessories included

Developer toolkit:

• Broad API support, sample codes

• Broad SW framework support

Bulk packaging (MOQ of 30)

No accessories

Same form factor as single box kit but at lower per cost per unit

Ideal to embed into designs

Developers, makers/enthusiasts, academia Start-ups, system integrators

Camera to be packaged as single unit for development or in bulk

for product integration

Boxed camera shortens development timelineBulk Pack camera provides easy migration

from prototype to productionValue Prop

Target Audience

Two Form Factors for Series D400 Depth Camera

13

14

Intel® RealSense™ SDK 2.0

Intel® RealSense™ Viewer

Intel® RealSense™ Depth Quality Test Tool for Intel RealSense Camera

Debug Tools

Examples

3rd Party Wrappers*Intel® RealSense™ Dynamic Calibration

Calibration Read/Write Data guidance

Device Firmware update tool for Intel® RealSense™ Technology

Open Source, Cross Platform support for rapid development Intel® RealSense™ Vision Libraries & Apps Available on GitHub and release.intel.com/develop

Tools

Capture,Multi Cam

Depth color Alignment

PointCloud

Intel® RealSense™ SDK 2.0 enables rapid prototyping & application development

* Other names and brands may be claimed as the property of others

* Some/all 3rd party wrappers may not be available at the same time. Please check GitHub for availability

Coming Soon

Intel® RealSense™ SDK 2.0 – Tools

Intel® RealSense™ Viewer

Depth Quality Test tool for Intel®

RealSense™ camera

Debug Tools

Capture, View and Manipulate Depth Streams

Playback/Record

3D PointCloud

Presets, Filtering, Optical Power

Test Setup with camera position and target

Control settings– resolution, gain, exposure, distance to target, error etc.

Calculate z-accuracy, standard deviation

Data Collection

FW logger

Terminal access

Camera Enumeration

Interact and Process your data seamlessly

15* Other names and brands may be claimed as the property of others

16

Intel® RealSenseTM sdk 2.0 - EXAMPLES

Capture

Depth Color Alignment

PointCloud

Access multiple streams – left & right imagers, depth and RGB Run multiple cameras Dynamic and Fixed Range Capture frames and write last frame to disk

Librealsense/examples:rs-capture

rs-multicamrs-capture-2disk

Librealsense/examples:rs-pointcloud View, Manipulate Pointcloud, add texture, color overlays

Librealsense/examples/depth-color-alignment

2 depth levels for background replacement

Librealsense/examples/opencv

Acquire, Display Color frame, Display IR frame

Embed Sample codes into your Applications with Ease

Intel® RealSense™ SDK 2.0 – 3rd party API Wrappers

Programming Lang/ Wrappers

Intel® RealSense™ wrappers supporting sample apps for capturing, viewing and manipulating depth & RGB streams

All RealSense APIs will be supported for platforms below

Example-displaying IR frame

1 Code snippet from IR_sample.cpp

2 Compile and run

3 Result

Access RealSense APIs on the framework of your choice

17* Other names and brands may be claimed as the property of others

18

Segment roadmap - Hardware2017 2018

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

RealSense™ Camera SR300

RealSense™ Camera R200

RealSense™ Camera ZR300

RealSense™ AIO Device based on LR200 and TM1 non-POR variants

Euclid

RealSense™ Depth Camera D400 series

Intel® RealSense™ Depth Camera D435*

Intel® RealSense™ Depth Camera D415*

Creative SR300 Dev Kit

Aaeon Robotic Dev Kit

Creative R200 Dev Kit

Intel R200 Dev Kit

ZR300 Dev Kit*

Available, Sell NowAvailable, Sell Now

POR, Not Launched

*Depth Cameras D415 and D435 will replace all current SKUs from previous generations.

20

Intel® RealSense™ Depth Camera D415

Product DescriptionThe Intel® RealSenseTM Depth Camera D415 is an USB-powered camera that includes depth sensors and a RGB sensor. It is ideal for makers, educators, hardware prototyping and software development. The camera is designed for ease of setup and portability. This camera is designed for to cover a wide range of uses including indoor/outdoor. It is ideal if you are wanting to add depth to your product or try it for the first time!

The Intel® RealSenseTM Depth Camera D415 comes with the Intel® RealSense™ SDK 2.0 enabling multiple computing languages and wrappers, sample code and tools.

Camera Features

Use Environment Indoor/Outdoor

Depth Technology Active IR Stereo (Rolling Shutter)

Depth FOV (H x V x D) 69.4 x 42.5 x 77

Depth Output Resolution &

Frame Rate

Up to 1280x720

Up to 90fps

Minimum Depth Distance (Min-Z) 0.3m

Maximum Range10m+. Varies depending on performance accuracy,

scene and light conditions

RGB Resolution 1080p @ 30fps

Physical

Form Factor Camera Peripheral

Connectors USB3.0

Product Dimension Length x Depth x Height.

99mm x 20mm x 23mm

Major Components

Camera Module Intel® RealSense™ Module D415

Vision Processor Card Intel® RealSense™ Vision Processor D4

Getting StartedTo buy: Please contact Intel® Authorized Distributor. • MM#: 961443 Single Units with camera, cable and tripod (ship to 55 countries)• MM# 962304 Bulk Packs of just the camera (MOQ 30)To develop: • Intel® RealSense™ SDK 2.0 (https://github.com/IntelRealSense)

Target Applications

Scanning Object or People tracking

Robotics Control via Motion

Depth Capture Object avoidance

Quick prototyping with depth Availability to embed into your product

21

Product DescriptionThe Intel® RealSenseTM Depth Camera D435 is an USB-powered camera that includes wider field of view depth sensors and a RGB sensor. It is ideal for makers, educators, hardware prototyping and software development. The camera is designed for ease of setup and portability. Perfect if you are doing capture with fast motion – global shutter allows for this. Virtual Reality is a good example.

Getting StartedTo buy: Please contact Intel® Authorized Distributor. • MM#: 961448 Single Units with camera, cable and tripod (ship to 55 countries)• MM# 962305 Bulk Packs of just the camera (MOQ 30) to developTo develop:• Intel® RealSense™ SDK 2.0 (https://github.com/IntelRealSense)

* Other names and brands may be claimed as the property of others

Intel® RealSense™ Depth Camera D435 Camera Features

Use Environment Indoor/Outdoor

Depth Technology Active IR Stereo (Global Shutter)

Depth FOV (H x V x D) 91.2 x 65.5 x 100.6

Depth Output Resolution &

Frame Rate

Up to 1280x720

Up to 90fps

Minimum Depth Distance (Min-Z) 0.2m

Maximum Range10m+. Varies depending on performance accuracy,

scene and light conditions

RGB Resolution 1080p @ 30fps

Physical

Form Factor Camera Peripheral

Connectors USB3.0

Product Dimension Length x Depth x Height.

90mm x 25mm x 25mm

Major Components

Camera Module Intel® RealSense™ Module D430 + RGB Camera

Vision Processor Card Intel® RealSense™ Vision Processor D4

Target Applications

Scanning/Depth Capture Object or People tracking

Robotics Control via Motion

Virtual reality Object avoidance

Quick prototyping with depth Availability to embed into your product

* Photo from FLIR Website

The Intel® RealSenseTM Depth Camera D435 comes Intel® RealSense™ SDK 2.0 enabling multiple computing languages and wrappers, sample code and tools.

D435 compared to D415The D435 is different from the D415 in it has global shutter IR cameras for better capture of fast moving scenes. It also has an additional 20⁰ depth field of view to capture more of the environment.

* Other names and brands may be claimed as the property of others

22

Intel® RealSense™ Development Kit sR300

Product DescriptionThe Intel® RealSenseTM Development Kit SR300 is a USB-powered camera that includes a depth sensor and an RGB sensor. It is ideal for makers, educators, hardware prototyping and software development. The development kit is designed for ease of setup and portability.

The Intel® RealSense™ SR300 Development Kit comes with Intel® RealSense™ Software Development Kit (SDK) for Windows

Camera Features

Use Environment Indoor

Depth Technology Coded Light

Depth Resolution & Frame Rate VGA @ 60 fps

Depth FOV (D×V×H) 90x59x73

Depth Output Resolution &

Frame RateUp to 640×480 @ 60 fps

Minimum Depth Distance (Min-Z) 0.2m

Maximum Range 1.5m. Varies depending on light conditions

RGB Resolution 1080p @ 30fps

Physical

Form Factor Camera Peripheral

Connectors USB3.0 Micro type B connector on peripheral

Box Dimension 103mm x 50mm x 156mm

Major Components

Camera Module Intel RealSense camera SR300

Interposer board Enable connection to host platform

Getting StartedTo buy: Please contact Intel® Authorized Distributor. • MM#: 945581 (ship to 55 countries)To develop: • Win: https://software.intel.com/sites/default/files/Getting_Started_0.pdf• Linux: https://github.com/IntelRealSense/librealsense

* Other names and brands may be claimed as the property of others

23