1© 2018 The MathWorks, Inc.

MATLAB® & Simulink®

Development, Simulation and Integration Platform

for Your ADAS Functions

Gaurav Tomar Automotive Industry Manager EMEA

Alexander Schreiber Principle Application Engineer and Technical Account Manager

Percepti

onControl

Planning

Control

Planning

Perception

3

Deploy MATLAB Algorithms and Applications

Analyze Data

Data

exploration

Preprocessing

Domain-specific

algorithms

Access Data

Sensors

Files

Databases

Deploy

Embedded

devices

Desktop apps

Enterprise

systems

Develop

AI model

Algorithm

development

Modeling &

simulation

4

Model-Based Design and Code Generation

for AEB Sensor Fusion

1.5M km of recorded data

3+ years of driving time

12 hours re-simulation

6

Enterprise IntegrationIntegrate MATLAB analytics into your technology stack

MathWorks Managed Clusters, Preconfigured Clusters in Amazon Web Services (AWS), Cloud Solutions from MathWorks Partners, Custom Infrastructure for AWS, Azure, and Others, Use MATLAB in the Cloud

7

Designing Autonomous Systems

Sense

PerceiveDecide

& Plan

Act

8

Computer Vision System ToolboxRobotics System Toolbox

▪ Segment and register lidar point clouds

▪ Lidar-Based SLAM: Localize robots and

build map environments using lidar sensors

Mapping of environments using sensor data

Designing Autonomous Systems

Sense

PerceiveDecide

& Plan

Act

9

Designing Autonomous Systems

▪ Object detection and tracking

▪ Semantic segmentation using deep learning

Understanding the environment using computer

vision and deep learning techniques

Neural Network ToolboxComputer Vision System ToolboxAutomated Driving System Toolbox

CamVid Database: Brostow, Gabriel J., Julien Fauqueur, and Roberto Cipolla. "Semantic object classes in video: A high-definition ground truth database." Pattern Recognition LettersVol 30, Issue 2, 2009, pp 88-97.

Sense

PerceiveDecide

& Plan

Act

10

Designing Autonomous Systems

▪ Interactively design synthetic driving scenarios

composed of roads and actors

(vehicles, pedestrians, etc.)

▪ Generate visual and radar detections of actors

Design synthetic driving scenarios to test

controllers and sensor fusion algorithms

Automated Driving System Toolbox

Driving Scenario Designer App

Sense

PerceiveDecide

& Plan

Act

11

Designing Autonomous Systems

▪ Use prebuilt blocks instead of starting from

scratch

▪ Simplified application-specific interfaces for

configuring model predictive controllers

▪ Flexibility to customize for your application

Model predictive control for adaptive cruise

control and lane-keeping algorithms

Model Predictive Control Toolbox

Sense

PerceiveDecide

& Plan

Act

12

Full Vehicle Simulation

Vehicle Dynamics Blockset

Ride & handling Chassis controls Automated Driving

Sense

PerceiveDecide

& Plan

Act

13

Customize MATLAB and Simulink for your automated driving

application

Web based ground

truth labeling▪ Consulting project with

Caterpillar

▪ 2017 MathWorks Automotive

Conference

Lidar ground truth

labeling▪ Joint presentation with Autoliv

▪ SAE Paper 2018-01-0043

▪ 2018 MathWorks Automotive

Conference

Lidar sensor model

for Unreal Engine▪ Joint paper with Ford

▪ SAE Paper 2017-01-0107

14

CaterpillarWeb-based ground truth labeling and deep learning with big data

Caterpillar presentation slides from MathWorks Automotive Conference 2017, Plymouth, Michigan

Caterpillar presentation recording from MathWorks Automotive Conference 2017, Plymouth, Michigan

16

Working with Customers: Interfaces to Virtual EnvironmentsCollaborative work together with FORD on synthesis of lidar point cloud data to test active safety

systems

SAE Paper 2017-01-0107

SAE web article: A Safer Scenario for Autonomous Driving and Active Safety Testing, July 2017

17

Working with Customers: Interfaces to Virtual EnvironmentsCollaborative work together with FORD on synthesis of lidar point cloud data to test active safety

systems

SAE Paper 2017-01-0107

SAE web article: A Safer Scenario for Autonomous Driving and Active Safety Testing, July 2017

18

Working with Customers: LiDAR Based Sensor Verification3D Bounding Box Labeling for Point Cloud Data

SAE Technical Paper 2018-01-0043, 2018

SAE web article: LiDAR Based Sensor Verification

19

Open Platform

Interoperability with open source frameworks

PlatformProductivity

Best in class tools for design of deep networks

Multi-platform Deployment

Design in MATLAB deploy to any processor

MATLAB as Deep Learning Framework

20

Open Platform

Interoperability with open source frameworks

PlatformProductivity

Best in class tools for design of deep networks

Multi-platform Deployment

Design in MATLAB deploy to any processor

MATLAB as Deep Learning Framework

21

Ground Truth Labeling

▪ Adding Ground Truth Information

▪ Solutions

– Ground Truth Labeler App

– Image Labeler App

▪ Semi-automated Labeling

– ROI Labels (rectangle, line)

– Scene Labels

– Pixel Labels

22

Example: Ground Truth Labeling of LiDAR Data

23

Example: Ground Truth Labeling of LiDAR Data

24

Example: Ground Truth Labeling of LiDAR Data

25

Two Approaches for Deep Learning

▪ Reusing existing feature

extraction

▪ Adapting to specific needs

▪ Requires

– Smaller training data set

– Lower training time

▪ Tailored and optimized to

specific needs

▪ Requires

– Larger training data set

– Longer training time

2. Fine-tune a pre-trained model (transfer learning)

1. Train a Deep Neural Network from Scratch

26

Neural Network Toolbox: Deep Learning Network Support

27

Transfer Learning

28

Transfer Learning

29

Transfer Learning

30

Accelerating Training (CPU, GPU, multi-GPU, Clusters)

More GPUs

Mo

re C

PU

s

31

Accelerating Training (CPU, GPU, multi-GPU, Clusters)

Multiple GPU support

More GPUs

Single GPU performance

32

Open Platform

Interoperability with open source frameworks

PlatformProductivity

Best in class tools for design of deep networks

Multi-platform Deployment

Design in MATLAB deploy to any processor

MATLAB as Deep Learning Framework

Open Platform

Interoperability with open source frameworks

33

Easy Access to Pretrained Modelswith a single line of code…

Complete list of models:

www.mathworks.com/solutions/deep-learning/models.html

Caffe*MODELS

TensorFlow*MODELS

*) Importers available through the Add-Ons Manager

34

Open Platform

Interoperability with open source frameworks

PlatformProductivity

Best in class tools for design of deep networks

Multi-platform Deployment

Design in MATLAB deploy to any processor

MATLAB as Deep Learning Framework

Multi-platform Deployment

Design in MATLAB deploy to any processor

35

Algorithm Design to Embedded Deployment Workflow

MATLAB algorithm

(functional reference)

Functional test1 Deployment

unit-test

2

Desktop

GPU

C++

Deployment

integration-test

3

Desktop

GPU

C++

Real-time test4

Embedded GPU

.mex .lib/.dll Cross-compiled

.lib

Build type

Call CUDA

from MATLAB

directly

Call CUDA from

(C++) hand-

coded main()

Call CUDA from (C++)

hand-coded main().

(Test in MATLAB on host) (Test generated code in

MATLAB on host + GPU)

(Test generated code within

C/C++ app on host + GPU)

(Test generated code within

C/C++ app on Tegra target)

36

GPU Coder Compilation Flow

Benefits

▪ MATLAB as single golden

reference

▪ Much faster conversion

from MATLAB to CUDA

▪ Elimination of manual

coding errors

▪ No expert-level expertise

in parallel computing

needed

GPU Coder

CUDA Kernel creation

Memory allocation

Data transfer minimization

• Library function mapping

• Loop optimizations

• Dependence analysis

• Data locality analysis

• GPU memory allocation

• Data-dependence analysis

• Dynamic memcpy reduction

37

GPU Coder Output

GPU memory

allocation

memcpy CPU -> GPU

Image

preprocessing

memcpy GPU -> GPU

Network inference

memcpy GPU -> GPU

Postprocessing

memcpy GPU -> CPU

GPU memory

deallocation

38

Algorithm Design to Embedded Deployment

MATLAB algorithm

(functional reference)

Functional test1 Deployment

unit-test

2

Tesla

GPU

C++

Deployment

integration-test

3

Tesla

GPU

C++

Real-time test4

Tegra GPU

.mex Cross-compiled

.lib

Build type

Call CUDA

from MATLAB

directly

Call CUDA from

(C++) hand-

coded main()

Call CUDA from (C++)

hand-coded main().

Cross-compiled on host

with Linaro toolchain

.lib/.dll

39

Deployment Unit Test within MATLAB

Benefits

▪ Leveraging the MATLAB infrastructure

▪ Re-using exiting MATLAB testbench

▪ Minimal changes needed

40

AlexNet Inference on NVIDIA Titan Xp

GPU Coder +

TensorRT (3.0.1)

GPU Coder +

cuDNN

Fra

me

s p

er

se

co

nd

Batch Size

CPU Intel(R) Xeon(R) CPU E5-1650 v4 @ 3.60GHz

GPU Pascal Titan Xp

cuDNN v7

Testing platform

MXNet (1.1.0)

GPU Coder +

TensorRT (3.0.1, int8)

TensorFlow (1.6.0)

41

Algorithm Design to Embedded Deployment

MATLAB algorithm

(functional reference)

Functional test1 Deployment

unit-test

2

Tesla

GPU

C++

Deployment

integration-test

3

Tesla

GPU

C++

Real-time test4

Tegra GPU

.mex Cross-compiled

.lib

Build type

Call CUDA

from MATLAB

directly

Call CUDA from

(C++) hand-

coded main()

Call CUDA from (C++)

hand-coded main().

Cross-compiled on host

with Linaro toolchain

.lib/.dll

42

Deployment to Tegra: Cross-Compiled with ‘lib’

Two small changes

1. Change build-type to ‘lib’

2. Select cross-compile toolchain

43

0

50

100

150

200

250

300

350

400

1 16 32 64 128 256

AlexNet Inference on Jetson TX2: Performance

GPU Coder +

cuDNN

Fra

me

s p

er

se

co

nd

Batch Size

C++ Caffe (1.0.0-rc5)

GPU Coder +

TensorRT (3.0.1)

44

GPU Coder: Deep Learning Network Support (with Neural Network TB)

SeriesNetwork DAGNetwork

GPU Coder: R2017b

Networks: MNist

Alexnet

YOLO

VGG

Lane detection

Pedestrian detection

GPU Coder: R2018a

Networks: GoogLeNet

ResNet

SegNet

DeconvNet

Object

detection

Semantic

segmentation

45

Semantic Segmentation

Running in MATLAB Generated Code from GPU Coder

46

Deploying to GPUs and CPUs

GPU

Coder

Deep Learning

Networks

NVIDIA

cuDNN

& TensorRT

Libraries

ARM

Compute

Library

Intel

MKL-DNN

Library

47

Deploying to GPUs and CPUs

GPU

Coder

Deep Learning

Networks

NVIDIA

cuDNN

& TensorRT

Libraries

ARM

Compute

Library

Intel

MKL-DNN

Library

Desktop CPU

Raspberry Pi board

48

Summary

▪ Manage large image sets

▪ Automate image labeling

▪ Design, Training,

Inference within MATLAB

▪ Acceleration with GPU’s

▪ Scale to clusters

▪ Easy access to models ▪ Automate compilation to

GPUs and CPUs using

GPU Coder:▪ 5x faster than TensorFlow

▪ 2x faster than MXNet