developing products with machine learning...developing products with machine learning cadence and...

© 2018 Arm Limited

Developing Products with Machine Learning Cadence and Arm Seminar 2018

Petach Tikvah, Israel

Phil Burr, Director, Embedded Portfolio, ArmJacques-Olivier Piednoir, R&D Vice President, CadenceJuly 2018

2 © 2018 Arm Limited

Agenda

• What is machine learning (ML)?

• Machine learning at the edge

• Implementing machine learning

• Software, ecosystem, and resources

• Machine learning in silicon design

3 © 2018 Arm Limited

What is Machine Learning?

Artificial intelligence

Machine learning

Deep learning

Algorithms: CNNs,

RNNs, etc.

• Additional terms

• Location• Cloud – processing done in data farms• Edge – processing done in local devices

(growing much faster than Cloud ML)

• Key components of machine learning• Model – a mathematical approximation of a

collection of input data• Training – in deep learning, datasets are used

to create a “model”• Inference – in deep learning, a “model” is

used to check against new data

4 © 2018 Arm Limited

Machine Learning “Training”

Training data

Model

For each piece of data used to train the model, millions of model parameters are adjusted.

The process is repeated many times until the model delivers satisfactory performance.

Neural Network

5 © 2018 Arm Limited

Machine Learning “Inference”

✓

Input Neural Network with Model Output

96.4% confidence

97.4% confidence

When new data is presented to the trained model, large numbers of multiply-add operations

are performed using the new data and the model parameters. The process is performed once.

6 © 2018 Arm Limited

Compute Requirements Differ for Training and Inference

Apply the model

Low latencyEfficiency

Low/mid performance

SecurityPrivacy

Train the model

High performance

Throughput oriented

Large data sets

High-performance compute, servers, GPU

Embedded systems, heterogeneous processing

Clo

ud

sid

eD

evic

e si

de

© 2018 Arm Limited

Machine Learningat the Edge

8 © 2018 Arm Limited

Bandwidth ReliabilityPower SecurityCost Latency

Why ML is Moving to the Edge

9 © 2018 Arm Limited

Keyword detection

Pattern training

Voice & image recognition

Object detection

Image enhancement

Autonomous drive

Increasing power and cost (silicon)

Incr

easi

ng

per

form

ance

(O

ps/

seco

nd

)

~100W

~1W

~1-10mW

~10W

Range of “Edge” Applications

10 © 2018 Arm Limited

Machine Learning Innovation

BRAGI 'The Dash PRO'

ReindeerCam

Hive View

DriveCore PlatformNauto for Safe Driving

EZVIZ C5Si Camera

11 © 2018 Arm Limited

Keyword detection

Pattern training

Voice and image recognition

Object detection

Image enhancement

Autonomous driving

Data center

Incr

easi

ng

per

form

ance

(o

ps/

seco

nd

)

Flexible, Scalable ML Solutions

Increasing power and cost (silicon)

12 © 2018 Arm Limited

Scalable Machine Learning Processor “Architecture”

Scalable Compatible Programmable

Sensors (2 GOP/s) Servers (150 TOP/s)

IoT Mobile Industrial Automotive Networking Server

Targeting Multiple Markets with Scalable Architecture

1~3 TOP/s > 70 TOP/s~20-50 TOP/s~20 GOP/s

Typical Market Performance Requirements

© 2018 Arm Limited

Implementing Machine Learning

14 © 2018 Arm Limited

Machine Learning Tradeoffs in Embedded SystemsNeural Networks

Machine learning

System

Considerations Number of outputs

Number of inputs

Transfer function

Number of hidden layers

Neurons perhidden layer

Accuracy

HW/SW

Memory

Data type

Hardware

Software

System Considerations

Latency

Privacy Security Scalability

Energy efficiency

Communicationbandwidth

Optimization of system and machine learning requirements

15 © 2018 Arm Limited

Heterogeneous Systems Enable Efficient ML

Sophisticated end node based on a heterogeneous architecture

Interconnect

Shared L2

AHB/AXI interconnect

Local memory

DMC

DDR

ApplicationsSubsystem

Always-on subsystemSubsystem

AHB interconnect

TimerSensor SRAM

Efficient “always-on” subsystem filters and

processes data using ML

Advanced ML usingApplications processor

Co

mp

ute

per

form

ance

Time

© 2018 Arm Limited

Software, Ecosystem, and Resources Support

17 © 2018 Arm Limited

Machine Learning Needs a Strong Ecosystem

Hardware Platforms

Apps and Services Frameworks and Models

18 © 2018 Arm Limited

Frameworks simplifying ML development

Arm NN software translates existing NN frameworks:

TensorFlow, Caffe, Android NNAPI, MXNet etc.Developers maintain existing workflow and toolsReduces overall development timeAbstracts away the complexities of underlying hardware

Arm NN

CMSIS-NN

CPU GPU

Compute Library

3rd-party IP

Partner IP driver and

SW functions

Compute Library

CPUArm

ML processor

Compute Library

NN Frameworks

better efficiency and performance for NN functions

CMSIS-NN 5x

faster than other open-source software (OSS)

Compute Library 15x

19 © 2018 Arm Limited

▪ EDA wrestles with many data-driven problems▪ Need to provide answers quickly, in-design solutions▪ Heavy degree of automation is needed▪ What-if analysis to explore alternative solutions faster

Scale• Larger designs• More rules and restrictions• More data (simulation, extraction, shapes,

techfiles)

Complexity• More complicated silicon technologies (FinFET)• More complicated design and electrical rules• More interactions between chip, package and board• Thermal related impact between devices and wires

One Impactsthe others

Productivity• More uncertainty leads to re-design and

missed schedules• Limited number of capable, trained design and

layout engineers• Larger complexity and scale… more activities in

same schedule

Design Challenges That Are Faced Today

One Impactsthe others

20 © 2018 Arm Limited

Data-Driven Solutions = ML, Analytics, DMRarely is ML used in a Vacuum in Real Products and Systems

Massive Parallelization(Cloud-Enabled)

Data Preparation

Model-Based Inference

Adaptation

Steps in Producing Intelligent Solutions

• Filter• Label• Augment• Reduce• Partition

• Calibrate• Learn• Optimize

Creating an Intelligent and Adaptive Product Requires More

than Machine Learning

• Select• Train• Test• Verify

Machine Learning

Machine Learning

Machine Learning Optimization

Analyticsand DM

Prepare Data Build Models Adapt to Uncertainty

We often say “Machine Learning,” but that implies some combination of analytics, ML, optimization, and distributed processing

21 © 2018 Arm Limited

Machine Learning at Cadence

Enablement• Hardware/software co-design

Inside• Better PPA, faster engines• Improved testing/diagnostics

Outside• Automated design flow• Productivity improvement

22 © 2018 Arm Limited

Intelligent Design and Optimization

Capture more abstractions of design intent and preferences

Analytics

Optimization Goals

Observe

Recommend

Action or Decision

CPU GPU

Massive Parallelization(Cloud-Enabled)

Key Technologies for Intelligent Solutions

Create knowledgebase that can be mined and statistically analyzed in future planning and design

23 © 2018 Arm Limited

Machine Learning for Library Characterization

Standard Cell

Extracted Netlist

Stimulus (Vector)

Generation

Circuit Simulation

Simulation Results

Timing Model Extraction

Timing Library

Challenges• Many cells, many process corners• Many new effects• Millions of simulations for each new standard cell library

Machine learning improves throughput:• Learn from previous process corners• Smart interpolation by extracting critical measurements• Critical corner identification

24 © 2018 Arm Limited

Summary

• Machine Learning is already enabling innovation

• ML at the edge is enhancing intelligence in even the smallest of devices

• Strong ecosystem of ML solutions

• ML is already enhancing silicon design

• Talk to Arm and Cadence about ML solutions

2525

Thank YouDankeMerci谢谢ありがとうGraciasKiitos감사합니다धन्यवादתודה

© 2018 Arm Limited

2626

The Arm trademarks featured in this presentation are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners.

www.arm.com/company/policies/trademarks

© 2018 Arm Limited