Upload File

tinyml meetup kick-off•hw: tpu, fpga, gpu, cpu edge ml •optimized algos and cnn-light •soc...

  • Home
  • Documents
  • tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:
17
tinyML meetup kick-off Enabling ultra-low power Machine Learning at the Edge Evgeni Gousev, Qualcomm AI Research Santa Clara June 27, 2020

Upload: others

Post on 04-Jul-2020

3 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML meetup kick-offEnabling ultra-low power Machine Learning at the Edge

Evgeni Gousev, Qualcomm AI ResearchSanta Clara

June 27, 2020

Page 2: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

Motivation: Problem Statement and Opportunity

1. Growing (urgent) need to drive tinyML acceleration and adoption throughout the whole ecosystem

• Use-cases – Apps – SW – Tools – Algos – HW – ASIC – Device - Fabs

2. TinyML is real and will be huge• Many pieces of the big puzzle are “popping up”. Someone (who if not us !)

needs to put them together

• More innovations and breakthroughs ahead

Page 3: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

What is tinyML ?

• (for now) tinyML is broadly defined as machine learning architectures, techniques, tools and approaches capable of performing on-device analytics for a variety of sensing modalities (vision, audio, motion, environmental, human health monitoring etc.) at “mW” (or below) power range targeting predominately battery operated devices (IoT, bioelectronics, …)

Page 4: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

What is tinyML ?

• Capable of performing on-device analytics for a variety of sensing modalities (vision, audio, environmental, human health monitoring etc.) at “mW” power range

• battery operated devices (IoT, bioelectronics, …)

• HW – algorithms – SW – Use cases/applications

Page 5: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

Why tinyML ?Data is a new oil(electricity) and ML is a way to produce it

Cloud ML

•DNN on the cloud

•HW: TPU, FPGA, GPU, CPU

Edge ML

•Optimized algos and CNN-light

•SoC (with NPUs/NSP accelerators)

Tiny ML

•CNN-micro

•MCU w/ HW accelerators

Data Sources:

Storage and sharing

User provided:1. Pics2. Audio3. Clicks/likes4. GPS, Location based

Real-time in the physical world

CMOS cameras

IRcameras

IMUs Audiomicsb

Environ/chemical

Temperature Optricalsensors

1%

4%

95%

Page 6: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML is “good enough” NOW… and more enhancements coming in the near future

SW

AlgosHWHW accelerators (digital) Quantization, compressionSmaller models (100s kB)

$ initial tinyML applications

- Compute in memory- Analog compute- Neuromorphic

- Novel algos/networks - 10s kB models

$$$ More tinyML apps and value creation

Enabling technologies: ULP sensors, novel memories, 3D, energy scavenging, ULP radio

Page 7: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:
Page 8: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML Summit-2019

Page 9: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML Summit 2019

• March 21-22 – hosted by Google

• 160 attendees

– Over 100 additional people on the interest list• unable to attend due to space constraints

• 17 technical presentations & 2 panels

• 29 posters

• 15 demos

www.tinymlsummit.org 9

http://www.tinymlsummit.org/
Page 10: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML-2019

Committee

Page 11: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

ANNOUNCEMENT: tinyML Summit 2020

www.tinyMLsummit.org 11

Wei XiongSamsung

Samsung, San Jose

http://www.tinymlsummit.org/
Page 12: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML-2020 Summit Objectives and Focus

• Continue to grow tinyML Global Ecosystem – 2x attendance size (wrt the 2019 Summit) while keeping the highest quality event

• Continue to build tinyML awareness

• Start connecting tinyML technologies to end-user products and applications

• More focus on: algorithms and end-user products and applications

• Bring more academic work of fundamental importance for tinyML ecosystem

• More organizational diversity

Page 13: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

Co-Chairs

Wei XiongSamsung

Technical Program Committee

Operations• Bette Copper• Ira Feldman

Organization

13

Page 14: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML Community “DNA”

• Highest Quality: prime tinyML global Community/Events

• Industry focused & driven, with strong academic participation & influence

• “Full stack”/E2E coverage: HW-SYS-Algo-SW-Apps

• Deeply technical with no marketing/sales pitches

• Diverse and collaborative (while respecting privacy)

• Building “tinyML Foundation”, a non-profit Org/Community

Page 15: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

tinyML Meetup Committee

Page 16: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

“Let’s make tinyML BIG !”

16

Page 17: tinyML meetup kick-off•HW: TPU, FPGA, GPU, CPU Edge ML •Optimized algos and CNN-light •SoC (with NPUs/NSP accelerators) Tiny ML •CNN-micro •MCU w/ HW accelerators Data Sources:

17

Objectives

1. Collaborative platform for the tinyML ecosystem• Networking and biz dev opportunities

2. Common tinyML “roadmap”• Shared vision

• Tech pipeline from pre-competitive R&D based on leading edge academic research

3. Growth engine• New use cases and applications

• Start-ups and VCs

4. Benchmarking and standards• To make sure we all speak the same language (e.g. open datasets for benchmarking)

• Esp important as the ecosystem pie grows; scale is impossible w/o standards

5. Workforce dev’t• Training

• Collaboration w/ academia


Today’s Schedule – 5/3/10 CNN Student News 27.1: Physical Geography of China, Mongolia & Taiwan 27.1 Vocabulary East and Southeast Asia Map HW: – Standards

Convolutional Neural Network - CNN · Introduction CNN Layers CNN Models Popular Frameworks Papers References Convolutional Neural Network - CNN Eduardo Todt, Bruno Alexandre Krinski

Ultra HD Video Scaling: Low-Power HW FF vs. CNN-based Super-Resolution

Today’s Schedule – 05/17/10 CNN Student News 29.3 Vocab Check 29.3 PPT: Southeast Asia Today HW:…

New Architecture of ML Systems 07 Hardware Accelerators · 2020. 7. 4. · 8 706.550 Architecture of Machine Learning Systems –07 HW Accelerators Matthias Boehm, Graz University

HW 1General Revision HW 2Percentages HW 3Volume HW 4Linear Relationships

Fused-Layer CNN Accelerators - Stony Brookcompas.cs.stonybrook.edu/~mferdman/downloads.php... · Fused-Layer CNN Accelerators Manoj Alwani Stony Brook University [email protected]

BIOGEOCHEMICAL BUDGET THROUGH THE STRAIT …doga.ogs.trieste.it/doga/echo/ecem07/Main/30Nov/Macias_D.pdf · HW+5 HW-5 HW-3 HW-1 HW+1 HW+3 HW+5 HW-5 HW-3 HW-1 HW+1 HW+3 HW+5 Two layer

2sRanking-CNN: A 2-stage ranking-CNN for …2.2.1 1st-stage Ranking-CNN Before explaining 1st-stage ranking-CNN, let’s briefly explain how ranking-CNN works. Ranking-CNN was proposed

Introduction to Accelerators: Evolution of Accelerators …uspas.fnal.gov/materials/11ODU/Lecture1a_Intro_to... ·  · 2015-03-05Introduction to Accelerators: Evolution of Accelerators

Startup Accelerators

ENGINEERING HW - Lost River Development - Horace, ND€¦ · x x x x x bcl bcl bcl bcl bcl bcl bcl bcl bcl bcl bcl bcl bcl bcl bcl hw hw hw hw hw hw hw hw hw hw hw hw hw hw hw drainage

Introduction to Accelerators: Evolution of Accelerators

EE382M.20 SOC Design HW Accelerators and Co-Processorsusers.ece.utexas.edu/~gerstl/ee382m...CoProcessors.pdf–Optional Vectra LX DSP registers §General Purpose AR Register File –32

Improving Scalability of Chip-MultiProcessors with Many HW ... · Improving Scalability of Chip-MultiProcessors with Many HW ACCelerators A Dissertation Presented by Nasibeh Teimouri

Today’s Schedule – 05/14/10 CNN Student News 29.2 Vocab Check 29.2 Part PPT: History and Culture of Mainland Southeast Asia HW: –29.3 Vocabulary –Complete

M1 accelerators

LINEAR ACCELERATORS

Today’s Schedule – 28.1 Turn in Unit 9 Maps CNN Student News – 28.1 Vocab check Current Event Presentations 28.1 PPT: China HW: – Read 28.2 – Complete

Accelerators and Detectors - School of Physics and …muheim/teaching/np3/lect-exp.pdfNuclear and Particle Physics Franz Muheim 1 Accelerators and Detectors Accelerators Linear Accelerators

HW 1-1 0 HW 1-2 3 HW 1-3 HW 2-1 3 HW 2-2 3 HW 2-3 3 9 HW 3 ...mt/132.1.01w/132-score.pdf · hw 4-1 hw 4-2 hw 4-3 hw 4-t 0 hw 5-1 hw 5-2 hw 5-3 hw 5-t 0. hw 2-1 3 hw 2-2 3 hw 2-3 3

Today’s Schedule – 05/06/10 CNN Student News 27.3 Vocab Check 27.3 PPT: China, Mongolia and Tibet Today NPR: Understanding the China/Tibet Conflict HW:

Today’s Schedule – 05/05/10 CNN Student News 27.2 Part 2: Culture of China Activity: Chinese Lanterns HW: – Email website to Ms. Hayden – TONIGHT! – 27.3

Accelerators Introduction

Today’s Schedule – 4/24 CNN Student News Current Event Presentations Complete Hinduism PPT PPT: India, History and Current Issues HW: – Read 25.2 – Complete

Today’s Schedule – 05/12/10 CNN Student News 28.3 Vocab Check 28.3 PPT: Japan and Koreas Today HW: – 29.1 Vocab – Study for Chapter 28 Quiz

CNN-PS: CNN-based Photometric Stereo for General Non ...openaccess.thecvf.com/...CNN-PS_CNN-based_Photometric_ECCV_2… · CNN-PS: CNN-based PhotometricStereo for General Non-Convex

Hardware Accelerators for ECC and HECC · A. Tisserand, CNRS{IRISA{CAIRN. Hardware Accelerators for ECC and HECC 7/35. Accelerator Speci cations encryption signature etc level HW

Performance Comparison of Hybrid CNN-SVM and CNN-XGBoost

REGISTERED - Design Eq...baratta street steps brick garage hw hw hw hw hw hw hw w w w w carport brick garage hw hw hw hw hw w w w hw hw hw carport brick garage brick garage w 16.66

Accelerators f or America’s F uture · INTRODUCTION Accelerators for America’s Future Chapter 1 Accelerators for Energy and the Environment Chapter 2 Accelerators for Medicine

Safe Overclocking for CNN Accelerators through Algorithm

Sankalp Semiconductor - Design And Reuse · Sankalp Semiconductor Confidential 12C IoT Edge Device ASOC Wireless Communication Special IP DSP HW Accelerators Security Sensor / Actuator

RENT A SEGWAY! SERVICE...May 12, 2006  · IndyCar Racing News (CC) ABC Wld ... CNN CNN Live Saturday CNN Live Saturday On the Story (CC) CNN Presents Larry King Live CNN Saturday

Architecture of ML Systems 07 Hardware Accelerators€¦ · 4 706.550 Architecture of Machine Learning Systems –07 HW Accelerators Matthias Boehm, Graz University of Technology,