trend and analysis of vision intelligence and ann ... · trend and analysis of vision intelligence...

1

Youngsu Kwon

Processor Research Group

ICT Materials and Components Laboratory, ETRI

Trend and Analysis of Vision Intelligence and

ANN Intelligence Processor

Sep. 26, 2016

2 Aldebaran: Automotive ADAS Fault-Tolerant Processor

Vision Perception is Crucial

2

AUDI zFAS with NVIDIA K1

Mobileye EyeQ4

Google: 3D Scanning with Ridar

and map-based self-locating

Tesla: Single-lens object

detection with Mobileye

New era of “fast” Automotive Processor...

but, is our current technology sufficient to

provide Safety in Self-Driving?


Aldebaran3: ISO26262-Compliant Vision Processor

3

8mm

4 Aldebaran: Automotive ADAS Fault-Tolerant Processor 4

8mm

Aldebaran3 Features

Quad-Core

ABC_CT with Cache-ECC

and Recovery

CAN 1.0A/B and FD

for fast auto network

Recognition Engine

(Now transitioning to ANN)

H.265 Video Encoder and

Decoder for Video Storage


Aldebaran3: ISO26262-Compliant 1GHz Processor

5

ABC_CT3 @c3

m2

ABC_CT2 @c2

m3

m4

@b1

IR

s0 s1

@ddr3c_ui

DDR3C_MC @sdr

SDR_MC

s8 s9

@b1

LP

SW0 128bit @b0

RES @re

m5 m6

VOM @b0

VIM_128 @b0

SW3 32bit @b1

m0 m1

ABC_CT1 @c1

ABC_CT0 @c0

32bit 32bit 32bit 32bit

128bit

m9

128bit 32bit 128bit 128bit

DMA @b0

m8

32bit

HPN @hpn

32bit

m10

SPN @spn

128bit

32bit 32bit

PMU GPIO

VC FMC

UART TIMER

PWM WDT

RTC I2C

AC97 INTC

SDIO VIM

MOTOR JTAG

@b1

MP

s10

32bit

@vim_e @vom_e

@can_e

SW1 64bit @b0

SJTAG @b1

m12 m13

SDC0 @sdc

s5

SW2 32bit @b1

s7

EMP @emp

32bit

m11

s6

32bit 32bit

32bit

32bit

32bit 32bit

s4

@b1

FMC @b1

SMC @b1

USBHS

@usb_e

iROM

iRAM

CAN

SPN

HPN

abts3_noc128

abts3_noc128

abts3_noc128

abts3_noc128

p_osc_clki PLL

clkpll0~6

CSG

CSEL

clk00 ~ clk16 clkpll0~6

AXI

AHB

APB

RESET FTM

PMU

RE0 @re

32bit

m14

RE1 @re

32bit

m15

SW5 32bit @b0

m16

VIM_32 @b0

32bit

@vim_e

SW4 128bit @b0

AB3


Aldebaran3: Automotive Vision Processor

6

Vertical Integration :

ECU-SW-System-Chip


Aldebaran5

ABC_ST ABC_ST ABC_ST

Aldebaran5: Automotive ANN Processor

7

ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST

ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST ABC_ST

ABNC

(x1024 Convolver)

NOVA

(4K Enc/Dec)

Ethernet Cam/ISP CAN-FD

QEI

Legacy

AB FT-NoC

ISO26262 ASIL D

Compliant Multi-Core

H.265 Video Recording

* AI Processor for Autonomous Vehicle & Driver Assistance

ANN for

Real-time Recognition

with Power Efficiency

AI processor for

Vision in EVs AB-NZVC

Near-Zero Voltage Core

for Power-off missions


Searching for a Practical Way to ANN Processor

8

Neuromorphic Research

Large Scale Neuromorphic

Sytem

Memristive Device

IBM C 2 S 2 Synapse Project

Carbon nanotube

Sensor Application

Silicon Retina

Electronic Nose

Silicon Cochleas

Sensorimotor System

Stanford Brainstorm

Univ . of Hidelberg Spikey

USCD NeuroDyn

Small Scale Neuromorphic

System

Digital

Analog

UCSD HiAER IFAT Project

Stanford Neurogrid

Univ . of Hidelberg FACETS and

BrainScale Project

Univ . of Manchester

SpiNNaker Project

Neuro Computing System

Spin Device

HRL Neuromorphic

Aldebaran-ANN Project in ETRI


Practicality

9

It's nice to imitate Nature,

But we also need to understand

How do we know which

details are important?

Which details are merely

the result of evolution, and

the constraints of

biochemistry?

For airplanes, we developed

aerodynamics and fluid

dynamics.

We figured out that feathers

and wing flapping weren't

crucial.

Question: What is the

equivalent of aerodynamics for

understanding intelligence?

Excerpts from Yann LeCun's

L'Avion III de Clement Ader, 1897

His Eole(Avion) took off from the ground in 1890,

13 years before the Wright Brothers, but you may

have never heard of it.


Neuromorphic Researches in Korea

10

Institu

te

Circuit

Sensor/

Circuit

Memristor

ExoBrain

(SW)

DeepView

(SW)

Neurmorphic

Vision

Processor

(Chip)

Conte

nts

Low Power

Neural Cell and

STDP-based

512 Neuron

Core

Memristor-

Synapse

integrated into

image sensor

Memristor

synapse for

neuromorphic

device

Exobrain is a

SW for

Knowledge

learning and

sharing

Real-time video

recognition for

vision

intelligence

Mobile ANN

Processor for

Vision

intelligence


Huddles ahead

11

State-of-the-Arts Hardware Configuration Power/Energy

Google,

Andrew Ng,

ICML 2012

1000 server machines

Total 16,000 Processor

1 billion connections

10 million images (200x200)

3 days Training

1.6 MW

Google,

Andrew Ng,

ICML 2013

3 Server Machine

[4GPUs, GTX680, 3Tflops, 300W]+

2xQuadCore CPU]/server 3-5 days Training

1.5 kW (36Tflops)

Oxford, VGG

ILSVRC 2014

1 server machine

[4GPUs, Titan Black, 5Tflops, 300w]

16-19 layers

1.2 million images 2-3 weeks Training

1.2 kW (20Tflops)

Over 15x109 Neuron computation per second requires

specialized computing techniques for single-chip applications.


Synapse Compiler

12

Synapse Compiler

Re-Configuration

Synapse Optimization

Synaptic Op. Scheduling

Neural Net. Optimization

Neuron 1

Neuron 2

Neuron 3

Dendrite

Axon

Synapse

NN Architecture Model

SNN, CNN/FCN circuits u-Neural Core

Hardware Neural Net. (HNN)

Localization (Obj. Region Proposal)

Quantized Synapse Neural Net

Learning/Recognition

Algorithm

AlexNet, VGG, GoogleNet, ResNet,

…etc.,

Neural Network Model

Memory Capacity & Bandwidth

Synapse Capacity

Processing Performance (GSOPS)

Target Perception Performance

Optimization

CONSTRAINTs

Neuro-Synaptic

Code Network Connection

Synapse Weight

Operation Scheduling

Internal NC Network

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

uNC uNC

Synapse Weights

Neural Op. Scheduling

Sensor

(ISP)

Aldebaran5 ANN Processor

Synapse Compiler produces a Neural Network by

restructuring, conversion, compression of state-of-the-art NNs

stream-in


Aldebaran5: A Practical ANN Processor

13

mdm

@b1clk

IR

sdd

@ddr3c_uiclk

DDR3C_MC

sir smp

@b1clk

LP

SW00 128bit @b0clk

NOVFMC @novclk

mvd mvi

VOM @b0clk

VIM_128 @b0clk

SW40 32bit @b1clk

mc0

ABC_CT @c0clk

$32K

32bit

128bit

128bit 128bit 128bit

DMA @b0clk

mnv

32bit

32bit 32bit

BootMode INTC

UART GIOCAP

PWM TIMER/WDT

RTC I2C

SPI

@b1clk

MP

slp

32bit

SW10 32bit @b0clk

SJTAG @b1clk

msi mjt

SDC0 @sdcclk

SW30 32bit @b1clk

32bit

32bit 32bit

sfm

@b1clk

FMC

iROM

iRAM

CAN

@can_e

QEI

p_osc_clki ABSYSF clk00-clk34

AXI

AHB

APB

PMURST

FTM

ISP @isp (clknet)

64bit

mip02

SW20 64bit @isp_busclk

64bit

mip03

32bit

sip26

mc1

ABC_CT @c1clk

32bit

mc3

ABC_CT @c3clk

32bit

mc4

ABC_CT @c4clk

32bit

mc5

ABC_ST @c5clk

32bit

SW11 32bit @b0clk

mc6

ABC_ST @c6clk

32bit

mc7

ABC_ST @c7clk

32bit

mc8

ABC_ST @c8clk

32bit

ADAS_IP @adas

32bit

mad0

SW21 32bit @adaclk

32bit

mad1

32bit

mad2

ABNC @ncclk

mnc

mc2

ABC_CT @c2clk

32bit

FT

M

FT

M

64bit

mip04

64bit

mip05

64bit

mip06

64bit

mip07

64bit

mip08

64bit

mip09

64bit

mip10

64bit

mip11

CDC

32bit

sspi

32bit

SW50 32bit @b0clk

mep

EMP @empclk

sep

14 Aldebaran: Automotive ADAS Fault-Tolerant Processor 14

trend and analysis of vision intelligence and ann ... · trend and analysis of vision intelligence...

Documents