"memory innovation for embedded vision systems," a presentation from samsung electronics
TRANSCRIPT
February 22nd, 2017 l JIN KIM l Samsung Electronics
Memory innovation for embedded vision systems
2/20
Disclaimer
This presentation is intended to provide information concerning memory industry. We do our best to make sure that information presented is accurate and fully up-to-date. However, the presentation may be subject to technical inaccuracies, information that is not up-to-date or typographical errors. As a consequence, Samsung does not in any way guarantee the accuracy or completeness of information provided on this presentation. Samsung reserves the right to make improvements, corrections and/or changes to this presentation at any time.
The information in this presentation or accompanying oral statements may include forward-looking
statements. These forward-looking statements include all matters that are not historical facts, statements regarding the Samsung Electronics' intentions, beliefs or current expectations concerning, among other things, market prospects, growth, strategies, and the industry in which Samsung operates. By their nature, forward-looking statements involve risks and uncertainties, because they relate to events and depend on circumstances that may or may not occur in the future. Samsung cautions you that forward looking statements are not guarantees of future performance and that the actual developments of Samsung, the market, or industry in which Samsung operates may differ materially from those made or suggested by the forward-looking statements contained in this presentation or in the accompanying oral statements. In addition, even if the information contained herein or the oral statements are shown to be accurate, those developments may not be indicative developments in future periods.
3/20
Contents
• Vision IoT, divergence
• Embedded vision memory
‒ Auto/VR/Mobile
• Innovative memory solutions
• Summary
4/20
Vision IoT, divergence
5/20
To divergence
DIVERGENCE CONVERGENCE
• Better smartphone UX with vision IoTs
• Need divergence of vision IoT
6/20
Changing role, enriching mobile eco
Service
Intelligent Hub
Data Explosion
6 10 15
22
31
'15 '16 '17 '18 '19 '20
10X
Data Generation
& Distribution Personalization
Cloud
Data Analytics, Deep Learning
Source: Global Mobile Data Traffic, CISCO’16
Active Interaction
7/20
For artificial intelligence, On-device • On-device service expansion
‒ Privacy/security/picture quality On-device A.I.(deep learning)
‒ Energy-efficiency/high-bandwidth/low-latency
Accuracy(# of layers)
Coverage
(10 layers)
(8 layers)
Object detect Medical
Natural Language
Audio
Translation
Self-Drive
Face pay
Number
Recognition
Cloud service
On-device service
Recommend
Super resolution
4-layer 5-layer 6-layer
8/20
Embedded vision memory : Auto/VR/Mobile
9/20
Vision IoT, more memory
Influence level
on memory Imaging
(CIS+memory)
Recognition (SoC+memory)
A.I. (Cloud+memory)
Wearable ★ ★
Automotive ★★ ★★ ★★★★
Camera ADAS ★★ ★★★
IP camera ★★ ★★
Robot/Drone ★ ★★ ★★★
Gateway/Edge ★ ★★
Cellular IoT/M2M ★
Smart Home ★ ★ ★
Retail ★ ★
Automotive
ADAS
Tele-
matics
Info/
Cluster
IP CAM
Smart
Gateway
Robotics
Drone
Smart Home
Thermo
-stat
Appliance Wearable
IoT (50B unit)
100M
C-IoT/
M2M
Retail
• Vision for Imaging Vision for Sensing and A.I.
• Top 10 IoT applications, driving memory consumption
Imaging: improved picture quality/high speed shooting/continuous shooting Recognition: 3D mapping/multi-camera/object/motion/biometrics/face recognition A.I.: autonomous/emotion recognition/disease diagnosis/VR
10/20
Automotive, evolving fast
0
20
40
60
'12 13 '14 15 '16 17
Mobile
Auto
0
2
4
6
8
10
'12 13 '14 15 '16 17
Mobile
Auto
DRAM Bandwidth DRAM Capacity
GB/s GB
Vision
Sensor
D/L Data
Processing
100
50
200
[GB/s]
300
‘21 ‘17 ‘19
OS , Apps
D/L Weight
8x
Number of objects to recognize
40ea(’13) ▶ 1,000ea (’18)
Bandwidth for Compute ADAS
• Autonomous driving adoption, faster than expected
• Compute ADAS drives high B/W (object detection/scene seg./depth extraction)
Source: Samsung
11/20
VR, high-performance & low-power • VR now : FHD 90fps, MTP* latency(~50ms), 2D audio VR sickness
• Immersive VR : 160FoV, 8K120fps, MTP(~20ms), 360 audio
Enthusiastic Game
Professional/Industrial
Military/Health Care
Education/Retail
VR Cinema
360 Video Streaming
Casual Game
8K120 streaming, ~51.2GB/s
4GB+
8K120 commercial, ~150GB/s
8GB+
8K120 gaming, 1TB+/s
16GB+
Teth
ere
d
All
In
On
e
Dro
p I
n
4K65 4K75 4K120 8K60 4K90
Display
17 13 9 11 Latency
’16 ’17 ’18 ’19 ’20
ms
TDP
[Watt]
100 300 200
All-in-One
Tethered
LP4/5(~34GB/s)
HBM2/GD6(~500GB/s)
LP5/WIO(~60GB/s)
GP
U [T
FL
OP
S]
8
3
1
∬
Drop-In
fps
Performance
Low
Power
10
<9
*MTP: Motion To Photon
Type & DRAM Requirements VR Ecosystem Readiness
Source: Samsung
12/20
Mobile, overcoming technology barrier
SOC Performance DRAM Performance
0.1
1
10
100
‘10 ‘12 ‘14 ‘16
(log)
‘18
Be
nch
ma
rk S
co
re (
3D
gra
ph
ic)
0
1.6
3.2
4.8
'10 '12 '14 '16 '18
Gbps
Time
‘12 ‘14 ‘16
• Need performance, but can’t increase it
Source: Samsung
13/20
Memory technology trend
Power Efficiency
[mW/GBps]
100%
80%
60%
40%
20%
2020 2016 2018
Performance
[Gbps/pin] 15
12
6
3
LP5
LP4X LP4
2016 2018 2020
DDR4
9
DDR5
GDDR5
LP4
• GDDR6 with over 14Gbps, beyond 10Gbps GDDR5
• LP5, 20% more power-efficient than LP4X
LP5
GDDR6
DDR5
LP4X
GDDR5
DDR4
LP3
DDR3
Source: Hotchips2016, Samsung
GDDR6
14/20
Innovative memory solutions
15/20
High Bandwidth Memory: HBM
PCB
DRAM
Buffer Logic Processor
Si Interposer
HBM
TSV Technology
1,024 I/O Architecture
Benefits
Microbump
8H stacked 20nm 8GB HBM
HBM
GDDR5
X 0.8
Power Efficiency
High Bandwidth 1TB/s
X 2.7
Performance
HBM
GDDR5
Source: Samsung
16/20
3D/2.5D SiP memory PKG • Mobile, small F/F, high-speed, low-power requirement 3D/2.5D SiP PKG
• Close collaboration with SoC
Stacked FBGA
FOPLP
Dual Flip Chip
2.5D Si Interposer
3D SiP
Performance
Form Factor
time
SiP
PoP
Wire Bump FOPLP TSV
Mobile Memory SVR/Gfx Memory
time
TSV
Wire
/FPGA
Wire/FBGA FC/TSV High-IO Interposer
High IO TSV (HBM)
Interposer Based Platform
DDR TSV
FC-CSP
Stacked FBGA
BOC
interposer Performance
Source: Samsung
17/20
A.I. mobile memory
[ source : ISSCC]
DDR Main Memory (DDR4/5)
PCIeGen-Z/CCIX/CAPI
Accelerator (GPU/FPGA/ASIC)
CPU
NPU
HBMs
LPDDR Main Memory (LP4/5)
AMBA/AHB
Accelerator (DSP/VPU/NPU)
AP
A.I . mem
iGPU iGPU
Inference (DSP/NPU/VPU) Training
(GPU/FPGA/ASIC) Trained
model
Output
Energy efficient solution requirement
Dedicated H/W (accelerator + memory)
• Vision IoT requires A.I. specific memory
‒ Deep-learning/parallel/inference processing
Deep-learning on IoT device
Source: Samsung
18/20
Memory-stacked Photography • Require DSLR-level performance (dual camera/pixel and high-speed shooting)
• Chance to processing in stacked memory
‒ Low power, thermal spread, super resolution, real time HDR and slow motion control
Source: ISSCC 2017, Sony
e.g. data rate control, high speed read-out, multi stream output, high speed binning
19/20
PIM for embedded vision IoT • Added value from reduced power consumption
‒ Reduce the unnecessary data transfer and frame rate control
• Possible collaboration with SoC/AP
AP CIS Display
AMBA AHB
Display
CIS AP
Added Value Memory B/W Traffic
VPU
Recognition Distortion FRC Correction
Pre/Post Processing In Memory
Source: Samsung
20/20
Summary
• Smartphone as a Intelligent Hub will continue to enrich mobile ecosystem
‒ Performance never be enough in mobile. The more we have, the better we use
‒ Mobile challenges with power optimization and continued innovation
• High performance, more memory are needed in embedded vision IoT
‒ Automotive ADAS drives high bandwidth memory
‒ Immersive VR requires high-performance and low-power
‒ Mobile memory is getting more power-efficient
• Close collaboration is essential
‒ Keeps innovating technology to correspond to the requirements
‒ Artificial Intelligent memory, CIS stacked memory, Vision processing in memory