ic technology what will the next node offer us? · 8/17/2019 · 2d baseline system off-chip dram...

TSMC Property © 2019 TSMC, Ltd

®

IC Technology –What Will the Next

Node Offer Us?

H.-S. Philip WongVice President, Corporate Research, TSMC

Willard R. & Inez Kerr Bell Professor, Stanford University

2 TSMC Property © 2019 TSMC, Ltd®

MOORE’S LAWTransistors per microprocessor

Source: Karl Rupp. 40 Years of Microprocessor Trend Data.

1010

109

108

107

106

105

104

1971 1980 1990 2000 2010 2017


MOORE’S LAW DENSITY AND COST PER FUNCTION

Source: G. Moore, Electronics, 1965

105

104

103

102

10

1105104103102101

Number of Components per Integrated Circuit

Rela

tive M

anufa

ctu

ring C

ost

/ C

om

ponent

1960

1970

1965


104

103

102

101

100

10-1

10-2

10-3

MOORE’S LAW IS WELL AND ALIVE

DENSITY: A NECESSARY ATTRIBUTE

1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020

Re

lati

ve

De

nsi

ty

Year

Standard cell inverter

High density SRAM

Logic gates

Transistor density (microprocessors)


IMAGINE: TRANSISTOR PERFORMANCE W/O DENSITY


• Not enough memory

• No multi-core chips

• No accelerators

• Wire delay slows big chips.

IMAGINE: TRANSISTOR PERFORMANCE W/O DENSITY


TECHNOLOGY LEADERSHIP

N7World’s first 7 nmParticipated in all the products on 7 nm


N7Best performance

Highest density

Extensive EUV layers

Design ecosystem ready

In risk production


N5 ( )P


N7 N3


N5 ( )P


THE ELEPHANT

IN THE ROOM

m 10-1 m

Bacteria

2 μmStrand of hair

0.1 mmTennis ball

10 cmVirus

50 nmCarbon

nanotube

1.2 nm

FinFET

Water molecule

0.28 nm

H H

O

-+ Hydrogen atom

0.1 nm

10-2 m 10-3 m 10-4 m 10-5 m 10-6 m 10-7 m 10-8 m 10-9 m 10-10 m


CONTINUOUS BENEFITS NODE AFTER NODE

MOORE’S LAW – A HISTORY OF INNOVATIONS

Dennard scaling

Strained Si, high-k / metal gate

FinFET / DTCO



MULTIPLE ROADS LEAD TO ROME

Innovations


INTEGRATING CHIPS INTO SYSTEMS

It may prove to be more economical to build

large systems out of smaller functions, which

are separately packaged and interconnected.

The availability of large functions, combined

with functional design and construction,

should allow the manufacturer of large

systems to design and construct a

considerable variety of equipment both

rapidly and economically.

Source: G. Moore, Electronics, 1965


CoWoS® SYSTEM INTEGRATION

Source: 2013 TSMC Technology Symposium

TSMC CoWoS® fully assembled test chip

1 SoC + 2 DRAMs


CoWoS® SYSTEM INTEGRATION

2500 mm2 interposer:

2 processors (600 mm2)

+ 8 HBM DRAM


Integrated Si/Package Area, Reticle

SYSTEM INTEGRATION TECHNOLOGIESI/O

Pin

Count


Package Size

Inte

rposer

Siz

e (

mm

2)

GP100(Courtesy of Nvidia)

7V580THeterogeneous Integration

(Courtesy of Xilinx)

7V2000THomogeneous Integration

(Courtesy of Xilinx)

XCVU440(Courtesy of Xilinx)

GV100(Courtesy of Nvidia)

mm2

CHIPLETS INTEGRATION REDUCES SYSTEM COST PER FUNCTION

2X

1X

1.5X


PC / Internet Mobile AI / 5GMini-ComputerTransistor Radio

SEMICONDUCTOR TECHNOLOGY EVOLVES

DRIVEN BY CHANGING APPLICATION LANDSCAPE

Invention of point-contact transistor

1947

Transistor Scaling Principle

1974

Intel 4004

1971

Invention of IC

1958

Pentium CPU

1995Flash Memory

1984Mobile phone

1973

3G

2002

4G

2009

iPhone

2007

FinFET

1999

7nm FinFET

2018

GPU (21B Transistors)

2017

5nm CMOS

2020

2050 and beyond


15%

85%

8%

92%

20%

80%

Memory Compute

Deep Learning Accelerators

Intel performance counter monitors 2 CPUs, 8-cores/ CPU + 128GB DRAM

DATA MOVEMENT HITS THE MEMORY WALL

ABUNDANT-DATA APPLICATIONS: ENERGY MEASUREMENTS

Source: S. Mitra (Stanford)

…ResNet-152

(CNN)

AlexNet

(CNN)

Language Model

(LSTM)


Network(application)

Type(LSTM/ CNN)

Training/ Inference

Model SizeMemory Usage

(GBytes)

ResNet(vision)

CNNTraining

120 MBytes21*

Inference 0.12

LanguageModel(NLP)

LSTMTraining

2.5 GBytes40*

Inference 2.5

* Training memory usage: Batch size 64, word size 64-bit, memory can increase with greater batch sizes, footprint of activations, weights, errors and gradients.

Source: M. Lee, W. Hwang, Prof. S. Mitra (Stanford), M. Aly (NTU, Singapore), Y. Wang, K. Akarvardar (TSMC)

DEEP NEURAL NETWORKS

REQUIRE LARGE MEMORY CAPACITY


ON-CHIP SRAM CAPACITY:

NEVER ENOUGH

0

10

20

30

40

50

60

Estim

ate

d O

n-c

hip

SR

AM

(M

B)

Launch Year

2006 201820122009 2015

Intel Xeon X5355 NVIDIA Tesla K40

NVIDIA Tesla V100

Intel Xeon E7-8890 v4

CPU

GPU

3.8 Gbytes

@

1.4 nm node

Source: W. Hwang, Prof. S. Mitra (Stanford)


CAN WE PUT LOTS OF MEMORY ON-CHIP?

WHAT KINDS OF MEMORY, FOR WHICH APPLICATION?


Source: “Inside Volta” , Nvidia GPU Tech. Conf. , May 10, 2017.

Heterogeneous Integration:GPU + High Bandwidth Memory (HBM2)

CoWoS Module

Superior processing power that equals to 100 CPUs

>300 B transistors

SUPER AI ACCELERATOR ENABLED BY CoWoS®

HBM2

HBM2HBM2

HBM2

GPU


COMPUTE-MEMORY INTEGRATION

Off-Chip DRAM

Si Logic Die

Printed Circuit Board

Limited I/O Connectivity

2D System (traditional baseline)

Source: W. Hwang, W. Wan, Y. Malviya, H. Li, M. Lee, M. Aly, H.-S. P. Wong, S. Mitra. Work in progress 2017 – 2019 w/ TSMC


2.5D System HBM-Type DRAM

Si Logic Die

Si InterposerMicron Scale Connectivity




HBM-Type DRAM

Si Logic Die

TSV + µBump Connectivity(Micron Scale)

3D TSV System




N3XT SystemHigh Density On-Chip Nonvolatile Memory

Dense ILV Connectivity(Nanometer Scale)

Si Logic Die

Energy Efficient Logic(Thin Device Layers)

High Speed On-ChipNonvolatile Memory

Energy Efficient MemoryAccess Transistors

Nonvolatile Memory Cells




Bottom Electrode

Top Electrode

oxide isolation

switching region

phase change material

PCMPhase change

memory

filament

oxygen ion

Top Electrode

Bottom Electrode

metaloxide

oxygen vacancy

RRAMResistive

switching random access memory

filament

Bottom Electrode

solid electrolyte

Active Top Electrode

metal atoms

CBRAMConductive

bridge random access memory

STT-MRAMSpin torque

transfer magnetic random access

memory

FERAMFerro-electric

random access memory

Ferroelectric layer

p-Si

n+ n+

Interface Layer

top gate

Source: H.-S. P. Wong, S. Salahuddin, Nature Nanotech (2015)

“NEW” MEMORIES FOR COMPUTE-MEMORY INTEGRATION

Soft Magnet

Pinned Magnet

tunnel barrier (oxide)

current

Random access, non-volatile, no erase before write, on-chip integration


Source: Stanford/NTU: M. Aly, S. Mitra, TSMC: Yih (Eric) Wang, K. Akarvardar, 2019

2D baselinesystem

AcceleratorCores

SRAM on-chip memory

NEW MEMORY: HIGH-BANDWIDTH,

HIGH-CAPACITY, ON-CHIP


2D baselinesystem

Off-chip DRAM (LPDDR3)• Capacity: 4 GBytes• Latency: 50 ns• BW: 12 GBytes/s• Read/write energy: 17 pJ/bit


AcceleratorCores

SRAM on-chip memory




2D baselinesystem



AcceleratorCores

SRAM on-chip memory



New system

AcceleratorCores

SRAM on-chip memory

Off-chip DRAM (LPDDR3)• Capacity: (4 GBytes minus New Mem. Cap.)• Latency: 50 ns • BW: 12 GBytes/s• Read/write energy: 17 pJ/bit


2D baselinesystem



AcceleratorCores

SRAM on-chip memory

New system

High Bandwidth, High Capacity both critical

AcceleratorCores

SRAM on-chip memory

Off-chip DRAM (LPDDR3)• Capacity: (4 GBytes minus New Mem. Cap.)• Latency: 50 ns • BW: 12 GBytes/s• Read/write energy: 17 pJ/bit

On-chip New memory• Capacity: sweep (up to 4 GBytes)• Latency: sweep (down to 3ns)• BW: sweep (up to 128 GBytes/s)• Read/write energy: 5 pJ/bit




5 ns memory access latency, 5 pJ/bit access energy

EDP benefits


NEW MEMORY ESSENTIAL REQUIREMENTON-CHIP CAPACITY MUST EXCEED DATA SIZE

Language model (LSTM)

2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

Band

wid

th (

GB

ytes/s

)

Band

wid

th (

GB

ytes/s

)

4.2x1.3x

1.3x 2.9x - 3.6x

12

0M

Byt

e

64 GBytes/s

Capacity1 MByte 4 GByte

128

10

Capacity

50x2.1x – 8x

2.1x – 8x1

5x –

30

x

2.5

GB

yte100 GBytes/s

1 MByte 4 GByte

10

128



2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

Band

wid

th (

GB

ytes/s

)

Capacity (MBytes)

112

128

96

80

64

48

32

16

1024 2048 3072 4096

Band

wid

th (

GB

ytes/s

)

Capacity (MBytes)

112

128

96

80

64

48

32

16

1024 2048 3072 4096

50

NEW MEMORY ESSENTIAL REQUIREMENTON-CHIP CAPACITY MUST EXCEED DATA SIZE

EDP benefits

5 ns memory access latency, 5 pJ/bit access energy



5 pJ/bit access energy

EDP benefits


NEW MEMORY ESSENTIAL REQUIREMENTHIGH BANDWIDTH MORE CRITICAL THAN LATENCY


2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

Band

wid

th (

GB

ytes/s

)

Band

wid

th (

GB

ytes/s

)

4.2x 4.1x

1.1x - 3x

20 n

s

64 GBytes/s

Latency (ns)3 50

128

10

Latency (ns)

50x 20x – 35x

1.1x – 20x

15

x –

30

x

10 n

s

100 GBytes/s

3 50

10

128

2.4

x –

3.9

x



2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

EDP benefits

5 pJ/bit access energy


Band

wid

th (

GB

ytes/s

)

Latency (ns)

112

128

96

80

64

48

32

16

5 10 40 80

50

1

20

Band

wid

th (

GB

ytes/s

)

Latency (ns)

112

128

96

80

64

48

32

16

5 10 40 80

4.2

20

NEW MEMORY ESSENTIAL REQUIREMENTHIGH BANDWIDTH MORE CRITICAL THAN LATENCY


Energy ✕ Execution Time

1971X

525X320X

159X63X

1

10

100

1000

10000

Lang.Model (LSTM) AlexNet (CNN) Captioning (LSTM) ResNet152 (CNN) VGG19 (CNN)

Sys

tem

-Leve

l B

enefits

N3XT Benefits: relative to 2D Baseline System (28nm silicon CMOS, LPDDR3)Inference: 16-bit data, batch size of 1

Workload: Inference on ML Accelerator

N3XT: UP TO ~2,000X

ENERGY EFFICIENCY BENEFITS

Source: Stanford/NTU: M. Aly, T. Wu, A. Bartolo, H.-S. P. Wong, S. Mitra et. al., Proc. IEEE 2019


N3XT SYSTEM

High Density On-Chip Nonvolatile Memory


Si Logic Die






1D carbon nanotube (CNT)

2D TMD (MoS2, WSe2, WS2…)

Source: S.-K. Su, … L.-J. Li (TSMC), Nature Nanotech., 2019.

Photo credit: B. Radisavljevic et al., Nature Nanotech., p. 147, 2011

NANOMETER-THIN TRANSISTOR CHANNEL

1 nm

< 1 nm

1

10

100

1,000

10,000

0 1 2 3 4

Mo

bilit

y(c

m2/V

-s)

Channel thickness (nm)

MoS2

WS2

WSe2 Si

Ge

CNT

Filled: electronOpen: hole

Photo credit: User Mstroeck on en.wikipedia


2D LAYERED MATERIALS (WS2, WSe2)

MoS2-e0.5

0.4

0.3

0.2

0.110 100 1000

Effective m

ass (

m0)

Mobility (cm2/V-s)

Source: C.-C. Cheng et al. (TSMC), Symp. VLSI Tech. 2019

WSe2-h ION (μA/μm)

200

400

600

800

20 nm 10 nm

Classical FET

G

S D

G

S D


5 nm Gate Length

SHORT-CHANNEL

CARBON NANOTUBE TRANSISTORS

10 nm Gate Length

Source: C. Qiu,…L-M. Peng (PKU), Science, 2017

VDS = -0.4V VDS = 0.4V

SS = 70 mV/Dec 70 mV/Dec

10-5

10-6

10-7

10-8

-1.0 -0.5 0.0 0.5

Vgs (V)

I ds

(A)

10-5

10-6

10-7

10-8

-1.0 -0.5 0.0 0.5

Vgs –Vt (V)

I ds

(A)

10-9

10-10

VDS = -0.1 V

SS = 73 mV/Dec

Lg = 5 nm


CARBON NANOTUBE COMPUTER

Source: M. Shulaker,… H.-S. P. Wong, S. Mitra (Stanford), Nature, 2013

instruction fetch

arithmetic block

data fetch

write-back


Kbit 6T SRAM (6144 CNFETs)

CARBON NANOTUBE FET CMOS SRAM

Source: P. Kanhaiya,… M. Shulaker (MIT), Symp. VLSI Tech., 2019


MEMORY INTEGRATION

ON LOGIC PLATFORM

Better transistor alone


Transistors integrated with memory in 3D

MEMORY INTEGRATION

ON LOGIC PLATFORM


Norm

aliz

ed

Den

sity

SYSTEM INTEGRATION

A CONTINUUM FROM FAR BACK-END TO FRONT-END

Source: IMEC

Interposer Chip-on-waferWafer-on-wafer

Monolithic 3D

107

108

106

105

104

103

102

101

100


SOCIETAL NEEDS FOR ADVANCED

TECHNOLOGY IS INSATIABLE

ADVANCED TECHNOLOGY

− A KEY DIFFERENTIATOR



Continuous transistor & memory advances

Memory logic integration


System integration with high connectivity


A CALL TO ACTION: EARLY ENGAGEMENT

SYSTEM ↔ TECHNOLOGY

ACADEMIA ↔ INDUSTRY RESEARCH


End of Talk

Questions?



Continuous transistor & memory advances

Memory logic integration


System integration with high connectivity

COMMITTED TO PROVIDING THE MOST

ADVANCED TECHNOLOGIES

ic technology what will the next node offer us? · 8/17/2019 · 2d baseline system off-chip dram...

Documents