iccd talk

1 Broadcom Proprietary and Confidential. © 2013 Broadcom Corporation. All rights reserved.

2015 International Conference On Computer AidedDesign

The Premier Conference Devoted to Technical Innovations in Electronic Design Automation

November 2 - 6, 2015Doubletree Hotel

Austin, TX

2 0 1 5 C O N F E R E N C E2 0 1 5 C O N F E R E N C E

- ICCAD IS NEXT WEEK! - Plan your time and make the most of the conference:View ICCAD's Agenda.

- SEE YOU IN AUSTIN -Be sure to check out all this great city has to offer!Visit austintexas.org/visit for more information.

- LEARN MORE ABOUT ICCAD - Interested in Attending ICCAD? Check out what peopleare saying about this conference:

C O N F E R E N C E H I G H L I G H T SC O N F E R E N C E H I G H L I G H T S

- NEW MOBILE APP FOR ICCAD2015 -Download the free free official Whova app for ICCAD!Following your download, sign up on Whova with the emailaddress that you used for the event RSVP or your socialmedia accounts. If you are asked to enter an eventpasscode when accessing the event, please use thefollowing passcode: iccboiccbo

Benefits include:Benefits include:Plan who to meet at ICCAD by browsing attendee profilesin advanceEasily discover professionals with common affiliations,education, shared networks and social profilesSend in-app messages and exchange digital business cardsand other contact infoReceive update notifications from organizersAccess agenda, GPS guidance, maps, and parking directions

HOME ABOUT ICCAD CONFERENCE REGISTRATION TRAVEL & STAY

RESOURCE CENTER CONTACT US

Special session : Dennard Scaling is History and Moore's Law is Aging: How to Break the Inevitable Power Wall? Partha Kundu

POWER AND MICROARCHITECTURE TRADEOFFS IN NEXT-GENERATION MANYCORES FROM HOMOGENOUS TO HETEROGENEOUS CORES AND EVERYTHING IN BETWEEN

Partha Kundu Technical Director Infrastructure & Networking Group 2015 International Conference On Computer Aided

Design



Austin, TX













Austin, TX










Special session : Dennard Scaling is History and Moore's Law is Aging: How to Break the Inevitable Power Wall?

Austin, TX, USA Nov 2, 2015





Austin, TX











STATE OF SERVER CPUS

going to ensure we’ll drive this transition together.” More than any other single advance, the factthat Xeon E5-2600 v3 series supports the newest form factor of enterprise-class memory module,will change the way servers look, act, and work. The DDR4 class has been waiting in the wings foryears, having been pioneered and even produced by Samsung.

- DDR4’s maximum component density has doubled over DDR3 to 8Gb

- DDR4’s bit rate has been boosted by 50% over DDR3 to 3.2 Gbps.

- DDR4’s standard voltage has been lowered from 1.5V to 1.2V.

DDR3’s memory bandwidth had been capped at 1866 MHz since early 2011, although 1600 MHzmodules remained prevalent. Meanwhile, DDR4 modules have been produced for over 2 years withbandwidths at 2133 MHz.

The “Advanced series” of E5 v3 will support 2133 MHz out of the gate, with five models startingwith the 10-core, 2.3 GHz E5-2650 v3, working up to the 12-core, 2.6 GHz E5-2690 v3. Nearly allof the “segment-optimized” series (high-performance, high-frequency, and workstation), except forthe 4-core models, support DDR4-2133, including the high-frequency 6-core, 3.4 GHz E5-2643v3,and the top-of-the-line 18-core, 2.3 GHz E5-2699 v3.

Every advancement in server memory architecture in the last three years has been designed forDDR4. But until today, there hasn’t been an Intel x86 server processor that would support it.

“As most people know, the memory vendors have stopped investing in DDR3 in terms of newcapabilities,” admitted McConnell. “DDR3 caps out at an 1866 [MHz] memory bandwidth speed. So we’re excited about the new bandwidth improvements we can get with DDR4, and we’re alsolooking at significant power efficiencies.”

A network of 18 coresWe’ve known since the onset of the multicore era that processor cores are not linearly scalable.After a certain number — which engineers in the last decade reliably pinned at 8 — simply stackingmore cores onto a die could either yield no performance improvement or actually impose aperformance cost.

So linear scalability is not the answer long-term. For Intel to make a processor more capable bystacking cores, it has had to change the architecture with which cores are stacked. With E5 v3,that change runs deeper than you might have expected.

Although we call Xeon E5 v3 an “18-core” processor, to betruthful, there’s just one model (“SKU”) where all 18 are present:the 2.3 GHz, 145W TDP E5-2699 v3. Nevertheless, each of thedie configurations in the v3 series is based on an 18-coredesign that may be scaled down for lower cores.

The 18-core, 2.3 GHz Xeon E5-2699 v3 is arranged somewhatdifferently from the 12-core, 2.7 GHz Xeon E5-2697 v2 (no, that’s no typo, the newer model has theslower clock speed). Without rethinking the microarchitecture of the core itself, Intel has changed2





Austin, TX











END OF DENNARD SCALING

00.20.40.60.81

1.21.41.61.82

1997 2000 2003 2006 2009

Ope

ra&n

gVo

ltage(V

)

IBMPowerPC405LPIntelXscale80200TransMetaCrusoeTM5800IntelItaniumMontecitoAtomSilverthorneVminVmax

3

DVFS less useful to achieve energy proportionality





Austin, TX











END OF DENNARD SCALING : WHAT CAN WE DO IN ARCHITECTURE?

•  Heterogeneous cores, Same ISA

•  Homogenous cores + specific accelerators

4





Austin, TX











OUTLINE OF TALK

•  Heterogeneous cores, Same ISA


5





Austin, TX











HETEROGENEOUS CORES, SAME ISA

Single ISA heterogenous multi-core architectures for multi-threaded applications, Kumar et al, ISCA ‘04

6





Austin, TX











SAME ISA, HETEROGENEOUS CORES

Single ISA heterogenous multi-core architectures for multi-threaded applications, Kumar et al, ISCA ‘04

7





Austin, TX











ARM’S BIG.LITTLE

http://www.eetimes.com/document.asp?doc_id=1279167&page_number=1

8





Austin, TX











ARM’S BIG.LITTLE

9





Austin, TX











Widget: Wisconsin decoupled grid execution tiles, Watanabe, ISCA’10

FLEXIBLE HETEROGENEITY : SAME ISA

10





Austin, TX











FLEXIBLE HETEROGENEITY : WIDGET UARCH

11





Austin, TX











WIDGET: STEERING HEURISTIC

§  Based on dependence-based steering [Palacharla97] §  Expose independent instr chains §  Consumer directly behind the producer §  Stall steering when no empty buffer is found

§  WiDGET: Power-performance goal §  Emphasize locality & scalability

12

Cluster0 Cluster1

OutstandingOps?

ProducerbufEmptybufwithincluster

Anyemptybuf Availbehindproducer? Availbehindeitherofproducers?

Emptybufineitherofclusters

0 1 2

Y Y NN

•  Consumer-pushoperandtransfers–  SendsteeredEUIDtotheproducerEU–  MulV-castresulttoallconsumers





Austin, TX











WIDGET

0.20.30.40.50.60.70.80.91

0.3 0.5 0.7 0.9 1.1 1.3

Normalized

ChipPo

wer

NormalizedPerformance

Neon

Mite

1EU

2EUs

3EUs

4EUs

5EUs

6EUs

7EUs

8EUs

•  Best-case: 2x of Neon, 21x of Mite •  1.5x the efficiency of Xeon for the same performance

13





Austin, TX











OUTLINE OF TALK

•  Heterogenous cores, Same ISA


14





Austin, TX











STATE OF HPC NETWORKING

15

Proprietary Interconnects still used in the highest performance systems Low overhead API required to improve interconnect systems going forward





Austin, TX











ADI-Ch ADI-Ch

Infiniband(transport)layer

MPIapps PGASapps

MPICH OpenMPI Gasnet openShmem

MPI ON INFINIBAND

16





Austin, TX











Accelerating High Performance Computing Applications Through MPI Offloading, Shainer et al, HPC Council whitepaper, 2011

MPI PROFILE ACROSS AMBER : A MOLECULAR DYNAMICS PACKAGE

17

Majority of time spent in AllReduce





Austin, TX











18

MPI_REDUCE

0 1 2 3

0

5 1 7 8 4 2

18 14

2 3

MPI_SUM

0 1 2 3

1

5 1 7 8 4 2

18 14

2 3

MPI_SUM

0 18 14 2 18 14 3 18 14

Example: perform average @ node0

Example: perform std dev. @ all nodes

MPI_ALLREDUCE

MPI_REDUCE +

MPI_BROADCAST





Austin, TX











19

TREE ALGORITHM : LARGE MESSAGES

0 1 (k-1)

n

m

……………………….

Recursive doubling: short messages

ALLREDUCE ALGORITHMS

Tree Algorithm : large messages





Austin, TX











libfabric

Communication Services

Connection Management

Address Vectors

Completion Srvcs

Event Queues

Counters

Data Transfer Services

Message Queues

Tag Matching

RMA

Atomics

Control Services

Discovery

OFI Provider

Address Vectors

Event Queues

Counters

Message Queues

Tag Matching

RMA

Atomics

Discovery

Trig

gere

d O

pera

tions

Tr

igge

red

Ope

ratio

ns

MPI SHMEM PGAS

Libfabric Enabled Applications

. . .

NIC TX Command Queues

RX Command Queues

Connection Management

LibFabrics ó Portals* Application & Software stack

20

*Portals Network Programming Interface: www.cs.sandia.gov/Portals/





Austin, TX











IBVERBS VS OFI VS. PORTALS

21





Austin, TX











22

BARRIER OFFLOADED USING PORTALS

CPU

NIC NIC

CPU

CtAlloc(.,&Ct1)CtWait(&Ct1,N,*event)

P1 P0

CPU

NIC

P2

CPU

NIC

PN

……. Put(..,P0,..)Put(..,P0,..)Put(..,P0,..)

Barrier

Ct1++

Ct1++

…..

Ct1++

(Ct1==N): EventPost(*event)

PN

P1 …….

P0





Austin, TX











23

SYNCHRONIZATION : OFFLOADED WITH PORTALS (SIMPLE)





Austin, TX










Special session : Dennard Scaling is History and Moore's Law is Aging: How to Break the Inevitable Power Wall? Partha Kundu 24

MPI_ALLREDUCE (TREE ALGORITHM) WITH PORTALS ATOMICS

0 1 (k-1)

n

m

Atomic(…,UpNode_n,...Sum,int_8……)

TriggeredAtomic(…,UpNode_m,….Sum,int_8,TrigCtHandle,k)

TriggeredPut(…,DownNode_n,..TrigCtHandle,(m-n-1))

TriggeredPut(..,DownNode_0,…,TrigCt,1)TriggeredPut(..,DownNode_1,…,TrigCt,1)…TriggeredPut(..,DownNode_k-1,…,TrigCt,1)

……………………….





Austin, TX











25

Short Messages (500ns message) At higher node counts Recursive Doubling does better than Triggered tree

Long Messages (1500ns message) Triggered Tree does best consistently

PERFORMANCE : HOST VS. TREE VS. RECURSIVE

Enabling Flexible Collective Communication Offload with Triggered Operations, Underwood et. al., IEEE High Performance Interconnects, 2011





Austin, TX











CONCLUSIONS

§ Application mixes low ILP and high ILP in phases, different programs

§ Heterogeneity (same ISA) reduces power

§ Accelerators that offload main CPU effectively allowing communication and computation need to be overlapped. Improve performance and power.

§ Accelerator API needs to close the semantic gap between application libraries and hardware

26

iccd talk

Documents