denovo † : rethinking hardware for disciplined parallelism

DeNovo†: Rethinking Hardware for Disciplined Parallelism

Byn Choi, Rakesh Komuravelli, Hyojin Sung, Rob Bocchino, Sarita Adve, Vikram Adve

Other collaborators:Languages: Adam Welc, Tatiana Shpeisman, Yang Ni (Intel)

Applications: John Hart, Victor Lu

†De Novo = From the beginning, anew

Motivation• Goal: Power-, complexity-, performance-scalable hardware• Today: shared-memory

– Directory-based coherence• Complex, unscalable

– Address, communication, coherence granularity is cache line• Software-oblivious, inefficient power, bandwidth, latency, area

– Difficult programming model• Data races, non-determinism, no safety/composability/modularity, …

– Can’t specify “what value can read return” a.k.a. memory model• Data races defy acceptable semantics; mismatched hardware/software

• Fundamentally broken for hardware & software



– Address, communication, coherence granularity is cache line• Inefficient in power, bandwidth, latency, area, especially for O-O codes


– Can’t specify “what value a read can return” a.k.a. memory model• Data races defy acceptable semantics; mismatched hardware/software


Banish shared memory?







Banish wild shared memory!







Banish wild shared memory!

Need disciplined shared memory!

What is Shared-Memory?

Shared-Memory =

Global address space +

Implicit, anywhere communication, synchronization


Wild Shared-Memory =


Implicit, anywhere communication, synchronization


Disciplined Shared-Memory =


Implicit, anywhere communication, synchronizationExplicit, structured side-effects

Disciplined Shared-Memory• Top-down view: prog model, language, … (earlier today)

– Use explicit effects for semantic guarantees • Data-race-freedom, determinism-by-default, controlled non-determinism

– Reward: simple semantics, safety, composability, …• Bottom-up view: hardware, runtime, …

– Use explicit effects + above guarantees for• Simple coherence and consistency• Software-aware address/comm/coherence granularity, data layout

– Reward: power-, complexity-, performance-scalable hardware• Top-down and bottom-up views are synergistic!

DeNovo Research Strategy1. Start with deterministic ( data-race-free) codes

― Common & best case, basis for extension to other codes2. Disciplined non-deterministic codes3. Wild non-deterministic, legacy codes

• Work with languages for best h/w-s/w interface– Current driver is DPJ– End-goal is language-oblivious interface

• Work with realistic applications– Current work with kd-trees

Progress Summary1. Deterministic codes

― Language level model with DPJ, translation to h/w interface― Design for simple coherence, s/w-aware comm. and layout― Baseline coherence implemented, rest underway

2. Disciplined non-deterministic codes― Language level model with DPJ (and Intel)

3. Wild non-deterministic, legacy codes– Just begun, will be influenced by above

Collaborations with languages & applications groups– DPJ; first scalable SAH quality kd-tree construction

Coherence and ConsistencyKey Insights• Guaranteed determinism

• Explicit effects


– Read should return value of last write in sequential order• From same task in this parallel phase, if it exists• Or from previous parallel phase• No concurrent conflicting writes in this phase

• Explicit effects


– Read should return value of last write in sequential order• From same task in this parallel phase, if it exists• Or from previous parallel phase• No concurrent conflicting writes in this phase

• Explicit effects – Compiler knows all regions written in this parallel phase– Cache can self-invalidate before next parallel phase

• Invalidates data in writeable regions not written by itself

Today's Coherence Protocols• Snooping: broadcast, ordered networks• Directory: avoid broadcast through indirection

– Complexity: Races in protocol – Overhead: Sharer list– Performance: All cache misses go through directory


– Complexity: Races in protocol • Race-free software (almost) race-free coherence protocol• No transient states, much simpler protocol

– Overhead: Sharer list– Performance: All cache misses go through directory



– Overhead: Sharer list• Explicit effects enable self-invalidations• No need for sharer lists

– Performance: All cache misses go through directory



– Overhead: Sharer list• Explicit effects enable self-invalidations• No need for sharer lists

– Performance: All cache misses go through directory• Directory only tracks one up-to-date copy, not sharers or serialization• Data copies can move from cache to cache without telling directory

Baseline DeNovo Coherence• Assume (for now): Private L1, shared L2; single word line• Directory tracks one current copy of line, not sharers• L2 data arrays double as directory

– Keep valid data or registered core id, no space overhead

• S/W inserts self-invalidates for regions w/ write effects • L1 states = invalid, valid, registered• No transient states: Protocol ≈ 3-state textbook pictures

– Formal specification and verification with Intel

DeNovo Region Granularity• DPJ regions too fine-grained

– Ensure accesses to individual objects/fields don’t interfere– Too many regions for hardware

• DeNovo only needs aggregate data written in phase– E.g., can summarize a field of entire data structure as one region

Can we aggregate to few enough regions,without excessive invalidations?

Evaluation Methodology• Modified Wisconsin GEMS + (Intel!) Simics simulator• 4 apps: LU, FFT, Barnes, kd-tree

– Converted DPJ regions into DeNovo regions by hand

• Compared DeNovo vs. MESI for single word lines– Goal: Does simplicity impact performance? E.g., are self-invalidations too conservative? (Efficiency enhancements are next step)

App Barnes FFT LU Kd-tree# Regions 8 3 2 4

Results for Baseline Coherence

• DeNovo comparable to MESI• Simple protocol is foundation for efficiency enhancements

LU FFT Barnes-Hut kd-tree (nested)

0

20

40

60

80

100

MESIDeNovo

Exec

utio

n tim

e N

orm

aliz

ed to

MES

I (%

)

Improving Performance & PowerInsight: Can always copy valid data to another cache

– w/o demand access, w/o going through directory – If later demand read sees it, it must be correct– No false sharing effects (no loss of “ownership”)

Simple line based protocol (with word based valid bits) Can get line from anywhere, not just from directory L2

― Point-to-point transfer, sender-initiated transfer Can transfer more than line at a time

― Point-to-point bulk transfer Transfer granularity can be region-driven

― AoS vs. SoA optimization is natural

Towards Ideal EfficiencyCurrent systems:

― Address, transfer, coherence granularity = fixed cache lineDenovo so far:

― Transfer is flexible, but address is still line, coherence is still wordNext step: Region-centered caches

– Use regions for memory layout• Region based “pool allocation,” fields of same region at fixed strides

– Cache banks devoted to regions– Regions accessed together give address, transfer granularity– Regions w/ same sharing behavior give coherence granularity

Applicable to main memory and pin bandwidthInteractions with runtime scheduler

Summary• Current shared-memory models fundamentally broken

– Semantics, programmability, hardware• Disciplined programming models solve these problems• DeNovo = hardware for disciplined programming

– Software-driven memory hierarchy– Coherence, consistency, communication, data layout, …– Simpler, faster, cooler, cheaper, …

• Sponsor interactions:– Nick Carter, Mani Azimi/Akhilesh Kumar/Ching-Tsun Chou– Non-deterministic model: Adam Welc/Shpeisman/Ni– SCC prototype exploration: Jim Held

Next steps• Phase 1:

– Full implementation, verification, results for deterministic codes– Design and results for disciplined non-determinism– Design for wild non-deterministic, legacy codes– Continue work with language and application groups– Explore prototyping on SCC

• Phase 2: – Design and simulation results for complete DeNovo system

running large applications– Language-oblivious hardware-software interface– Prototype and tech transfer

denovo † : rethinking hardware for disciplined parallelism

Documents