rathijit sen david a. wood reuse-based online models for caches 6/20/2013 acm sigmetrics 2013 @ cmu,...

RATHIJIT SENDAVID A. WOOD

Reuse-based Online Models for Caches

6/20/2013ACM SIGMETRICS 2013 @ CMU, Pittsburgh, PA

1

2

The Problem


Caches: power vs performance

Reconfigurable caches e.g., IvyBridge

The Problem: Which configuration to select?

e.g., to get the best energy-efficiency?

Core

Core

Core

Core

Core

Core

Core

Core

LLC LLC

LLC LLC

LLC LLC

LLC LLC

DRAM

Miss Fetch

3

Cache Performance Prediction


We propose a frameworkh = (r · B) · φ

h: hit ratio r: reuse-distance distribution (novel hardware support) B: stochastic Binomial matrix φ: hit function (LRU, PLRU, RANDOM, NMRU)

Case study: Energy-Delay Product (EDP) within 7% of minimum

4

Agenda


The Problem

Framework Locality (r) Matrix transformations (B) Hit functions (φ) h = (r · B) · φ

Hardware support

Case Study

5

Cache Overview


Limited storage Sets of (usually 64-byte) blocks #blocks/set = associativity (#ways) Set Index + Address tags identify data

b b b b b b b b

b b b b b b b b

b b b b b b b b

b b b b b b b b

Associativity (A)

Sets (S)

AddressTag

Match?

Y HitMiss

N

6

Last-Level Cache (LLC)

Workload Variation

2MB 4MB 8MB 16MB 32MB0

5

10

15

20

25

30

Mis

s / 1

000

Inst

ructi

on swim


ammp, blackscholes, bodytrack, fluidanimate, freqmine, swaptions

equake, gafort, wupwise

apache

mgrid

zeus

oltpjbb

fma3d

7

Bad configurations hurt!


EDP (energy-delay product)

blac body flui freq swapammp equa fma3 gafo mgri swim wupw apac jbb oltp zeus1

1.5

2

2.5

3

3.5Max. EDP

Rela

tive

to m

in. E

DP

27% worse

218% worse

MinimumMaximum

8

Problem Summary


Reconfigurable caches

Multiple replacement policies

Goal: Online miss-ratio prediction

b b b b b b b b

b b b b b b b b

b b b b b b b b

b b b b b b b b

Associativity (A)

Sets (S)

9

Indexing Assumption


Mapping of unique addresses to cache setsAssumption: independent, uniform [Smith, 1978]

Unique accesses as Bernoulli trials

(Partial) Hashing POWER4, POWER5, POWER6, Xeon Simple XOR-based function [similar to Cypher, 2008]

10

Agenda


The Problem


Hardware support

Case Study

11

Temporal Locality Metrics


Unique Reuse Distance (URD) #unique intervening addresses x y z z y x : URD(x)=2 Stack Distance [Mattson, 1970] – 1 Large cache large distances to track

Absolute Reuse Distance (ARD) #intervening addresses x y z z y x : ARD(x)=4

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r

Size?

12

Per-set Locality, r(S)


r(S) is “compressed” as S (#sets) increases Less of the tail is important

0 4 8 12 16 20 24 28 320

0.1

0.2

0.3

0.4

0.5

0.6S=2^14S=2^13S=2^12S=2^11S=2^10

Per-set URD (unique reuse distance)

Prob

abili

ty

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r

x x

x x

#sets: S #sets: S > S

0 4 8 12 16 20 24 28 320

0.2

0.4

0.6

0.8

1

S=2^14S=2^13S=2^12S=2^11S=2^10

Per-set URD (unique reuse distance)

Cum

ulati

ve P

rob.

13

Agenda


The Problem


Hardware support

Case Study

14

Generalized stochastic Binomial matrices [Strum, 1977]r(S) = r(1) · B(1 – 1/S, 1/S)

Composition:r(S) = r(S) · B(1 – S/S, S/S)

0

0

0

0

0 0

0

0

0

0

0 0 0

0

0

0

0

Estimating per-set locality


■ ■ ■ ■ ■ ■ ■ ■ i

P(URD=i)

k

ir

B

P(k successes in i trials) i.e.,P(k of i to the same set)

0

0

0 0

0

0

0

0

0

0

0

1

15

Computation reuse & speedup


“Shorter” tail smaller matrices

r(1)

r(214)

r(213)

r(212)

r(211)

r(210)

r(210)

r(214)

r(213)

r(212)

r(211)

r(1)

Now: computeLater: hardware support

Size?

Poisson Approximation

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r

16

Size of r(210)?


Prediction with r(210) limited to URD < n

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2MB 4MB 8MB 16MB 32MB

0

0.05

0.1

0.15

0.2

0.25

0.3n=32 n=64 n=128n=256 n=512 Actual

Mis

s Ra

tio

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r

17

Agenda


The Problem


Hardware support

Case Study

18

Hit Function, φ


φk: P(x will hit|URD(x)=k)

Monotonically decreasing model Intuition: larger URD same or larger eviction probability

φ0 = 1φk ≤ φk-1

φ = 0

x

Not x

x

∞

19

Hit Function, φ


Example: A=8

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 320

0.10.20.30.40.50.60.70.80.9

1LRUPLRUNMRURANDOM

Unique Reuse Distance

Hit

Prob

abili

ty

20

Formulating φ


φ(LRU): step-function (r · B) · φ(LRU) [Smith, 1978], [Hill & Smith, 1989]

φ(PLRU): Assumes on average, traffic evenly divided between subtrees

φ(RANDOM): Estimates #intervening misses using ARD

φ(NMRU): similar to φ(RANDOM) except φ1=1

21

Agenda


The Problem


Hardware support

Case Study

22

Prediction Accuracy


LRU, PLRU(A=2), NMRU(A=2): exact per-set modelOthers: approximate per-set model

-1% 0% 1% 2% 3% 4% 5% 6%0

0.10.20.30.40.50.60.70.80.9

1

LRU PLRU RANDOM NMRU

abs((predicted-actual)/actual) miss ratio

Cum

ulati

ve P

roba

bilit

y

23

Overheads


r = r · B : 6 80 μsec Binomial Poisson approximation for each row of B

h = (r · B) · φ : 20 30 μsec Average over 24 configurations B applied 8 times

24

Agenda


The Problem


Hardware support

Case Study

25

Computation reuse & speedup


“Shorter” tail smaller matrices

r(1)

r(214)

r(213)

r(212)

r(211)

r(210)

r(210)

r(214)

r(213)

r(212)

r(211)

r(1)

Now: computeLater: hardware support

Size=512

Poisson Approximation

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r

Now

26

Insights


x y z z y x : URD(x)=2

Unique “remember” addresses Only cardinality, not full addresses

Bloom filter for compact (approximate) representation

r(210) is seen by any set of a cache with S=210 Filter address stream

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r

27

Reference address register

access

insert

Set Filter Control Logic

filtered access

load hitinc

reset

read

read

1024-bit Bloom Filter2 hash fns

9-bit Counter

inc

512-entry Histogram

array

Hardware Support for estimating r(210)


Start Sample

Addr match?

Unique?

Remember

End Sample

N

Y (not hit)

Y

28

Agenda


The Problem


Hardware support

Case Study

+ way counters

ACM SIGMETRICS 2013 @ CMU, Pittsburgh, PA

29

LRU Way Counters [Suh, et al. 2002]

6/20/2013

One counter per logical way (stack position)Determining logical position is hard

not totally (re-)ordered with every access heuristics, e.g., for PLRU [Kedzierski, et al. 2010]

Other Limitations Inclusion property Fixed #sets

S = S : special case of reuse frameworkS S ? Use B

provided, enough tail of r(S) is available

30

Min. EDP configuration


EDP within 7% of minimumReuse models outperform PLRU way counters in most cases

blac body flui freq swapammpequa fma3 gafo mgri swim wupw apac jbb oltp zeus1

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08Reuse ModelPLRU Way Counters

Rela

tive

to m

in. E

DP

31

Summary


The Problem: Online miss-rate estimation for reconfigurable caches

We propose a frameworkh = (r · B) · φ

h: hit-ratio r: reuse-distance distribution (novel hardware support) B: stochastic Binomial matrix φ: hit function (LRU, PLRU, RANDOM, NMRU)

Case study: EDP within 7% of minimum

Future work: More policies, applications/case studies


32

Also in the paper

6/20/2013

r: lossy summarization of the address trace

Estimation for ARD

Optimizations for LRU

Conditions for PLRU eviction

More details on models & evaluation

Reuse-based Online Models for Caches


33

Questions?

34

Example LLC performance


OLTP (TPC-C + IBM DB2)

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2-w

4-w

8-w

16-w

32-w

2MB 4MB 8MB 16MB 32MB

0

0.1

0.2

0.3

0.4RANDOMNMRUPLRULRU

Mis

s Ra

tio


35

Estimating cache performance

6/20/2013

Hit ratio = hits/access

∑ P(URD=i) · P(hit|URD=i)

= ·

Miss ratio = misses/access = 1 – hit ratio

Miss rate = misses/instruction = miss ratio x access/instruction

■ ■ ■ ■ … ■ ■ i

P(URD=i)

r … i

P(hit|URD=i)

φ

i

36

URD vs ARD


x x

z0z1 z2 z3 zk-1

{z0}* {z0,z1}* {z0,z1,z2}* {z0,z1,z2,...,zk-1}*

dk = dk-1 +1/rikApproximation:

∞dk

rathijit sen david a. wood reuse-based online models for caches 6/20/2013 acm sigmetrics 2013 @ cmu,...

Documents