what do you mean by “cache friendly”?...what do you mean by “cache friendly”? – code::dive...
TRANSCRIPT
![Page 1: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/1.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 1/205
What Do You Mean by “Cache Friendly”?
Björn Fahller
![Page 2: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/2.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 2/205
typedef uint32_t (*timer_cb)(void*);struct timer { uint32_t deadline; timer_cb callback; void* userp; struct timer* next; struct timer* prev;};
static timer timeouts = { 0, NULL, NULL, &timeouts, &timeouts };
timer* schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer* iter = timeouts.prev; while (iter != &timeouts !& is_after(iter!>deadline, deadline)) iter = iter!>prev; add_behind(iter, deadline, cb, userp);}
![Page 3: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/3.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 3/205
typedef uint32_t (*timer_cb)(void*);struct timer { uint32_t deadline; timer_cb callback; void* userp; struct timer* next; struct timer* prev;};
static timer timeouts = { 0, NULL, NULL, &timeouts, &timeouts };
timer* schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer* iter = timeouts.prev; while (iter != &timeouts !& is_after(iter!>deadline, deadline)) iter = iter!>prev; add_behind(iter, deadline, cb, userp);}
![Page 4: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/4.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 4/205
typedef uint32_t (*timer_cb)(void*);struct timer { uint32_t deadline; timer_cb callback; void* userp; struct timer* next; struct timer* prev;};
static timer timeouts = { 0, NULL, NULL, &timeouts, &timeouts };
timer* schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer* iter = timeouts.prev; while (iter != &timeouts !& is_after(iter!>deadline, deadline)) iter = iter!>prev; add_behind(iter, deadline, cb, userp);}
![Page 5: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/5.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 5/205
typedef uint32_t (*timer_cb)(void*);struct timer { uint32_t deadline; timer_cb callback; void* userp; struct timer* next; struct timer* prev;};
static timer timeouts = { 0, NULL, NULL, &timeouts, &timeouts };
timer* schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer* iter = timeouts.prev; while (iter != &timeouts !& is_after(iter!>deadline, deadline)) iter = iter!>prev; add_behind(iter, deadline, cb, userp);}
void cancel_timer(timer* t) { t!>next!>prev = t!>prev; t!>prev!>next = t!>next; free(t);}
![Page 6: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/6.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 6/205
What Do You Mean by “Cache Friendly”?
Björn Fahller
![Page 7: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/7.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 7/205
Simplistic model of cache behaviour
Includes
![Page 8: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/8.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 8/205
Simplistic model of cache behaviour
Includes
● The cache is small
![Page 9: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/9.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 9/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines
![Page 10: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/10.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 10/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast
![Page 11: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/11.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 11/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast● and data acess miss is very slow
![Page 12: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/12.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 12/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast● and data acess miss is very slow
Excludes
![Page 13: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/13.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 13/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast● and data acess miss is very slow
Excludes
● Multiple levels of caches
![Page 14: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/14.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 14/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast● and data acess miss is very slow
Excludes
● Multiple levels of caches● Associativity
![Page 15: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/15.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 15/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast● and data acess miss is very slow
Excludes
● Multiple levels of caches● Associativity● Threading
![Page 16: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/16.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205
Simplistic model of cache behaviour
Includes
● The cache is small● and consists of fixed size lines● and data access hit is very fast● and data acess miss is very slow
Excludes
● Multiple levels of caches● Associativity● Threading
All models are wrong, but some are useful
![Page 17: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/17.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 17/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x3A100x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 18: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/18.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 18/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x3A100x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 19: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/19.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 19/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x3A100x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 20: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/20.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 20/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x3A100x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 21: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/21.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 21/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x3A100x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 22: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/22.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 22/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x3A100x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 23: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/23.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 23/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memorySimplistic model of cache behaviour
![Page 24: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/24.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 24/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
![Page 25: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/25.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 25/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
![Page 26: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/26.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 26/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
![Page 27: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/27.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 27/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
![Page 28: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/28.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 28/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
0x4010
![Page 29: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/29.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 29/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40100x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
0x4010
![Page 30: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/30.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 30/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
Simplistic model of cache behaviour
![Page 31: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/31.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 31/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
Simplistic model of cache behaviour
![Page 32: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/32.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 32/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
Simplistic model of cache behaviour
![Page 33: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/33.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 33/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
Simplistic model of cache behaviour
![Page 34: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/34.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 34/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
Simplistic model of cache behaviour
![Page 35: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/35.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 35/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
Simplistic model of cache behaviour
![Page 36: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/36.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 36/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
0x4080
Simplistic model of cache behaviour
![Page 37: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/37.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 37/205
const int* hot = 0x4001;const int* cold = 0x4042;int* also_cold = 0x4080;
int a = *hot;int c = *cold;*also_cold = a;also_cold[1] = c;
0x40000x4FF0
cache
0x40000x40100x40200x40300x40400x40500x40600x40700x40800x40900x40A00x40B00x40C00x40D00x40E00x40F0
memory
0x4040
0x4080
0x4080
Simplistic model of cache behaviour
![Page 38: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/38.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 38/205
Analysis of implementation
int main() { std!:random_device rd; std!:mt19937 gen(rd()); std!:uniform_int_distribution<uint32_t> dist;
for (int k = 0; k < 10; !+k) { timer* prev = nullptr; for (int i = 0; i < 20'000; !+i) { timer* t = schedule_timer( dist(gen), [](void*){return 0U;}, nullptr); if (i & 1) cancel_timer(prev); prev = t; } while (shoot_first()) ; } }
![Page 39: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/39.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 39/205
Analysis of implementation
int main() { std!:random_device rd; std!:mt19937 gen(rd()); std!:uniform_int_distribution<uint32_t> dist;
for (int k = 0; k < 10; !+k) { timer* prev = nullptr; for (int i = 0; i < 20'000; !+i) { timer* t = schedule_timer( dist(gen), [](void*){return 0U;}, nullptr); if (i & 1) cancel_timer(prev); prev = t; } while (shoot_first()) ; } }
![Page 40: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/40.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 40/205
Analysis of implementation
int main() { std!:random_device rd; std!:mt19937 gen(rd()); std!:uniform_int_distribution<uint32_t> dist;
for (int k = 0; k < 10; !+k) { timer* prev = nullptr; for (int i = 0; i < 20'000; !+i) { timer* t = schedule_timer( dist(gen), [](void*){return 0U;}, nullptr); if (i & 1) cancel_timer(prev); prev = t; } while (shoot_first()) ; } }
![Page 41: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/41.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 41/205
Analysis of implementation
int main() { std!:random_device rd; std!:mt19937 gen(rd()); std!:uniform_int_distribution<uint32_t> dist;
for (int k = 0; k < 10; !+k) { timer* prev = nullptr; for (int i = 0; i < 20'000; !+i) { timer* t = schedule_timer( dist(gen), [](void*){return 0U;}, nullptr); if (i & 1) cancel_timer(prev); prev = t; } while (shoot_first()) ; } }
![Page 42: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/42.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 42/205
Analysis of implementation
int main() { std!:random_device rd; std!:mt19937 gen(rd()); std!:uniform_int_distribution<uint32_t> dist;
for (int k = 0; k < 10; !+k) { timer* prev = nullptr; for (int i = 0; i < 20'000; !+i) { timer* t = schedule_timer( dist(gen), [](void*){return 0U;}, nullptr); if (i & 1) cancel_timer(prev); prev = t; } while (shoot_first()) ; } }
bool shoot_first() { if (timeouts.next != &timeouts) return false;
timer* t = timeouts.next; t!>callback(t!>userp); cancel_timer(t); return true;}
![Page 43: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/43.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 43/205
Analysis of implementation
valgrind --tool=callgrind –-cache-sim=yes –-dump-instr=yes --branch-sim=yes
![Page 44: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/44.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 44/205
Analysis of implementation
valgrind --tool=callgrind –-cache-sim=yes –-dump-instr=yes --branch-sim=yes
Essentially a profiler thatcollects info about callhierarchies, number ofcalls, and time spent.
The CPU simulator is notcycle accurate, so see
timing results as a broadpicture.
![Page 45: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/45.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 45/205
Analysis of implementation
valgrind --tool=callgrind –-cache-sim=yes –-dump-instr=yes --branch-sim=yes
Essentially a profiler thatcollects info about callhierarchies, number ofcalls, and time spent.
The CPU simulator is notcycle accurate, so see
timing results as a broadpicture.
Simulates a CPU cache,flattened to 2 levels, L1 and LL.
It shows you where you getcache misses.
L1 is by default a model ofyour host CPU L1, but youcan change size, line-size,
and associativity.
![Page 46: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/46.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 46/205
Analysis of implementation
valgrind --tool=callgrind –-cache-sim=yes –-dump-instr=yes --branch-sim=yes
Essentially a profiler thatcollects info about callhierarchies, number ofcalls, and time spent.
The CPU simulator is notcycle accurate, so see
timing results as a broadpicture.
Simulates a CPU cache,flattened to 2 levels, L1 and LL.
It shows you where you getcache misses.
L1 is by default a model ofyour host CPU L1, but youcan change size, line-size,
and associativity.
Collects statistics perinstruction instead of per
source line. Can helppinpointing bottlenecks.
![Page 47: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/47.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 47/205
Analysis of implementation
valgrind --tool=callgrind –-cache-sim=yes –-dump-instr=yes --branch-sim=yes
Essentially a profiler thatcollects info about callhierarchies, number ofcalls, and time spent.
The CPU simulator is notcycle accurate, so see
timing results as a broadpicture.
Simulates a CPU cache,flattened to 2 levels, L1 and LL.
It shows you where you getcache misses.
L1 is by default a model ofyour host CPU L1, but youcan change size, line-size,
and associativity.
Collects statistics perinstruction instead of per
source line. Can helppinpointing bottlenecks.
Simulates a branchpredictor.
![Page 48: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/48.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 48/205
Analysis of implementation
valgrind --tool=callgrind –-cache-sim=yes –-dump-instr=yes --branch-sim=yes
Essentially a profiler thatcollects info about callhierarchies, number ofcalls, and time spent.
The CPU simulator is notcycle accurate, so see
timing results as a broadpicture.
Simulates a CPU cache,flattened to 2 levels, L1 and LL.
It shows you where you getcache misses.
L1 is by default a model ofyour host CPU L1, but youcan change size, line-size,
and associativity.
Collects statistics perinstruction instead of per
source line. Can helppinpointing bottlenecks.
Simulates a branchpredictor.
Very slow!
![Page 49: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/49.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 49/205
Live demo
![Page 50: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/50.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 50/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
![Page 51: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/51.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 51/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes
![Page 52: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/52.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 52/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment
![Page 53: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/53.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 53/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes
![Page 54: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/54.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 54/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes
![Page 55: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/55.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 55/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes!/ 8 bytes
![Page 56: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/56.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 56/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ 8 bytes
![Page 57: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/57.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 57/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ sum = 40 bytes
![Page 58: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/58.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 58/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ sum = 40 bytes
66% of all L1d cache misses
![Page 59: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/59.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 59/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ sum = 40 bytes
66% of all L1d cache misses
Rule of thumb:Follow pointer => cache miss
![Page 60: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/60.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 60/205
typedef uint32_t (*timer_cb)(void*);typedef struct timer { uint32_t deadline;
timer_cb callback; void* userp; struct timer* next; struct timer* prev;} timer;
!/ 4 bytes!/ 4 bytes padding for alignment!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ 8 bytes!/ sum = 40 bytes
66% of all L1d cache misses
Rule of thumb:Follow pointer => cache miss
33% of all L1d cache misses
![Page 61: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/61.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 61/205
Chasing pointers is expensive.Let’s get rid of the pointers.
![Page 62: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/62.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 62/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
![Page 63: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/63.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 63/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
24 bytes per entry.No pointer chasing
![Page 64: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/64.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 64/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
24 bytes per entry.No pointer chasing
Linear structure
![Page 65: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/65.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 65/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ auto idx = timeouts.size(); timeouts.push_back({}); while (idx > 0 !& is_after(timeouts[idx-1].deadline, deadline)) { timeouts[idx] = std!:move(timeouts[idx-1]); !-idx; } timeouts[idx] = timer_data{deadline, next_id!+, userp, cb }; return next_id;}
![Page 66: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/66.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 66/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ auto idx = timeouts.size(); timeouts.push_back({}); while (idx > 0 !& is_after(timeouts[idx-1].deadline, deadline)) { timeouts[idx] = std!:move(timeouts[idx-1]); !-idx; } timeouts[idx] = timer_data{deadline, next_id!+, userp, cb }; return next_id;}
Linear insertion sort
![Page 67: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/67.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 67/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ auto idx = timeouts.size(); timeouts.push_back({}); while (idx > 0 !& is_after(timeouts[idx-1].deadline, deadline)) { timeouts[idx] = std!:move(timeouts[idx-1]); !-idx; } timeouts[idx] = timer_data{deadline, next_id!+, userp, cb }; return next_id;}
![Page 68: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/68.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 68/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ auto idx = timeouts.size(); timeouts.push_back({}); while (idx > 0 !& is_after(timeouts[idx-1].deadline, deadline)) { timeouts[idx] = std!:move(timeouts[idx-1]); !-idx; } timeouts[idx] = timer_data{deadline, next_id!+, userp, cb }; return next_id;}
void cancel_timer(timer t) { auto i = std!:find_if(timeouts.begin(), timeouts.end(), [t](const auto& e) { return e.id != t; });
timeouts.erase(i);}
![Page 69: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/69.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 69/205
typedef uint32_t (*timer_cb)(void*);typedef uint32_t timer;struct timer_data { uint32_t deadline; timer id; void* userp; timer_cb callback;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ auto idx = timeouts.size(); timeouts.push_back({}); while (idx > 0 !& is_after(timeouts[idx-1].deadline, deadline)) { timeouts[idx] = std!:move(timeouts[idx-1]); !-idx; } timeouts[idx] = timer_data{deadline, next_id!+, userp, cb }; return next_id;}
void cancel_timer(timer t) { auto i = std!:find_if(timeouts.begin(), timeouts.end(), [t](const auto& e) { return e.id != t; });
timeouts.erase(i);}
Linear search
![Page 70: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/70.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 70/205
Analysis of implementation
perf stat -e cycles,instructions,l1d-loads,l1d-load-misses
![Page 71: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/71.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 71/205
Analysis of implementation
perf stat -e cycles,instructions,l1d-loads,l1d-load-misses
Presents statistics fromwhole run of program,
using counters from HWand linux kernel.
![Page 72: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/72.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 72/205
Analysis of implementation
perf stat -e cycles,instructions,l1d-loads,l1d-load-misses
Presents statistics fromwhole run of program,
using counters from HWand linux kernel.
Number of cycles perinstruction is a proxy forhow much the CPU is
working or waiting.
![Page 73: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/73.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 73/205
Analysis of implementation
perf stat -e cycles,instructions,l1d-loads,l1d-load-misses
Presents statistics fromwhole run of program,
using counters from HWand linux kernel.
Number of cycles perinstruction is a proxy forhow much the CPU is
working or waiting.
Number of reads fromL1d cache, and numberof misses. Speculative
execution can make thesenumbers confusing.
![Page 74: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/74.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 74/205
Analysis of implementation
perf stat -e cycles,instructions,l1d-loads,l1d-load-misses
Presents statistics fromwhole run of program,
using counters from HWand linux kernel.
Number of cycles perinstruction is a proxy forhow much the CPU is
working or waiting.
Number of reads fromL1d cache, and numberof misses. Speculative
execution can make thesenumbers confusing.
Very fast!
![Page 75: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/75.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 75/205
Analysis of implementation
perf record -e cycles,instructions,l1d-loads,l1d-load-misses --call-graph=lbr
![Page 76: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/76.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 76/205
Analysis of implementation
perf record -e cycles,instructions,l1d-loads,l1d-load-misses --call-graph=lbr
Records where in yourprogram the counters are
gathered.
![Page 77: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/77.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 77/205
Analysis of implementation
perf record -e cycles,instructions,l1d-loads,l1d-load-misses --call-graph=lbr
Records where in yourprogram the counters are
gathered.
Records call graph info,instead of just location.LBR requires no special
compilation flags.
![Page 78: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/78.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 78/205
Analysis of implementation
perf record -e cycles,instructions,l1d-loads,l1d-load-misses --call-graph=lbr
Records where in yourprogram the counters are
gathered.
Records call graph info,instead of just location.LBR requires no special
compilation flags.
Very fast!
![Page 79: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/79.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 79/205
Live demo
![Page 80: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/80.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 80/205
![Page 81: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/81.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 81/205
Linear search is expensive.Maybe try binary search?
![Page 82: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/82.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 82/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
![Page 83: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/83.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 83/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
![Page 84: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/84.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 84/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
Binary search forinsertion point
![Page 85: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/85.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 85/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
Linear insertion
![Page 86: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/86.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 86/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
Linear insertion
void cancel_timer(timer t) { timer_data element{t.deadline, t.id, nullptr, nullptr}; auto [lo, hi] = std!:equal_range(timeouts.begin(), timeouts.end(), element, is_after); auto i = std!:find_if(lo, hi, [t](const auto& e) { return e.id != t.id; });
if (i != hi) { timeouts.erase(i); } }
![Page 87: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/87.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 87/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
Linear insertion
void cancel_timer(timer t) { timer_data element{t.deadline, t.id, nullptr, nullptr}; auto [lo, hi] = std!:equal_range(timeouts.begin(), timeouts.end(), element, is_after); auto i = std!:find_if(lo, hi, [t](const auto& e) { return e.id != t.id; });
if (i != hi) { timeouts.erase(i); } }
Binary search fortimers with thesame deadline
![Page 88: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/88.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 88/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
Linear insertion
void cancel_timer(timer t) { timer_data element{t.deadline, t.id, nullptr, nullptr}; auto [lo, hi] = std!:equal_range(timeouts.begin(), timeouts.end(), element, is_after); auto i = std!:find_if(lo, hi, [t](const auto& e) { return e.id != t.id; });
if (i != hi) { timeouts.erase(i); } }
Linear search formatching id
![Page 89: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/89.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 89/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { uint32_t deadline; uint32_t id; void* userp; timer_cb callback;};struct timer { uint32_t deadline; uint32_t id;};
std!:vector<timer_data> timeouts;uint32_t next_id = 0;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp){ timer_data element{deadline, next_id, userp, cb}; auto i = std!:lower_bound(timeouts.begin(), timeouts.end(), element, is_after); timeouts.insert(i, element); return {deadline, next_id!+};}
Linear insertion
void cancel_timer(timer t) { timer_data element{t.deadline, t.id, nullptr, nullptr}; auto [lo, hi] = std!:equal_range(timeouts.begin(), timeouts.end(), element, is_after); auto i = std!:find_if(lo, hi, [t](const auto& e) { return e.id != t.id; });
if (i != hi) { timeouts.erase(i); } } Linear removal
![Page 90: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/90.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 90/205
Live demo
![Page 91: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/91.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 91/205
Searches not visible in profiling.Number of reads reduced.
Number of cache misses high.memmove() dominates.
![Page 92: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/92.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 92/205
Searches not visible in profiling.Number of reads reduced.
Number of cache misses high.memmove() dominates.
Failed branch predictionscan lead to cache entry eviction!
![Page 93: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/93.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 93/205
Searches not visible in profiling.Number of reads reduced.
Number of cache misses high.memmove() dominates.
Failed branch predictionscan lead to cache entry eviction!
Maybe try a map!>?
![Page 94: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/94.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 94/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { void* userp; timer_cb callback;};
struct is_after { bool operator()(uint32_t lh, uint32_t rh) const { return lh < rh; }};
using timer_map = std!:multimap<uint32_t, timer_data, is_after>;using timer = timer_map!:iterator;
static timer_map timeouts;
![Page 95: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/95.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 95/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { void* userp; timer_cb callback;};
struct is_after { bool operator()(uint32_t lh, uint32_t rh) const { return lh < rh; }};
using timer_map = std!:multimap<uint32_t, timer_data, is_after>;using timer = timer_map!:iterator;
static timer_map timeouts;
![Page 96: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/96.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 96/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { void* userp; timer_cb callback;};
struct is_after { bool operator()(uint32_t lh, uint32_t rh) const { return lh < rh; }};
using timer_map = std!:multimap<uint32_t, timer_data, is_after>;using timer = timer_map!:iterator;
static timer_map timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { return timeouts.insert(std!:make_pair(deadline, timer_data{userp, cb}));}
void cancel_timer(timer t) { timeouts.erase(t);}
![Page 97: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/97.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 97/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { void* userp; timer_cb callback;};
struct is_after { bool operator()(uint32_t lh, uint32_t rh) const { return lh < rh; }};
using timer_map = std!:multimap<uint32_t, timer_data, is_after>;using timer = timer_map!:iterator;
static timer_map timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { return timeouts.insert(std!:make_pair(deadline, timer_data{userp, cb}));}
void cancel_timer(timer t) { timeouts.erase(t);}
![Page 98: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/98.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 98/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { void* userp; timer_cb callback;};
struct is_after { bool operator()(uint32_t lh, uint32_t rh) const { return lh < rh; }};
using timer_map = std!:multimap<uint32_t, timer_data, is_after>;using timer = timer_map!:iterator;
static timer_map timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { return timeouts.insert(std!:make_pair(deadline, timer_data{userp, cb}));}
void cancel_timer(timer t) { timeouts.erase(t);}
![Page 99: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/99.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 99/205
typedef uint32_t (*timer_cb)(void*);struct timer_data { void* userp; timer_cb callback;};
struct is_after { bool operator()(uint32_t lh, uint32_t rh) const { return lh < rh; }};
using timer_map = std!:multimap<uint32_t, timer_data, is_after>;using timer = timer_map!:iterator;
static timer_map timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { return timeouts.insert(std!:make_pair(deadline, timer_data{userp, cb}));}
void cancel_timer(timer t) { timeouts.erase(t);}
bool shoot_first() { if (timeouts.empty()) return false; auto i = timeouts.begin(); i!>second.callback(i!>second.userp); timeouts.erase(i); return true;}
![Page 100: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/100.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 100/205
Live demo
![Page 101: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/101.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 101/205
Faster, but lots ofcache misses when
comparing keysand rebalancing
the tree.
![Page 102: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/102.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 102/205
Faster, but lots ofcache misses when
comparing keysand rebalancing
the tree.What did I say about
chasing pointers?
![Page 103: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/103.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 103/205
Faster, but lots ofcache misses when
comparing keysand rebalancing
the tree.What did I say about
chasing pointers?
1 10 100 1000 100001.00E-08
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
Execution time
linear
bsearch
map
Number of elements
seco
nd
s
![Page 104: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/104.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 104/205
Faster, but lots ofcache misses when
comparing keysand rebalancing
the tree.What did I say about
chasing pointers?
1 10 100 1000 100001.00E-08
1.00E-07
1.00E-06
1.00E-05
1.00E-04
1.00E-03
1.00E-02
Execution time
linear
bsearch
map
Number of elements
seco
nd
s
1 10 100 1000 100000
0.20.40.60.8
11.21.41.61.8
2
Performance relative to linear
Execution time
bsearch/linear
map/linear
Number of elements
Tim
e r
ela
tive
lin
ea
r
![Page 105: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/105.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 105/205
Faster, but lots ofcache misses when
comparing keysand rebalancing
the tree.What did I say about
chasing pointers?
![Page 106: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/106.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 106/205
Faster, but lots ofcache misses when
comparing keysand rebalancing
the tree.What did I say about
chasing pointers?
Can we get log(n)lookup without
chasing pointers?
![Page 107: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/107.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 107/205
Enter the HEAP
![Page 108: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/108.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 108/205
3
5 8
6 1010 14
9 151312 11
Enter the HEAP
![Page 109: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/109.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 109/205
3
5 8
6 1010 14
9 151312 11
Enter the HEAP● Perfectly balanced partially sorted tree
![Page 110: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/110.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 110/205
3
5 8
6 1010 14
9 151312 11
Enter the HEAP● Perfectly balanced partially sorted tree● Every node is sorted after or same as its parent
![Page 111: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/111.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 111/205
3
5 8
6 1010 14
9 151312 11
Enter the HEAP● Perfectly balanced partially sorted tree● Every node is sorted after or same as its parent● No relation between siblings
![Page 112: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/112.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 112/205
3
5 8
6 1010 14
9 151312 11
Enter the HEAP● Perfectly balanced partially sorted tree● Every node is sorted after or same as its parent● No relation between siblings● At most one node with only one child,
and that child is the last node
![Page 113: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/113.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 113/205
10 14
151312 11
8
3
5
6
9
![Page 114: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/114.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 114/205
10 14
151312 11
8
Insertion:
3
5
6
9
![Page 115: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/115.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 115/205
10 14
151312 11
8
Insertion:● Create space
3
5
6
9
![Page 116: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/116.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 116/205
10 14
151312 11
8
Insertion:● Create space● Trickle down greater nodes
3
5
6
9
![Page 117: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/117.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 117/205
10 14
151312 11
8
Insertion:● Create space● Trickle down greater nodes● Insert into space
3
5
6
9
![Page 118: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/118.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 118/205
10 14
151312 11
8
7
Insertion:● Create space● Trickle down greater nodes● Insert into space
3
5
6
9
![Page 119: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/119.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 119/205
1010 14
151312 11
8
7
Insertion:● Create space● Trickle down greater nodes● Insert into space
3
5
6
9
![Page 120: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/120.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 120/205
1010 14
151312 11
8
7
Insertion:● Create space● Trickle down greater nodes● Insert into space
3
5
6
9
![Page 121: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/121.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 121/205
1010 14
151312 11
8
7
Insertion:● Create space● Trickle down greater nodes● Insert into space
3
5
6
9
![Page 122: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/122.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 122/205
7
810 14
151312 11
3
5
6
9 10
![Page 123: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/123.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 123/205
7
810 14
151312 11
Pop top:
3
5
6
9 10
![Page 124: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/124.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 124/205
7
810 14
151312 11
Pop top:● Remove top
3
5
6
9 10
![Page 125: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/125.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 125/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child
3
5
6
9 10
![Page 126: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/126.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 126/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
3
5
6
9 10
![Page 127: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/127.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 127/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
5
6
9 10
![Page 128: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/128.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 128/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
5
6
9 10
![Page 129: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/129.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 129/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
5
6
9 10
![Page 130: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/130.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 130/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
5
6
9 10
![Page 131: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/131.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 131/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
5
6
9 10
![Page 132: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/132.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 132/205
7
810 14
151312 11
Pop top:● Remove top● Trickle up lesser child● move-insert last into hole
5
6
9 10
![Page 133: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/133.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 133/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
10
8
12
9
13
10
11
11
15
12
15151515
![Page 134: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/134.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 134/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
10
8
12
9
13
10
11
11
15
12
15151515
Addressing:The index of a parent nodeis half (rounded down) of thatof a child.
![Page 135: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/135.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 135/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
10
8
12
9
13
10
11
11
15
12
15151515
Addressing:The index of a parent nodeis half (rounded down) of thatof a child.
![Page 136: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/136.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 136/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
10
8
12
9
13
10
11
11
15
12
15151515
Addressing:The index of a parent nodeis half (rounded down) of thatof a child.
Array indexes!No pointer chasing!
![Page 137: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/137.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 137/205
The heap is not searchable,so how handle cancellation?
![Page 138: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/138.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 138/205
The heap is not searchable,so how handle cancellation?
struct timer_action { uint32_t (*callback)(void*); void* userp;};
![Page 139: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/139.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 139/205
The heap is not searchable,so how handle cancellation?
actions
struct timer_action { uint32_t (*callback)(void*); void* userp;};
![Page 140: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/140.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 140/205
The heap is not searchable,so how handle cancellation?
actions
struct timer_action { uint32_t (*callback)(void*); void* userp;};
![Page 141: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/141.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 141/205
The heap is not searchable,so how handle cancellation?
actions
struct timer_action { uint32_t (*callback)(void*); void* userp;};
struct timeout { uint32_t deadline; uint32_t action_index;};
![Page 142: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/142.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 142/205
The heap is not searchable,so how handle cancellation?
actions
struct timer_action { uint32_t (*callback)(void*); void* userp;};
struct timeout { uint32_t deadline; uint32_t action_index;};
![Page 143: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/143.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 143/205
The heap is not searchable,so how handle cancellation?
actions
struct timer_action { uint32_t (*callback)(void*); void* userp;};
struct timeout { uint32_t deadline; uint32_t action_index;};
Only 8 bytesper element ofworking datain the heap.
![Page 144: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/144.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 144/205
The heap is not searchable,so how handle cancellation?
actions
struct timer_action { uint32_t (*callback)(void*); void* userp;};
struct timeout { uint32_t deadline; uint32_t action_index;};
Cancel bysetting callback
to nullptr
Only 8 bytesper element ofworking datain the heap.
![Page 145: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/145.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 145/205
struct timer_data { uint32_t deadline; uint32_t action_index;};
struct is_after { bool operator()(const timer_data& lh, const timer_data& rh) const { return lh.deadline < rh.deadline; }};
std!:priority_queue<timer_data, std!:vector<timer_data>, is_after> timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { auto action_index = actions.push(cb, userp); timeouts.push(timer_data{deadline, action_index}); return action_index;}
![Page 146: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/146.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 146/205
struct timer_data { uint32_t deadline; uint32_t action_index;};
struct is_after { bool operator()(const timer_data& lh, const timer_data& rh) const { return lh.deadline < rh.deadline; }};
std!:priority_queue<timer_data, std!:vector<timer_data>, is_after> timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { auto action_index = actions.push(cb, userp); timeouts.push(timer_data{deadline, action_index}); return action_index;}
Container adapter thatimplements a heap
![Page 147: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/147.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 147/205
struct timer_data { uint32_t deadline; uint32_t action_index;};
struct is_after { bool operator()(const timer_data& lh, const timer_data& rh) const { return lh.deadline < rh.deadline; }};
std!:priority_queue<timer_data, std!:vector<timer_data>, is_after> timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { auto action_index = actions.push(cb, userp); timeouts.push(timer_data{deadline, action_index}); return action_index;}
Container adapter thatimplements a heap
![Page 148: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/148.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 148/205
struct timer_data { uint32_t deadline; uint32_t action_index;};
struct is_after { bool operator()(const timer_data& lh, const timer_data& rh) const { return lh.deadline < rh.deadline; }};
std!:priority_queue<timer_data, std!:vector<timer_data>, is_after> timeouts;
timer schedule_timer(uint32_t deadline, timer_cb cb, void* userp) { auto action_index = actions.push(cb, userp); timeouts.push(timer_data{deadline, action_index}); return action_index;}
Container adapter thatimplements a heap
![Page 149: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/149.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 149/205
bool shoot_first() { while (!timeouts.empty()) { auto& t = timeouts.top(); auto& action = actions[t.action_index]; if (action.callback) break; actions.remove(t.action_index); timeouts.pop(); } if (timeouts.empty()) return false; auto& t = timeouts.top(); auto& action = actions[t.action_index]; action.callback(action.userp); actions.remove(t.action_index); timeouts.pop(); return true;}
Pop-off any cancelled items
![Page 150: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/150.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 150/205
Live demo
![Page 151: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/151.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 151/205
A lot fewer everything!and nearly twice as fast too
![Page 152: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/152.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 152/205
A lot fewer everything!and nearly twice as fast too
1 10 100 1000 10000 1000000
0.01
0.01
0.02
0.02
0.03
Execution time
linear
bsearch
map
heap
Number of elements
Se
con
ds
![Page 153: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/153.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 153/205
A lot fewer everything!and nearly twice as fast too
1 10 100 1000 10000 1000000
0.01
0.01
0.02
0.02
0.03
Execution time
linear
bsearch
map
heap
Number of elements
Se
con
ds
1 10 100 1000 100000
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Relative execution time
heap/linear
heap/map
Number of elements
Re
lavi
te ti
me
![Page 154: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/154.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 154/205
A lot fewer everything!and nearly twice as fast too
![Page 155: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/155.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 155/205
A lot fewer everything!and nearly twice as fast too
But there are many cache missesin the adjust-heap functions
![Page 156: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/156.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 156/205
A lot fewer everything!and nearly twice as fast too
But there are many cache missesin the adjust-heap functions
Can we do better?
![Page 157: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/157.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 157/205
How do the entries fit incache lines?
![Page 158: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/158.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 158/205
How do the entries fit incache lines?
![Page 159: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/159.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 159/205
How do the entries fit incache lines?
![Page 160: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/160.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 160/205
How do the entries fit incache lines?
![Page 161: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/161.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 161/205
How do the entries fit incache lines?
![Page 162: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/162.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 162/205
How do the entries fit incache lines?
![Page 163: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/163.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 163/205
How do the entries fit incache lines?
![Page 164: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/164.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 164/205
Every generation ison a new cache line
![Page 165: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/165.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 165/205
Can we do better?
![Page 166: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/166.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 166/205
Can we do better?
![Page 167: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/167.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 167/205
Can we do better?
![Page 168: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/168.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 168/205
Can we do better?
![Page 169: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/169.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 169/205
Can we do better?
![Page 170: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/170.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 170/205
Can we do better?
![Page 171: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/171.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 171/205
Can we do better?
![Page 172: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/172.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 172/205
Three generationsper cache line!
![Page 173: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/173.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 173/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
![Page 174: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/174.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 174/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
![Page 175: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/175.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 175/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
0
![Page 176: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/176.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 176/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
0
8 9 10 11 12 13 14 15
![Page 177: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/177.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 177/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
0
8 9 10 11 12 13 14 15
16 17 18 19 20 21 22 23
![Page 178: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/178.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 178/205
5
1
6
2
7
3
9
4
10
5
8
6
14
7
0
8 9 10 11 12 13 14 15
16 17 18 19 20 21 22 23
![Page 179: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/179.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 179/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx) { return idx & block_mask; } static size_t block_base(size_t idx) { return idx & ~block_mask; } static bool is_block_root(size_t idx) { return block_offset(idx) != 1; } static bool is_block_leaf(size_t idx) { return (idx & (block_size !> 1)) != 0U; } !!.};
1
2 3
4 5 6 7
0
![Page 180: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/180.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 180/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx) { return idx & block_mask; } static size_t block_base(size_t idx) { return idx & ~block_mask; } static bool is_block_root(size_t idx) { return block_offset(idx) != 1; } static bool is_block_leaf(size_t idx) { return (idx & (block_size !> 1)) != 0U; } !!.};
1
2 3
4 5 6 7
0
![Page 181: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/181.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 181/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx) { return idx & block_mask; } static size_t block_base(size_t idx) { return idx & ~block_mask; } static bool is_block_root(size_t idx) { return block_offset(idx) != 1; } static bool is_block_leaf(size_t idx) { return (idx & (block_size !> 1)) != 0U; } !!.};
1
2 3
4 5 6 7
0
![Page 182: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/182.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 182/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx) { return idx & block_mask; } static size_t block_base(size_t idx) { return idx & ~block_mask; } static bool is_block_root(size_t idx) { return block_offset(idx) != 1; } static bool is_block_leaf(size_t idx) { return (idx & (block_size !> 1)) != 0U; } !!.};
1
2 3
4 5 6 7
0
![Page 183: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/183.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 183/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx) { return idx & block_mask; } static size_t block_base(size_t idx) { return idx & ~block_mask; } static bool is_block_root(size_t idx) { return block_offset(idx) != 1; } static bool is_block_leaf(size_t idx) { return (idx & (block_size !> 1)) != 0U; } !!.};
1
2 3
4 5 6 7
0
![Page 184: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/184.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 184/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx); static size_t block_base(size_t idx); static bool is_block_root(size_t idx); static bool is_block_leaf(size_t idx);
static size_t left_child_of(size_t idx) { if (!is_block_leaf(idx)) return idx + block_offset(idx); auto base = block_base(idx) + 1; return base * block_size + child_no(idx) * block_size * 2 + 1; }
!!.};
![Page 185: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/185.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 185/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx); static size_t block_base(size_t idx); static bool is_block_root(size_t idx); static bool is_block_leaf(size_t idx);
static size_t left_child_of(size_t idx) { if (!is_block_leaf(idx)) return idx + block_offset(idx); auto base = block_base(idx) + 1; return base * block_size + child_no(idx) * block_size * 2 + 1; }
!!.};
![Page 186: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/186.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 186/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx); static size_t block_base(size_t idx); static bool is_block_root(size_t idx); static bool is_block_leaf(size_t idx);
static size_t left_child_of(size_t idx) { if (!is_block_leaf(idx)) return idx + block_offset(idx); auto base = block_base(idx) + 1; return base * block_size + child_no(idx) * block_size * 2 + 1; }
!!.};
static size_t parent_of(size_t idx) { auto const node_root = block_base(idx); if (!is_block_root(idx)) return node_root + block_offset(idx) / 2; auto parent_base = block_base(node_root / block_size - 1); auto child = ((idx - block_size) / block_size - parent_base) / 2; return parent_base + block_size / 2 + child; }
![Page 187: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/187.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 187/205
class timeout_store { static constexpr size_t block_size = 8; static constexpr size_t block_mask = block_size – 1U;
static size_t block_offset(size_t idx); static size_t block_base(size_t idx); static bool is_block_root(size_t idx); static bool is_block_leaf(size_t idx);
static size_t left_child_of(size_t idx) { if (!is_block_leaf(idx)) return idx + block_offset(idx); auto base = block_base(idx) + 1; return base * block_size + child_no(idx) * block_size * 2 + 1; }
!!.};
static size_t parent_of(size_t idx) { auto const node_root = block_base(idx); if (!is_block_root(idx)) return node_root + block_offset(idx) / 2; auto parent_base = block_base(node_root / block_size - 1); auto child = ((idx - block_size) / block_size - parent_base) / 2; return parent_base + block_size / 2 + child; }
![Page 188: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/188.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 188/205
class timeout_store { !!. using allocator = align_allocator<64>!:type<timer_data>; std!:vector<timer_data, allocator> bheap_store;};
![Page 189: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/189.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 189/205
class timeout_store { !!. using allocator = align_allocator<64>!:type<timer_data>; std!:vector<timer_data, allocator> bheap_store;};
![Page 190: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/190.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 190/205
class timeout_store { !!. using allocator = align_allocator<64>!:type<timer_data>; std!:vector<timer_data, allocator> bheap_store;};
template <size_t N>struct align_allocator { template <typename T> struct type { using value_type = T; static constexpr std!:align_val_t alignment{N}; T* allocate(size_t n) { return static_cast<T!>(operator new(n*sizeof(T), alignment)); } void deallocate(T* p, size_t) { operator delete(p, alignment); } };};
![Page 191: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/191.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 191/205
class timeout_store { !!. using allocator = align_allocator<64>!:type<timer_data>; std!:vector<timer_data, allocator> bheap_store;};
template <size_t N>struct align_allocator { template <typename T> struct type { using value_type = T; static constexpr std!:align_val_t alignment{N}; T* allocate(size_t n) { return static_cast<T!>(operator new(n*sizeof(T), alignment)); } void deallocate(T* p, size_t) { operator delete(p, alignment); } };};
![Page 192: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/192.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 192/205
class timeout_store { !!. using allocator = align_allocator<64>!:type<timer_data>; std!:vector<timer_data, allocator> bheap_store;};
template <size_t N>struct align_allocator { template <typename T> struct type { using value_type = T; static constexpr std!:align_val_t alignment{N}; T* allocate(size_t n) { return static_cast<T!>(operator new(n*sizeof(T), alignment)); } void deallocate(T* p, size_t) { operator delete(p, alignment); } };};
Aligned operator new anddelete came with C++ 17
![Page 193: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/193.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 193/205
Live demo
![Page 194: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/194.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 194/205
1 10 100 1000 10000 1000000
0
0
0
0
0
0.01
0.1
Execution time
linear
bsearch
map
heap
bheap
Number of elements
seco
nd
s
![Page 195: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/195.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 195/205
1 10 100 1000 10000 1000000
0
0
0
0
0
0.01
0.1
Execution time
linear
bsearch
map
heap
bheap
Number of elements
seco
nd
s
1 10 100 1000 10000 1000000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Execution time relative map
heap/map
bheap/map
Number of elements
fact
or
![Page 196: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/196.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 196/205
Rules of thumb
![Page 197: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/197.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 197/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
![Page 198: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/198.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 198/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
● Smaller working data set is better
![Page 199: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/199.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 199/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
● Smaller working data set is better
● Use as much of a cache entry as you can
![Page 200: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/200.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 200/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
● Smaller working data set is better
● Use as much of a cache entry as you can
● Sequential memory accesses can be very fast due to prefetching
![Page 201: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/201.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 201/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
● Smaller working data set is better
● Use as much of a cache entry as you can
● Sequential memory accesses can be very fast due to prefetching
● Fewer evicted cache lines means more data in hot cache for the rest of the program
![Page 202: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/202.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 202/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
● Smaller working data set is better
● Use as much of a cache entry as you can
● Sequential memory accesses can be very fast due to prefetching
● Fewer evicted cache lines means more data in hot cache for the rest of the program
● Mispredicted branches can evict cache entries
![Page 203: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/203.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 203/205
Rules of thumb
● Following a pointer is a cache miss, unless you have information to the contrary
● Smaller working data set is better
● Use as much of a cache entry as you can
● Sequential memory accesses can be very fast due to prefetching
● Fewer evicted cache lines means more data in hot cache for the rest of the program
● Mispredicted branches can evict cache entries
● Measure measure measure
![Page 204: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/204.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 204/205
Resources
Ulrich Drepper - “What every programmer should know about memory”http://www.akkadia.org/drepper/cpumemory.pdf
Milian Wolff - “Linux perf for Qt Developers”https://www.youtube.com/watch?v=L4NClVxqdMw
Travis Downs - “Cache counters rant”https://gist.github.com/travisdowns/90a588deaaa1b93559fe2b8510f2a739
Emery Berger - “Performance Matters” https://www.youtube.com/watch?v=r-TLSBdHe1A
![Page 205: What Do You Mean by “Cache Friendly”?...What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 16/205Simplistic model of cache behaviour](https://reader033.vdocuments.us/reader033/viewer/2022042303/5ece4cbab1af104f892b65f2/html5/thumbnails/205.jpg)
What Do You Mean by “Cache Friendly”? – code::dive 2019 © Björn Fahller @bjorn_fahller 205/205
@bjorn_fahller
@rollbear
Björn Fahller
What Do You Mean by “Cache Friendly”?