the art and science of memory allocaon - sbuporter/courses/cse506/s16/slides/malloc.pdf · the art...
TRANSCRIPT
CSE506:Opera.ngSystems
TheArtandScienceofMemoryAlloca4onDonPorter
1
CSE506:Opera.ngSystems
LogicalDiagram
MemoryManagement
CPUScheduler
User
Kernel
Hardware
BinaryFormats
Consistency
SystemCalls
Interrupts Disk Net
RCU FileSystem
DeviceDrivers
Networking Sync
MemoryAllocators Threads
Today’sLecture
2
CSE506:Opera.ngSystems
Lecturegoal• Thislecturesisaboutalloca4ngsmallobjects– Futurelectureswilltalkaboutalloca4ngphysicalpages
• Understandhowmemoryallocatorswork– Inbothkernelandapplica4ons
• Understandtrade-offsandcurrentbestprac4ces
3
CSE506:Opera.ngSystems
libc.soheap
BigPicture
int main () { struct foo *x = malloc(sizeof(struct foo)); ... void * malloc (ssize_t n) { if (heap empty) mmap(); // add pages to heap find a free block of size n; }
4
VirtualAddressSpace
0 0xffffffff
Code(.text) heapstackheap
(empty)n
CSE506:Opera.ngSystems
Today’sLecture• Howtoimplementmalloc()ornew – Notethatnewisessen4allymalloc+constructor– malloc()ispartoflibc,andexecutesintheapplica4on
• malloc()getspagesofmemoryfromtheOSviammap()andthensub-dividesthemfortheapplica4on
• Thenextlecturewilltalkabouthowthekernelmanagesphysicalpages– Forinternaluse,ortoallocatetoapplica4ons
5
CSE506:Opera.ngSystems
Bumpallocator
• malloc(6)• malloc(12)• malloc(20)• malloc(5)
6
CSE506:Opera.ngSystems
Bumpallocator• Simply“bumps”upthefreepointer• Howdoesfree()work?Itdoesn’t– Well,youcouldtrytorecyclecellsifyouwanted,butcomplicatedbookkeeping
• Controversialobserva4on:Thisisidealforsimpleprograms– Youonlycareaboutfree()ifyouneedthememoryforsomethingelse
7
CSE506:Opera.ngSystems
Assumememoryislimited• Hoard:best-of-breedconcurrentallocator– Userapplica4ons– Seminalpaper
• We’llalsotalkabouthowLinuxallocatesitsownmemory
8
CSE506:Opera.ngSystems
Overarchingissues• Fragmenta4on• Alloca4onandfreelatency– Synchroniza4on/Concurrency
• Implementa4oncomplexity• Cachebehavior– Alignment(cacheandword)– Coloring
9
CSE506:Opera.ngSystems
Fragmenta4on• Undergradreview:Whatisit?Whydoesithappen?• Whatis– Internalfragmenta4on?
• Wastedspacewhenyouroundanalloca4onup
– Externalfragmenta4on?• Whenyouendupwithsmallchunksoffreememorythataretoosmalltobeuseful
• Whichkinddoesourbumpallocatorhave?
10
CSE506:Opera.ngSystems
Hoard:Superblocks• Atahighlevel,allocatoroperatesonsuperblocks– Chunkof(virtually)con4guouspages– Allobjectsinasuperblockarethesamesize
• Agivensuperblockistreatedasanarrayofsame-sizedobjects– Theygeneralizeto“powersofb>1”;– Inusualprac4ce,b==2
11
CSE506:Opera.ngSystems
Superblockintui4on256byte
objectheap
4KBpage
(Freespace)
4KBpage
next next next
next next next
Free next
FreelistinLIFOorder
Eachpageanarrayofobjects
Storelistpointersinfreeobjects!
12
CSE506:Opera.ngSystems
SuperblockIntui4on
malloc (8);
1) Findthenearestpowerof2heap(8)
2) Findfreeobjectinsuperblock
3) Addasuperblockifneeded.Goto2.
13
CSE506:Opera.ngSystems
malloc(200)256byte
objectheap
4KBpage
(Freespace)
4KBpage
next next next
next next next
Free next
Pickfirstfreeobject
14
CSE506:Opera.ngSystems
Superblockexample• Supposemyprogramallocatesobjectsofsizes:– 4,5,7,34,and40bytes.
• HowmanysuperblocksdoIneed(ifb==2)?– 3–(4,8,and64bytechunks)
• IfIallocatea5byteobjectfroman8bytesuperblock,doesn’tthatyieldinternalfragmenta4on?– Yes,butitisboundedto<50%– Giveupsomespacetoboundworstcaseandcomplexity
15
CSE506:Opera.ngSystems
High-levelstrategy• Allocateaheapforeachprocessor,andonesharedheap– Note:notthreads,butCPUs– CanonlyuseasmanyheapsasCPUsatonce– Requiressomewaytofigureoutcurrentprocessor
• Tryper-CPUheapfirst• Ifnofreeblocksofrightsize,thentryglobalheap– Whytrythisfirst?
• Ifthatfails,getanothersuperblockforper-CPUheap
16
CSE506:Opera.ngSystems
Example:malloc()onCPU0
17
CPU0Heap CPU1Heap
GlobalHeapFirst,tryper-CPUheap
Second,tryglobalheap
Ifglobalheapfull,grow
per-CPUheap
CSE506:Opera.ngSystems
Bigobjects• Ifanobjectsizeisbiggerthanhalfthesizeofasuperblock,justmmap()it– Recall,asuperblockisontheorderofpagesalready
• Whataboutfragmenta4on?– Example:4097byteobject(1page+1byte)– Argument:Moretroublethanitisworth
• Extrabookkeeping,poten4alconten4on,andpoten4albadcachebehavior
18
CSE506:Opera.ngSystems
Memoryfree• Simplyputbackonfreelistwithinitssuperblock• Howdoyoutellwhichsuperblockanobjectisfrom?– Supposesuperblockis8k(2pages)
• Andalwaysmappedatanaddressevenlydivisibleby8k
– Objectataddress0x431a01c– Justmaskoutthelow13bits!– Camefromasuperblockthatstartsat0x431a000
• Simplemathcantellyouwhereanobjectcamefrom!
19
CSE506:Opera.ngSystems
LIFO• Whyareobjectsre-allocatedmost-recentlyusedfirst?– Aren’tallgoodOSheuris4csFIFO?– Morelikelytobealreadyincache(hot)– Recallfromundergradarchitecturethatittakesquiteafewcyclestoloaddataintocachefrommemory
– Ifitisallthesame,let’strytorecycletheobjectalreadyinourcache
20
CSE506:Opera.ngSystems
HoardSimplicity• Thebookkeepingforallocandfreeisstraighsorward– Manyallocatorsarequitecomplex(lookingatyou,slab)
• Overall:(#CPUs+1)heaps
– Perheap:1listofsuperblocksperobjectsize(22—211)
– Persuperblock:• Needtoknowwhich/howmanyobjectsarefree
– LIFOlistoffreeblocks
21
CSE506:Opera.ngSystems
CPU0Heap,Illustrated
22OneoftheseperCPU(andoneshared)
FreeList:
Order: 2
FreeList:
3
FreeList:
4
FreeList:
5
FreeList:
11
...
FreeList:LIFOorder
Somesizescanbeempty
CSE506:Opera.ngSystems
Locking• Onallocandfree,locksuperblockandper-CPUheap• Why?– AnobjectcanbefreedfromadifferentCPUthanitwasallocatedon
• Alterna4ve:– Wecouldaddmorebookkeepingforobjectstomovetolocalsuperblock
– Reintroducefragmenta4onissuesandlosesimplicity
23
CSE506:Opera.ngSystems
Howtofindthelocks?• Again,pagealignmentcaniden4fythestartofasuperblock
• Andeachsuperblockkeepsasmallamountofmetadata,includingtheheapitbelongsto– Per-CPUorsharedHeap– Andheapincludesalock
24
CSE506:Opera.ngSystems
Lockingperformance• Acquiringandreleasingalockgenerallyrequiresanatomicinstruc4on– Tenstoafewhundredcyclesvs.afewcycles
• Wai4ngforalockcantakethousands– Dependsonhowgoodthelockimplementa4onisatmanagingconten4on(spinning)
– Blockinglocksrequiremanyhundredsofcyclestocontextswitch
25
CSE506:Opera.ngSystems
Performanceargument• Commoncase:alloca4onsandfreesarefromper-CPUheap
• Yes,grabbingalockaddsoverheads– Butbeverthanthefragmentedorcomplexalterna4ves– Andlockinghurtsscalabilityonlyunderconten4on
• Uncommoncase:allCPUscontendtoaccessoneheap– Hadtoallcomefromthatheap(onlyfreescrossheaps)– Bizarreworkload,probablywon’tscaleanyway
26
CSE506:Opera.ngSystems
Cachelinealignment• Linesarethebasicunitatwhichmemoryiscached• Cachelinesarebiggerthanwords– Word:32-bitsor64-bits– Cacheline–64—128bytesonmostCPUs
27
CSE506:Opera.ngSystems
UndergradArchitectureReview
CPU0
Cache
ldw0x1008
CPUloadsoneword(4bytes)
MemoryBus
CacheMiss
0x1000
RAM
Cacheoperatesatlinegranularity(64
bytes)
28
CSE506:Opera.ngSystems
CacheCoherence(1)
CPU0
Cache
MemoryBus
0x1000
RAM
CPU1
Cache
ldw0x1010
Linessharedforreadinghaveasharedlock 29
CSE506:Opera.ngSystems
CacheCoherence(2)
CPU0
Cache
MemoryBus
0x1000
RAM
CPU1
Cache
ldw0x1010
Linestobewrivenhaveanexclusivelock
stw0x1000 Copiesoflineevicted
0x1000
30
CSE506:Opera.ngSystems
Simplecoherencemodel• Whenamemoryregioniscached,CPUautoma4callyacquiresareader-writerlockonthatregion– Mul4pleCPUscanshareareadlock– Writelockisexclusive
• Programmercan’tcontrolhowlongtheselocksareheld– Ex:astorefromaregisterholdsthewritelocklongenoughtoperformthewrite;heldfromthereun4lthenextCPUwantsit
31
CSE506:Opera.ngSystems
Objectfoo(CPU0writes)
Objectbar(CPU1writes)
Falsesharing
• Theseobjectshavenothingtodowitheachother– Atprogramlevel,privatetoseparatethreads
• Atcachelevel,CPUsarefigh4ngforawritelock
Cacheline
32
CSE506:Opera.ngSystems
FalsesharingisBAD• Leadstopathologicalperformanceproblems– Super-linearslowdowninsomecases
• Ruleofthumb:anyperformancetrendthatismorethanlinearinthenumberofCPUsisprobablycausedbycachebehavior
33
CSE506:Opera.ngSystems
Strawman• Roundeverythinguptothesizeofacacheline• Thoughts?– Wastestoomuchmemory;abitextreme
34
CSE506:Opera.ngSystems
Hoardstrategy(pragma4c)• Roundinguptopowersof2helps– Onceyourobjectsarebiggerthanacacheline
• Localityobserva4on:thingstendtobeusedontheCPUwheretheywereallocated
• Forsmallobjects,alwaysreturnfreetotheoriginalheap– Rememberideaaboutextrabookkeepingtoavoidsynchroniza4on:someallocatorsdothis• Savelocking,butintroducefalsesharing!
35
CSE506:Opera.ngSystems
Hoardsummary• Reallynicepieceofwork• Establishesnicebalanceamongconcerns• Goodperformanceresults
36
CSE506:Opera.ngSystems
Part2:Linuxkernelallocators• malloc()andfriends,butinthekernel
• Focustodayondynamicalloca4onofsmallobjects– Laterclassonmanagementofphysicalpages– Andalloca4onofpagerangestoallocators
37
CSE506:Opera.ngSystems
kmem_caches• Linuxhasakmallocandkfree,butcachespreferredforcommonobjecttypes
• LikeHoard,agivencacheallocatesaspecifictypeofobject– Ex:acacheforfiledescriptors,acacheforinodes,etc.
• UnlikeHoard,objectsofthesamesizenotmixed– Allocatorcandoini4aliza4onautoma4cally– Mayalsoneedtoconstrainwherememorycomesfrom
38
CSE506:Opera.ngSystems
Caches(2)• Cachescanalsokeepacertain“reserve”capacity– Noguarantees,butallowsperformancetuning– Example:IknowI’llhave~100listnodesfrequentlyallocatedandfreed;targetthecachecapacityat120elementstoavoidexpensivepagealloca4on
– Oyencalledamemorypool
• Universalinterface:canchangeallocatorunderneath• Kernelhaskmallocandkfreetoo– Implementedoncachesofvariouspowersof2(familiar?)
39
CSE506:Opera.ngSystems
Superblockstoslabs• Thedefaultcacheallocator(atleastasofearly2.6)wastheslaballocator
• Slabisachunkofcon4guouspages,similartoasuperblockinHoard
• Similarbasicideas,butsubstan4allymorecomplexbookkeeping– Theslaballocatorcamefirst,historically
40
CSE506:Opera.ngSystems
Complexitybacklash• I’llspareyouthedetails,butslabbookkeepingiscomplicated
• 2groupsupset:(guesseswho?)– Usersofverysmallsystems– Usersoflargemul4-processorsystems
41
CSE506:Opera.ngSystems
Smallsystems• Think4MBofRAMonasmalldevice(thermostat)• Assystemmemorygets4ny,thebookkeepingoverheadsbecomealargepercentoftotalsystemmemory
• Howbadisfragmenta4onreallygoingtobe?– Note:notsurethishasbeencarefullystudied;mayjustbeintui4on
42
CSE506:Opera.ngSystems
SLOBallocator• SimpleListOfBlocks• Justkeepafreelistofeachavailablechunkanditssize
• Grabthefirstonebigenoughtowork– Splitblockifleyoverbytes
• Nointernalfragmenta4on,obviously• Externalfragmenta4on?Yes.Tradedforlowoverheads
43
CSE506:Opera.ngSystems
Largesystems• Forverylarge(thousandsofCPU)systems,complexallocatorbookkeepinggetsoutofhand
• Example:slabstrytomigrateobjectsfromoneCPUtoanothertoavoidsynchroniza4on– Per-CPU*Per-CPUbookkeeping
44
CSE506:Opera.ngSystems
SLUBAllocator• TheUnqueuedSlabAllocator• AmuchmoreHoard-likedesign– Allobjectsofsamesizefromsameslab– Simplefreelistperslab– Nocross-CPUnonsense
• NowthedefaultLinuxcacheallocator
45
CSE506:Opera.ngSystems
Conclusion• Differentalloca4onstrategieshavedifferenttrade-offs– Noone,perfectsolu4on
• Allocatorstrytoop4mizeformul4plevariables:– Fragmenta4on,lowfalseconflicts,speed,mul4-processorscalability,etc.
• Understandtradeoffs:HoardvsSlabvs.SLOB
46
CSE506:Opera.ngSystems
Miscnotes• Whenisasuperblockconsideredfreeandeligibletobemovetotheglobalbucket?– Seefigure2,free(),line9– Essen4allyaconfigurable“emptyfrac4on”
• Isa"usedblock"countstoredsomewhere?– Notclear,butprobably
47