SustainableMemoryUseAllocation&(Implicit) Deallocation(mostlyinJava)
COMP412FALL2017
Copyright2017,KeithD.Cooper&ZoranBudimlić,allrightsreserved.StudentsenrolledinComp412atRiceUniversityhaveexplicitpermissiontomakecopiesofthesematerialsfortheirpersonaluse.Facultyfromeducationalinstitutionsmayusethesematerialsfornonprofiteducationalpurposes,providedthiscopyrightnoticeispreserved.
Preliminaries
Today’slectureAutomaticmemorymanagement→ Procedure-localmemorymanagement→ “GarbageCollection”
→ ReferenceCounting→ CopyingCollectors→ Short-PauseCollectors
→ HowyoushouldprogramdefensivelynowthatyouknowSecondpart:Javamemorymodel,allocation,&recycling→ Howitworks→ Howitaffectsyourcode’sruntime
→ Howyoushouldprogramdefensivelynowthatyouknow
COMP412,Fall2017 1
WheredoobjectsliveinJava?
COMP140 &COMP215• Studentsencouragedtoignoretheissueofwhereobjects,variables,andmethodslive
• Theimplementation(PythonorJava)takescareofthesedetails
• Fundamentally,abstractionisagoodthing— rightuptothepointwhereitcausesproblems
• AtsomepointinyourJavacareer,performancewillmatter– COMP412labs– Atthatpoint,youneedtopayattentiontodetails
• Today’slectureisaboutdetails
COMP412,Fall2017 2
JavaWorld
Fred Kate
JuliaSystem.out.
println()
Wheredoobjectslive?
TheJavaSystemmapsJavaWorldontoProcessorResources• Processorhasfiniteresources• Javasuggeststhatyouhave“enough”resources
• Mapping“enough”onto“what’sthere”isthejoboftheJavacompilerandruntime(JVM)
Knowinghowthatmappingworkscanhelpyouunderstandthebehaviorofyourprograms,andsuggestwaystoimprovetheprogram’sbehavior.
COMP412,Fall2017 3
JavaWorld
Fred Kate
JuliaSystem.out.
println()
0 1 2 k
RAM
ProcessorCore
ProcessorCore
ProcessorCore ���
Fundamentals
Intheexample,whatneedsstorage?• Thetwoclasses(Point&C)• Point’slocalmembers(x,y,& draw)• C’slocalmembers(s,t,& m)• m’slocalvariables(a,b,& p)
COMP412,Fall2017 4
ClassPoint{publicintx,y;publicvoiddraw();
}ClassC{
int s,t;publicvoidm(){int a,b;Pointp=newPoint();a=…;b=…;p.draw();
}}
Aclassicexample
Fundamentals
Intheexample,whatneedsstorage?• Thetwoclasses(Point&C)• Point’slocalmembers(x,y,& draw)• C’slocalmembers(s,t,& m)• m’slocalvariables(a,b,& p)MemoryintheJavaruntimeisdivided,broadlyspeaking,intoaHeapandacollectionofStacks• Oneheapperprogram(large)• Onestackperthread(smaller)
COMP412,Fall2017 5
ClassPoint{publicintx,y;publicvoiddraw();
}ClassC{
int s,t;publicvoidm(){int a,b;Pointp=newPoint();a=…;b=…;p.draw();
}}
Aclassicexample
Heap
STAC
K 0
STAC
K 1
STAC
K 2
STAC
K n…
Point C
“new”point
“Helloworld!”
StringPool
p:a:b:
JVMMemoryLayout
Conceptually,Javamemoryislaidoutalongtheselines
• Whenrunningcodecreatesavariable,itgoesintothethread’sstack• Whenrunningcodecreatesaclassoranobject(e.g.,withanew),itgoesintotheheap• Codelivesofftotheleft (mightconsideritpartoftheheap)
COMP412,Fall2017 6
So,canaprogramrunoutofheapspace? (toomanynews)
Whathappens?⇒ Theruntimesystemtriestorecyclespaceontheheap
Heap
Stacks
Growthsp
ace
forstacks
Globa
ls
Code
⇒ Yes.Emphaticallyyes
SustainableMemoryManagement
Whentheheaprunsoutofspace,thesystemcopes• Scourstheheaplookingforobjectsthatarenolongerofinterest– Technicaltermis“live”– Anobjectisconsideredliveiff itcanbereachedfromtherunningcode
• Startfromallthenamesintherunningcode– Variablesareonthestack1
– Globalnamessuchasdeclaredorimportedclasses– Eachobjectonthestackhasadeclarationwhichrevealsitsstructure– Youcanimaginechasingdownchainsofreferencestofindallliveobjects2➝ That’showitwasdoneforalongtime…
• Moderngarbagecollectorsaremorenuanced– Theystillstartfromthebeginning:local&globalnames– Mostmoderncollectorsare“copying”collectors
COMP412,SupplementalMaterial
AKA“GarbageCollection”
1Localsofthecurrentmethodareonthestack.Localsofthemethodthatcalleditarebelowthecurrentmethodonthestack.Localsofthemethodthatcalledthatmethodarebelow…,andsoon.That’swhytheruntimeusesastack2H.Schorr andW.MWaite,“Anefficientmachine-independentprocedureforgarbagecollectioninvariousliststructures,Comm.ACM,10(8),1967,pages501—506 7
Taxonomy
COMP412,Fall2017 8
Garbage Collectors
Reference-Counters
Trace-Based
Stop-the-World Short-Pause
Mark-and-Sweep
Mark-and-Compact
Basic Baker’s Basic Cheney’s
Incremental Partial
Generational Train
ReferenceCounting
ProsSimpleimplementationConservativeEasytoadapttoreal-time
ConsFragmentationSelf-referencinggarbageLocalityeffectsOverheadperpointermodificationMemoryoverhead
COMP412,Fall2017 9
RootObject
A(1)
E(1)D(2)
B(2)
C(1)
ReferenceCounting
ProsSimpleimplementationConservativeEasytoadapttoreal-time
ConsFragmentationSelf-referencinggarbageLocalityeffectsOverheadperpointermodificationMemoryoverhead
COMP412,Fall2017 10
A(0)
E(1)D(2)
B(2)
C(1)
RootObject
ReferenceCounting
ProsSimpleimplementationConservativeEasytoadapttoreal-time
ConsFragmentationSelf-referencinggarbageLocalityeffectsOverheadperpointermodificationMemoryoverhead
COMP412,Fall2017 11
E(1)D(2)
B(1)
C(0)
RootObject
ReferenceCounting
ProsSimpleimplementationConservativeEasytoadapttoreal-time
ConsFragmentationSelf-referencinggarbageLocalityeffectsOverheadperpointermodificationMemoryoverhead
COMP412,Fall2017 12
E(1)D(1)
B(1)
RootObject
Taxonomy
COMP412,Fall2017 13
Garbage Collectors
Reference-Counters
Trace-Based
Stop-the-World Short-Pause
Mark-and-Sweep
Mark-and-Compact
Basic Baker’s Basic Cheney’s
Incremental Partial
Generational Train
14
MarkandSweep
1. Free =notholdinganobject;availableforallocation.2. Unreached =Holdsanobject,buthasnotyetbeenreachedfromthe
rootset.3. Unscanned =Reachedfromtherootset,butitsreferencesnotyet
followed.4. Scanned =Reachedandreferencesfollowed.
15
Marking
1. AssumeallobjectsinUnreached state.2. Startwiththerootset.PuttheminstateUnscanned.3. while Unscanned objectsremaindo
examineoneoftheseobjects;makeitsstatebeScanned;addallreferencedobjectstoUnscannediftheyhavenotbeenthere;
end;
16
Sweeping
• PlaceallobjectsstillintheUnreached stateintotheFree state.• PlaceallobjectsinScanned stateintotheUnreached state.– Toprepareforthenextmark-and-sweep.
• Handlesgarbagecyclesproperly
17
MarkandSweep
• Problem:Takestimeproportionaltotheheapsize.– BecauseyoumustvisitallobjectstoseeiftheyareUnreached.
• Baker’salgorithmkeepsalistofallallocatedchucksofmemory,aswellastheFree list.• Keychange:Inthesweep,lookonlyatthelistofallocatedchunks.• ThosethatarenotmarkedasScanned aregarbageandaremovedtotheFree list.• ThoseintheScanned stateareputintheUnreached state.– Forthenextcollection.
Taxonomy
COMP412,Fall2017 18
Garbage Collectors
Reference-Counters
Trace-Based
Stop-the-World Short-Pause
Mark-and-Sweep
Mark-and-Compact
Basic Baker’s Basic Cheney’s
Incremental Partial
Generational Train
19
Issue:WhyCompact?
• Compact =movereachableobjectstocontiguousmemory.• Locality --- fewerpagesorcache-linesneededtoholdtheactivedata.• Fragmentation --- availablespacemustbemanagedsothereisspacetostorelargeobjects.
20
BasicMark-and-Compact
1. Markreachableobjectsasbefore.2. Maintainatable(hash?)fromreachedchunkstonewlocationsfor
theobjectsinthosechunks.– Scanchunksfromlowendofheap.– Maintainpointerfree thatcountshowmuchspaceisusedbyreached
objectssofar.
3. Moveallreachedobjectstotheirnewlocations,andalsoretargetallreferencesinthoseobjectstothenewlocations.– Usethetableofnewlocations.
4. Retargetrootreferences.
21
Example:Mark-and-Compact
free
22
Example:Mark-and-Compact
free
23
Example:Mark-and-Compact
free
24
Example:Mark-and-Compact
free
25
Example:Mark-and-Compact
free
26
Example:Mark-and-Compact
free
27
Example:Mark-and-Compact
free
28
Example:Mark-and-Compact
free
29
Example:Mark-and-Compact
free
Taxonomy
COMP412,Fall2017 30
Garbage Collectors
Reference-Counters
Trace-Based
Stop-the-World Short-Pause
Mark-and-Sweep
Mark-and-Compact
Basic Baker’s Basic Cheney’s
Incremental Partial
Generational Train
GarbageCollectionviaCopying
CopyingCollectors• Acopyingcollectordividestheheapintotwoormorepools• Newobjectsareallocatedinthecurrentpool• Whenthecurrentpoolisfull,executionpausesandthecollector:– copiesallliveobjectsfromthecurrentpooltotheemptypool– swapsthedesignationscurrentandemptyUnreachableobjectsarenotcopied,sothenewpoolhasfreespace
COMP412,Fall2017 31
HEAP
CurrentPool EmptyPool
1 4
53
2
6
BEFORE
GarbageCollectionviaCopying
CopyingCollectors• Acopyingcollectordividestheheapintotwoormorepools• Newobjectsareallocatedinthecurrentpool• Whenthecurrentpoolisfull,executionpausesandthecollector:– copiesallliveobjectsfromthecurrentpooltotheemptypool– swapsthedesignationscurrentandemptyUnreachableobjectsarenotcopied,sothenewpoolhasfreespace
COMP412,Fall2017 32
HEAP
CurrentPool EmptyPool
1 4
53
2
6
Objectsleftintheemptypoolarediscardedenmasse HE
AP
EmptyPool CurrentPool
1 4 532 6
BEFORE
AFTER
Cheney’sCopyingAllocator
AshotgunapproachtoGC.2heaps:Allocatespaceinone,copytosecondwhenfirstisfull,thenswaproles.Maintaintableofnewlocations.Assoonasanobjectisreached,giveitthenextfreechunkinthesecondheap.Asyouscanobjects,adjusttheirreferencestopointtosecondheap.Advantages:
O(live)complexityNounreachablegarbageVeryfastallocation
Issues:Only½memorycanbeused“Stoptheworld”toperformgarbagecollectionObjectshavetobe“movable”
34
TheObjectLife-Cycle
• “Mostobjectsdieyoung.”– ButthosethatsurviveoneGCarelikelytosurvivemany.
• TailorGCtospendmoretimeonregionsoftheheapwhereobjectshavejustbeencreated.– Givesabetterratioofreclaimedspaceperunittime.
Taxonomy
COMP412,Fall2017 35
Garbage Collectors
Reference-Counters
Trace-Based
Stop-the-World Short-Pause
Mark-and-Sweep
Mark-and-Compact
Basic Baker’s Basic Cheney’s
Incremental Partial
Generational Train
36
GenerationalGarbageCollection
• DividetheheapintopartitionsP0,P1,…– Eachpartitionholdsolderobjectsthantheonebeforeit.
• CreatenewobjectsinP0,untilitfillsup.• GarbagecollectP0 only,andmovethereachableobjectstoP1.• WhenP1 fills,garbagecollectP0 andP1,andputthereachableobjectsinP2.• Ingeneral:WhenPi fills,collectP0,P1,…,Pi andputthereachableobjectsinP(i +1).
Summary
• Referencecounting:conservative,smallpause,fragmented,cyclicgarbage• Mark&Sweep:nocyclicgarbagebutlongpause,O(heap)• Mark&Compact:nofragmentation,O(heap),longpause,needsmovableobjects• Copying(Cheney’s):fastallocation,needsmovableobjects,wastes½heap,O(live),longpause• Generational:short(er)pause,needswritebarrier,canbeusedforrealtime• Concurrent:needsthreadsynchronization• Incremental:runinparallelwithexecution,needssynchronization,canbeusedforrealtime• Hybrid:copyingfortheyoungestgeneration,mark&compact forolderones
COMP412,Fall2017 37
Ifyouwantperformance,payattentiontogarbage• Collectorlocatesliveobjectsbywalkingoutfromvariables– Whenyouaredonewithanobject,setthevariabletoNULL– Leavingthereferencetotheheapobjectwillkeepitlive
• Storagecan“leak”,orbecomeun-recyclable– Leaveapointertoalargedatastructureonthestack,orinaglobal,…– orforgotteninanotherobject,thathappenstobelive– Leadstoextracollectionsand,eventually,anoutofmemoryerror
ImplicationsforProgramming
COMP412,Fall2017 38
Thisisthetakeawaymessage!
38Heap
Stack
“Helloworld!”
StringPoolGlobals
Ifperformancereallymatters,payattentiontosizeofthepool– Javausesavariantofagenerationalcopyingcollector– Allnew objectsareallocatedintoEden– Edeniscopied,whenfull,intooneofStable0 orStable1– WhenStableistoofull,itisaddedtotheLongTermPool
InCOMP412,weoffereda5%bonusforthefastestlab1inalanguage• IntheJavalabs,thetopthreeorfourwereseparatedbythebehaviorofthegarbagecollector• Thefastestlabhadnomajorcollections,&fewerminorcollections
HEAP
Eden Stable0 Stable1 LongTermPool
Minorcollection
Majorcollection
ImplicationsforProgramming
COMP412,Fall2017 39
majorminor
swap
COMP412,Fall2017 40
Doesthisstuffmatter?
PerformanceofOneStudent’sJavaCode,COMP412,Lab1
0.000
5.000
10.000
15.000
20.000
25.000
30.000
- 25,000.00 50,000.00 75,000.00 100,000.00 125,000.00
Standard
BigHeap
SmallHeap
majorcollection(2xinspeed)
waytoomanycollections
