sustainable memory use - rice university · →how it affects your code’s runtime ... knowing how...

SustainableMemoryUseAllocation&(Implicit) Deallocation(mostlyinJava)

COMP412FALL2017

Copyright2017,KeithD.Cooper&ZoranBudimlić,allrightsreserved.StudentsenrolledinComp412atRiceUniversityhaveexplicitpermissiontomakecopiesofthesematerialsfortheirpersonaluse.Facultyfromeducationalinstitutionsmayusethesematerialsfornonprofiteducationalpurposes,providedthiscopyrightnoticeispreserved.

Preliminaries

Today’slectureAutomaticmemorymanagement→ Procedure-localmemorymanagement→ “GarbageCollection”

→ ReferenceCounting→ CopyingCollectors→ Short-PauseCollectors

→ HowyoushouldprogramdefensivelynowthatyouknowSecondpart:Javamemorymodel,allocation,&recycling→ Howitworks→ Howitaffectsyourcode’sruntime

→ Howyoushouldprogramdefensivelynowthatyouknow

COMP412,Fall2017 1

WheredoobjectsliveinJava?

COMP140 &COMP215• Studentsencouragedtoignoretheissueofwhereobjects,variables,andmethodslive

• Theimplementation(PythonorJava)takescareofthesedetails

• Fundamentally,abstractionisagoodthing— rightuptothepointwhereitcausesproblems

• AtsomepointinyourJavacareer,performancewillmatter– COMP412labs– Atthatpoint,youneedtopayattentiontodetails

• Today’slectureisaboutdetails

COMP412,Fall2017 2

JavaWorld

Fred Kate

JuliaSystem.out.

println()

Wheredoobjectslive?

TheJavaSystemmapsJavaWorldontoProcessorResources• Processorhasfiniteresources• Javasuggeststhatyouhave“enough”resources

• Mapping“enough”onto“what’sthere”isthejoboftheJavacompilerandruntime(JVM)

Knowinghowthatmappingworkscanhelpyouunderstandthebehaviorofyourprograms,andsuggestwaystoimprovetheprogram’sbehavior.

COMP412,Fall2017 3

JavaWorld

Fred Kate

JuliaSystem.out.

println()

0 1 2 k

RAM

ProcessorCore

ProcessorCore

ProcessorCore ��

Fundamentals

Intheexample,whatneedsstorage?• Thetwoclasses(Point&C)• Point’slocalmembers(x,y,& draw)• C’slocalmembers(s,t,& m)• m’slocalvariables(a,b,& p)

COMP412,Fall2017 4

ClassPoint{publicintx,y;publicvoiddraw();

}ClassC{

int s,t;publicvoidm(){int a,b;Pointp=newPoint();a=…;b=…;p.draw();

}}

Aclassicexample

Fundamentals

Intheexample,whatneedsstorage?• Thetwoclasses(Point&C)• Point’slocalmembers(x,y,& draw)• C’slocalmembers(s,t,& m)• m’slocalvariables(a,b,& p)MemoryintheJavaruntimeisdivided,broadlyspeaking,intoaHeapandacollectionofStacks• Oneheapperprogram(large)• Onestackperthread(smaller)

COMP412,Fall2017 5

ClassPoint{publicintx,y;publicvoiddraw();

}ClassC{

int s,t;publicvoidm(){int a,b;Pointp=newPoint();a=…;b=…;p.draw();

}}

Aclassicexample

Heap

STAC

K 0

STAC

K 1

STAC

K 2

STAC

K n…

Point C

“new”point

“Helloworld!”

StringPool

p:a:b:

JVMMemoryLayout

Conceptually,Javamemoryislaidoutalongtheselines

• Whenrunningcodecreatesavariable,itgoesintothethread’sstack• Whenrunningcodecreatesaclassoranobject(e.g.,withanew),itgoesintotheheap• Codelivesofftotheleft (mightconsideritpartoftheheap)

COMP412,Fall2017 6

So,canaprogramrunoutofheapspace? (toomanynews)

Whathappens?⇒ Theruntimesystemtriestorecyclespaceontheheap

Heap

Stacks

Growthsp

ace

forstacks

Globa

ls

Code

⇒ Yes.Emphaticallyyes

SustainableMemoryManagement

Whentheheaprunsoutofspace,thesystemcopes• Scourstheheaplookingforobjectsthatarenolongerofinterest– Technicaltermis“live”– Anobjectisconsideredliveiff itcanbereachedfromtherunningcode

• Startfromallthenamesintherunningcode– Variablesareonthestack1

– Globalnamessuchasdeclaredorimportedclasses– Eachobjectonthestackhasadeclarationwhichrevealsitsstructure– Youcanimaginechasingdownchainsofreferencestofindallliveobjects2➝ That’showitwasdoneforalongtime…

• Moderngarbagecollectorsaremorenuanced– Theystillstartfromthebeginning:local&globalnames– Mostmoderncollectorsare“copying”collectors

COMP412,SupplementalMaterial

AKA“GarbageCollection”

1Localsofthecurrentmethodareonthestack.Localsofthemethodthatcalleditarebelowthecurrentmethodonthestack.Localsofthemethodthatcalledthatmethodarebelow…,andsoon.That’swhytheruntimeusesastack2H.Schorr andW.MWaite,“Anefficientmachine-independentprocedureforgarbagecollectioninvariousliststructures,Comm.ACM,10(8),1967,pages501—506 7

Taxonomy

COMP412,Fall2017 8

Garbage Collectors

Reference-Counters

Trace-Based

Stop-the-World Short-Pause

Mark-and-Sweep

Mark-and-Compact

Basic Baker’s Basic Cheney’s

Incremental Partial

Generational Train

ReferenceCounting

ProsSimpleimplementationConservativeEasytoadapttoreal-time

ConsFragmentationSelf-referencinggarbageLocalityeffectsOverheadperpointermodificationMemoryoverhead

COMP412,Fall2017 9

RootObject

A(1)

E(1)D(2)

B(2)

C(1)

ReferenceCounting



COMP412,Fall2017 10

A(0)

E(1)D(2)

B(2)

C(1)

RootObject

ReferenceCounting



COMP412,Fall2017 11

E(1)D(2)

B(1)

C(0)

RootObject

ReferenceCounting



COMP412,Fall2017 12

E(1)D(1)

B(1)

RootObject

Taxonomy

COMP412,Fall2017 13

Garbage Collectors

Reference-Counters

Trace-Based


Mark-and-Sweep

Mark-and-Compact


Incremental Partial

Generational Train

14

MarkandSweep

1. Free =notholdinganobject;availableforallocation.2. Unreached =Holdsanobject,buthasnotyetbeenreachedfromthe

rootset.3. Unscanned =Reachedfromtherootset,butitsreferencesnotyet

followed.4. Scanned =Reachedandreferencesfollowed.

15

Marking

1. AssumeallobjectsinUnreached state.2. Startwiththerootset.PuttheminstateUnscanned.3. while Unscanned objectsremaindo

examineoneoftheseobjects;makeitsstatebeScanned;addallreferencedobjectstoUnscannediftheyhavenotbeenthere;

end;

16

Sweeping

• PlaceallobjectsstillintheUnreached stateintotheFree state.• PlaceallobjectsinScanned stateintotheUnreached state.– Toprepareforthenextmark-and-sweep.

• Handlesgarbagecyclesproperly

17

MarkandSweep

• Problem:Takestimeproportionaltotheheapsize.– BecauseyoumustvisitallobjectstoseeiftheyareUnreached.

• Baker’salgorithmkeepsalistofallallocatedchucksofmemory,aswellastheFree list.• Keychange:Inthesweep,lookonlyatthelistofallocatedchunks.• ThosethatarenotmarkedasScanned aregarbageandaremovedtotheFree list.• ThoseintheScanned stateareputintheUnreached state.– Forthenextcollection.

Taxonomy

COMP412,Fall2017 18

Garbage Collectors

Reference-Counters

Trace-Based


Mark-and-Sweep

Mark-and-Compact


Incremental Partial

Generational Train

19

Issue:WhyCompact?

• Compact =movereachableobjectstocontiguousmemory.• Locality --- fewerpagesorcache-linesneededtoholdtheactivedata.• Fragmentation --- availablespacemustbemanagedsothereisspacetostorelargeobjects.

20

BasicMark-and-Compact

1. Markreachableobjectsasbefore.2. Maintainatable(hash?)fromreachedchunkstonewlocationsfor

theobjectsinthosechunks.– Scanchunksfromlowendofheap.– Maintainpointerfree thatcountshowmuchspaceisusedbyreached

objectssofar.

3. Moveallreachedobjectstotheirnewlocations,andalsoretargetallreferencesinthoseobjectstothenewlocations.– Usethetableofnewlocations.

4. Retargetrootreferences.

21

Example:Mark-and-Compact

free

22


free

23


free

24


free

25


free

26


free

27


free

28


free

29


free

Taxonomy

COMP412,Fall2017 30

Garbage Collectors

Reference-Counters

Trace-Based


Mark-and-Sweep

Mark-and-Compact


Incremental Partial

Generational Train

GarbageCollectionviaCopying

CopyingCollectors• Acopyingcollectordividestheheapintotwoormorepools• Newobjectsareallocatedinthecurrentpool• Whenthecurrentpoolisfull,executionpausesandthecollector:– copiesallliveobjectsfromthecurrentpooltotheemptypool– swapsthedesignationscurrentandemptyUnreachableobjectsarenotcopied,sothenewpoolhasfreespace

COMP412,Fall2017 31

HEAP

CurrentPool EmptyPool

1 4

53

2

6

BEFORE

GarbageCollectionviaCopying

CopyingCollectors• Acopyingcollectordividestheheapintotwoormorepools• Newobjectsareallocatedinthecurrentpool• Whenthecurrentpoolisfull,executionpausesandthecollector:– copiesallliveobjectsfromthecurrentpooltotheemptypool– swapsthedesignationscurrentandemptyUnreachableobjectsarenotcopied,sothenewpoolhasfreespace

COMP412,Fall2017 32

HEAP

CurrentPool EmptyPool

1 4

53

2

6

Objectsleftintheemptypoolarediscardedenmasse HE

AP

EmptyPool CurrentPool

1 4 532 6

BEFORE

AFTER

Cheney’sCopyingAllocator

AshotgunapproachtoGC.2heaps:Allocatespaceinone,copytosecondwhenfirstisfull,thenswaproles.Maintaintableofnewlocations.Assoonasanobjectisreached,giveitthenextfreechunkinthesecondheap.Asyouscanobjects,adjusttheirreferencestopointtosecondheap.Advantages:

O(live)complexityNounreachablegarbageVeryfastallocation

Issues:Only½memorycanbeused“Stoptheworld”toperformgarbagecollectionObjectshavetobe“movable”

34

TheObjectLife-Cycle

• “Mostobjectsdieyoung.”– ButthosethatsurviveoneGCarelikelytosurvivemany.

• TailorGCtospendmoretimeonregionsoftheheapwhereobjectshavejustbeencreated.– Givesabetterratioofreclaimedspaceperunittime.

Taxonomy

COMP412,Fall2017 35

Garbage Collectors

Reference-Counters

Trace-Based


Mark-and-Sweep

Mark-and-Compact


Incremental Partial

Generational Train

36

GenerationalGarbageCollection

• DividetheheapintopartitionsP0,P1,…– Eachpartitionholdsolderobjectsthantheonebeforeit.

• CreatenewobjectsinP0,untilitfillsup.• GarbagecollectP0 only,andmovethereachableobjectstoP1.• WhenP1 fills,garbagecollectP0 andP1,andputthereachableobjectsinP2.• Ingeneral:WhenPi fills,collectP0,P1,…,Pi andputthereachableobjectsinP(i +1).

Summary

• Referencecounting:conservative,smallpause,fragmented,cyclicgarbage• Mark&Sweep:nocyclicgarbagebutlongpause,O(heap)• Mark&Compact:nofragmentation,O(heap),longpause,needsmovableobjects• Copying(Cheney’s):fastallocation,needsmovableobjects,wastes½heap,O(live),longpause• Generational:short(er)pause,needswritebarrier,canbeusedforrealtime• Concurrent:needsthreadsynchronization• Incremental:runinparallelwithexecution,needssynchronization,canbeusedforrealtime• Hybrid:copyingfortheyoungestgeneration,mark&compact forolderones

COMP412,Fall2017 37

Ifyouwantperformance,payattentiontogarbage• Collectorlocatesliveobjectsbywalkingoutfromvariables– Whenyouaredonewithanobject,setthevariabletoNULL– Leavingthereferencetotheheapobjectwillkeepitlive

• Storagecan“leak”,orbecomeun-recyclable– Leaveapointertoalargedatastructureonthestack,orinaglobal,…– orforgotteninanotherobject,thathappenstobelive– Leadstoextracollectionsand,eventually,anoutofmemoryerror

ImplicationsforProgramming

COMP412,Fall2017 38

Thisisthetakeawaymessage!

38Heap

Stack

“Helloworld!”

StringPoolGlobals

Ifperformancereallymatters,payattentiontosizeofthepool– Javausesavariantofagenerationalcopyingcollector– Allnew objectsareallocatedintoEden– Edeniscopied,whenfull,intooneofStable0 orStable1– WhenStableistoofull,itisaddedtotheLongTermPool

InCOMP412,weoffereda5%bonusforthefastestlab1inalanguage• IntheJavalabs,thetopthreeorfourwereseparatedbythebehaviorofthegarbagecollector• Thefastestlabhadnomajorcollections,&fewerminorcollections

HEAP

Eden Stable0 Stable1 LongTermPool

Minorcollection

Majorcollection

ImplicationsforProgramming

COMP412,Fall2017 39

majorminor

swap

COMP412,Fall2017 40

Doesthisstuffmatter?

PerformanceofOneStudent’sJavaCode,COMP412,Lab1

0.000

5.000

10.000

15.000

20.000

25.000

30.000

- 25,000.00 50,000.00 75,000.00 100,000.00 125,000.00

Standard

BigHeap

SmallHeap

majorcollection(2xinspeed)

waytoomanycollections

sustainable memory use - rice university · →how it affects your code’s runtime ... knowing how...

Documents