Dynamic Storage Dynamic Storage AllocationAllocation

Bradley HerrupBradley HerrupCS 297 Security and Programming CS 297 Security and Programming

LanguagesLanguages

Storing InformationStoring Information

Information needs to be stored somewhereInformation needs to be stored somewhere

Two Areas of StorageTwo Areas of Storage The Stack – Program StorageThe Stack – Program Storage

Organized and Structured storage mechanism with a relative Organized and Structured storage mechanism with a relative standardized layoutstandardized layout

The Heap – Data StorageThe Heap – Data StorageRandom allocation of space for the storage of Data Random allocation of space for the storage of Data maintained by the Operating Systemmaintained by the Operating System

Can exist in many different parts of memory it is a virtual Can exist in many different parts of memory it is a virtual construct that exists simultaneously in Cache and RAMconstruct that exists simultaneously in Cache and RAM

A Little HistoryA Little History

1960-19701960-1970 Interested in the mapping of logical systems to physical memoryInterested in the mapping of logical systems to physical memory Developing a area to maintain data and separate the the OS from other Developing a area to maintain data and separate the the OS from other

programsprograms1970-19801970-1980

Looking at the ability to refine Heap storage algorithmsLooking at the ability to refine Heap storage algorithms Ability to trace the heap using programs allows scientists to create Ability to trace the heap using programs allows scientists to create

heuristics and statistics to examine the heapheuristics and statistics to examine the heap1980-19901980-1990

Redefinition of Heap structure to incorporate more higher-level data Redefinition of Heap structure to incorporate more higher-level data structures and Objectsstructures and Objects

Focus moves away from fine-tuning the Heap and the Stack to Focus moves away from fine-tuning the Heap and the Stack to increasing OSes using different methodologies, namely increasing the increasing OSes using different methodologies, namely increasing the density of hardware such that if the heap runs out, the OS can just density of hardware such that if the heap runs out, the OS can just request that more memory be allocated from RAMrequest that more memory be allocated from RAM

New vs. Old Views New vs. Old Views

Old ViewsOld Views Worry about present heap managementWorry about present heap management Considered to be a solved or an unsolvable problemConsidered to be a solved or an unsolvable problem

New ViewsNew Views Look at the effect of the Heap over long periods of Look at the effect of the Heap over long periods of

timetimeComputers are on significantly longer than 20 years ago and Computers are on significantly longer than 20 years ago and are running much larger programsare running much larger programs

Determine scenarios in which the heap acts in certain Determine scenarios in which the heap acts in certain ways to best be able to handle the heap in those ways to best be able to handle the heap in those scenariosscenarios

Don’t solve one BIG problem, solve a bunch of little onesDon’t solve one BIG problem, solve a bunch of little ones

The AllocatorThe Allocator

The process repsonsible for maintaining the The process repsonsible for maintaining the HeapHeap Keeps track of:Keeps track of:

Used MemoryUsed MemoryFree MemoryFree Memory

Passes available blocks of memory to programs on Passes available blocks of memory to programs on the Stack where actual information can be storedthe Stack where actual information can be stored

What the Allocator can NOT do:What the Allocator can NOT do: Compact memoryCompact memory Reallocate memory without a programs direct consentReallocate memory without a programs direct consent Move data around to free up necessary spaceMove data around to free up necessary space

FragmentationFragmentation

Allocator creates and deletes information Allocator creates and deletes information wherever in the heap wherever enough free wherever in the heap wherever enough free space exists causing space exists causing FragmentationFragmentation

Internal FragmentationInternal Fragmentation Occurs when the available block size is larger than Occurs when the available block size is larger than

the data needing to be assigned to that areathe data needing to be assigned to that area

External FragmentationExternal Fragmentation Occurs when free memory exists but cannot be used Occurs when free memory exists but cannot be used

because object is larger than anyone particular blockbecause object is larger than anyone particular block

Faults and FixesFaults and Fixes

Majority of programs testing heap allocator are Majority of programs testing heap allocator are simulations that do not emulate real lifesimulations that do not emulate real life

Results in false positivesResults in false positives Do not test long term processors, i.e. many OS processesDo not test long term processors, i.e. many OS processes

Can lead to a lot of fragmentation over long period of timeCan lead to a lot of fragmentation over long period of time

Allocators create/destroy at randomAllocators create/destroy at random In Real Life Programs creation and destruction of data is not In Real Life Programs creation and destruction of data is not

necessarily randomnecessarily random Most programs either:Most programs either:

Create Lots of Data Structures at the beginning of a program and Create Lots of Data Structures at the beginning of a program and manipulate said structuresmanipulate said structuresContinuously create new structures and then destroy them but in Continuously create new structures and then destroy them but in groupsgroups

Methods of Testing the HeapMethods of Testing the Heap

Tracing the HeapTracing the HeapRunning Benchmark and simulations to Running Benchmark and simulations to create probabilities and statistics to fine-create probabilities and statistics to fine-tune heap allocationtune heap allocation ““A single death is a tragedy. A million deaths A single death is a tragedy. A million deaths

is a statistic” – Joseph Stalinis a statistic” – Joseph StalinRelevant because it only goes to show that a Relevant because it only goes to show that a statistic does not tell us what exactly is causing the statistic does not tell us what exactly is causing the problemproblemTheses Statistics are on short term programs Theses Statistics are on short term programs cannot account for long term heap stabilitycannot account for long term heap stability

Ramps, Peaks, and Plateaus Ramps, Peaks, and Plateaus

Example of Ramp Heap Allocation

Grobner Software

Example of Peak Heap Allocation

GCC Compiler

Example of Plateau Heap Allocation

Perl Script

Examination of Real Life Programs led to three distinct families of Examination of Real Life Programs led to three distinct families of heap usageheap usageEnables developers to create general allocators that will be able to Enables developers to create general allocators that will be able to handle different types of programs in different wayshandle different types of programs in different waysIt is more important to cover the manipulation of the heap at the It is more important to cover the manipulation of the heap at the peaks then in the troughspeaks then in the troughs

Strategies and PoliciesStrategies and Policies

Optimize allocations to minimize wait cyclesOptimize allocations to minimize wait cycles Authors believe that over a billion cycles are Authors believe that over a billion cycles are

sacrificed and squandered hourly due to lack of sacrificed and squandered hourly due to lack of efficiency in Allocation efficiency in Allocation

Placement choice is key to determining the Placement choice is key to determining the optimal location algorithm for data in the heapoptimal location algorithm for data in the heap Have to be worried about memory overheadHave to be worried about memory overhead Also time overheadAlso time overhead

““Put blocks where they won’t cause Put blocks where they won’t cause fragmentation later” fragmentation later”

Splitting and CoalescingSplitting and Coalescing

Two main ways of manipulating Two main ways of manipulating space in the heapspace in the heap Splitting: if a memory block is too big, Splitting: if a memory block is too big,

split it to fit necessary datasplit it to fit necessary data Coalescing: if a memory block becomes Coalescing: if a memory block becomes

empty and either of its neighbors are empty and either of its neighbors are empty join both blocks together to empty join both blocks together to provide larger blockprovide larger block

Profiling of AllocatorsProfiling of Allocators

Minimize Time overheadMinimize Time overhead

Minimize Holes and FragmentsMinimize Holes and Fragments

Exploitation of Common PatternsExploitation of Common Patterns

Use of Splitting and CoalescenceUse of Splitting and Coalescence

Fits: when a block of a size is reused are Fits: when a block of a size is reused are block of the same size used preferentiallyblock of the same size used preferentially

Splitting ThresholdsSplitting Thresholds

Taxonomy of Allocating AlgorithmsTaxonomy of Allocating Algorithms

Algorithms are classified by the Algorithms are classified by the mechanism they use for keeping track mechanism they use for keeping track of areas:of areas: Sequential FitsSequential Fits Buddy SystemsBuddy Systems Indexed FitsIndexed Fits Bitmapped FitsBitmapped Fits

Low-Level TricksLow-Level Tricks

Header Fields and AlignmentHeader Fields and Alignment Used to store block related informationUsed to store block related information

Typically the Size of block of memoryTypically the Size of block of memoryRelationship with neighboring blocksRelationship with neighboring blocksIn Use BitIn Use Bit

Typically add 10% overhead to memory usageTypically add 10% overhead to memory usage

Boundary tags (aka Footer)Boundary tags (aka Footer) In Use BitIn Use Bit Neighboring cells informationNeighboring cells information 10% overhead as well10% overhead as well

Lookup TablesLookup Tables Instead of indexing blocks by address, index by pages of blocks Instead of indexing blocks by address, index by pages of blocks

that are the same sizethat are the same size

Sequential FitsSequential Fits

Best FitBest Fit Find the smallest free block large enough to satisfy a requestFind the smallest free block large enough to satisfy a request An exhaustive search O(Mn)An exhaustive search O(Mn) If there are equal blocks which to use?If there are equal blocks which to use? Good Memory Usage in the endGood Memory Usage in the end

First FitFirst Fit Find the first block large enough to satisfy mem allocationFind the first block large enough to satisfy mem allocation Question as to what order to look in memoryQuestion as to what order to look in memory

Next FitNext Fit Optimization of First Fit: using roving pointer to keep track of last Optimization of First Fit: using roving pointer to keep track of last

block used to cut down on execution timeblock used to cut down on execution time

Segregated Free ListsSegregated Free Lists

Simple Segregated StorageSimple Segregated Storage No splitting of free blocks is doneNo splitting of free blocks is done All heap broken down into size blocks of Power-two All heap broken down into size blocks of Power-two All blocks of same size indexed by size and not by addressAll blocks of same size indexed by size and not by address No Header overhead(all blocks same size)No Header overhead(all blocks same size)

Segregated FitsSegregated Fits Array of Free ListsArray of Free Lists If block does not exist take blocks from other list and split or If block does not exist take blocks from other list and split or

coalesce empty blocks to fitcoalesce empty blocks to fitThree Types:Three Types:

Exact ListsExact Lists Strict Size with RoundingStrict Size with Rounding Size Classes with Range ListsSize Classes with Range Lists

Buddy SystemBuddy System

Variant of segregated listsVariant of segregated lists

Split entire heap into twoSplit entire heap into two Split each half into two(does not need to be Split each half into two(does not need to be

power of two)power of two)And so on…if two blocks need to coalesce to And so on…if two blocks need to coalesce to make room requires that binary pair both be emptymake room requires that binary pair both be empty

Log(n) timeLog(n) time

Can also be done using Self-Balancing Can also be done using Self-Balancing Binary tree or Fibonacci sequence treeBinary tree or Fibonacci sequence tree

Indexed FitsIndexed Fits

More abstract algorithm that fits more More abstract algorithm that fits more to a policyto a policy

Policy being a rule set like all blocks Policy being a rule set like all blocks at 8 bytesat 8 bytes

One example uses a cartesian tree One example uses a cartesian tree sorted by block size and address sorted by block size and address

Has log(n) search time Has log(n) search time

Bitmapped FitsBitmapped Fits

Bitmap used to record which parts of the Bitmap used to record which parts of the heap are in usheap are in usBitmap being a simple vector of 1-bit flags Bitmap being a simple vector of 1-bit flags with one bit corresponding to each word of with one bit corresponding to each word of the heap areathe heap areaNot used conventionallyNot used conventionallyOn a 3% overhead vs 20%On a 3% overhead vs 20%Search times are linear but using Search times are linear but using heuristics can get it down to approximatly heuristics can get it down to approximatly O(log N)O(log N)

Heap Allocators for Multiprocessor Heap Allocators for Multiprocessor ArchitecturesArchitectures

Things to consider:Things to consider: Is there a Shared Cache?Is there a Shared Cache? Need a global heap that is accessible to all of the Need a global heap that is accessible to all of the

processors or an OS that can handle:processors or an OS that can handle:ContentionContention

False SharingFalse Sharing

SpaceSpace

Applying and Using this for Garbage Applying and Using this for Garbage CollectionCollection

The efficiency of the garbage collector is The efficiency of the garbage collector is directly related to the efficiency of the directly related to the efficiency of the allocator algorithmallocator algorithmAdditionally if some of the low-level tricks Additionally if some of the low-level tricks are used it enables the garbage collector are used it enables the garbage collector to even more effectively decipher what is to even more effectively decipher what is used and what is notused and what is notCan also participate in maintaining the Can also participate in maintaining the heap by coalescing neighboring blocks heap by coalescing neighboring blocks together to free up spacetogether to free up space

ConclusionsConclusions

Heap organization is still an area worth Heap organization is still an area worth researchingresearchingBecause there is no defined organization Because there is no defined organization to the heap it is not nearly as easy enough to the heap it is not nearly as easy enough to exploit the heap as it is the stackto exploit the heap as it is the stackBy optimizing the heap’s efficiency it is By optimizing the heap’s efficiency it is possible to speed up the average possible to speed up the average computer without having to increase the computer without having to increase the size of the hardwaresize of the hardware

Works CitedWorks Cited

Emery Berger. Emery Berger. Scalable Memory Management. Scalable Memory Management. University of Massachusetts, Dept of Computer University of Massachusetts, Dept of Computer Science, 2002.Science, 2002.Wilson, Paul, Johnstone, Mark, Neely, Michael, Wilson, Paul, Johnstone, Mark, Neely, Michael, and Boles, David. and Boles, David. Dynamic Storage Allocation: A Dynamic Storage Allocation: A Survey and Critical ReviewSurvey and Critical Review. University of Texas . University of Texas at Austin, Department of Computer Science, at Austin, Department of Computer Science, Austin, TX. 1995.Austin, TX. 1995.Wilson, Paul. Wilson, Paul. Uniprocessor Garbage Collection Uniprocessor Garbage Collection Techniques.Techniques. ACM. ACM.