Dynamic Memory Management

Professor Jennifer Rexfordhttp://www.cs.princeton.edu/~jrex

Goals of Todays LectureDynamic memory managementGarbage collection by the run-time system (Java)Manual deallocation by the programmer (C, C++)Challenges of manual deallocation Arbitrary request sizes in an arbitrary orderComplex evolution of heap as a program runsDesign decisions for the K&R implementationCircular linked-list of free blocks with a first fit allocationCoalescing of adjacent blocks to create larger blocks

Memory Layout: Heapchar* string = hello;int iSize;

char* f(){ char* p; scanf(%d, &iSize); p = malloc(iSize); return p;}TextBSSStackHeapNeeded when required memory size is not known before the program runsRoDataData

Allocating & Deallocating MemoryDynamically allocating memoryProgrammer explicitly requests space in memorySpace is allocated dynamically on the heapE.g., using malloc in C, and new in JavaDynamically deallocating memoryMust reclaim or recycle memory that is never used againTo avoid (eventually) running out of memory

GarbageAllocated block in heap that will not be accessed again Can be reclaimed for later use by the program

Option #1: Garbage CollectionRun-time system does garbage collection (Java)Automatically determines objects that cant be accessedAnd then reclaims the resources used by these objectsObject x = new Foo();Object y = new Bar();x = new Quux();

if (x.check_something()) { x.do_something(y);}System.exit(0);Object Foo() is never used again!

Challenges of Garbage CollectionDetecting the garbage is not always easyif (complex_function(y)) x = Quux();Run-time system cannot collect all of the garbageDetecting the garbage introduces overheadKeeping track of references to objects (e.g., counter)Scanning through accessible objects to identify garbageSometimes walking through a large amount of memoryCleaning the garbage leads to bursty delaysE.g., periodic scans of the objects to hunt for garbageLeading to unpredictable freeze of the running programVery problematic for real-time applications though good run-time systems avoid long freezes

Option #2: Manual DeallocationProgrammer deallocates the memory (C and C++)Manually determines which objects cant be accessedAnd then explicitly returns the resources to the heapE.g., using free in C or delete in C++AdvantagesLower overheadNo unexpected pauses More efficient use of memoryDisadvantagesMore complex for the programmerSubtle memory-related bugsSecurity vulnerabilities in the (buggy) code

Manual Deallocation Can Lead to BugsDangling pointersProgrammer frees a region of memory but still has a pointer to itDereferencing pointer reads or writes nonsense valuesint main(void) { char *p; p = malloc(10); free(p); putchar(*p);}May print nonsense character.

Manual Deallocation Can Lead to BugsMemory leakProgrammer neglects to free unused region of memorySo, the space can never be allocated againEventually may consume all of the available memoryvoid f(void) { char *s; s = malloc(50); return;}

int main(void) { while (1) f(); return 0;}Eventually, malloc() returns NULL

Manual Deallocation Can Lead to BugsDouble freeProgrammer mistakenly frees a region more than onceLeading to corruption of the heap data structure or premature destruction of a different object

int main(void) { char *p, *q; p = malloc(10); free(p); q = malloc(10); free(p); }Might free the space allocated to q!

Challenges for Malloc and FreeMalloc() may ask for an arbitrary number of bytesMemory may be allocated & freed in different orderCannot reorder requests to improve performancechar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p2

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p2

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p2p3

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p5, p2p3

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p5, p2p3

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p5, p2p3p4

Heap: Dynamic Memory #include void *malloc(size_t size); void free(void *ptr);00xffffffffStack}HeapHeapchar *p1 = malloc(3);char *p2 = malloc(1);char *p3 = malloc(4);free(p2);char *p4 = malloc(6);free(p3);char *p5 = malloc(2);free(p1);free(p4);free(p5);p1p5, p2p3p4

Goals for Malloc and FreeMaximizing throughputMaximize number of requests completed per unit timeNeed both malloc() and free() to be fastMaximizing memory utilizationMinimize the amount of wasted memoryNeed to minimize size of data structures

Strawman #1: free() does nothingGood throughput, but poor memory utilizationStrawman #2: malloc() finds the best fitGood memory utilization, but poor throughput

Keeping Track of Free BlocksMaintain a collection of free blocks of memoryAllocate memory from one of the blocks in the free listDeallocate memory by returning the block to the free listAsk the OS for additional block when more are neededDesign questionsHow to keep track of the free blocks in memory?How to choose an appropriate free block to allocate?What to do with the left-over space in a free block?What to do with a block that has just been freed?freefreefree

Need to Minimize FragmentationInternal fragmentationAllocated block is larger than malloc() requestedE.g., malloc() imposes a minimum size (e.g., 64 bytes)

External fragmentationEnough free memory exists, but no block is big enoughE.g., malloc() asks for 128 contiguous bytes

64646433

Simple K&R-Like ApproachMemory allocated in multiples of a base sizeE.g., 16 bytes, 32 bytes, 48 bytes, Linked list of free blocksMalloc() and free() walk through the list to allocate and deallocateMalloc() allocates the first big-enough blockTo avoid sequencing further through the listMalloc() splits the free blockTo allocate what is needed, and leave the rest availableLinked list is circularTo be able to continue where you left offLinked list in the order the blocks appear in memoryTo be able to coalesce neighboring free blocks

Allocate Memory in Multiples of Base SizeAllocate memory in multiples of a base sizeTo avoid maintaining very tiny free blocksTo align memory on size of largest data type (e.g., long) Requested size is rounded upAllocation in units of base_sizeRound:(nbytes + base_size 1)/base_sizeExample:Suppose nbytes is 37And base_size is 16 bytesThen (37 + 16 1)/16 is 52/16 which rounds down to 316165

Linked List of Free BlocksLinked list of free blocks

Malloc() allocates a big-enough block

Free() adds newly-freed block to the listAllocatedNewlyfreed

First-Fit AllocationHandling a request for memory (e.g., malloc)Find a free block that satisfies the requestMust have a size that is big enough, or biggerSimplest approach: first fitSequence through the linked listStop upon encountering a big enough free blockExample: request for 64 bytesFirst-fit algorithm stops at the 128-byte block

483212864256

Splitting an Oversized Free BlockSimple case: perfect fitMalloc() asks for 128 bytes, and free block has 128 bytesSimply remove the free block from the list

Complex case: splitting the blockMalloc() asks for 64 bytes, and free block has 128 bytes4832128642564832642566464

Circular Linked List of Free BlocksAdvantages of making free list a circular listAny element in the list can be the beginningDont have to handle the end of the list as specialPerformance optimizationMake the head be where last block was foundMore likely to find big enough blocks later in the list

48326425664new head

Maintaining Free Blocks in OrderKeep list in order of increasing addressesMakes it easier to coalesce adjacent free blocksThough, makes calls to free() more expensiveNeed to insert the newly-freed block in the right placeInuseInuseInuseFree list

Coalescing Adjacent Free BlocksWhen inserting a block in the free listLook left and look right for neighboring free blocksInuseInuseLeftRight

An Implementation ChallengeNeed information about each free blockStarting address of the block of memoryLength of the free blockPointer to the next block in the free listWhere should this information be stored?Number of free blocks is not known in advanceSo, need to store the information on the heapBut, wait, this code is what manages the heap!!!Cant call malloc() to allocate storage for this informationCant call free() to relinquish the storage, either

Store Information in the Free BlockStore the information directly in the free blockSince the memory isnt being used for anything anywayAnd allows data structure to grow and shrink as neededEvery free block has a headerSize of the free blockPointer to (i.e., address of) the next free block

Challenge: programming outside the type system

ConclusionsElegant simplicity of K&R malloc and freeSimple header with pointer and size in each free blockSimple circular linked list of free blocksRelatively small amount of code (~25 lines each)Limitations of K&R functions in terms of efficiencyMalloc requires scanning the free listTo find the first free block that is big enoughFree requires scanning the free listTo find the location to insert the to-be-freed block

Dereferencing a NULL pointer typically causes a seg-fault.This program would just keep running, but consume every more memory, and push other processes out of main memory, making the whole system sluggish.Avoiding internal fragmentation: probably want a range of block sizesAvoiding external fragmentation: need malloc() to be smart in selecting a good fitDownside: some internal fragmentation