1. 11/26/2015IT 3271 Memory Management (Ch 14) n Dynamic memory allocation Language systems provide an important hidden player: Runtime memory manager – Activation records – Objects – Explicit allocations: new, malloc, etc. – Implicit allocations: strings, file buffers, arrays with dynamically varying size, etc. It is a 800-pages topic, but here we want to do it in one/two class!

2. 11/26/2015IT 3272 Memory Model n For now, assume that the OS grants each running program one or more fixed-size regions of memory for dynamic allocation n We will model these regions as an array – Our textbook shows an examples of memory management code in Java Yes! It’s OS’s job!!

3. 11/26/2015IT 3273 Stacks Of Activation Records n Activation records must be allocated dynamically n Allocate on call and de-allocate on return n Stack of activation records: push and pop A simple memory management problem

4. 11/26/2015IT 3274 A Stack Illustration An empty stack of 8 words. grow reg.

5. 11/26/2015IT 3275 The manager program calls m.push(3). Some program P needs 3 words P gets this pointer

6. 11/26/2015IT 3276 Some other program Q needs 2 words The manager program calls m.push(2). P gets this pointer Q gets this pointer wasted

7. 11/26/2015IT 3277 Heap n Stack is easy to do, but Allocations and deallocations may come in any order? n A heap is a pool of blocks of memory At the beginning there is only one block (but not always the case) There are many approaches for doing this…

8. 11/26/2015IT 3278 How to deal blocks to the requests? A pool of free blocks are linked, initially containing one big free block, then fragmented into small blocks – Search the free list for first adequate block – If there is extra space in the block, return the unused portion to front of the free list – Allocate requested portion (at the lower end) The easiest approach: First Fit for request To free a block, just add to the front of the free list

9. 11/26/2015IT 3279 Heap implementation in an array Size of this block Pointer to next block, -1 means no next block.

10. 11/26/2015IT 32710 Size of this free block Size of this block for P Some program P requests 4 words P gets this pointer

11. 11/26/2015IT 32711 Size of this block Q gets this pointer Some program Q requests 2 words

12. 11/26/2015IT 32712 P is done

13. 11/26/2015IT 32713 Some program R requests 1 word R gets this pointer

14. 11/26/2015IT 327 14 A Problem n Consider this sequence: n The manager needs to coalesce adjacent free blocks. 1.p1=m.allocate(4); 2.p2=m.allocate(4); 3.m.deallocate(p1); 4.m.deallocate(p2); 5.p3=m.allocate(7); 0 5 5 5 5 5 5 p1 p2 5 5 p2  top How?

15. 11/26/2015IT 32715 Improvement strategy n Quick lists, a separate list, same sized blocks n Delayed coalescing Keep separate free lists for popular (small) block sizes

16. 11/26/2015IT 32716 p1=m.allocate(4); p2=m.allocate(1); m.deallocate(p1); p3=m.allocate(5); The final allocation will fail because of fragmentation. Another problem: Fragmentation DeFragmentation

17. 11/26/2015IT 32717 Heap Mechanisms (to manage heaps.) n Three major issues: – Placement—where to allocate a block – Splitting—when and how to split large blocks – Coalescing—when and how to recombine Many variety and different refinements

18. 11/26/2015IT 32718 Placement: Where to allocate a block n (FF) First Fit from a stack (FIFO) of free list n (BF) Best Fit … n (NF) Next Fit … Some mechanisms use a more scalable data structure like a Balanced Binary Tree

19. 11/26/2015IT 32719 Splitting: When and how to split large blocks n Split to requested size n Sometimes, less splitting gives better results e.g., allocate more than requested or rounding up allocation size to some multiple, for example 2 n

20. 11/26/2015IT 32720 Coalescing: When and how to recombine adjacent free blocks n Several varieties: – No coalescing – Eager coalescing – Delayed coalescing

21. 11/26/2015IT 32721 Current Heap Links n Some systems track current heap links n A current heap link is a memory address (location in the heap) where the running program may use a = new X(i); b = new X(j); a = b; abab What if

22. 11/26/2015IT 32722 Tracing Current Heap Links IntList a = new IntList(null); int b = 2; int c = 1; a = a.cons(b); a = a.cons(c); Where are the current heap links in this picture? 1 2 3 123

23. 11/26/2015IT 32723 To Find Current Heap Links n Start with the root set: memory locations outside of the heap with links into the heap We have to check: – Active activation records (if on the stack) – Static variables, etc. n For each memory location in the set, look at the allocated block it points to, and add all the memory locations in that block n Repeat until no new locations are found {a b c } They are not in the active activation record.

24. 11/26/2015IT 32724 Possible Errors In Current Heap Links 1. Exclusion errors: a memory location that actually is a current heap link is left out. (This type of error is not allowed) Current Activation Record p z:200 r[3] Static variables s x:123 y:100 456 a b c d 2000 3000 4000 Heap Current heap links p, r, a

25. 11/26/2015IT 32725 Possible Errors In Current Heap Links 2. Unused inclusion errors: a memory location is included, but the program never actually uses the value stored there Current Activation Record p z:200 r[3] Static variables s x:123 y:100 456 a b c d 2000 3000 4000 Heap Current heap links s, p, r, a, c, d How do I know this is the case?

26. 11/26/2015IT 32726 Possible Errors In Current Heap Links 3. Used inclusion errors (mistaken): a memory location is included, but the program uses the value stored there as something other than a heap address (e.g. int) Current Activation Record p z:200 r[3] Static variables s x:123 y:100 456 a b c d 2000 3000 4000 Heap Current heap links s, p, r, a, c, y, z How do I know this is the case?

27. 11/26/2015IT 32727 Errors Are Unavoidable n For heap manager purposes, exclusion errors are unacceptable Therefore, we must include a location if it might be used as a heap link, and n This makes unused inclusion errors unavoidable n Depending on the language, used inclusions errors (mistaken) may also be unavoidable

28. 11/26/2015IT 32728 An application for current heap links: Heap Compaction/Defragmentation n Manager can re-allocate blocks: – Copy the block to a new location – Update all links to (or into) that block n So it can compact the heap, moving all allocated blocks to one end, leaving one big free block and no fragmentation Q P P Q S R T R S T

29. 11/26/2015IT 32729 n There are so many errors caused by improper de- allocation (e.g., Bad or without class destructors in C/C++) n It is a burden on the programmer … Therefore, let the language system do the job automatically --- Garbage Collection

30. 11/26/2015IT 32730 Garbage Collection 1. Mark and sweep 2. Copying 3. Reference counting Three Major Approaches

31. 11/26/2015IT 32731 Mark And Sweep uses current heap links in a two-stage process: – Mark: find the live heap links and mark all the heap blocks pointed by them – Sweep: make a pass over the heap and return unmarked blocks to the free pool - both kinds of inclusion errors are tolerated, - blocks are not moved, fragmentation may remain

32. 11/26/2015 IT 327 Mark And Sweep P S R T P S R T Mark: live heap links (in red) Sweep: return unmarked blocks to the free pool blocks are not moved / fragmentations may remain Free blocks 32

33. 11/26/2015IT 32733 Copying Collection n divide memory in half, and uses only one half at a time n When one half becomes full, find live heap links, and copy live blocks to the other half n Fragmentation is eliminated n Moves blocks Cannot tolerate used inclusion errors. Why? Because that used variable will be mistaken as a link and hence modified

34. 11/26/2015IT 327 34 Q P P Q S R T R S T Copying Collection  Moves blocks: cannot tolerate used inclusion errors (mistake) X=700 X=100

35. 11/26/2015IT 32735 Reference Counting n Each block has a counter of heap links to it n Incremented when a heap link is copied, decremented when a heap link is discarded n When counter goes to zero, block is garbage and can be freed n Does not use current heap links

36. 11/26/2015IT 32736 Reference Counting

37. 11/26/2015IT 32737 Reference Counting Problem Reference counters are not zero But all blocks are garbage. Garbage cycles can’t be identified

38. 11/26/2015IT 32738 Reference Counting n Problem 1: cycles of garbage n Problem 2: overhead of updating reference counters is high n One advantage: more uniform. I.e., cost is incremental, no big pause while collecting

39. 11/26/2015IT 32739 Garbage Collecting Refinements n Generational collectors – Divide block into generations according to age – Garbage collect in younger generations more often (using previous methods) n Incremental collectors – Collect garbage a little at a time – Avoid the uneven performance like mark-and- sweep and copying collectors

40. 11/26/2015IT 32740 Languages with Garbage Collectors n Required: Java, ML n encouraged: Ada n difficult to have: C, C++ – But not impossible, even for C and C++ – There are libraries that replace the usual malloc / free with a garbage-collecting manager

41. 11/26/2015IT 32741 Conclusion n Memory management is an important hidden player in language systems n Performance and reliability are critical n Different techniques are difficult to compare, since every run of every program makes different memory demands n Still an active area of language systems research and experimentation