Experience with Safe Memory Management in Cyclone Michael Hicks University of Maryland, College Park Joint work with Greg Morrisett - Harvard Dan Grossman - UW Trevor Jim - AT&T

Cyclone Derived from C, having similar goals –Exposes low-level data representations, provides fine-grained operations But safe –Restrictions to C (e.g., (int *)1 not allowed) –Additions and types to regain flexibility Today: balancing safety and flexibility when managing memory

Goal: Programmer Control Many memory management choices –Garbage collection –Stack allocation –malloc/free –Reference counting (Linux, COM) –Arenas (bulk free) (Apache, LCC) Depends on the application

Unifying Theme: Region types Conceptually divide memory into regions –Different kinds of regions (e.g., not just bulk-free) Associate every pointer with a region Prevent dereferencing pointers into dead regions int *`r x; // x points into region `r *x = 3; // deref allowed if `r is live (inference often obviates annotations `r) Liveness by type & effects system (Tofte&Talpin)

Outline Motivation and basic approach Regions description –Basics: LIFO arenas, stack and heap regions –Unique and reference-counted pointers –Dynamic arenas Programming experience Experimental measurements Conclusions

LIFO Arenas Dynamic allocation mechanism Lifetime of entire arena is scoped –At conclusion of scope, all data allocated in the arena is freed.

LIFO Arena Example FILE *infile = … Image *i; if (tag(infile) == HUFFMAN) { region h; // region `r created struct hnode *`r huff_tree; huff_tree = read_tree(h,infile); // allocates with h i = decode_image(infile,huff_tree,…); // region `r deallocated upon exit of scope } else …

Stack and Heap Regions Stack regions –Degenerate case of LIFO arena which does not allow dynamic allocation –Essentially activation records Heap region –A global region `H that is always live –Like a LIFO arena that never dies; objects reclaimed by a garbage collector

Scoped Regions Summary See PLDI `02 paper for more details Region Variety Allocation (objects) Deallocation (what) (when) Aliasing (objects) Stackstaticwhole region exit of scope free LIFO Arenadynamic Heapsingle objects GC

Benefits No runtime access checks Arena/stacks –costs are constant-time region allocation region deallocation object creation –useful for Temporary data (e.g., local variables) Callee-allocates data (rprintf) Lots of C-style code

Limitations Lack of control over memory usage –Spurious retention of regions and their objects –Fragmentation –Extra space required by the garbage collector Lack of control over CPU usage –Garbage collection is “one-size-fits-all” Hard to tune –Cannot avoid GC in some cases: LIFO arenas not expressive enough E.g., objects with overlapping lifetimes

Overcoming the Limitations Allow greater control over lifetimes –Object lifetimes Unique pointers and reference-counted pointers –Arena lifetimes Dynamic arenas But not for nothing... –Restrictions on aliasing –Possibility of memory leaks

Unique Region Distinguished region name `U Individual objects can be freed manually An intraprocedural, flow-sensitive analysis –ensures that a unique pointer is not used after it is consumed (i.e. freed) –treats copies as destructive; i.e. only one usable copy of a pointer to the same memory –Loosely based on affine type systems

Unique Pointer Example void foo() { int *`U x = malloc(sizeof(int)); int *`U y = x; // consumes x *x = 5; // disallowed free(y); // consumes y *y = 7;// disallowed }

Temporary Aliasing Problem: Non-aliasing too restrictive Partial solution: Allow temporary, lexically- scoped aliasing under acceptable conditions –Makes unique pointers easier to use –Increases code reuse

Alias construct extern void f(int *`r x); // `r any scoped region void foo() { int *`U x = malloc(sizeof(int)); *x = 3; { alias int *`r y = x; // `r fresh f(y); // y aliasable, but x consumed } // x unconsumed free(x); }

Alias inference extern void f(int *`r x); // `r any scoped region void foo() { int *`U x = malloc(sizeof(int)); *x = 3; f(x); // alias inserted here automatically free(x); }

Reference-counted Pointers Distinguished region `RC Objects allocated in `RC have hidden reference-count field Aliasing tracked as with unique pointers. Explicit aliasing/freeing via `a *`RC alias_refptr(`a *`RC); void drop_refptr(`a *`RC);

Reference-counting Example struct conn * `RC cmd_pasv(struct conn * `RC c) { struct ftran * `RC f; int sock = socket(...); f = alloc_new_ftran(sock,alias_refptr(c)); c->transfer = alias_refptr(f); listen(f->sock, 1); f->state = 1; drop_refptr(f); return c; }

Regions Summary Region Variety Allocation (objects) Deallocation (what) (when) Aliasing (objects) Stackstaticwhole region exit of scope free LIFOdynamic Dynamicmanual Heapsingle objects GC Uniquemanual restricted Refcounted

Ensuring Uniformity and Reuse Many different idioms could be hard to use –Duplicated library functions –Hard-to-change application code We have solved this problem by –Using region types as a unifying theme –Region polymorphism with kinds E.g., functions independent of arguments’ regions –All regions can be treated as if lexical Temporarily, under correct circumstances Using alias and open constructs

Boa web server BetaFTPDftp server Epic image compression Kiss-FFTportable fourier transform MediaNetstreaming overlay network CycWebweb server CycSchemescheme interpreter Programming Experience

Application Characteristics

MediaNet Datastructures

Platform –Dual 1.6 GHz AMD Athlon MP GB RAM Switched Myrinet –Linux (RedHat) Software –C code: gcc –Cyclone code: cyclone 0.8 –GC: BDW conservative collector 6.2  4 –malloc/free: Lea allocator Experimental Measurements

CPU time –Most applications do not benefit from switching from BDW GC to manual approach –MediaNet is the exception Memory usage –Can reduce memory footprint and working set size by 2 to 10 times by using manual techniques Bottom Line

Throughput: Webservers

Throughput: MediaNet

Memory Usage: Web (I)

Memory Usage: Web (II)

Memory Usage: Web (III)

Memory Usage: MediaNet (4 KB packets)

Related Work Regions –ML-Kit (foundation for Cyclone’s type system) –RC –Reaps –Walker/Watkins Uniqueness –Wadler, Walker/Watkins, Clean –Alias types, Calculus of Capabilities, Vault –Destructive reads (e.g., Boyland)

Future Work Tracked pointers sometimes painful; want –Better inference (e.g. for alias) –Richer API (restrict; autorelease) Prevent leaks –unique and reference-counted pointers Specified aliasing –for doubly-linked lists, etc. Concurrency

Conclusions High degree of control, safely: Sound mechanisms for programmer- controlled memory management –Region-based vs. object-based deallocation –Manual vs. automatic reclamation Region-annotated pointers within a simple framework –Scoped regions unifying theme (alias,open) –Region polymorphism, for code reuse

More Information Cyclone homepage – Has papers, benchmarks from this paper, and free distribution –Read about it, write some code!