Detecting Memory Errors using Compile Time Techniques Nurit Dor Mooly Sagiv Tel-Aviv University

Memory errors u Hard to detect –point of failure is not point of error –difficult to reproduce –Depends on the system’s architecture u Many result from pointer misuse u Other types: out of bound reference

Reference beyond duration int * g() {int i; return &i; } main() {int *p; p = g(); *p = 5; }

Dereference of NULL pointers main() {list *p,*q,*r; p = (list *) malloc(sizeof(list));... q = p->next; /* q = NULL */ r = q->next;/* <== error */ }

Usage of dead storage main() {int *x,*z; x = (int *)malloc(sizeof(int)); free(x); z = (int *)malloc(sizeof(int)); if (x==z) printf(“ unexpected equality”); } usage of deallocated storage

Dereference of NULL pointers typedef struct element { int value; struct element *next; } Elements bool search(int value, Elements *c) { Elements *elem; for ( elem = c; c != NULL;elem = elem->next;) if (elem->val == value) return TRUE; return FALSE NULL dereference

Dereference of NULL pointers typedef struct element { int value; struct element *next; } Elements bool search(int value, Elements *c) { Elements *elem; for ( elem = c; elem != NULL;elem = elem->next;) if (elem->val == value) return TRUE; return FALSE

Memory leakage Elements* reverse(Elements *c) { Elements *h,*g; h = NULL; while (c!= NULL) { g = c->next; h = c; c->next = h; c = g; } return h; leakage of address pointed-by h

Memory leakage Elements* reverse(Elements *c) { Elements *h,*g; h = NULL; while (c!= NULL) { g = c->next; c->next = h; h = c; c = g; } return h;

Cleanness u Rules that a program must obey u Does not depend on a program’s specification u Precondition rules for each statement type u Some cleanness rules are integrated into the programming language: –Type checking –Array bound accesses –Java dereference u Other cleanness rules are programmer responsibility

Run-Time vs. Static Property Run-Time Conservative Static Manual runs  Depends on test cases  Assures against bugs  Interferes production  False alarms  Scales for large programs ??

Innovation of this research u Theoretical –Define memory cleanness for a subset of C programs –Study techniques needed for a conservative static tool –Invent a new shape analysis algorithm u Empirical –Implementation –comparison to other techniques

Program analysis u Static techniques for computing approximations of the possible run-time states u Used mainly in compilers u Areas: –Data flow –Control flow –Type analysis …

Shape graph u Example u Characteristics –finite representation –“sharing” of run-time locations by: “pointed-to by” and “reachable from” variables c elem c NULL

Shape analysis u Initialization - empty shape graph u Iteratively apply every program statement and condition u Stop when no more shape graphs can be derived

Cleanness checking via shape analysis u Compute a set of possible shape graphs before every program statement u Check cleanness condition of every statement against any possible shape graph u Cleanness conditions are generated in a syntax directed fashion u Report violations with the “witness” shape graph

Abstract interpretation state  ’ Operational semantics statement s abstract representation  state  concretization Abstract semantics statement s abstract representation  ’ abstraction

Dereference of NULL pointers typedef struct element { int value; struct element *next; } Elements bool search(int value, Elements *c) { Elements *elem; for ( elem = c; c != NULL;elem = elem->next;) if (elem->val == value) return TRUE; return FALSE NULL dereference

Example c elem c c elem= elem  next NULL

c elem NULL  c c elem   elem= elem  next

c NULL c elem   c NULL c elem c NULL elem   X

c NULL c elem c NULL elem   X c c NULL elem NULL  X c elem 

Motivation u Conservative static analysis for cleanness checking u Use existing pointer-analysis techniques u Minimal false alarms u Which information is needed? u Is user input necessary?

Differences from SRW98 u NULL node u Stack variables Important for statements like p=&a u Each shape-node is represented by –stack variable (unique) –pointed-to by variables –reachable variables u Set of graphs instead of one combined graph

Sample Checks u Statement type – p = q – p = q ->sel – p->sel = q u Cleanness Rules –Unintilized pointer –Unallocated pointer –Usage of dead storage –Dereference of NULL –Memory leakage (failure to release unreachable heap space)

Simple Statements p = q u Dynamic (Run-time) condition –q must be initialized (allocated or NULL) –q not pointing to a released address –address held in p is reachable from a different variable u Shape Graph (static) condition –q must point to a node or to the NULL –q not pointing to a “freed” node –node pointed-by p is reachable from a different variable

Simple statement - example p = q;  X (q is uninitialized) q r q  q X (node was freed) X (memory leakage) p q p

Dereference Statement i = p ->val u Dynamic (Run-time) condition p must not be NULL u Shape Graph (static) condition p must point to a non NULL node

Dereference statement - example i = p  val p p  X (p not allocated)

Core techniques u Flow sensitivity u Interpret conditions u Must alias p = malloc; q = p; …. p = malloc; free(p); *q = 5; if (p!=NULL) *p = 5; p=NULL; q=&p; *q=&i; *p=5;

Core techniques - more u Relations between variables –Example: current = first  prev = NULL u Data Shape -Example: acyclic lists NULL terminating tree

Implementation u PAG (Program Analysis Generator) –C front-end –Supply transfer functions and abstract representation u Input –C program under restrictions »no recursion »no pointer arithmetic or casting u Output –graphical presentation of shape graphs –list of potential cleanness violations

Points-To analysis u Program analysis that computes information regarding the pointers in the program u Point-to pairs (p,a) p = &a;  “ p points-to a” u Heap treatment (p,heap l ): l: p= malloc(...)  “ p points-to heap l - heap address allocate at this statement”

Empirical results sec / leakage false alarms Program Shape Analysis Points-to search.c0.02/00.01/5 null_deref.c 0.03/00.02/5 delete.c0.05/00.01/7 del_all.c0.02/00.01/6 insert.c 0.02/00.03/7 merge.c2.08/00.01/8 reverse.c0.03/00.01/7 fumble.c0.04/00.02/6 rotate.c0.01/00.01/5 swap.c0.01/00.01/5

Empirical results sec / reference+dereference false alarms Program Shape Analysis Points-to search.c0.02/0 0.01/0 null_deref.c 0.03/0 0.02/0 delete.c0.05/0 0.01/0 del_all.c0.02/0 0.01/4 insert.c 0.02/0 merge.c2.08/0 0.01/5 reverse.c0.03/0 0.01/0 fumble.c0.04/0 0.02/0 rotate.c0.01/0 0.01/1 swap.c0.01/0 0.03/1

False alarms u Infeasible paths –Sedgewick_tree treeinsert(int v ){ Tree *f,*p; p = root; f = p; while (p != NULL) { f = p; if (v key) p = p->l; else p = p->r; } p = MALLOC; p->key = v; p->r = NULL; p->l = NULL; if (v key) f->l = p; else f->r = p;}

False alarms u Abstraction not precise enough –acyclic lists –trees u Infeasible paths

Advantage u Detection of non trivial bugs u Easy to use: –Minimal false alarms (No false alarms on many linked list programs) –Minimal user interactions (No annotations) –Graphical output of control-flow graph and shape graphs u Significantly faster than verification tools

Challenges u Scaling for large programs –Annotations –Cheaper preprocessing –Better interprocedural analysis –Other programming languages –Ignore unlikely cases - losing conservative u Other data structures (trees, cyclic lists) u Applications that can benefit from this

Other Accomplishments u Locating array memory leaks in Java (Ran Shaham) u A parametric algorithm for shape analysis (Sagiv, Reps, Wilhelm 99) u An algorithm for analyzing mobile code (Nielson, Nielson, Sagiv 99) u A generic “yacc-like” tool for program analysis (Tal Lev-Ami)

Ongoing work u Interprocedural shape analysis (Noam Rinetskey) u Hardware support for cleanness checking (Roi Amir) u Slicing programs (Eran Yahav)

Previous work u Run-Time tools –check cleanness on a given input –detect errors found on a given input –Examples: Safe-C, Purify u Static checking tools –check cleanness on all possible inputs (compile-time) –can detect all potential errors (but may decide to ignore some) –Examples: LCLint, Extended Static Checking