1. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 1 High Coverage Detection of Input-Related Security Faults Eric Larson and Todd Austin August 7, 2003 University of Michigan

2. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 2 Introduction Failing to properly bound input data can be exploited by malicious users –bugs found in Windows –especially important for network data Common security exploits –array references –string library functions Exploitable bugs are often difficult to find –precise input is often necessary to expose the bug –bug may not produce an error in the output

3. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 3 Static vs. Dynamic Bug Finding Approaches Compile-time (static) bug detection +no dependence on input +can prove that a particular operation is safe in some cases –often computationally infeasible  scope is limited Run-time (dynamic) bug detection +can analyze all variables (including those on the heap) +execution is on a real path  fewer false alarms –depends on program input

4. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 4 Overview of Our Approach Dynamic approach to detecting input-related security faults Program instrumentation tracks input derived data –possible range of integer variables –maximum size and termination of strings Dangerous operations are checked over entire range of possible values Found 16 bugs in 8 programs, including 2 known high security faults in OpenSSH Relaxes constraint that the user provides an input that exposes the bug

5. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 5 Testing Process Source Code Instrumentation specification Instrumented Executable Error reports Compile (GCC w/MUSE) Run test suite Debug and fix errors

6. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 6 Detecting Array Buffer Overflows Interval constraint variables are introduced when external inputs are read –Holds the lower and upper bounds of each input value –Initial values encompass the entire range of values –Control points narrow the bounds –Arithmetic operations adjust the bounds Potentially dangerous operations are checked: –array indexing –controlling a loop (to prevent DoS attacks) –arithmetic operations (overflow)

7. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 7 Array Buffer Overflow Example Code SegmentValue of xInterval Constraint on x unsigned int x; int array[5]; scanf(“%d”, &x); if (x > 4) fatal(“bounds”); x++; a = array[x]; 22332233 0  x  MAX_UINT  x    x  5 1  x  5 ERROR! When x = 5, array reference is out of bounds!

8. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 8 Detecting Dangerous String Operations Strings are shadowed by: – max_str_size : largest possible size of the string – known_null : set if string is known to contain a null character Checking string operations: –source string will fit into the destination –source strings are guaranteed to be null terminated Integers that store string lengths are shadowed by: –base address of corresponding string –difference between its value and actual string length Operations involving a string length can narrow the maximum string size

9. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 9 String Fault Detection Example Code SegmentString max_str_sizeknown_null char *bad_strcopy(char *src) { char *dest; char temp[16]; if (strlen(src) > 16) return NULL; strncpy(temp, src, 16); dest = (char *)malloc(16); strcpy(dest, temp); return dest; } src temp src temp dest MAX_INT 16 17 16 TRUE FALSE TRUE FALSE ERROR! temp may not be null terminated during strcpy

10. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 10 String Fault Detection Example Code SegmentString max_str_sizeknown_null char *bad_strcopy(char *src) { char *dest; if (strlen(src) > 16) return NULL; dest = (char *)malloc(16); strcpy(dest, src); return dest; } src dest MAX_INT 17 16 TRUE FALSE ERROR! src may not fit into dest during strcpy

11. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 11 Implementation Our technique was implemented in MUSE –general-purpose instrumentation tool –implemented in gcc at the abstract syntax tree (AST) level –simplification phase removes C nuances –instrumented code is not optimized (future work) Shadowed state for stored in hash tables –separate tables for arrays and integers –hash tables are indexed by address –pointers are shadowed by base address Debug tracing mode can help find source of error

12. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 12 Results ProgramDescription Defects Found Add’l False Alarms anagramanagram generator20 ksgraph partitioning40 yacr2channel router21 betaftpdfile transfer protocol daemon11 gaim (v0.59.8)instant messaging client11 ghttpdweb server32 openssh (v3.0.2)secure shell client / server31 thttpd (v2.20c)web server01 TOTAL 167

13. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 13 Performance Results ProgramOriginal (seconds) Instrumented (seconds) IncreaseUseless Instr. anagram0.1117.7916273.7% ks8.751923.6221950.1% yacr20.5596.7917675.2% betaftpd0.081.091381.2% ghttpd0.346.702096.7% openssh0.020.381978.8% thttpd0.328.472677.8%

14. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 14 Future Work Improve performance by eliminating unnecessary instrumentation calls –Interprocedural dataflow analysis will determine which variables never hold input data –Inline instrumentation to avoid call overhead and hash table lookups Add symbolic analysis support to find more defects and reduce false alarms Address these common scenarios: –pointer walking (manual string handling) –multiple string concatenation into a single buffer

15. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 15 Conclusion Our dynamic approach shadows variables derived from input with additional state –Integers: upper and lower bounds –Strings: maximum string size and known null flag Found 16 bugs in 8 programs –2 known high security faults in OpenSSH Run-time performance overhead is high –Instrumentation has not been optimized

16. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 16 Questions and Answers

17. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 17 Manipulating Interval Constraints RuleInput Interval Constraint a’ = x’ + ya’.lb = max(MIN_VAL(a’), x’.lb + y) a’.ub = min(MAX_VAL(a’), x’.ub + y) a’ = x’ + y’a’.lb = max(MIN_VAL(a’), x’.lb + y’.lb) a’.ub = min(MAX_VAL(a’), x’.ub + y’.ub) if (x’ < y’) (CONDITION IS TRUE) x’.lb = x’.lb x’.ub = min(x’.ub, y’.ub - 1) y’.lb = max(y’.lb, x’.lb + 1) y’.ub = y’.ub while (x’ < y)TRUE: x’.lb = x’.lb, x.ub = min(x’.ub, y-1) FALSE: x’.lb = max(x’.lb, y), x.ub = x’.ub Ticked variables (a’, x’, y’) hold input data. y does not hold input data.

18. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 18 Array Creation Rules Rule actual_sizemax_str_size known _null s = argv[i]strlen(s)+1INT_MAXTRUE char s[n]nnFALSE s = malloc(n)nnFALSE s = malloc(n’’) ( n’’ is a string length) n’’ (n’’.string).max_str_size + n’’.size_diff FALSE NOTE: Pointers to the middle of the array will have shadowed state containing the base address

19. Advanced Computer Architecture Lab University of Michigan USENIX Security ’03 Slide 19 String Functions strcpy(d,s)Assert: s.known_null == TRUE Assert: s.max_str_size <= SIZE(d) d.max_str_size = s.max_str_size; d.known_null = TRUE; strncpy(d,s,n)Assert: s.known_null == TRUE Assert: n <= SIZE(d) d.max_str_size = MIN(s.max_str_size, n); d.known_null = (s.max_str_size <= n); SIZE(d) = MAX(d.actual_size, d.max_str_size)