1. Michael Ernst, page 1 Learning and repair tools background Michael Ernst MIT Lab for Computer Science http://sdg.lcs.mit.edu/~mernst/ Joint work with Jake Cockrell, Adam Czeisler, Bill Griswold, Josh Kataoka, and David Notkin

2. Michael Ernst, page 2 Constraint learning = Dynamic invariant detection Goal: recover invariants from programs A (likely) specification for the program Technique: run the program, generalize over observed values hypothesize every possible invariant discard the ones that are falsified Many checks, each one cheap Artifact: Daikon http://pag.csail.mit.edu/daikon/

3. Michael Ernst, page 3 Examples Detect invariants (as in assert s or specifications) x > abs(y) x = 16*y + 4*z + 3 array a contains no duplicates for each node n, n = n.child.parent graph g is acyclic if ptr  null then *ptr > i Even simpler properties can be useful

4. Michael Ernst, page 4 Uses of dynamic invariant detection Verifying safety properties [Vaziri 98] [Nimmer 02] Automatic theorem proving [Win 02] Identifying refactoring opportunities [Kataoka 01] Predicate abstraction [Dodoo 02] Generating test cases [Xie 03] [Gupta 03] Selecting and prioritizing test cases [Harder 03] Explaining test failures [Groce 03] Predicting incompatibilities in component upgrades [McCamant 03] Error detection [Raz 02] [Hangal 02] [Pytlik 03] [Mariani 04] [Brun 04] Error isolation [Xie 02] [Liblit 03] Choosing modalities [Lin 04]

5. Michael Ernst, page 5 Ways to obtain invariants Programmer-supplied Static analysis: examine the program text [Cousot 77, Gannod 96] properties are guaranteed to be true pointers are intractable in practice Dynamic analysis: run the program complementary to static techniques

6. Michael Ernst, page 6 Basic technique Run the program, generalize over observed values hypothesize every possible invariant discard the ones that are falsified many checks, each one cheap Why does this work so well? Richer properties Eliminate irrelevant properties Optimizations

7. Michael Ernst, page 7 Improved invariant relevance Add desired invariants: 1. Implicit values 2. Unused polymorphism Eliminate undesired invariants (and improve performance): 3. Unjustified properties 4. Redundant invariants 5. Incomparable variables

8. Michael Ernst, page 8 1. Implicit values Goal: relationships over non-variables array: length, sum, min, max array and scalar: element at index, subarray number of calls to a procedure

9. Michael Ernst, page 9 Derived variables Successfully produces desired invariants Adds many new variables slowdown irrelevant invariants Staged derivation and invariant inference avoid deriving meaningless values avoid computing tautological invariants

10. Michael Ernst, page 10 2. Unused polymorphism Variables declared with general type, used with more specific type Example: given a generic list that contains only integers, report that the contents are sorted Also applicable to subtype polymorphism

11. Michael Ernst, page 11 Unused polymorphism example class MyInteger { int value; … } class Link { Object element; Link next; … } class List { Link header; … } List myList = new List(); for (int i=0; i<10; i++) myList.add(new MyInteger(i)); Desired invariant: in class List, header. closure( next ).element.value : sorted by 

12. Michael Ernst, page 12 Polymorphism elimination Pass 1: front end outputs object ID, runtime type, and all known fields Pass 2: given refined type, front end outputs more fields Effective for programs tested so far Sound for deterministic programs

13. Michael Ernst, page 13 3. Unjustified properties Given three samples for x: x = 7 x = – 42 x = 22 Potential invariants: x  0 x  22 x  – 42

14. Michael Ernst, page 14 Statistical checks Check hypothesized distribution To show x  0 for v values of x in range of size r, probability of no zeroes is Range limits (e.g., x  22): same number of samples as neighbors (uniform) more samples than neighbors (clipped)

15. Michael Ernst, page 15 Duplicate values Array sum program: // Sum array b of length n into variable s. i := 0; s := 0; while i  n do { s := s+b[i]; i := i+1 } b is unchanged inside loop Problem: at loop head, –88  b[n – 1]  99 –556  sum(b)  539 Reason: more samples inside loop

16. Michael Ernst, page 16 Disregard duplicate values Idea: count a value if its var was just modified Front end outputs modification bit per value compared techniques for eliminating duplicates Result: eliminates undesired invariants

17. Michael Ernst, page 17 4. Redundant invariants Given: 0  i  j Redundant: a[i]  a[0..j] max(a[0..i])  max(a[0..j]) Redundant invariants are logically implied Implementation contains many such tests

18. Michael Ernst, page 18 Suppress redundancies Avoid deriving variables: suppress 25-50% equal to another variable nonsensical Avoid checking invariants: false invariants: trivial improvement true invariants: suppress 90% Avoid reporting trivial invariants: suppress 25%

19. Michael Ernst, page 19 5. Unrelated variables Problem: the following are of no interest bool b; int *p; b < p int myweight, mybirthyear; myweight < mybirthyear

20. Michael Ernst, page 20 Limit comparisons Check relations only over comparable variables declared program types Lackwit [O’Callahan 97] : value flow analysis based on polymorphic type inference

21. Michael Ernst, page 21 Comparability results Comparisons: declared types: 60% as many comparisons Lackwit: 5% as many comparisons; scales well Runtime: 40-70% improvement Few differences in reported invariants

22. Michael Ernst, page 22 Ignore redundant invariants Redundant invariants are not instantiated or checked Many invariants are implied by others As long as the antecedents are true, the consequent need be neither instantiated nor checked An invariant must be created when its antecedent is falsified (x = y) Ù odd(x) Þ odd(y) If a sample is seen where x ¹ y, the odd(y) invariant must be created The new invariant must be true over all past samples (which are no longer available) The new invariant must be checked over future samples

23. Michael Ernst, page 23 Online inference optimizations Constants Equality sets Program point and variable hierarchy Suppression