1. Static Program Analysis via Three-Valued Logic Thomas Reps University of Wisconsin Joint work with M. Sagiv (Tel Aviv) and R. Wilhelm (U. Saarlandes)

2. x = 3; y = 1/(x-3); x = 3; px = &x; y = 1/(*px-3); x = 3; p = (int*)malloc(sizeof int); *p = x; q = p; y = 1/(*q-3); need to track values other than 0 need to track pointers need to track heap-allocated storage

3. Flow-Sensitive Points-To Analysis a d b c f e a d b c f e a d b c f e a d b c f e a d b c f e a d b c f e 33 22 11 44 a = &e 55 c = &f *b = c b = ab = a d = *a p = &q; p = q; p = *q; *p = q; pq p r1r1 r2r2 q r1r1 r2r2 q s1s1 s2s2 s3s3 p p s1s1 s2s2 q r1r1 r2r2 pq p r1r1 r2r2 q r1r1 r2r2 q s1s1 s2s2 s3s3 p p s1s1 s2s2 q r1r1 r2r2

4. What About Malloc? Each malloc site  malloc-site variable r p int *p, *q, *r; a: p = (int*)malloc(sizeof(int)); b: q = (int*)malloc(sizeof(int)); r = p; q malloc$b malloc$a

5. What About Malloc? Each malloc site  malloc-site variable /* Create a List of length n */ List head; List *p = &head; for (int i = 0; i < n; i++) { *p = (List)malloc(sizeof(List*)); p = &((*p)  next); } typedef struct list_cell { int val; struct list_cell *next; } *List; p head

6. Shape Analysis [Jones and Muchnick 1981] Characterize dynamically allocated data –Identify may-alias relationships –x points to an acyclic list, cyclic list, tree, dag, … –“disjointedness” properties x and y point to structures that do not share cells –show that data-structure invariants hold Account for destructive updates through pointers

7. Applications: Software Tools Static detection of memory errors –dereferencing NULL pointers –dereferencing dangling pointers –memory leaks Static detection of logical errors –Is a data-structure invariant restored?

8. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships Dynamic storage allocation –No bound on the size of run-time data structures Data-structure invariants typically only hold at the beginning and end of operations –Want to verify that data-structure invariants are re-established

9. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

10. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

11. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

12. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

13. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

14. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

15. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

16. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

17. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

18. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

19. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

20. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

21. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

22. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

23. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; 123 NULL x yt

24. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

25. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

26. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

27. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

28. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL Materialization

29. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

30. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

31. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

32. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

33. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

34. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

35. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

36. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

37. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

38. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

39. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

40. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

41. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt

42. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

43. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

44. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

45. Example: In-Situ List Reversal List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x  next; y  next = t; } return y; } typedef struct list_cell { int val; struct list_cell *next; } *List; x yt NULL

46. Idea for a List Abstraction represents x y t NULL x y t x y t x y t

47. x yt x y t x y t return y t = y y  next = t y = x x = x  next x != NULL x yt NULL x yt x y t x y t x y t x y t x y t x y t x y t x y t x y t x y t

48. Properties of reverse(x) On entry, x points to an acyclic list On each iteration, x & y point to disjoint acyclic lists All the pointer dereferences are safe No memory leaks On exit, y points to an acyclic list On exit, x = = NULL All cells reachable from y on exit were reachable from x on entry, and vice versa On exit, the order between neighbors in the y-list is opposite to their order in the x-list on entry

49. A ‘Yacc’ for Shape Analysis: TVLA Parametric framework –Some instantiations  known analyses –Other instantiations  new analyses Applications beyond shape analysis –Partial correctness of sorting algorithms –Safety of mobile code –Deadlock detection in multi-threaded programs –Partial correctness of mark-and-sweep gc alg. –Correct usage of Java iterators

50. A ‘Yacc’ for Static Analysis: TVLA Parametric framework –Some instantiations  known analyses –Other instantiations  new analyses Applications beyond shape analysis –Partial correctness of sorting algorithms –Safety of mobile code –Deadlock detection in multi-threaded programs –Partial correctness of mark-and-sweep gc alg. –Correct usage of Java iterators

51. Formalizing “... ” Informal: x Formal: x Summary node

52. The French Recipe for Program Verification [Cousot & Cousot] Concrete operational semantics –  st  :    Collecting semantics –  st  c : 2   2  Abstract semantics –  : 2   A,  : A  2 ,  (  (a)) = a,  (  (C))  C –  st  # : A  A –Upper approximation Sound results But may produce false alarms

53. Using Relations to Represent Linked Lists

54. u1u1 u2u2 u3u3 u4u4 x y

55. Formulas: Queries for Observing Properties Are x and y pointer aliases?  v: x(v)  y(v)

56. x y u1u1 u2u2 u3u3 u4u4 Are x and y Pointer Aliases?  v: x(v)  y(v) x y u1u1  1 Yes

57. Predicate-Update Formulas for “y = x” x’(v) = x(v) y’(v) = x(v) t’(v) = t(v) n’(v 1,v 2 ) = n(v 1,v 2 )

58. x u1u1 u2u2 u3u3 u4u4 y’(v) = x(v) 10001000 y Predicate-Update Formulas for “y = x”

59. Predicate-Update Formulas for “x = x  n” x’(v) =  v 1 : x(v 1 )  n(v 1,v) y’(v) = y(v) t’(v) = t(v) n’(v 1, v 2 ) = n(v 1, v 2 )

60. x u1u1 u2u2 u3u3 u4u4 y x’(v) =  v 1 : x(v 1 )  n(v 1,v)   x Predicate-Update Formulas for “x = x  n”

61. Why is Shape Analysis Difficult? Destructive updating through pointers –p  next = q –Produces complicated aliasing relationships Dynamic storage allocation –No bound on the size of run-time data structures Data-structure invariants typically only hold at the beginning and end of operations –Need to verify that data-structure invariants are re-established

62. Two- vs. Three-Valued Logic 01 Two-valued logic {0,1} {0}{1} Three-valued logic {0}  {0,1} {1}  {0,1}

63. Two- vs. Three-Valued Logic Two-valued logicThree-valued logic

64. Two- vs. Three-Valued Logic 01 Two-valued logic {0}{1} Three-valued logic {0,1}

65. Two- vs. Three-Valued Logic 01 Two-valued logic ½ 01 Three-valued logic 0  ½ 1  ½

66. Boolean Connectives [Kleene]

67. Canonical Abstraction u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234        

68. Canonical Abstraction u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234

69. Canonical Abstraction u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234   

70. Property-Extraction Principle Questions about a family of two-valued stores can be answered conservatively by evaluating a formula in a three-valued store Formula evaluates to 1  formula holds in every store in the family Formula evaluates to 0  formula does not hold in any store in the family Formula evaluates to 1/2  formula may hold in some; not hold in others  

71. Are x and y Pointer Aliases? u1u1 u x y  v: x(v)  y(v)    Yes 1

72. Is Cell u Heap-Shared?  v 1,v 2 : n(v 1,u)  n(v 2,u)  v 1  v 2 u Yes 1  1 1 1 

73. Maybe Is Cell u Heap-Shared?  v 1,v 2 : n(v 1,u)  n(v 2,u)  v 1  v 2 u1u1 u x y 1/2   1

74. The Embedding Theorem y x u1u1 u 34 u2u2 y x u1u1 u 234 y x u1u1 u3u3 u2u2 u4u4 x y u 1234  v: x(v)  y(v) Maybe No

75. The Embedding Theorem If a structure B can be embedded in a structure S by an onto function f, such that basic predicates are preserved, i.e., p B (u 1,.., u k )  p S (f(u 1 ),..., f(u k )) Then every formula  is preserved: –If    = 1 in S, then    = 1 in B –If    = 0 in S, then    = 0 in B –If    = 1/2 in S, then    could be 0 or 1 in B

76. Embedding u1u1 u2u2 u3u3 u4u4 x u5u5 u6u6 u 12 u 34 u 56 x u 123 u 456 x

77. Canonical Abstraction: An Embedding Whose Result is of Bounded Size u1u1 u2u2 u3u3 u4u4 x u1u1 x u 234

78. Predicate-Update Formulas for “y = x” y ’ (v) = x(v) Old: u1u1 u x y New: u1u1 u x 

79. Predicate-Update Formulas for “x = x  n” x ’ (v) =  v 1 : x(v 1 )  n(v 1,v) y Old: u1u1 u x y New: u1u1 u x  

80. x yt NULL return y t = y y  next = t y = x x = x  next x != NULL x yt NULL x yt u1u1 u x y u1u1 u x y ’ (v) = x(v) 1010

81. x yt NULL x y t x y t return y t = y y  next = t y = x x = x  next x != NULL x yt NULL x yt x y t x y t x y t x y t x y t x y t x y t x y t x y t x y t

82. x’ (v) =  v 1 : x(v 1 )  n(v 1,v) Naïve Transformer (x = x  n) x y Evaluate update formulas y x  

83. Cyclic versus Acyclic Lists  x 31 7191 u1u1 u x u1u1 u x

84. How Are We Doing? Conservative Convenient But not very precise  –Advancing a pointer down a list loses precision –Cannot distinguish an acyclic list from a cyclic list

85. The Instrumentation Principle Increase precision by storing the truth-value of some chosen formulas

86. Is Cell u Heap-Shared?  v 1,v 2 : n(v 1,u)  n(v 2,u)  v 1  v 2 u

87. is = 0 Example: Heap Sharing  x 31 7191 is(v) =  v 1,v 2 : n(v 1,v)  n(v 2,v)  v 1  v 2 u1u1 u x u1u1 u x is = 0

88. Example: Heap Sharing  x 31 7191 is(v) =  v 1,v 2 : n(v 1,v)  n(v 2,v)  v 1  v 2 is = 0 is = 1 u1u1 u x u1u1 u x is = 0 is = 1

89. Example: Cyclicity  x 31 7191 c(v) =  v 1 : n(v,v 1 )  n * (v 1,v) c = 0 is = 0c = 1 u1u1 u x u1u1 u x c = 0 c = 1

90. Is Cell u Heap-Shared?  v 1,v 2 : n(v 1,u)  n(v 2,u)  v 1  v 2 u1u1 u x y is = 0 No! 1/2   1 Maybe

91. Formalizing “... ” Informal: x y Formal: x y

92. Formalizing “... ” Informal: x y t2t2 t1t1 Formal: x y t2t2 t1t1

93. Formalizing “... ” Informal: x y Formal: x y reachable from variable x reachable from variable y r[x]r[x] r[y]r[y] r[x]r[x] r[y]r[y]

94. Formalizing “... ” Informal: x y t2t2 t1t1 Formal: t2t2 t1t1 r[x],r[t 1 ] r[y],r[t 2 ] r[x],r[t 1 ] r[y],r[t 2 ] x y r[y]r[y] r[x]r[x]r[x]r[x] r[y]r[y]

95. Updating Auxiliary Information t1t1 {r[t 1 ],r[x],r[y]}{p[t 1 ],r[x],r[y]} x {p[x],p[y]} {r[x],r[y]} y t1t1 {r[t 1 ],r[x]}{p[t 1 ],r[x]} x {p[x]} {r[x]} y = NULL

96. Automatic Generation of Update Formulas for Instrumentation Predicates Where do we get the predicate-update formulas to update the extra predicates?

97. Automatic Generation of Update Formulas for Instrumentation Predicates Originally, user provided –Update formulas for core predicates:  c,st (v) –Definitions of instrumentation predicates:  p (v) –Update formulas for instrumentation predicates: p,st (v) Now: p,st created from  p and the  c,st dddd Consistently defined?

98. doubly-linked(v) reachable-from-variable-x(v) acyclic-along-dimension-d(v) tree(v) dag(v) AVL trees: –balanced(v), left-heavy(v), right-heavy(v) –... but not via height arithmetic Useful Instrumentation Predicates Need FO + TC

99. Materialization x = x  n Informal: x y y x x = x  n Formal: x y x y x y x = x  n y x

100. Naïve Transformer (x = x  n) x y Evaluate update formulas y x

101. Best Transformer (x = x  n) x y y x y x  y x y x ...... Evaluate update formulas y x y x......

102. “Focus”-Based Transformer (x = x  n) x y y x y x Focus(x  n) “Partial  ” y x y x Evaluate update formulas y x y x 