1. Dual Analysis for Proving Safety and Finding Bugs Corneliu Popeea Wei-Ngan Chin Technische Universität MünchenNational University of Singapore Symposium on Applied Computing – SAC 2010, Sierre, Switzerland

2. 2 Why do Program Analysis? Programs have bugs Program analysis for proving safety finding bugs

3. 3 Automated Approaches Program testing: –detects presence of bugs –examines some paths, not all (under-approximation of the program semantics) –compromises on program safety Verification: –examines all paths (over-approximation of the program semantics) –may report false bugs

4. 4 Combination of Testing and Verification Search for both bugs and safety proofs Combination of under- and over-approximation Projects at Microsoft Research –Synergy, Smash: analysis of Windows device drivers

5. 5 Our Methodology Extend a static analyzer aimed at proving safety to discover (true) bugs –based only on over-approximation analysis –dual simultaneous analyses

6. 6 Quicksort Example void quicksort (float a[], int l, int h) { if (l < h) { int p = partition (a,l,h); quicksort (a,l,p-1); quicksort (a,p+1,h); }} int partition (…) { … v = a[l]; … a[m] = v; …} WANTED 2 conditions on inputs a,I,h NeverBug - guarantees safety of bound checks MustBug - guarantees execution triggers a bug

7. 7 Concrete Semantics and Abstraction Three possible outcomes for a program execution: ok - for a safe/successful execution err - for an unsafe execution (due to a failed assertion) loop - for a non-terminating execution Compute over-approximations for inputs leading to: OK - all possibly safe executions ERR- all possibly unsafe executions Example: OK=true, ERR=true OKERR

8. 8 Forward Reasoning Rules Formulated using Hoare-style triples: Rule for function call: Summaries of recursive functions: –computed by abstract interpretation

9. 9 Partial Correctness and Bug Finding NEVER_BUG= OK Æ : ERR Any input satisfying this condition is guaranteed to lead to ok or loop. MUST_BUG= ERR Æ : OK Any input satisfying this condition is guaranteed to lead to err or loop. MAY_BUG = OK Æ ERR Any input satisfying this condition may lead to either ok, err or loop. Partial correctnessBug finding OKERR NEVER BUG MUST BUG MAY BUG

10. 10 Quicksort Example void quicksort (float a[], int l, int h) { … } Compute OK and ERR using a disjunctive abstract domain: Characterization of quicksort inputs: NEVER_BUG = OK MUST_BUG = ERR

11. 11 Detect Non-termination Some inputs may be outside both OK and ERR. LOOP = : OK Æ : ERR Any input satisfying this condition is guaranteed to lead to loop. Example: void ex_fig9() { int x,y; x:=0; y:=0; l5: while (y>=0) {y:=y+x;} l6: assert(false); } Loop summary: {OK: (x<0 Ç y<0), ERR: false Function summary: {OK: false, ERR.l5.LOOP: true, ERR.l6: false} LOOP OKERR }, LOOP: (x ¸ 0 Æ y ¸ 0)}

12. 12 Dualyzer: Prototype Implementation Written in the Haskell language: –uses a Presburger arithmetic solver: Omega library –disjunctive fixed-point analyzer Objectives: –prove program safety + confirm true bugs Test programs: –small programs: binary search, queens, quick sort –numerical benchmarks: Fast Fourier Transform, LU decomposition, Linpack

13. 13 Dualyzer and Blast [Henzinger-Jhala-et-al POPL02] Correct programs: can prove safety of bound checks? Faulty programs: can identify true bugs? Faulty Programs Dualyzer MayMust(secs) bsort012.44 initarr011.12 quick-sort012.24 sentinel011.31 Correct Programs BlastDualyzer Proved? (secs) Proved? (secs) bsearch*0.1√3.1 bsort*0.1√0.8 initarr√1.1√0.2 queens√3.4√1.4 quick-sort√28√1.5 sentinel*1.3*0.1 FFT*0.5√13.5 LU√7.2√14.3 SOR√2.1√3.5 Linpack*408√38.9

14. 14 Verisec Benchmark [Ku-Hart-Chechik-Lie ASE07] Small challenging testcases: –actual vulnerabilities from CVE database –corrected versions of these testcases Dualyzer found 2 unknown bugs in the corrected testcases: –off-by-one buffer error [SpamAssassin testcase] –non-termination bug [Samba testcase]

15. 15 Related Work Static analysis / Program testing / Model checking. Various goals: –proving safety. –finding bugs. –proving safety + finding bugs. AnalysisDirectionApproximationCompos.TerminatesGoal Suzuki et al. [POPL77], Xu et al. [PLDI00]BWunderNOYESsafety Cousot et al.[POPL78], ASTREE[PLDI03]FWoverNOYESsafety VeriSoft [POPL97]FWunderNO bugs Saturn [POPL05]FWover+underYES bugs(1) Slam [SPIN01], BLAST [POPL02]FW+BWover+symNO safety+bugs(1) SMASH [POPL10]BW+FWover+under+symYESNOsafety+bugs DualyzerFWoverYES safety+bugs(2) A bug that is reported indicates: a true bug (unconditionally): bugs either a true bug or a false positive: bugs(1) a true bug, or non-termination: bugs(2)

16. 16 Conclusion Dual analyses based on over-approximation –prove program safety –find bugs Find non-termination inputs Prototype implementation

17. Thank you for attention! Questions?

18. 18 Additional slides follow

19. 19 Example void g(int[] a, ref int x) { if (x<=0) then () else { a[x]:=0; //assert(0 <= x < a) x:=x-1; g(a,x) }}

20. 20 Constraint Abstractions - Dual Analysis void g(int[] a, ref int x) { if (x<=0) then () else { a[x]:=0; x:=x-1; g(a,x) }} Constraint abstractions: gOK(a,x,x') = (x · 0 Æ x'=x) Ç (x>0 Æ 0 · x<a Æ 9 x 1 ¢ (x 1 =x-1 Æ gOK(a,x 1,x'))) gERR(a,x) = x>0 Æ (x<0 Ç x ¸ a Ç 0 · x<a Æ 9 x 1 ¢ (x 1 =x-1 Æ gERR(a,x 1 ))) Results of fixpoint approximation: gOK(a,x,x') = (x · 0 Æ x'=x) Ç (1 · x < a Æ x'=0) gERR(a,x) = (x>0 Æ x ¸ a) Function summary: {gOK, gERR}. For general recursion, the ERR cons-abs. may depend on OK. OKERR

21. 21 Precise Error Tracing Report the label of calls that lead to a bug void foo(int x, int y) { if x=y then { l3: error } else () } void goo(int x) { l1: foo(x,x+1); l2: foo(x,3); } {OK: x  y, ERR.l3: x=y} {OK: x  3, ERR.l1.l3: false, ERR.l2.l3: x=3} OKERR.l2.l3 ERR.l1.l3

22. 22 Synergy : software model checker + testing –discovers a true BUG; –proves that the program is SAFE; –times-out during refinement loop (ABORT). Examples from SYNERGY [Gulavani-et-al FSE06] ProgramsSYNERGYDualyzer MayMust ex_fig1BUG01 ex_fig3SAFE00 ex_fig4BUG01 ex_fig6SAFE00 ex_fig7BUG01 ex_fig8SAFE00 ex_fig9ABORT01(LOOP)

23. 23 Related Work Static analysis / Program testing / Model checking. AnalysisDirectionApproximationCompos.TerminatesGoal Suzuki et al. [POPL77], Xu et al. [PLDI00]BWunderNOYESsafety Cousot et al.[POPL78], ASTREE[PLDI03]FWoverNOYESsafety VeriSoft [POPL97]FWunderNO bugs DART [PLDI05], EXE [CCS06]FWunder+symNO bugs Saturn [POPL05]FWover+underYES bugs(1) Syntox [PLDI93]FW+BWover+underNOYESsafety+bugs(2) Slam [SPIN01], BLAST [POPL02]FW+BWover+symNO safety+bugs(1) Synergy [FSE06]FW+BWover+under+symNO safety+bugs Gulwani [PLDI08]constr-based inv-generationYES safety+bugs SMASH [POPL10]BW+FWover+under+symYESNOsafety+bugs DualyzerFWoverYES safety+bugs(2)