Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT.

Slides:

Advertisements

Similar presentations

BEST FIRST SEARCH - BeFS

Advertisements

Hybrid BDD and All-SAT Method for Model Checking Orna Grumberg Joint work with Assaf Schuster and Avi Yadgar Technion – Israel Institute of Technology.

Lights Out Issues Questions? Comment from me.

A Technique for Parallel Reachability Analysis of Java Programs Raghuraman R. Sridhar Iyer G. Sajith.

CS 267: Automated Verification Lecture 2: Linear vs. Branching time. Temporal Logics: CTL, CTL*. CTL model checking algorithm. Counter-example generation.

CS 267: Automated Verification Lecture 8: Automata Theoretic Model Checking Instructor: Tevfik Bultan.

Course Outline Traditional Static Program Analysis Software Testing

Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:

Abstraction and Modular Reasoning for the Verification of Software Corina Pasareanu NASA Ames Research Center.

State Space 3 Chapter 4 Heuristic Search. Three Algorithms Backtrack Depth First Breadth First All work if we have well-defined: Goal state Start state.

1 Symbolic Execution for Model Checking and Testing Corina Păsăreanu (Kestrel) Joint work with Sarfraz Khurshid (MIT) and Willem Visser (RIACS)

1/20 Generalized Symbolic Execution for Model Checking and Testing Charngki PSWLAB Generalized Symbolic Execution for Model Checking and Testing.

Carnegie Mellon University Java PathFinder and Model Checking of Programs Guillaume Brat, Dimitra Giannakopoulou, Klaus Havelund, Mike Lowry, Phil Oh,

Rigorous Software Development CSCI-GA Instructor: Thomas Wies Spring 2012 Lecture 13.

Analyzing Gene Relationships for Down Syndrome with Labeled Transition Graphs Neha Rungta, Hyrum Carroll, Eric Mercer, Mark Clement, and Quinn Snell Computer.

CS 267: Automated Verification Lecture 10: Nested Depth First Search, Counter- Example Generation Revisited, Bit-State Hashing, On-The-Fly Model Checking.

Introducing BLAST Software Verification John Gallagher CS4117.

SMU SRG reading by Tey Chee Meng: Automatic Patch-Based Exploit Generation is Possible: Techniques and Implications by David Brumley, Pongsin Poosankam,

1 of 24 Automatic Extraction of Object-Oriented Observer Abstractions from Unit-Test Executions Dept. of Computer Science & Engineering University of Washington,

CS171 Introduction to Computer Science II Graphs Strike Back.

Assembly Code Verification Using Model Checking Hao XIAO Singapore University of Technology and Design.

SAT and Model Checking. Bounded Model Checking (BMC) A.I. Planning problems: can we reach a desired state in k steps? Verification of safety properties:

The Software Model Checker BLAST by Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala and Rupak Majumdar Presented by Yunho Kim Provable Software Lab, KAIST.

Testing and Analysis of Device Drivers Supervisor: Abhik Roychoudhury Author: Pham Van Thuan 1.

Thread-modular Abstraction Refinement Tom Henzinger Ranjit Jhala Rupak Majumdar [UC Berkeley] Shaz Qadeer [Microsoft Research]

Research Related to Real-Time Strategy Games Robert Holte November 8, 2002.

FunState – An Internal Design Representation for Codesign A model that enables representations of different types of system components. Mixture of functional.

Penn ESE 535 Spring DeHon 1 ESE535: Electronic Design Automation Day 22: April 23, 2008 FSM Equivalence Checking.

1 Completeness and Complexity of Bounded Model Checking.

Efficient Software Model Checking of Data Structure Properties Paul T. Darga Chandrasekhar Boyapati The University of Michigan.

272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 4: SMT-based Bounded Model Checking of Concurrent Software.

Data Structures and Programming.  John Edgar2.

Thread-modular Abstraction Refinement Thomas A. Henzinger, et al. CAV 2003 Seonggun Kim KAIST CS750b.

Verifying Concurrent Message- Passing C Programs with Recursive Calls Sagar Chaki, Edmund Clarke, Nicholas Kidd, Thomas Reps, and Tayssir Touili.

Formal Verification of fFSM Model Sachoun Park, Gihwon Kwon Department of Computer Science Kyonggi University, Korea IWFST, Shanghai, China,

Graphs II Robin Burke GAM 376. Admin Skip the Lua topic.

PRESTO: Program Analyses and Software Tools Research Group, Ohio State University STATIC ANALYSES FOR JAVA IN THE PRESENCE OF DISTRIBUTED COMPONENTS AND.

Parallel and Distributed Computing in Model Checking Diana DUBU (UVT) Dana PETCU (IeAT, UVT)

Department of Computer Science A Static Program Analyzer to increase software reuse Ramakrishnan Venkitaraman and Gopal Gupta.

1 A Static Analysis Approach for Automatically Generating Test Cases for Web Applications Presented by: Beverly Leung Fahim Rahman.

A* Lasso for Learning a Sparse Bayesian Network Structure for Continuous Variances Jing Xiang & Seyoung Kim Bayesian Network Structure Learning X 1...

Distributed Verification of Multi-threaded C++ Programs Stefan Edelkamp joint work with Damian Sulewski and Shahid Jabbar.

Encoded PC: Self Protection from Buffer Overflow Attacks Akhilesh Tyagi Depts: Elec. & Computer Engg; Computer Science Iowa State University.

EXAMPLE 3 Find the inverse of a 3 × 3 matrix Use a graphing calculator to find the inverse of A. Then use the calculator to verify your result. 2 1 – 2.

Yang Liu, Jun Sun and Jin Song Dong School of Computing National University of Singapore.

Inferring Specifications to Detect Errors in Code Mana Taghdiri Presented by: Robert Seater MIT Computer Science & AI Lab.

Race Checking by Context Inference Tom Henzinger Ranjit Jhala Rupak Majumdar UC Berkeley.

P ROBLEM Write an algorithm that calculates the most efficient route between two points as quickly as possible.

Type Systems CS Definitions Program analysis Discovering facts about programs. Dynamic analysis Program analysis by using program executions.

Lazy Abstraction Jinseong Jeon ARCS, KAIST CS750b, KAIST2/26 References Lazy Abstraction –Thomas A. Henzinger et al., POPL ’02 Software verification.

Lecture 7: Uninformed Search Dr John Levine Algorithms and Complexity February 13th 2006.

Development of Formally Verified Erlang Programs a case study Thomas Arts Clara Benac Earle Computer Science Lab Stockholm, Sweden.

External Program Model Checking Stefan Edelkamp, Shahid Jabar, Dino Midzic, Daniel Rikowski and Damian Sulewski Computer Science Department University.

Model Checking Java Programs using Structural Heuristics

Symbolic Execution with Abstract Subsumption Checking Saswat Anand College of Computing, Georgia Institute of Technology Corina Păsăreanu QSS, NASA Ames.

Parallel External Directed Model Checking with Linear I/O Shahid Jabbar Stefan Edelkamp Computer Science Department University of Dortmund, Dortmund, Germany.

OPTIMIZING DSP SCHEDULING VIA ADDRESS ASSIGNMENT WITH ARRAY AND LOOP TRANSFORMATION Chun Xue, Zili Shao, Ying Chen, Edwin H.-M. Sha Department of Computer.

Model Checking for Simple Java Programs Taehoon Lee, Gihwon Kwon Department of Computer Science Kyonggi University, Korea IWFST, Shanghai, China,

I/O Efficient Directed Model Checking Shahid Jabbar and Stefan Edelkamp, Computer Science Department University of Dortmund, Germany.

Informed Search CSE 473 University of Washington.

/ PSWLAB Thread Modular Model Checking by Cormac Flanagan and Shaz Qadeer (published in Spin’03) Hong,Shin Thread Modular Model.

/ PSWLAB Evidence-Based Analysis and Inferring Preconditions for Bug Detection By D. Brand, M. Buss, V. C. Sreedhar published in ICSM 2007.

Large Scale External Directed Liveness Checking Stefan Edelkamp Shahid Jabbar Computer Science Department University of Dortmund, Dortmund, Germany.

Zaiben Chen et al. Presented by Lian Liu. You’re traveling from s to t. Which gas station would you choose?

Hybrid BDD and All-SAT Method for Model Checking

Operating Systems (CS 340 D)

Neha Rungta and Eric G. Mercer Software Model Checking Lab

Abstraction, Verification & Refinement

Predicate Abstraction

3-2 Solving Inequalities Using Addition and Subtraction

Presentation transcript:

Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT

Verification and Validation, CS Dept, BYU2 Software Model Checking  Motivation Ariane 5 Comair debacle  Verifying Software Models A transition graph for the model is created A predefined property is verified ex. Reachability  Problem Number of behaviors is exponential with every increment This causes a state explosion problem

Verification and Validation, CS Dept, BYU3 Approaches  Traditional techniques to counter it Parallel or Distributed Model Checking Predicate Abstraction Disk based Algorithm Heuristics for Guided search  Heuristics Find a counterexample before memory runs out Property based heuristics Structure based heuristics  Structure of program can be use to guide the search

Verification and Validation, CS Dept, BYU4 Current Structural heuristics  Stefan Edelkamp and Tilman Mehler  Finds a short and easy to understand Error trail  Minimal operations to reach g from s is FSM distance  This distance is admissible and consistent  Build control flow graph (CFG) with just PC values  Willem Visser and Alex Groce  Specific only to Java

Verification and Validation, CS Dept, BYU5 01 main main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Underestimation Example

Verification and Validation, CS Dept, BYU main Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU main foo Underestimation Example 06 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU main foo error Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU9 error Underestimation Example main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG

Verification and Validation, CS Dept, BYU10 error Underestimation Example main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG

Verification and Validation, CS Dept, BYU11 error Underestimation Example main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG

Verification and Validation, CS Dept, BYU12 Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG error main foo error

Verification and Validation, CS Dept, BYU13 Underestimation Example foo 3 steps main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG error main foo error

Verification and Validation, CS Dept, BYU14 True Distance should be …. error main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU15 error main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be ….

Verification and Validation, CS Dept, BYU16 error main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be ….

Verification and Validation, CS Dept, BYU17 error main foo error main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be ….

Verification and Validation, CS Dept, BYU18 Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts error main foo error

Verification and Validation, CS Dept, BYU19 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be …. error main foo error

Verification and Validation, CS Dept, BYU20 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be …. error main foo error

Verification and Validation, CS Dept, BYU21 8 steps main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be …. error main foo error

Verification and Validation, CS Dept, BYU22 Solution: Interprocedural CFG  All the nodes in the ICFG that are part of a subroutine will be indexed on two things PC Value Return address to where the subroutine will return when it encounters a return statement

Verification and Validation, CS Dept, BYU23 01 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU24 01 (init) 02 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU25 01 (init) 02 (init) 06 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU26 01 (init) 02 (init) 06 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU27 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU28 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) 06 (05) 07 (05) 08 (05) 04 (init) 05 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU29 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) 06 (05) 07 (05) 08 (05) 04 (init) 05 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

Verification and Validation, CS Dept, BYU30 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) 06 (05) 07 (05) 08 (05) 04 (init) 05 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts 8 steps

Verification and Validation, CS Dept, BYU31 Nested Function Calls  x → f → g  y → f → g  Same problem as before main: 1 call x 2 call y error f: 7 call g 8 rts g: 9 xyz a rts x: 3 call f 4 rts y: 5 call f 6 rts fgx 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main xfg y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error)

Verification and Validation, CS Dept, BYU32 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU33 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU34 Abstract states from the stack PC: 09 0a (08) 08 (04) 04 (02) 02 (init) abstract states generated from the stack s a0 s a1 s a2 s a3

Verification and Validation, CS Dept, BYU35 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU36 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU37 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU38 Marking returns statically foo prologue beq epilogue return

Verification and Validation, CS Dept, BYU39 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU40 Improved ICFG Algorithm Calculating the Heuristic: D = 0 FSM ((a,8),error) = 4 fgx 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main xfg y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error) PC: 09

Verification and Validation, CS Dept, BYU41 Improved ICFG Algorithm PC: 09 Calculating the Heuristic: D = 0 FSM ((a,8),error) = 4 FSM ((a,8), (rts,8) = 1 1 < 4 D += 1 xfg 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error)

Verification and Validation, CS Dept, BYU42 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

Verification and Validation, CS Dept, BYU43 Improved ICFG Algorithm PC: 0a D = 11 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error) xfg

Verification and Validation, CS Dept, BYU44 Results: Number of states generated BFSDFSFSMImproved ICFG Hyman’s mutex Naïve dining phil (threads) 47, ,19614,140 Moody dining phil (threads) 225,26944,238555,60928,565 Lazy dining phil (threads) 317,13156,685>2.86 mil50,984 Bulls and cows 27,61328,014 28,007

Verification and Validation, CS Dept, BYU45 Conclusions  Small overhead allowed use of more static information  The Dynamic call stack with static analysis gave a better estimate  Testing shows an significant improvement in FSM distance  The Improved ICFG algorithm can be used on any graph  The algorithm is admissible and consistent

Verification and Validation, CS Dept, BYU46 QUESTIONS