Presentation is loading. Please wait.

Presentation is loading. Please wait.

Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT.

Published by Modified over 8 years ago

Similar presentations

Presentation on theme: "Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT."— Presentation transcript:

1 Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT

2 Verification and Validation, CS Dept, BYU2 Software Model Checking  Motivation Ariane 5 Comair debacle  Verifying Software Models A transition graph for the model is created A predefined property is verified ex. Reachability  Problem Number of behaviors is exponential with every increment This causes a state explosion problem

3 Verification and Validation, CS Dept, BYU3 Approaches  Traditional techniques to counter it Parallel or Distributed Model Checking Predicate Abstraction Disk based Algorithm Heuristics for Guided search  Heuristics Find a counterexample before memory runs out Property based heuristics Structure based heuristics  Structure of program can be use to guide the search

4 Verification and Validation, CS Dept, BYU4 Current Structural heuristics  Stefan Edelkamp and Tilman Mehler  Finds a short and easy to understand Error trail  Minimal operations to reach g from s is FSM distance  This distance is admissible and consistent  Build control flow graph (CFG) with just PC values  Willem Visser and Alex Groce  Specific only to Java

5 Verification and Validation, CS Dept, BYU5 01 main main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Underestimation Example

6 Verification and Validation, CS Dept, BYU6 01 02 main Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

7 Verification and Validation, CS Dept, BYU7 01 02 main foo Underestimation Example 06 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

8 Verification and Validation, CS Dept, BYU8 01 02 03 04 05 main foo error Underestimation Example 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

9 Verification and Validation, CS Dept, BYU9 error Underestimation Example 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG

10 Verification and Validation, CS Dept, BYU10 error Underestimation Example 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG

11 Verification and Validation, CS Dept, BYU11 error Underestimation Example 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG

12 Verification and Validation, CS Dept, BYU12 Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG error 01 02 03 04 05 main foo error 06 07 08

13 Verification and Validation, CS Dept, BYU13 Underestimation Example foo 3 steps main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts Edelkamp’s FSM Heuristic: Shortest Distance from current state to checking for error in the CFG error 01 02 03 04 05 main foo error 06 07 08

14 Verification and Validation, CS Dept, BYU14 True Distance should be …. error 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

15 Verification and Validation, CS Dept, BYU15 error 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be ….

16 Verification and Validation, CS Dept, BYU16 error 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be ….

17 Verification and Validation, CS Dept, BYU17 error 01 02 03 04 05 main foo error 06 07 08 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be ….

18 Verification and Validation, CS Dept, BYU18 Underestimation Example main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts error 01 02 03 04 05 main foo error 06 07 08

19 Verification and Validation, CS Dept, BYU19 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be …. error 01 02 03 04 05 main foo error 06 07 08

20 Verification and Validation, CS Dept, BYU20 main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be …. error 01 02 03 04 05 main foo error 06 07 08

21 Verification and Validation, CS Dept, BYU21 8 steps main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts True Distance should be …. error 01 02 03 04 05 main foo error 06 07 08

22 Verification and Validation, CS Dept, BYU22 Solution: Interprocedural CFG  All the nodes in the ICFG that are part of a subroutine will be indexed on two things PC Value Return address to where the subroutine will return when it encounters a return statement

23 Verification and Validation, CS Dept, BYU23 01 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

24 Verification and Validation, CS Dept, BYU24 01 (init) 02 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

25 Verification and Validation, CS Dept, BYU25 01 (init) 02 (init) 06 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

26 Verification and Validation, CS Dept, BYU26 01 (init) 02 (init) 06 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

27 Verification and Validation, CS Dept, BYU27 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

28 Verification and Validation, CS Dept, BYU28 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) 06 (05) 07 (05) 08 (05) 04 (init) 05 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

29 Verification and Validation, CS Dept, BYU29 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) 06 (05) 07 (05) 08 (05) 04 (init) 05 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts

30 Verification and Validation, CS Dept, BYU30 01 (init) 02 (init) 03 (init) 06 (03) 07 (03) 08 (03) 06 (05) 07 (05) 08 (05) 04 (init) 05 (init) main: 01: ldx #1 02: call foo 03: add x,1 04: call foo 05: check for error foo: 06: pshx 07: pulx 08: rts 8 steps

31 Verification and Validation, CS Dept, BYU31 Nested Function Calls  x → f → g  y → f → g  Same problem as before main: 1 call x 2 call y error f: 7 call g 8 rts g: 9 xyz a rts x: 3 call f 4 rts y: 5 call f 6 rts fgx 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main xfg y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error)

32 Verification and Validation, CS Dept, BYU32 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

33 Verification and Validation, CS Dept, BYU33 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

34 Verification and Validation, CS Dept, BYU34 Abstract states from the stack 08 04 02 PC: 09 0a (08) 08 (04) 04 (02) 02 (init) abstract states generated from the stack s a0 s a1 s a2 s a3

35 Verification and Validation, CS Dept, BYU35 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

36 Verification and Validation, CS Dept, BYU36 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

37 Verification and Validation, CS Dept, BYU37 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

38 Verification and Validation, CS Dept, BYU38 Marking returns statically foo prologue beq epilogue return

39 Verification and Validation, CS Dept, BYU39 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

40 Verification and Validation, CS Dept, BYU40 Improved ICFG Algorithm Calculating the Heuristic: D = 0 FSM ((a,8),error) = 4 fgx 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main xfg y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error) 08 04 02 PC: 09

41 Verification and Validation, CS Dept, BYU41 Improved ICFG Algorithm 08 04 02 PC: 09 Calculating the Heuristic: D = 0 FSM ((a,8),error) = 4 FSM ((a,8), (rts,8) = 1 1 < 4 D += 1 xfg 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error)

42 Verification and Validation, CS Dept, BYU42 Improved ICFG Algorithm ICFG_Algorithm(state S) s a0 = icfgState(S) // are abstracted states in the call stack of S d = 0 for i = 0 to n do s rtn = rtn(s ai ) if FSM(s ai, error) < FSM(s ai,s rtn ) then d = d + FSM(s ai,error) return d d = d + FSM(s ai, s rtn ) + 1 return d

43 Verification and Validation, CS Dept, BYU43 Improved ICFG Algorithm PC: 0a D = 11 1:call x (init) 3:call f (2) 7:call g (4) 9 (8) 2:call y (init) 5:call g (error) 7:call g (6) error main y f a:rts (8) 8:rts (4) 4:rts (2) 8:rts (6) 6:rts (error) xfg

44 Verification and Validation, CS Dept, BYU44 Results: Number of states generated BFSDFSFSMImproved ICFG Hyman’s mutex 4528700636481560 Naïve dining phil (threads) 47,2468062152,19614,140 Moody dining phil (threads) 225,26944,238555,60928,565 Lazy dining phil (threads) 317,13156,685>2.86 mil50,984 Bulls and cows 27,61328,014 28,007

45 Verification and Validation, CS Dept, BYU45 Conclusions  Small overhead allowed use of more static information  The Dynamic call stack with static analysis gave a better estimate  Testing shows an significant improvement in FSM distance  The Improved ICFG algorithm can be used on any graph  The algorithm is admissible and consistent

46 Verification and Validation, CS Dept, BYU46 QUESTIONS

Download ppt "Improving Error Discovery using Guided Search Neha Rungta & Eric Mercer Computer Science Department Brigham Young University, Provo UT."

Similar presentations

BEST FIRST SEARCH - BeFS

BEST FIRST SEARCH - BeFS
Hybrid BDD and All-SAT Method for Model Checking Orna Grumberg Joint work with Assaf Schuster and Avi Yadgar Technion – Israel Institute of Technology.

Hybrid BDD and All-SAT Method for Model Checking Orna Grumberg Joint work with Assaf Schuster and Avi Yadgar Technion – Israel Institute of Technology.
Lights Out Issues Questions? Comment from me.

Lights Out Issues Questions? Comment from me.
A Technique for Parallel Reachability Analysis of Java Programs Raghuraman R. Sridhar Iyer G. Sajith.

A Technique for Parallel Reachability Analysis of Java Programs Raghuraman R. Sridhar Iyer G. Sajith.
CS 267: Automated Verification Lecture 2: Linear vs. Branching time. Temporal Logics: CTL, CTL*. CTL model checking algorithm. Counter-example generation.

CS 267: Automated Verification Lecture 2: Linear vs. Branching time. Temporal Logics: CTL, CTL*. CTL model checking algorithm. Counter-example generation.
CS 267: Automated Verification Lecture 8: Automata Theoretic Model Checking Instructor: Tevfik Bultan.

CS 267: Automated Verification Lecture 8: Automata Theoretic Model Checking Instructor: Tevfik Bultan.
Course Outline Traditional Static Program Analysis Software Testing

Course Outline Traditional Static Program Analysis Software Testing
Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:

Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:
Abstraction and Modular Reasoning for the Verification of Software Corina Pasareanu NASA Ames Research Center.

Abstraction and Modular Reasoning for the Verification of Software Corina Pasareanu NASA Ames Research Center.
State Space 3 Chapter 4 Heuristic Search. Three Algorithms Backtrack Depth First Breadth First All work if we have well-defined: Goal state Start state.

State Space 3 Chapter 4 Heuristic Search. Three Algorithms Backtrack Depth First Breadth First All work if we have well-defined: Goal state Start state.
1 Symbolic Execution for Model Checking and Testing Corina Păsăreanu (Kestrel) Joint work with Sarfraz Khurshid (MIT) and Willem Visser (RIACS)

1 Symbolic Execution for Model Checking and Testing Corina Păsăreanu (Kestrel) Joint work with Sarfraz Khurshid (MIT) and Willem Visser (RIACS)
1/20 Generalized Symbolic Execution for Model Checking and Testing Charngki PSWLAB Generalized Symbolic Execution for Model Checking and Testing.

1/20 Generalized Symbolic Execution for Model Checking and Testing Charngki PSWLAB Generalized Symbolic Execution for Model Checking and Testing.
Carnegie Mellon University Java PathFinder and Model Checking of Programs Guillaume Brat, Dimitra Giannakopoulou, Klaus Havelund, Mike Lowry, Phil Oh,

Carnegie Mellon University Java PathFinder and Model Checking of Programs Guillaume Brat, Dimitra Giannakopoulou, Klaus Havelund, Mike Lowry, Phil Oh,
Rigorous Software Development CSCI-GA 3033-011 Instructor: Thomas Wies Spring 2012 Lecture 13.

Rigorous Software Development CSCI-GA Instructor: Thomas Wies Spring 2012 Lecture 13.
Analyzing Gene Relationships for Down Syndrome with Labeled Transition Graphs Neha Rungta, Hyrum Carroll, Eric Mercer, Mark Clement, and Quinn Snell Computer.

Analyzing Gene Relationships for Down Syndrome with Labeled Transition Graphs Neha Rungta, Hyrum Carroll, Eric Mercer, Mark Clement, and Quinn Snell Computer.
CS 267: Automated Verification Lecture 10: Nested Depth First Search, Counter- Example Generation Revisited, Bit-State Hashing, On-The-Fly Model Checking.

CS 267: Automated Verification Lecture 10: Nested Depth First Search, Counter- Example Generation Revisited, Bit-State Hashing, On-The-Fly Model Checking.
Introducing BLAST Software Verification John Gallagher CS4117.

Introducing BLAST Software Verification John Gallagher CS4117.
SMU SRG reading by Tey Chee Meng: Automatic Patch-Based Exploit Generation is Possible: Techniques and Implications by David Brumley, Pongsin Poosankam,

SMU SRG reading by Tey Chee Meng: Automatic Patch-Based Exploit Generation is Possible: Techniques and Implications by David Brumley, Pongsin Poosankam,
1 of 24 Automatic Extraction of Object-Oriented Observer Abstractions from Unit-Test Executions Dept. of Computer Science & Engineering University of Washington,

1 of 24 Automatic Extraction of Object-Oriented Observer Abstractions from Unit-Test Executions Dept. of Computer Science & Engineering University of Washington,
CS171 Introduction to Computer Science II Graphs Strike Back.

CS171 Introduction to Computer Science II Graphs Strike Back.

Similar presentations

Ads by Google