Verification & Validation By: Amir Masoud Gharehbaghi

2 V&V Techniques Outline Overview Selective History Theorem Proving Model Checking Hardware Verification Assertion-Based Verification Conclusions

3 V&V Techniques What is Verification (Validation) Functional Verification:  Task of establishing that a given design accurately implements the intended behavior

4 V&V Techniques Verification&Validation Techniques Simulation-based  Apply inputs to design, simulate (or run), and check the results Formal  Mathematically proof the correctness of system against the properties Semi-formal  Combine simulation and formal verification

5 V&V Techniques V&V Techniques Comparison Simulation-based Easy to use Fast  Low coverage Formal Perfect coverage  Not easy to use  Not applicable for large designs

6 V&V Techniques Using Formal Methods Writing formal specifications Proving properties about the specification  Theorem proving Deriving implementation from a given specification  Refinement Verifying properties for a given implementation  Property checking

7 V&V Techniques Selective History Early 1960’s  Suggestions: McCarthy and Dijkstra Late 1960’s and Early 1970’s  Proof systems: Floyd-Hoar, Boyer-Moore,… Late 1970’s  Temporal Logic for reactive systems (Pnueli, …) Early 1980’s  Model checking (Clarke, Emerson, …) Late 1980’s  Symbolic model checking using BDDs

8 V&V Techniques Selective History (cont.) 1990’s -> mostly hardware  Non-BDD based model checking  Satisfiability  Equivalence checking  Symbolic simulation & symbolic trajectory evaluation 2000’s  Assertion-based verification  Software model checking  Probabilistic verification  Automated theorem proving  Hybrid systems verification

9 V&V Techniques Theorem Proving Formally specify the system in a logic system Formally specify the properties of system Prove the correctness of properties of system in a proof system

10 V&V Techniques Theorem Provers & Logic Systems First-Order Logic  ACL2  Nqthm High-Order Logic  HOL  PVS

11 V&V Techniques Temporal Logic First-Order Logic + Temporal Operations Linear Temporal Logic (LTL) Computational Tree Logic (CTL)

12 V&V Techniques LTL Temporal Operations X: next F: finally G: globally U: until

13 V&V Techniques LTL Example p  X p alert  F halt G (alert  F halt ) G (alert  (alarm U halt ) )

14 V&V Techniques CTL Path Operations A: always E: there exists Combine with temporal operations of LTL:  AX, AF, AG, AU  EX, EF, EG, EU

15 V&V Techniques CTL Example AG p AF halt E ( alaram U halt ) AX alarm  EF close

16 V&V Techniques Properties Safety  Something never occurs. Liveness  Something will ultimately occur. Reachability  Some particular situation can be reached. Fairness  Something will (not) occur infinitely often. Properties are checked under certain conditions

17 V&V Techniques Model Checking Model is a state machine (or automata) Property is defined in a temporal logic CTL model checking  O(|Q| * |p|) LTL model checking  O (|Q| * 2^|p| )  |Q|: number of states  |p|: number of sub-formulas in property

18 V&V Techniques State Explosion Problem Both in LTL and CTL:  An automata is generated (explicitly) Number of states grow exponentially

19 V&V Techniques Symbolic Model Checking Symbolically (implicitly) represent states (and transition between states) Use Binary Decision Diagram (BDD) to represent state variables Uses CTL properties

20 V&V Techniques Reduced Order BDD (ROBDD) A directed acyclic graph (DAG) with two leaf nodes (1,0) Represent Boolean functions  Compact  Canonical  Efficient operations (linear or quadratic)  Simple to use

21 V&V Techniques Satisfiability (SAT) Checking Satisfiability Checking:  Check existence of a combination of values for a Boolean function that function is 1 Check that ~f is unsatisfiable

22 V&V Techniques Bounded Model Checking Search for counter example by unfolding system in time until a bound is reached. Use SAT checkers What about unbounded model checking?

23 V&V Techniques Symbolic Simulation Simulate with symbolic (not explicit) values.  Inputs: expressions  Outputs: expressions Originally based on BDD.

24 V&V Techniques Symbolic Trajectory Evaluation Check properties of A=>C form.  A: input variables’ values over time  C: expected output variables’ values over time Symbolically simulate with given input values (A). Check that expected results (C) to be compatible (subset of) simulated output results.

25 V&V Techniques Equivalence Checking Check equivalent behavior between two designs  Same level of abstraction  Different levels of abstraction Combinational Sequential

26 V&V Techniques Equivalence Checking Methods Combinational  Compare the canonical representation of two circuits. (may be not feasible)  Use SAT checker  … Sequential  Find equivalent FFs and Compare combinational circuits between them.  Construct the multiplicative state machine and check the equivalency of outputs in all states.  Bounded model checking

27 V&V Techniques Assertion-Based Verification Assertion: property Do property checking during simulation  Embed in design  Check in run-time

28 V&V Techniques Assertion Languages OVL: Open Verification library PSL: Property Specification Language  Formerly “Sugar” SystemVerilog …

29 V&V Techniques Coverage Percentage of design covered during simulation  Code Statement Path Condition …  Signal  …

30 V&V Techniques Conclusions Verification is a serious bottleneck for current designs  Up to 80 percent of design time Formal methods cannot be applied to real designs Simulation cannot guarantee correctness of designs Embedded system verification containing Hw/Sw requires new techniques