Download presentation
Presentation is loading. Please wait.
1
Introduction to Formal Property Verification (FPV)
Erik Seligman CS 510, Lecture 8, January 2009
2
Agenda Introduction To FPV The FPV Process Running FPV Using Jasper
FPV Hints
3
Agenda Introduction To FPV The FPV Process Running FPV Using Jasper
FPV Hints
4
Definitions Assertion Assumption Cover Point
Statement that must be true in all cases. Assumption Assertion treated as always-true constraint for FPV. Cover Point Condition that must be reachable for valid proof env Formal Property Verification (FPV) Mathematical proofs, not simulation Proves assertions: all possible test vectors
5
Motivation for Formal Property Verification
Simulation: spot coverage of design space Formal Methods is a family of techniques for attaining very high confidence in the function of logic Formal Methods are static - and require no set-up of test cases. They furthermore establish correctness in a comprehensive way (when applicable) Improved verification quality, reduced cost
6
Motivation for Formal Property Verification
Formal Property Verification (ideal case): full coverage of design space Simulation: spot coverage of design space Formal Methods is a family of techniques for attaining very high confidence in the function of logic Formal Methods are static - and require no set-up of test cases. They furthermore establish correctness in a comprehensive way (when applicable) Improved verification quality, reduced cost
7
Motivation for Formal Property Verification
Formal Property Verification (ideal case): full coverage of design space Formal Property Verification (real life): full coverage in some areas Simulation: spot coverage of design space
8
Major Benefits of FPV for ASIC Projects
Improving Design Process
9
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic
10
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions
11
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting
12
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench)
13
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench) Find Corner Cases Missed in Simulation
14
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench) Find Corner Cases Missed in Simulation Quickly Verify Design Changes
15
Major Benefits of FPV for ASIC Projects
Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench) Find Corner Cases Missed in Simulation Quickly Verify Design Changes “Peace of Mind”
16
Useful Assertions for FPV
Focus on high-level intent Assertions = “executable comments” Add insight to design Micro-assert on a couple of RTL lines less useful assign foo = bar & baz; A1: assert property (foo == bar & baz); Don’t be afraid of some modeling code Auxiliary calculations / wires are fine Provide `ifdef to exclude from synthesis Full reference models in areas of concern Smaller “shadow models” often very useful
17
Agenda Introduction To FPV The FPV Process Running FPV Using Jasper
FPV Hints
18
Prerequisites for FPV RTL design with assertions
Clocks must be identified Critical since FPV runs over time Clocks are ‘special’: driven 1/0/1/0/… Need explicit ratios if multiple clocks Reset pattern must be identified FPV resets model to known state at start Common method: single rst signal (easy) More complex design may have reset sequence Hold RST 10 cycles, then PowerGood for 5 cycles, …
19
Verilog RTL with Assertions
FPV Run Verilog RTL with Assertions FPV
20
Verilog RTL with Assertions
FPV Run Verilog RTL with Assertions FPV Passing Assertions
21
Verilog RTL with Assertions Bounded-Passing Assertions
FPV Run Verilog RTL with Assertions FPV Passing Assertions Bounded-Passing Assertions
22
Bounded vs Full Proofs Which of these do you need?
“Assertion can NEVER be violated” “Assertion can never be violated by any possible simulation of length up to <n>” Bounded proof usually easier for tools Use cover point proofs to judge good bound Bound == lengths of interesting scenarios Some coverage lost vs full proofs But often at point of diminishing ROI Consider modifying starting state Fill queue at start of proof…?
23
Verilog RTL with Assertions Bounded-Passing Assertions
FPV Run Verilog RTL with Assertions FPV Passing Assertions Bounded-Passing Assertions Failing Assertions
24
Verilog RTL with Assertions Bounded-Passing Assertions
FPV Run Verilog RTL with Assertions FPV Unknown Assertions Passing Assertions Bounded-Passing Assertions Failing Assertions
25
Edit RTL: Fix bugs, add assumptions
FPV Debug Loop Edit RTL: Fix bugs, add assumptions Verilog RTL with Assertions Passing Assertions Bounded-Passing Assertions FPV Failing Assertions Unknown Assertions Analyze Failures: RTL error, assertion error, or assumption needed?
26
Edit RTL: Fix bugs, add assumptions
FPV Debug Loop Edit RTL: Fix bugs, add assumptions Verilog RTL with Assertions Passing Assertions Bounded-Passing Assertions FPV Failing Assertions Unknown Assertions Analyze Failures: RTL error, assertion error, or assumption needed? This is where FPVers spend their time!
27
Cover Points and FPV Cover Point is opposite of assertion Example
“Good”: FPV creates example trace “Bad”: FPV proves point unreachable Also may aid simulation coverage checks Example cover property (opcode == `ADD); If it passes, FPV reports trace with ADD op If it fails, ADD op cannot exist in FPV env Maybe bad assumption prevents ADD op All proofs are suspect unless this was expected Add cover points when doing FPV! Good tools auto-generate some
28
Agenda Introduction To FPV The FPV Process Running FPV Using Jasper
FPV Hints
29
Running Jasper Just run ‘JG’ from command line
Can use –batch to run without GUI Runs are automatically logged See jgproject/jg.log GUI hints Right-clicking usually gets useful options Pass over button with cursor for name
30
Basic Jasper Command File
# Load into JG using "source <file>.tcl". # load model analyze -clear analyze -sva traffic_start.v elaborate -top traffic # set clocks and resets clock -clear clock clk reset -clear reset rst # Set engine mode & run proofs set_engine_mode {H D B3 H2} prove -all
31
Proof Results
32
View Violation Trace How would this work for liveness properties, like a |-> ##[0:$] b?
![View Violation Trace How would this work for liveness properties, like a |-> ##[0:$] b](http://slideplayer.com/slide/2559219/9/images/32/View+Violation+Trace+How+would+this+work+for+liveness+properties%2C+like+a+%7C-%3E+%23%23%5B0%3A%24%5D+b.jpg "View Violation Trace How would this work for liveness properties, like a |-> ##[0:$] b")
33
Violation Trace: “Why”?
34
Violation Trace: Why? Again
35
Violation Trace for Liveness
Trace shows possible infinite violation loop
36
Alternate JG Debug Tool: The Visualizer
37
Using The Visualizer
38
Visualizing Constraints
39
Visualize Options
40
Replot With Constraints
41
Adding More Constraints
42
Replot Again
43
Agenda Introduction To FPV The FPV Process Running FPV Using Jasper
FPV Hints
44
Failing Assertions Your first FPV run on a block *will* fail
Nobody writes right assumptions in advance! Always something you didn’t think of Thus most of FPVers time is debug This is OK– debug process gives insight Often debugging one assert can help identify other issues More assumptions improve counterexample
45
Assumption Creation Loop: Majority of FV Time
Run FV Analyze Failures Add Assumes Mindset change from sim; prepare team! Early runs have many false negatives More assumptions == more interesting CEX Interesting bugs not found on first run Several rounds of assumes deep traces Be sure to check assumptions too, in simulation or FV
46
Example: Adding an Assumption
In one unit, many assertions failed Mid-transaction address changes Needed input assumption to prove Bug found when assumption fired in simulation! Address Bus RTL Under Test (several cycles per transaction) ASSUME (held for n cycles)
47
Assumption Count Exploding?
Possible bad choice of boundary Effectively reimplementing neighbor block? Consider increasing hierarchy level Add upper level & many blackboxes? Also consider simplifying problem Only cover certain modes PCIE: prove for x16, not x4, x8? Restrict data Will one bit test most major logic? Are fully general payloads needed?
48
Example: FV too hard? MPE0 MRA0 MPE1 MRA1 MPE = Memory Protocol Engine
MRA = Memory Read Arbiter
49
Correct Hierarchy Makes FV Easy
MPE0 MRA0 MPE1 MRA1 MSB
50
Pick Your Battles FPV can be effort-intensive
Need good understanding of requirements Should concentrate on high-risk areas FPV owner needs deep understanding Tool pokes at unusual behaviors, not typical Very different from simulation If not author of block, need to study intensely Block owner should be available for questions Don’t assign random intern to FPV!
51
Effective FPV At Intel (Info from public DAC 2008 talk)
Chipsets: Found numerous bugs missed in simulation (PCIe, memory controllers) Also uncovered flaw in one validation env CPU designs expanding this usage mode Competitive: Recent project devoted 8% of validation resources in front-end design, found 8% of bugs 30-35% of bugs found by assertion FV were unlikely to be found in simulation
52
References / Further Reading
Jasper documentation in /pkgs/jasper/current/doc on ECE systems
Similar presentations
