Introduction to Formal Property Verification (FPV) Erik Seligman CS 510, Lecture 8, January 2009

Agenda Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Agenda Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Definitions Assertion Assumption Cover Point Statement that must be true in all cases. Assumption Assertion treated as always-true constraint for FPV. Cover Point Condition that must be reachable for valid proof env Formal Property Verification (FPV) Mathematical proofs, not simulation Proves assertions: all possible test vectors

Motivation for Formal Property Verification Simulation: spot coverage of design space Formal Methods is a family of techniques for attaining very high confidence in the function of logic Formal Methods are static - and require no set-up of test cases. They furthermore establish correctness in a comprehensive way (when applicable) Improved verification quality, reduced cost

Motivation for Formal Property Verification Formal Property Verification (ideal case): full coverage of design space Simulation: spot coverage of design space Formal Methods is a family of techniques for attaining very high confidence in the function of logic Formal Methods are static - and require no set-up of test cases. They furthermore establish correctness in a comprehensive way (when applicable) Improved verification quality, reduced cost

Motivation for Formal Property Verification Formal Property Verification (ideal case): full coverage of design space Formal Property Verification (real life): full coverage in some areas Simulation: spot coverage of design space

Major Benefits of FPV for ASIC Projects Improving Design Process

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench)

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench) Find Corner Cases Missed in Simulation

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench) Find Corner Cases Missed in Simulation Quickly Verify Design Changes

Major Benefits of FPV for ASIC Projects Improving Design Process Force Designer to Think Through Logic Help Identify Hidden Assumptions Bug Hunting Unit-Level Validation (before testbench) Find Corner Cases Missed in Simulation Quickly Verify Design Changes “Peace of Mind”

Useful Assertions for FPV Focus on high-level intent Assertions = “executable comments” Add insight to design Micro-assert on a couple of RTL lines less useful assign foo = bar & baz; A1: assert property (foo == bar & baz); Don’t be afraid of some modeling code Auxiliary calculations / wires are fine Provide `ifdef to exclude from synthesis Full reference models in areas of concern Smaller “shadow models” often very useful

Agenda Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Prerequisites for FPV RTL design with assertions Clocks must be identified Critical since FPV runs over time Clocks are ‘special’: driven 1/0/1/0/… Need explicit ratios if multiple clocks Reset pattern must be identified FPV resets model to known state at start Common method: single rst signal (easy) More complex design may have reset sequence Hold RST 10 cycles, then PowerGood for 5 cycles, …

Verilog RTL with Assertions FPV Run Verilog RTL with Assertions FPV

Verilog RTL with Assertions FPV Run Verilog RTL with Assertions FPV Passing Assertions

Verilog RTL with Assertions Bounded-Passing Assertions FPV Run Verilog RTL with Assertions FPV Passing Assertions Bounded-Passing Assertions

Bounded vs Full Proofs Which of these do you need? “Assertion can NEVER be violated” “Assertion can never be violated by any possible simulation of length up to <n>” Bounded proof usually easier for tools Use cover point proofs to judge good bound Bound == lengths of interesting scenarios Some coverage lost vs full proofs But often at point of diminishing ROI Consider modifying starting state Fill queue at start of proof…?

Verilog RTL with Assertions Bounded-Passing Assertions FPV Run Verilog RTL with Assertions FPV Passing Assertions Bounded-Passing Assertions Failing Assertions

Verilog RTL with Assertions Bounded-Passing Assertions FPV Run Verilog RTL with Assertions FPV Unknown Assertions Passing Assertions Bounded-Passing Assertions Failing Assertions

Edit RTL: Fix bugs, add assumptions FPV Debug Loop Edit RTL: Fix bugs, add assumptions Verilog RTL with Assertions Passing Assertions Bounded-Passing Assertions FPV Failing Assertions Unknown Assertions Analyze Failures: RTL error, assertion error, or assumption needed?

Edit RTL: Fix bugs, add assumptions FPV Debug Loop Edit RTL: Fix bugs, add assumptions Verilog RTL with Assertions Passing Assertions Bounded-Passing Assertions FPV Failing Assertions Unknown Assertions Analyze Failures: RTL error, assertion error, or assumption needed? This is where FPVers spend their time!

Cover Points and FPV Cover Point is opposite of assertion Example “Good”: FPV creates example trace “Bad”: FPV proves point unreachable Also may aid simulation coverage checks Example cover property (opcode == `ADD); If it passes, FPV reports trace with ADD op If it fails, ADD op cannot exist in FPV env Maybe bad assumption prevents ADD op All proofs are suspect unless this was expected Add cover points when doing FPV! Good tools auto-generate some

Agenda Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Running Jasper Just run ‘JG’ from command line Can use –batch to run without GUI Runs are automatically logged See jgproject/jg.log GUI hints Right-clicking usually gets useful options Pass over button with cursor for name

Basic Jasper Command File # Load into JG using "source <file>.tcl". # load model analyze -clear analyze -sva traffic_start.v elaborate -top traffic # set clocks and resets clock -clear clock clk reset -clear reset rst # Set engine mode & run proofs set_engine_mode {H D B3 H2} prove -all

Proof Results

View Violation Trace How would this work for liveness properties, like a |-> ##[0:$] b?

Violation Trace: “Why”?

Violation Trace: Why? Again

Violation Trace for Liveness Trace shows possible infinite violation loop

Alternate JG Debug Tool: The Visualizer

Using The Visualizer

Visualizing Constraints

Visualize Options

Replot With Constraints

Adding More Constraints

Replot Again

Agenda Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Failing Assertions Your first FPV run on a block *will* fail Nobody writes right assumptions in advance! Always something you didn’t think of Thus most of FPVers time is debug This is OK– debug process gives insight Often debugging one assert can help identify other issues More assumptions improve counterexample

Assumption Creation Loop: Majority of FV Time Run FV Analyze Failures Add Assumes Mindset change from sim; prepare team! Early runs have many false negatives More assumptions == more interesting CEX Interesting bugs not found on first run Several rounds of assumes  deep traces Be sure to check assumptions too, in simulation or FV

Example: Adding an Assumption In one unit, many assertions failed Mid-transaction address changes Needed input assumption to prove Bug found when assumption fired in simulation! Address Bus RTL Under Test (several cycles per transaction) ASSUME (held for n cycles)

Assumption Count Exploding? Possible bad choice of boundary Effectively reimplementing neighbor block? Consider increasing hierarchy level Add upper level & many blackboxes? Also consider simplifying problem Only cover certain modes PCIE: prove for x16, not x4, x8? Restrict data Will one bit test most major logic? Are fully general payloads needed?

Example: FV too hard? MPE0 MRA0 MPE1 MRA1 MPE = Memory Protocol Engine MRA = Memory Read Arbiter

Correct Hierarchy Makes FV Easy MPE0 MRA0 MPE1 MRA1 MSB

Pick Your Battles FPV can be effort-intensive Need good understanding of requirements Should concentrate on high-risk areas FPV owner needs deep understanding Tool pokes at unusual behaviors, not typical Very different from simulation If not author of block, need to study intensely Block owner should be available for questions Don’t assign random intern to FPV!

Effective FPV At Intel (Info from public DAC 2008 talk) Chipsets: Found numerous bugs missed in simulation (PCIe, memory controllers) Also uncovered flaw in one validation env CPU designs expanding this usage mode Competitive: Recent project devoted 8% of validation resources in front-end design, found 8% of bugs 30-35% of bugs found by assertion FV were unlikely to be found in simulation

References / Further Reading http://www.aracnet.com/~eseligma/docs/dvcon_2006.pdf Jasper documentation in /pkgs/jasper/current/doc on ECE systems http://www.systemverilog.org/pdf/3b_AssertionsUserTechnology.pdf http://www.aycinena.com/index2/index3/iccd%202006%20verification%20panel.html http://oskitech.com/papers/datta-mc-vlsi08.pdf