1. Pre-verified IP? Metrics and methods to determine if your IP has been pre-verified’
Moshik Rubin

2. Agenda The end customer profile and the resulting challenges
Introduction to Metric Driven Verification with OVM
Metric Driven Verification IPs
Summary 2
September 19, 2018
dsfdsfdsafaf 2

3. Our Customer’s Customer And the Resulting Design Challenge
2010
25 Years
1985
Design a 2 lb. radio…
Over a 10 year period…
that will run for ½ hour…
and sell for $4000…
for use by New York stock brokers…
to raise their social status.
Design a 4 oz. radio/computer/camera/TV/stereo/GPS…
Over a 9 month period…
that will run for 2 days…
and sell for $0 (after rebate)…
to connect every human on the planet…
to instantly exchange every form of information:
voice, text, web, photo, video,…
dsfdsfdsafaf

4. Back then…Analog Piece Parts
What’s Inside?
Back then…Analog Piece Parts
2010
1985
dsfdsfdsafaf

5. …Every Conceivable Interface
What’s Inside? Today…
Everything You Got
…Every Conceivable Interface
DSI
Digital Serial
Interface
DSI Host
DSI Device
Display
Storage
Speaker
Keyboard
SLIMbus
Serial Low-power
Inter-chip Media bus
SLIMbus Host/Master
SLIMbus Device
CSI
Camera Serial
CSI Host
CSI Device
Camera
Microphone
UniPro
UniPro Host
UniPro Device
Application Processor
dsfdsfdsafaf

6. Really…More than You Got …All Your Partner’s IP too
What’s Inside?
Really…More than You Got
…All Your Partner’s IP too
Interfaces Make or Break:
Block Realization
SoC Realization
System Realization
DSI
Digital Serial
Interface
DSI Host
DSI Device
Display
Storage
Speaker
Keyboard
SLIMbus
Serial Low-power
Inter-chip Media bus
SLIMbus Host/Master
SLIMbus Device
CSI
Camera Serial
CSI Host
CSI Device
Camera
Microphone
UniPro
UniPro Host
UniPro Device
Application Processor
And you no longer have control over the design of these interfaces
dsfdsfdsafaf

7. Which Brings Us Back to the Verification Challenge…
How to Verify IP Protocol Compliance and Manage SoC-Level complexity?
Interfaces Make or Break:
Block Realization
SoC Realization
System Realization
Engine X
And you no longer have control over the design of these interfaces
Cadence Metric-Driven Verification Flow
Management requires Measurement
Measurement requires Metrics
Done
Signoff?
Measure & Analyze
Execute
Construct
Plan
Spec
Metric Definitions
OVM
VIP
Simulation, Formal, Acceleration
Failure and Metric Analysis
The Fuel that feeds the MDV Engine
Is Verification IP including:
- vPlans - Test Suites
- Coverage Maps - Protocol Checks
Creating all the above requires lots of:
- Protocol Expertise
- Time
“A 1 line change in the spec cost me 2 engineers for 6 months”
- Major company Verification Manager regarding SATA protocol
dsfdsfdsafaf

8. Agenda The end customer profile and the resulting challenges
Introduction to Metric Driven Verification with OVM
Metric Driven Verification IPs
Summary 8
September 19, 2018
dsfdsfdsafaf 8

9. Traditional Approach: Directed Testing
Verification engineer writes a directed test for each item in Test Plan:
Rework when design changes
DUT
Point out that the traditional Verif Planning process usually indicates what test cases need to be written; but the underlying reason for said test cases, should be due to design analysis. Usually the classical test plan will then just document what directed tests need to be written. The test plan should be the set of Goals to achieve, but most plans document the means by which to achieve those goals.
Since manual testing is tedious and labor-prone, the typical method of copy-pasting snippets in test cases results in relatively low variability from 1 test case to the next.
Automation Significant manual effort to write all the tests
Automation Manual effort needed to verify each goal was reached
Completeness No coverage of non-goal scenarios …especially problematic for unanticipated cases
Visibility No measures of functional coverage completeness
dsfdsfdsafaf

10. Metric-Driven Verification (MDV) Planning with unified verification metrics
Functional
specification . .
metric-based plan
Yes No
Done
Signoff?
Metric definitions
reuse
checks
assertions
coverage
OVM
VIP
Plan
Failure and metric analysis
Measure and analyze
Construct
Execute
[ Animation requires 5 manual clicks to fill out the regions around the circle.]
(Discuss the metric-driven verification story.)
(Emphasize the impact of coordinating “across” design teams and the improved ability to manage the project. Next slide will emphasize impact of consistent metrics from “specification to closure.”)
(Emphasize ability to include hardware/software co-simulation metrics as one way to help close the gaps of the traditional verification flow seen earlier.)
Simulation, formal, acceleration
dsfdsfdsafaf 10 10

11. Even for non- goal states!
Coverage Driven Verification Defining Coverage “Goals” Enables Automation
Focus moves to reaching goal areas (versus execution of test lists):
Add constraints to target a specific corner case
Simply changing seeds generates new stimulus
DUT
Constrained-random stimulus generation explores goal areas (& beyond)
Coverage shows which goals have been exercised and which need attention
Self-Checking ensures proper DUT response.
Automation – Constrained-random stimulus accelerates hitting coverage goals and exposing bugs. Coverage and checking results indicate effectiveness of each simulation, which enables scaling many parallel runs.
Even for non- goal states!
dsfdsfdsafaf

12. Agenda The end customer profile and the resulting challenges
Introduction to Metric Driven Verification with OVM
Metric Driven Verification IPs
Summary 12
September 19, 2018
dsfdsfdsafaf 12

13. From Specification to Coverage Metrics Through Verification Plan
Specification => Verification Plan => Coverage and checks Metrics
Synchronizes spec changes with verification plan
Functional
&
Design
Specs
Synchronizes verification plan changes with coverage metrics
Verification
Plan
Coverage
Metrics
Tracks which verification plan features have coverage metrics implemented
Copyright © 2009, PCI-SIG, All Rights Reserved

14. Standard Specification => Verification Plan (PCIe gen3 example)
Associated Checks
Associated Coverage
Associate Verification plan items with the spec annotations
Annotate the spec with areas to monitor with coverage, checks and specific testcases
Capture relevant meta data and location in spec to track changes
Copyright © 2009, PCI-SIG, All Rights Reserved

15. Verification Plan => Coverage Metrics (AMBA AXI example)
Bucket Analysis shows test detail & results
Graphical view of constrained-random test sequences
Protocol tests performed
(in this case, channel transitions)
Directly correlated to protocol specification
Identifies gaps in simulation regressions
Metrics grade verification progress
© 2009 Cadence Design Systems, Inc. All rights reserved.

16. Module => System with multiple vPlans
SoC top vPlan view
AXI Master vPlan
AXI Slave vPlan
PCIe vPlan
User specific vPlan
September 19, Cadence Confidential: Cadence Internal Use Only

17. Summary
Today’s SoC complexity requires integration of multiple Standard Interfaces IPs
Compliance to the standard and Interoperability testing requires advance protocol knowledge and high investment
Effective verification management requires metrics and measurable goals
MDV Verification IP reduces the protocol knowledge barrier and increase predictability and productivity
dsfdsfdsafaf