Interactive 2-D Projection Cross Coverage Viewer for Coverage Hole Analysis Keisuke Shimizu ClueLogic Tomohisa Iida Verifore, Inc. Tomohisa Iida Verifore, Inc.
Outline Introduction and motivation Related work Cross coverage and views Our 2-D viewer “Room for improvement” factor Results Limitations and future improvement Summary
Introduction Constrained-random verification (CRV) methodology such as VMM and OVM becomes widely accepted The random nature of the methodology requires functional coverage metrics to monitor what has been tested Reaching 100% coverage is one of the goals Identifying the root cause of the coverage holes is crucial Especially when dealing with multi-dimensional cross coverage that tends to have a huge coverage space
Motivation Synopsys Unified Report Generator (URG) presents cross coverage comprehensively, but Not easy to grasp the big picture of the coverage Not easy to recognize hidden patterns of the coverage holes URG focuses on displaying the entire coverage results We wanted a tool to discover coverage holes and their patterns
Related Work Hole analysis for functional coverage data [Lachish, DAC ‘02] Defining coverage views to improve functional coverage analysis [Asaf, DAC ‘04] Selection Projection Grouping We focus on interactivity so that the users can see the coverage results from different view points
Cross Coverage Example Cross coverage with 3 coverage points Each coverage point has 4 bins This makes 4x4x4 cross bins
Coverage Hole Analysis Determine the areas that have not been tested Is there a pattern here?
Coverage View: List View Exhaustive, but loses patterns This is what URG basically provides List View
List View: URG URG puts coverage holes together URG
Coverage View: 2-D Projection Add bin counts of coverage points you do not focus on Projection
Coverage View: 2-D Projection Collapse multiple dimensions into two Projection
Coverage View: 2-D Projection We get C(n, 2) projections, where n is # of coverpoints in a cross Projection
Coverage View: Bin Selection Another way to see the coverage point relation Selection
Coverage View: Bin Selection Select a bin of a coverage point Selection
Coverage View: Bin Selection Can be combined with the 2-D projection Selection
How the 2-D Viewer Works
Coverage Example Design under Test SoC interconnect
Verification Environment VMM-based CRV
Coverage Data 256 coverage group instances 196 cross coverage instances 4,789 coverage bins in total (excluding illegal bins) Cross coverage example: Initiator vs. Target vs. Direction (Read/Write) 36x26x2 bins (including illegal bins)
Projection: Initiator vs. Target
Projection: Initiator vs. Direction
Projection: Target vs. Direction
Which coverage group to tackle first? Motivation Now that we can analyze coverage holes more efficiently But which coverage group should we look at first? Traditional “coverage score” measures how many bins are covered within a coverage group This metric does not show relative importance on the overall coverage
Coverage Group Comparison Which coverage group is more important? Coverage report displays the coverage group A to the top because it is least covered, but… Coverage GroupABC Number of total bins Number of bins hit28136 Coverage score20%40%80%
“Room for Improvement” Factor Coverage GroupABC Number of total bins Number of bins hit28136 Coverage score20%40%80% “Room for Improvement” factor4%6%17% Coverage score shows coverage group A is least covered, but … Coverage group A would increase overall coverage merely by 4%, whereas coverage group C would increase by 17% Assuming W g =number of bins in coverage group g (see next page)
The Factor Definition SShows the impact on the overall coverage “How many more percentages the overall coverage would increase if this coverage group were covered ” C g is the score of coverage group g in percent g set of coverage groups W g is the weight associated with coverage group g
Actual Data The coverage group having the lowest coverage score does not necessarily mean the one that has the most impact on the overall coverage We need both metrics
Results Using the 2-D Viewer, we found several verification/coverage bugs including: Illegal access to targets Boundary crossing Access to a reserved area Missing burst length values Illegal byte enable patterns
Byte Enable Coverage
Limitations There exists non-obvious coverage holes New approach is required to uncover the root causes of the hard to spot holes Currently only VCS coverage database is supported
Future Improvement Rewrite the 2-D Viewer preprocessor using Unified Coverage Database API UCAPI – to support VCS without worrying about future database format changes UCDB API – to support simulators other than VCS Add “grouping feature” of coverage bins 3-D Viewer?
Summary The interactive 2-D Viewer gives variable views for multi-dimensional cross coverage hole analysis The interactive features sometimes reveals unexpected coverage hole patterns The “room for improvement” factor enables us to identify the coverage groups which make the most impact on the overall coverage score
References S. Asaf, E. Marcus, and A. Ziv. Defining coverage views to improve functional coverage analysis. In Proceedings of the 41 st Conference on Design Automation, pages 41–44, Jun J. Bergeron, E. Cerny, A. Hunter, and A. Nightingale. Verification Methodology Manual for SystemVerilog. Springer, Cadence Design Systems, Inc. and Mentor Graphics, Inc. Open Verification Methodology User Guide Product Version 2.0.1, Oct Cadence Design Systems, Inc. and Mentor Graphics, Inc. UCDB API Reference Version 1.1, Feb O. Lachish, E. Marcus, S. Ur, and A. Ziv. Hole Analysis for Functional Coverage Data. In Proceedings of the 39 th Conference on Design Automation, pages , Jun Synopsys, Inc. Unified Coverage Database API Reference Manual Version B , Dec Synopsys, Inc. Unified Coverage Reporting User Guide Version B , Dec 2008.