1. Interactive 2-D Projection Cross Coverage Viewer for Coverage Hole Analysis Keisuke Shimizu ClueLogic keisuke@cluelogic.com Tomohisa Iida Verifore, Inc. tomo.iida@verifore.jp Tomohisa Iida Verifore, Inc. tomo.iida@verifore.jp

2. Outline  Introduction and motivation  Related work  Cross coverage and views  Our 2-D viewer  “Room for improvement” factor  Results  Limitations and future improvement  Summary

3. Introduction  Constrained-random verification (CRV) methodology such as VMM and OVM becomes widely accepted  The random nature of the methodology requires functional coverage metrics to monitor what has been tested Reaching 100% coverage is one of the goals  Identifying the root cause of the coverage holes is crucial Especially when dealing with multi-dimensional cross coverage that tends to have a huge coverage space

4. Motivation  Synopsys Unified Report Generator (URG) presents cross coverage comprehensively, but Not easy to grasp the big picture of the coverage Not easy to recognize hidden patterns of the coverage holes  URG focuses on displaying the entire coverage results  We wanted a tool to discover coverage holes and their patterns

5. URG

6. Related Work  Hole analysis for functional coverage data [Lachish, DAC ‘02]  Defining coverage views to improve functional coverage analysis [Asaf, DAC ‘04] Selection Projection Grouping  We focus on interactivity so that the users can see the coverage results from different view points

7. Cross Coverage Example  Cross coverage with 3 coverage points Each coverage point has 4 bins This makes 4x4x4 cross bins

8. Coverage Hole Analysis  Determine the areas that have not been tested  Is there a pattern here?

9. Coverage View: List View  Exhaustive, but loses patterns  This is what URG basically provides List View

10. List View: URG  URG puts coverage holes together URG

11. Coverage View: 2-D Projection  Add bin counts of coverage points you do not focus on Projection

12. Coverage View: 2-D Projection  Collapse multiple dimensions into two Projection

13. Coverage View: 2-D Projection  We get C(n, 2) projections, where n is # of coverpoints in a cross Projection

14. Coverage View: Bin Selection  Another way to see the coverage point relation Selection

15. Coverage View: Bin Selection  Select a bin of a coverage point Selection

16. Coverage View: Bin Selection  Can be combined with the 2-D projection Selection

17. How the 2-D Viewer Works

18. Intermediate XML

19. Coverage Example  Design under Test SoC interconnect

20. Verification Environment  VMM-based CRV

21. Coverage Data  256 coverage group instances 196 cross coverage instances 4,789 coverage bins in total (excluding illegal bins)  Cross coverage example: Initiator vs. Target vs. Direction (Read/Write) 36x26x2 bins (including illegal bins)

22. Projection: Initiator vs. Target

23. Bin Selection

24. Projection: Initiator vs. Direction

25. Projection: Target vs. Direction

26. Which coverage group to tackle first?  Motivation Now that we can analyze coverage holes more efficiently But which coverage group should we look at first?  Traditional “coverage score” measures how many bins are covered within a coverage group This metric does not show relative importance on the overall coverage

27. Coverage Group Comparison  Which coverage group is more important? Coverage report displays the coverage group A to the top because it is least covered, but… Coverage GroupABC Number of total bins1020170 Number of bins hit28136 Coverage score20%40%80%

28. “Room for Improvement” Factor Coverage GroupABC Number of total bins1020170 Number of bins hit28136 Coverage score20%40%80% “Room for Improvement” factor4%6%17%  Coverage score shows coverage group A is least covered, but …  Coverage group A would increase overall coverage merely by 4%, whereas coverage group C would increase by 17% Assuming W g =number of bins in coverage group g (see next page)

29. The Factor Definition SShows the impact on the overall coverage “How many more percentages the overall coverage would increase if this coverage group were covered ” C g is the score of coverage group g in percent g  set of coverage groups W g is the weight associated with coverage group g

30. Actual Data  The coverage group having the lowest coverage score does not necessarily mean the one that has the most impact on the overall coverage  We need both metrics

31. Results  Using the 2-D Viewer, we found several verification/coverage bugs including: Illegal access to targets Boundary crossing Access to a reserved area Missing burst length values Illegal byte enable patterns

32. Byte Enable Coverage

33. Limitations  There exists non-obvious coverage holes New approach is required to uncover the root causes of the hard to spot holes  Currently only VCS coverage database is supported

34. Future Improvement  Rewrite the 2-D Viewer preprocessor using Unified Coverage Database API UCAPI – to support VCS without worrying about future database format changes UCDB API – to support simulators other than VCS  Add “grouping feature” of coverage bins  3-D Viewer?

35. Summary  The interactive 2-D Viewer gives variable views for multi-dimensional cross coverage hole analysis The interactive features sometimes reveals unexpected coverage hole patterns  The “room for improvement” factor enables us to identify the coverage groups which make the most impact on the overall coverage score

36. References  S. Asaf, E. Marcus, and A. Ziv. Defining coverage views to improve functional coverage analysis. In Proceedings of the 41 st Conference on Design Automation, pages 41–44, Jun 2004.  J. Bergeron, E. Cerny, A. Hunter, and A. Nightingale. Verification Methodology Manual for SystemVerilog. Springer, 2005.  Cadence Design Systems, Inc. and Mentor Graphics, Inc. Open Verification Methodology User Guide Product Version 2.0.1, Oct 2008.  Cadence Design Systems, Inc. and Mentor Graphics, Inc. UCDB API Reference Version 1.1, Feb 2009.  O. Lachish, E. Marcus, S. Ur, and A. Ziv. Hole Analysis for Functional Coverage Data. In Proceedings of the 39 th Conference on Design Automation, pages 807-812, Jun 2002.  Synopsys, Inc. Unified Coverage Database API Reference Manual Version B-2008.12, Dec 2008.  Synopsys, Inc. Unified Coverage Reporting User Guide Version B-2008.12, Dec 2008.

37. Live Demo