Presentation on theme: "Paper Review: Hole Analysis For Functional Coverage Data by Oded Lachish, Eitan Marcus, Shmuel Ur, Avi אבשלום אלידע 0-320690 23 לאוניד פרואנצ."— Presentation transcript:
Paper Review: Hole Analysis For Functional Coverage Data by Oded Lachish, Eitan Marcus, Shmuel Ur, Avi Ziv @IL.IBM אבשלום אלידע 0-320690 23 לאוניד פרואנצ ' נקו 21052137-3 CAD of VLSI Systems (046880) 21/01/03
Functional Coverage Functional Coverage -- A means of: –Measuring the efficiency of Functional Verification. –Finding how and in what directions to improve Test Generation in order to increase coverage. –And as a result, reach more corner verification cases and CATCH MORE BUGS !!! Compare to older, less effective coverage methods (toggle, line, cross-state), Functional Coverage models closely the design functionality.
Functional Coverage: Motivation Just testing is not enough: –Manual testing (MT) time-consuming and only gives us relatively simple cases. –Efficient bug-finding with random testing (RT), but RT machine-resource consuming and needs to be tracked closely for efficiency with a dedicated tool … The 1M$ question of verification: what level of coverage are we achieving?! Without dedicated tool, cannot answer this.
Functional Coverage in a Nutshell Define list (space) of complete functionality of design. Comprises of verification tasks i.e, cases we expect to reach. Track testing results (mostly RT, also MT) and check which tasks were reached. (Groups of) tasks not reached are referred to as “holes”. In order to “plug” the holes: –Change/improve test-generator or testing-environment as needed or, – Define restriction on coverage model if hole is not real case.
Functional Coverage – Cntd. An Iterative Process: Resume testing to check improvement in total tasks covered report coverage again Cases-reached / Total-Case-Space = % Coverage
Example Coverage Model: A Network Switch Receive Port Define attribute groups. Rx-port consists of pairs. Define attributes, for each attr. its possible values:
ValuesAttribute Ethernet, IP-over-Eth, Layer4-o-IP-o-Eth, Eth- Control, … Pkt Type too short(0-63byte), small(64-127), medium(128-511), large(512-1023), extra- large(1024-1534), too long(1535 byte and up) Pkt Length unicast, multicast, broadcast, unknown (destination only or source too) Layer2 Cast intact, wrong, missing?CRC in Receive Port – Packet Received
ValuesAttribute reject, own device, other device, CPU, multicast/broadcast Forward to accept, length: too short/long, CRC, collision, rx error … reject reason pass as received, append, recalculateCRC out Receive Port – Decision
Hole Analysis Concise presentation of coverage data crucial for effective analysis of coverage results. –Mere list of tasks not reached – all but meaningless! Group together multiple single-element holes, Projected Holes: n-dimensional holes. More significance and clarity than mere list of uncovered tasks. –Case space: has N=7 dimensions. –0-dimensional hole: one point in case-space not reached: didn’t happen. –n-dimensional hole: no event reached for any of the cases where N-n+1 attributes were at a fixed value: hole of n=5, no IP pkts were rejected for any reason! –Group together 0-dim. holes to more significant n-dim. holes.
Hole Analysis: Coverage Data Presentation Aggregated Holes Group holes together based on Hamming Distance = 1. (Hamming dist. = # of different attributes). Example with 2 attributes (1-covered, 0-uncovered) Rej collisionRej on LengthRej on Bad CRCAccept 1100 port0 own dev 0011 port1 0011 port2 1101 Other dev 1111 CPU better: Partitioned Holes.
Conclusions Coverage emerging as critical tool in verif. flow: Close tracking of testing environment effectiveness, closely linked to RT. Assessment (in %) of degree of above effectiveness, gives a feel of how much more verif. needs to be done. Considering ever-increasing size&complexity of today’s industry designs, keeping verif. up to the task without diverse&informative coverage tool – next to impossible. Diverse&informative: in-depth Hole-Analysis, smart grouping of holes to “serve” for the verif. engineer.
Food for Thought Modeling of functional coverage closely coupled to many subjects studied in the course: Data structures use BDD for compact representation of sets attributes. Checking whether a task was covered satisfiability. Coverage and Formal-Verification: Two Sisters Formal: state a rule check for it’s tautology. Coverage: state a rule in from of “task didn’t happen” check that it is unsatisfied. *) No mumbo-jumbo clip-art or meaningless graphics were used in the making of this paper review. Technion, Israel, January 2003. A.E, L.F.