SLAM: SLice And Merge – Effective Test Generation for Large Systems ICCAD’13 Review Reviewer: Chien-Yen Kuo

Introduction  More logic squeezed into a single chip  Verification by test cases  Evaluated by functionality coverage  Consideration of system-level verification  Interaction between components  Stress on common resources

Test templates  To produce test cases more efficiently, use test template-based generator  A template consists of statements written in high- level language  Each statement represents a hardware transaction  E.g., memory access of CPU, interruption from external I/O device to CPU  A test generator parses statements and generate stimuli for corresponding components

Related works  Top-level scenario composed of several components (scenarios)  Component independency is needed  Operating system  Additional booting and running time  Interleaving  Additional scenario requirements  Irritation  A special case of SLAM

SLAM: SLice And Merge  Fact: usually a test template is associated with only a subset of full system  Assumption: exercising a stimulus(test case) on a subset of model does not undermine its effectiveness (prove in experiments)

Slicing  Pre-defined/ Random/ Constrained-random slicing  Each slice is assigned with a different test template

Merging  Integrate SLAM into test generator  Take more than one template/slice as input  Alternatively parse statements from each template  Maintenance  History of generated statements(?)  Current stimuli being generated  Current state of components

Verification cycle  For each function (unit, I/O bridge, or feature), write test templates for various scenarios  Target various scenarios in one template  Target various functionalities and stress the common resources in one template

Benefits of SLAM  Reducing manual effort on template generation  No complex template needed, use SLAM to generate complex test cases  Cover unexpected scenarios  Reducing regression time  Since stimuli of templates exercise slices in parallel

Experiments  Implemented in IBM System-level Test- Generator  Use a subset of template library  CACHE  RESERVATION  PREFETCH  NEW FEATURE  Coverage model used by System p verification team  400,000+ events divided into 8,000+ classes

Simulation on subsets of system  On a chip with 64 hardware threads  Vary the number of participating threads  Observations  Saturated event accumulation  Larger subset does not necessarily lead to larger coverage

Coverage growth  On a chip with 32 hardware threads  Two combinations of templates  NEW FEATURE and PREFETCH  NEW FEATURE and RESERVATION  6 test cases per template vs. 12 SLAM test cases

Unique event coverage  On a chip with 32 hardware threads  Average results of two combinations of templates  NEW FEATURE and PREFETCH  NEW FEATURE and CACHE  Observation  A class of events are almost always hit by SLAM tests but not hit by any test of individual template

Conclusions  SLAM, a method for test generation for large systems  Divide systems into slices, each of which is the bench of a scenario  Run test cases in parallel  Reaching additional coverage events  Used as part of the system level verification methodology for IBM System p servers

Comment and suggestions  The benefits should be solid  Suggestions  Be more specific  Slicing methods  Related works  Explanations on experimental results (example?)  Put related work review in introduction  Cite more in introduction