Presenter: Jyun-Yan Li Design Fault Directed Test Generation for Microprocessor Validation Deepak A. Mathaikutty, Sandeep K. Shukla FERMAT Lab, Virginia Tech, Blacksburg Sreekumar V. Kodakara, David Lilja The University of Minnesota, Minneapolis Ajit Dingankar Validation Tools, Intel Corporation, Folsom Design, Automation & Test in Europe Conference & Exhibition, DATE '07

Functional validation of modern microprocessors is an important and complex problem. One of the problems in functional validation is the generation of test cases that has higher potential to find faults in the design. We propose a model based test generation framework that generates tests for design fault classes inspired from software validation. There are two main contributions in this paper. Firstly, we propose a microprocessor modeling and test generation framework that generates test suites to satisfy modified condition decision coverage (MCDC), a structural coverage metric that detects most of the classified design faults as well as the remaining faults not covered by MCDC. 2

Secondly, we show that there exists good correlation between types of design faults proposed by software validation and the errors/bugs reported in case studies on microprocessor validation. We demonstrate the framework by modeling and generating tests for the microarchitecture of VESPA, a 32-bit microprocessor. In the results section, we show that the tests generated using our framework's coverage directed approach detects the fault classes with 100% coverage, when compared to model-random test generation 3

Simulation is widely used to validate large system  Depend on the quality of tests  Coverage metric measures the quality 。 Software validation  Statement, branch and path coverage 。 State machine representation  State, transition and path coverage 。 Functionality Effectiveness of coverage directed test generation  Type of faults 。 Can occur  Strength of the coverage metric 。 Detect faults 4

5 coverage Micro architecture coverage [12] Micro architecture coverage [12] random Extended fault class [7] Extended fault class [7] Design Fault Directed Test Generation for Microprocessor Validation This paper: coverage of formal models [5] coverage of formal models [5] Graph-based [9] Graph-based [9] MCDC [6] MCDC [6] Generate assembler test program directly for PowerPC For superscaler processor Build graph model of processor and generate test program to detect fault 9 fault classes related to boolean expressions Strategy of coverage metric Functional coverage

Fault classExample Expression Negation Fault (ENF) Term Omission Fault (TOF) Term Negation Fault (TNF) Literal Omission Fault (LOF) Literal Insertion Fault (LIF) Literal Negation Fault (LNF) Literal Reference Fault (LRF) Disjunctive Operator Reference Fault (ORF[+]) Conjunctive Operator Reference Fault (ORF[-]) 6

Statement coverage  Invoke executable statement Branch coverage  Execute both the true and false Modified condition Decision Coverage (MCDC)  Test each of condition within a decision  Affect outcome of the decision independently  m conditions, m+1 test cases 7 The same with T2 The same with T1

Correlate the bug categories in [2, 13] and map into the fault class Detect all the design fault except LIF  None of the errors observed was classified as a LIF 8

Modeling language as a metamodel  Describe architectural and microarchitectural 9 Validate the Microcode Validate the RTL implementation Statement, branch, MCDC Generate golden result Simulation result

Processor is specified  Register/memory-level description 。 Registers, their relationship and memory schematic  Instruction set capture 。 Instruction behavior Construct  Function blocks, if-else block, statement sequence and loop, register map and memory layout  Ex: 10 Register map Register/memory reference or immediate value jump-if-not-zero (JNZ) instruction

Construct  Configure pipe stage, provide ports with memory and registers, insert instruction registers and describe the control logic  Combined to complex stages by basic components as MUX, ALU, INC, etc  Ex: 11 Instruction fetch stage Instruction memory Z PC Increment by 4 PC

Constraint Satisfaction Problem (CPS)  Build Program Flow Graph (PFG)  Static Single-Assignment(SSA) analysis for multiple incoming path  Coverage Constraint Generator (CCG) generates constrains for decisions Test Case Generator (TCG)  ILOG solver generates test case  Binary generator initializes registers, memory locations and variables 12

Convert the architectural and micro-architectural model into Program Flow Graph (PFG)  Architectural model 。 Function block -> hierarchical states 。 Statement -> state 。 If-else -> decision 。 Identifier, registers, memory location -> variables  Micro-architectural model 。 Basic components -> bunch of if-else sequences If a state with multiple incoming transition  Resolve in the Static Single-Assignment (SSA) analysis 13

JNZ instruction IF stage 14 variable decision state

Help CSP to generate constrains for complete run Focus on states with multiple incoming transitions  If no change the value 。 the state will be ignored 。 Insert additional decisions and variables  Ex: 15 SSA result SSA analysis Ignore S3 Rename variables

ILOG is a commercial solver designed for performance and scalability  Generate test case that satisfying every constraint variable  Ex: 16

Two set of inputs  Coverage annotation 。 expedite the reachability of the coverage goal 。 Setting the attribute MARK associated with a state and decision  Coverage type 。 Statement coverage 。 Branch coverage 。 MCDC 。 Design Fault coverage 17

18 Arrange D by graph depth Graph G Problematic decision point D Node d Find a path p form root to d Boolean operator? Remove the duplicate test cases N Y

The fault detection capability of MCDC is 7/9 classes coverage  Literal omission fault (LOF) can be satisfied by MCDC 。 Every literal will affect the output of decision 。 Ex:  Literal Reference Fault (LRF) coverage by LRF_list 。 A list of variables to tell CCG which variable can be replaced for resulting in a fault 。 If LRF_list is not provided, the CCG replaces every variabled 。 Lead to large set of test case 。 Recommend LRF_list with a variable and mark it 19

How many  test cases are generated 。 Not describes in the paper  Conditions in a decision 。 330 unique decisions, 400 unique conditions 。 1~10 conditions in a decision  Fault in each fault class 20

Environment  VESPA 32-bit processor, 5-stage pipeline  50 decisions are chosen and replaced randomly for LRF  The number of test cases are the same for each coverage metrics 21

A metamodel-based modeling framework to capture structure and behavior  Architecture – registers, ISA  Micro-architecture – pipeline structure, control logic TGF results in test case for covering design faults  CSP formulation  TCG My comment  Overview the test case generating flow for code coverage  Some algorithm is not specification 22