2. 1 Techniques for automated localization and correction of design errors Jaan Raik Tallinn University of Technology

3. 2 Design error debug “There has never been an unexpectedly short debugging period in the history of computers.” Steven Levy

4. 3 Designs are getting bigger

5. 4 25-30 % annually decreasing cost per function 15 percent annual growth of the market for IC But …  The cost of chip design keeps on growing. In 1981, development of a leading-edge CPU cost 1 M$ …today it costs more than 300 M$ !!! Why do the costs increase ??? Designs are getting costlier

6. 5 Design automation crisis productivity gap –58% versus 21% annually transistors on the die Tehnology’s capabilities Designer’s productivity today time 406070 30402 30 2 3 2 System design Logic design Physical design Simulation Schematic entry Placement & routing Hierarchy, generators Logic synthesis High-level synthesis / System-level synthesis Specialized high-level synthesis < 1979 ~ 1983 1986 1988-92 1992-95 ~1996-... Person months / 20 000 logic gates 10 2 30502

7. 6 Verification and debugging Debug = Localization + Correction ~2/3 of development time for verification ~2/3 of verification time for debug Thus nearly half of the development cycle Specify Design DetectLocaliseCorrect Verification Debug Development time:

8. Bugs are getting „smarter“ 7 CREDES Summer School, June 2-3, 2011, Tallinn, Estonia

9. Traditional debug flow 8 Verification Design Spec Error! Counter-examples (waveforms), failed assertions,... ??? Too much information Too little information

10. Automated debug flow 9 Verification Design Spec Error! Corrected design, Repair log,... Error localization Error correction

11. Outline Verification basics Automated debug at the gate-level RTL debug methods –Localization: SAT; correction: resynthesis –Localization: path tracing; correction: mutation General discussion, future trends Prototype tools, on-going activities CREDES Summer School, June 2-3, 2011, Tallinn, Estonia 10

12. 11 Verification “To err is human - and to blame it on a computer is even more so.” Robert Orben

13. 12 Verification versus test The goal of verification is to check if a system is designed correctly. Validation is similar to verification but we check on a prototype device, not a model. By (manufacturing) test we understand checking every instance of a produced chip against manufacruring defects.

14. 13 Abstraction levels and verification

15. 14 Difficulties in verification Errors may be in implementation, specification or verification environment (constraints) No way to detect bugs in the spec, because reference object is missing. Thus: verification by redundancy. Problem: How to assess verification quality i.e. coverage? (except in equivalence checking)

16. 15

17. 16 Verification flow

18. 17 Dynamic verification

19. 18 Dynamic verification Based on simulation Code coverage Assertions, functional coverage

20. 19 Formal verification

21. 20 Dynamic vs formal verification

22. 21 Automated debug techniques “Logic is a poor model of cause and effect.” Gregory Bateson

23. 22 Concept of design error: –Mostly modeled in implementation, sometimes in specification Main applications: –Checking the synthesis tools –Engineering change, incremental synthesis –Debugging Debugging design errors

24. What leads to debugging? Design behavior doesn’t match expected behavior When does this occur? During simulation of design Formal tools (property/equivalence check) Checkers identify the mismatch 23 Debugging design errors

25. 24 Design error diagnosis Classification of methods: –Structure-based/specification-based –Explicit/Implicit fault model (model-free) –Single/multiple error assumption –Simulation-based/symbolic

26. 25 Debugging combinational logic Thoroughly studied in 1990s Many works by Aas, Abadir, Wahba & Borrione, others Also studied, at TUT (Ubar & Jutman) –Used structural BDDs for error localization

27. 26 Explicit error model (Abadir) functional errors of gate elements –gate substitution –extra gate –missing gate –extra inverter –missing inverter connection errors of signal lines –extra connection –missing connection –wrong connection

28. 27 Missing gate error (Abadir)

29. 28 Mapping stuck-at faults to design errors Abadir: Complete s-a test detects all single gate replacements (AND,OR,NAND,NOR), extra gates (simple case), missing gates (simple case) and extra wires.

30. Combinational fault diagnosis 0110 T 6 0010011 Fault F 5 located FaultsF 1 andF 4 are not distinguishable Fault localization by fault table No match, diagnosis not possible Test responses: 29

31. 30 Mapping stuck-at faults to design errors

32. 31 Distribution of design errors

33. 32 Explicit model: disadvantages High number of errors to model Some errors still not modeled

34. 33 Implicit design error models Do not rely on structure Circuit under verification as a black box I/O pin fault models

35. 34 Design error correction Classification: –Error matching approach –Resynthesis approach

36. 35 Design error correction Happens in a loop: –An error is detected and localized –Correction step is applied –Corrected design must be reverified –... Until the design passes verification

37. 36 Ambiguity of error location Since there is more than one way to synthesize a given function, it is possible that there is more than one way to model the error in an incorrect implementation correction can be made at different locations

38. Crash course on SAT 37 CREDES Summer School, June 2-3, 2011, Tallinn, Estonia

39. Digitaalsüsteemide verifitseerimise kursus 38 Satisfiability aka SAT SAT: a Boolean function is satisfiable iff there exists a variable assignment to make it evaluate to TRUE The Boolean function must be represented as a CNF:

40. Digitaalsüsteemide verifitseerimise kursus 39 Satisfiability aka SAT SAT is transformed to CNF (i.e. product of sums). Sums are called terms. If a term has max 2 literals, then 2-SAT 2-SAT is solved in polynomial time  3-SAT is an NP-complete problem N-SAT can be reduced to 3-SAT

41. Digitaalsüsteemide verifitseerimise kursus 40 SAT for circuits Characteristic function Build CNF for logic gates using implication: a  b = ¬a + b ab abab 001 011 100 111

42. Digitaalsüsteemide verifitseerimise kursus 41 Implications for AND-gate: ¬a  ¬c & ¬b  ¬c & ¬c  ¬a  ¬b Characteristic function for AND as a CNF: (a+ ¬c) (b+ ¬c) (c+ ¬a+ ¬b) & a b c SAT for circuits

43. Digitaalsüsteemide verifitseerimise kursus 42 Implications for OR-gate: a  c & b  c & c  a  b Characteristic function for OR as a CNF: (¬a + c) (¬b + c) (¬c + a + b) 1 a b c SAT for circuits

44. Digitaalsüsteemide verifitseerimise kursus 43 Characteristic function for the circuit: (a+¬d)(b+¬d)(d+¬a+¬b)(¬c+¬e)(c+e)(¬d+f)(¬e+f)(¬f+d+e) 1 c e f & a b d SAT for circuits

45. 44 SAT-based RTL debug Mux-enrichment –Muxes added to RTL code blocks –Mux select values select free inputs for the symptom blocks –Synthesis is applied to find logic expressions generating the signatures for these free inputs Cardinality constraints Test vector constraints Smith, Veneris, et al., TCAD, 2005

46. 45 SAT-based RTL debug a) Mux enrichment, b) cardinality constraints

47. 46 SAT-based RTL debug SAT provides locations of signals where errors can be corrected Multiple errors considered! They also provide the partial truth table of the fix Correction by resynthesis This is also a disadvantage: –Why should we want to replace a bug with a more difficult one?

48. Path tracing for localization One of the first debug methods Backtracing mismatched outputs (sometimes also matched outputs) Dynamic slicing → critical path tracing (RTL) 47

49. Mutation-based correction Locate error suspects by backtracing Correct by mutating the faulty block (replace by a different function from a preset library) An error-matching approach 48

50. Testbench-based approach 49 1. Identify injection location 1. Identify injection location 2. Apply mutation operators accordingly 2. Apply mutation operators accordingly Original system description Injected system description if (fn==1) else if (fn==2)... if (fn==4) else if (fn==5)... if (fn==1) else if (fn==2)... if (fn==4) else if (fn==5)... 1 1 2 2 4 4 5 5

51. Arithmetic Operator Replacement (AOR) Set of arithmetic operators = {addition, subtraction, multiplication, division, modulo} Replace each occurrence of arithmetic operator with all the other operators in the set a = b + c; a = b – c; a = b * c; a = b / c; a = b % c; 50

52. Logical Connector Replacement (LCR) Set of logical connectors = {and, nand, nor, or, xor} Replace each occurrence of logical connector with all the other connectors in the set if (a & b) … if !(a & b) … if !(a | b) … if (a | c) … if (a ^ c) … 51

53. Relational Operator Replacement (ROR) Set of relational operators = {equal, not_equal, greater_than, less_than, greater_than_or_equal, less_or_equal_then} Replace each occurrence of relational operator with all the other operators in the set if (a == b) … if (a != b) … if (a > b) … if (a < b) … if (a >= c) … if (a <= c) … 52

54. Unary Operator Injection (OUI) Set of unary operators = {negative, inversion} Replace each occurrence of unary operator with the other operator in the set 53 a = !b; a = ~b;

55. More mutation examples Constant value mutation Replacing signals with other signals Mutating control constructs..... CREDES Summer School, June 2-3, 2011, Tallinn, Estonia 54

56. Approaches for SW & HW Vidroha Debroy and W. Eric Wong, Using Mutation to Automatically Suggest Fixes for Faulty Programs, Software Testing, Verification and Validation Conf., June 2010. Raik, J.; Repinski, U.; et al. High-level design error diagnosis using backtrace on decision diagrams. 28th Norchip Conference 15-16 November 2010. 55

57. Motivational example 56 a-b b:=a-b

58. Motivational example 57 Passed sequence Failed sequence

59. Motivational example 58 ready b ready:=1 ready:=0 res=1 state:=s1state:=s5state:=s3state:=s2state:=s0state:=s1a=ba≠ba>b a=a  b a:=in1 b:=in2 ready b ready:=0 res=1 state:=s1state:=s2state:=s4state:=s2state:=s0state:=s1a≠b abab a:=in1 b:=in2 b:=a  b Backtrace cone: Passed sequence Backtrace cone: Failed sequence

60. Statistical analysis Ranking according to suspiciousness: 59 Suspiciousness score Circuit blocks

61. Fault localization experiments 60 Step1: Critical path tracing of mismatched outputs (max Failed) Step2: Max ratio (Failed/Passed+Failed) of backtrace cones

62. Advantages & open questions Mutation-based repair is readable Helps keeping user in the loop Provides a „global“ repair, for all stimuli How does this backtracing based method perform in the case of multiple errors? What would be a good fault model for high-level design errors? 61

63. Future trends The quality of localization and correction is dependent on input stimuli Thus, diagnostic test generation needed Readable, small correction prefered: –Correction holds normally only wrt given input vectors (e.g. Resynthesis) –Why should we replace an easily detectable bug with a more difficult one?! 62

64. Idea: HLDD-based correction A canonical form of high-level decision diagrams (HLDD) using characteristic polynomials It allows fast probabilistic proof of equivalence of two different designs. Idea: Extend it towards correction 63

65. Prototype tools, activities CREDES Summer School, June 2-3, 2011, Tallinn, Estonia 64

66. DIAMOND Kick-off, Tallinn, February 2-3, 2010 65 FP7 Project DIAMOND Start January 2010, duration 3 years Total budget 3.8M € –EU contribution 2.9M € Effort 462.5 PM The IBM logo is a registered trademark of International Business Machines Corporation (IBM) in the United States and other countries.

67. 66 The DIAMOND concept Specification ImplementationPost-Silicon Design Flow Design errors, soft errors,... Holistic fault models Fault diagnosis Fault correction Reliable Nanoelectronics Systems

68. 67 FORENSIC FoREnSiC – Formal Repair Engine for Simple C For debugging system-level HW Idea by TUG, UNIB and TUT at DATE’10 Front-end converting simple C descriptions to flowchart model completed 1st release expected by the end of 2011

69. 68 Forensic Flow

70. 69 APRICOT: Design Verification Extensions of BDD  HLDD  THLDD

71. APriCoT Verification System –Assertion/Property checkIng, Code coverage & Test generation –The tools run on a uniform design model based on high-level decision diagrams. –The functionality includes currently test generation, code coverage analysis, assertion-checking, mutation analysis and design error localization 70

72. ZamiaCAD: IDE for HW Design ZamiaCAD is an Eclipse-based development environment for hardware designs Design entry Analysis Navigation Simulation Scalable! Co-operation with IBM Germany, R. Dorsch 71

73. 72 To probe further... Functional Design Errors in Digital Circuits: Diagnosis, Correction and Repair K. H. Chang, I. L. Markov, V. Bertacco............................................... Publisher: Springer Pub Date: 2009