1. Jieyi Long and Seda Ogrenci Memik Dept. of EECS, Northwestern Univ. Jieyi Long and Seda Ogrenci Memik Dept. of EECS, Northwestern Univ. Automated Design of Self-Adjusting Pipelines

2. 2 Outline Introduction Introduction Self-Adjusting Pipeline (SAP) Self-Adjusting Pipeline (SAP) Systematic Design Framework of SAP Systematic Design Framework of SAP Experiment: Microprocessor Pipeline Experiment: Microprocessor Pipeline Conclusions Conclusions

3. 3 Introduction Aggressive Scaling Down Aggressive Scaling Down –Process variation (PV) –Circuit behaviors are harder to predict Steadily Increasing Integration Capacity Steadily Increasing Integration Capacity –Complexity of the designs amplified from generation to generation from generation to generation Challenges for Traditional CAD Flow Challenges for Traditional CAD Flow –Synthesis: has to be conservative –Simulation: challenging due to the variations –Verification: very tricky due to the complexity Novel Design Methodologies are Methodologies areNeeded!

4. 4 Introduction Self-Adjusting Architecture Self-Adjusting Architecture –A promising methodology to address the above mentioned challenges –Offers a way handle uncertainty after manufacturing manufacturing –Still in its infancy –Automated design tools Examples of Self-Adjusting Architectures Examples of Self-Adjusting Architectures –Razor [Ernst et al. MICRO2003] –SACTA [Long et al. ICCAD2007]

5. 5 Self-Adjusting Pipeline (SAP) Impact of Process Variation Impact of Process Variation Due to PV, balanced designs are actually NOT balanced! clk R1R1 R2R2 R3R3 Traditionally, pipeline stages are designed to have the same nominal delay (i.e., balanced) The impacts of PV on different stages are different

6. 6 Self-Adjusting Pipeline (SAP) Impact of Process Variation Impact of Process Variation FMAX Model [Bowman et al, TCAD 2007]: With the same nominal delay, pipeline stages having 1)larger number of independent critical paths and 2)smaller logic depth have higher probability to have longer delay clk R1R1 R2R2 More Vulnerable Less Vulnerable R3R3

7. 7 Self-Adjusting Pipeline (SAP) Allocate Execution Time on a Need Basis Allocate Execution Time on a Need Basis Our solution: Create dynamic clock skews to satisfy the actual need of the stages clk R1R1 R2R2 R3R3

8. 8 Self-Adjusting Pipeline (SAP) How to obtain the actual execution time of each stage? How to obtain the actual execution time of each stage? Razor: Detect after execution Razor: Detect after execution Our solution: Measure and predict! Our solution: Measure and predict! clk R1R1 R2R2 R3R3

9. 9 Self-Adjusting Pipeline (SAP) How to fix the timing error? How to fix the timing error? Our solution: Measure and predict! Our solution: Measure and predict! –We predict the error before it manifests itself, so we might have time to fix it clk R1R1 R2R2 R3R3

10. 10 T max Self-Adjusting Pipeline (SAP) Two supporting circuit elements Two supporting circuit elements –Delay sensor [based on Ghosh et al. TCAD 2007] –Adjustable skew buffer Delay Sensor TDTD V REF CLK P Sawtooth TAH

11. 11 Self-Adjusting Pipeline (SAP) Two supporting circuit elements Two supporting circuit elements –Delay sensor [based on Ghosh et al. TCAD 2007] –Adjustable skew buffer Adjustable Skew Buffer

12. 12 Systematic Design Framework of SAP Objective Function: Average Performance Objective Function: Average Performance Frequency bins: [f 1, f 2 ], …, [f i, f i+1 ], …, [f n, f n+1 ] Yield y i of bin [f i, f i+1 ] : The fraction of chips falling into the bin Speed Binning Speed Binning Performance Metric of a Set of Chips Performance Metric of a Set of Chips BP = ∑ i =1 f i · y i BP = ∑ i =1 f i · y i Batch Performance [Das et al., ASGI 2007]

13. 13 Systematic Design Framework of SAP Variables Variables Cannot be too early in the stage, otherwise the prediction will not be accurate Cannot be too late in the stage, otherwise we do not have time to fix the error Locations of the delay sensors: Locations of the delay sensors: Cannot be too small, otherwise the timing error in the first stage is not fixed Nominal delay of the adjustable skew buffer Nominal delay of the adjustable skew buffer Cannot be too large, otherwise there might be a timing error in the second stage

14. 14 Systematic Design Framework of SAP Automated Delay Sensor Insertion and Clock Skew Buffer Configuration Given: 1) a two back-to-back pipeline stages, where the first stage is more vulnerable to process variation than the second more vulnerable to process variation than the second 2) max tolerable delay of each internal node in the pipeline Determine: 1) the location of the delay sensors, 2) nominal delay of the adjustable skew buffers, such that the Batched Performance is maximized. Problem Definition

15. 15 Systematic Design Framework of SAP Mixed-Integer Programming Formulation Directed Acyclic Graph (DAG) representation of the pipelines Directed Acyclic Graph (DAG) representation of the pipelines –Gates → vertices –Registers → primary I/O vertices –Interconnects → directed edges

16. 16 Systematic Design Framework of SAP Mixed-Integer Programming Formulation Directed Acyclic Graph (DAG) representation of the pipelines Directed Acyclic Graph (DAG) representation of the pipelines –Primary Path: a path between a primary input and a primary output vertices Coverage Requirement: each primary path must be covered by one and only one delay sensor Coverage Requirement: each primary path must be covered by one and only one delay sensor –The edges with delay sensor on it form a cut of the DAG

17. 17 Systematic Design Framework of SAP Mixed-Integer Programming Formulation We assign a decision variable x i to each vertex We assign a decision variable x i to each vertex –The decision variables specifies the locations of the sensors: A sensor is on edge (v i, v j ), iff x i – x j = 1

18. 18 Systematic Design Framework of SAP Mixed-Integer Programming Formulation Constraints specifying the Coverage Requirement Constraints specifying the Coverage Requirement x i – x j ≥ 0, for each edge (v i, v j ) x p = 1, for all v p in PI 1 x p = 0, for all v p in PI 2 x q = 0, for all v q in PO 1 or PO 2

19. 19 Systematic Design Framework of SAP Mixed-Integer Programming Formulation Forbidden Vertex Set V F Forbidden Vertex Set V F –The delay from the underlying vertex and any primary output is less than the worst case delay of the OR-MUX chain x f = 0, for all v p in V F

20. 20 Systematic Design Framework of SAP Mixed-Integer Programming Formulation Objective Function: Batch Performance Objective Function: Batch Performance Pr(f) : The probability that the pipeline stage meets the timing constraints at frequency f Pr(f) : The probability that the pipeline stage meets the timing constraints at frequency f BP = ∑ i =1 f i · y i = ∑ i =1 f i · (Pr(f i+1 ) - Pr(f i )) = ∑ i =1 f i · (Pr(f i+1 ) - Pr(f i ))

21. 21 Systematic Design Framework of SAP Mixed-Integer Programming Formulation Analysis of Pr(f) Analysis of Pr(f) We consider two situations: 1)At least one error is predicted 2)No error is predicted Some definitions: D i : accumulative delay at node i D i m : the maximum tolerable delay at node i α(x) = 1 if x > 0, 0 otherwise

22. 22 Systematic Design Framework of SAP Mixed-Integer Programming Formulation At least one error is predicted At least one error is predicted If the sensor on edge (v i, v j ) detects an error α((x i - x j )(D i - D i m )) = 1 At least one sensor detects an error R 1 = ∑ α((x i - x j )(D i - D i m )) > 0 (v i, v j )

23. 23 Systematic Design Framework of SAP Mixed-Integer Programming Formulation At least one error is predicted At least one error is predicted The skew buffer will be reconfigured to generate a skew of amount δ For stage 1, the effective clock cycle time becomes (1/f +δ), we should have For stage 2, the effective clock cycle time becomes (1/f –δ) For each v k ∈ PO 1, α(1/f + δ – D k ) = 1 For each v k ∈ PO 2, α(1/f – δ – D k ) = 1

24. 24 Systematic Design Framework of SAP Mixed-Integer Programming Formulation At least one error is predicted At least one error is predicted The skew buffer will be reconfigured to generate a skew of amount δ Timing correctness requirement: R 2 = ( ∏ α(1/f + δ – D k ) ) ( ∏ α(1/f – δ – D k ) ) = 1 v k ∈ PO 1 v k ∈ PO 2

25. 25 Systematic Design Framework of SAP Mixed-Integer Programming Formulation At least one error is predicted At least one error is predicted The probability of error being fixed: Pr(R 1 > 0 and R 2 = 1)

26. 26 Systematic Design Framework of SAP Mixed-Integer Programming Formulation No error is predicted No error is predicted R 3 = ( ∏ α(1/f – D k ) ) ( ∏ α(1/f – D k ) ) = 1 v k ∈ PO 1 v k ∈ PO 2 there is actually no timing error:

27. 27 Systematic Design Framework of SAP Mixed-Integer Programming Formulation The probability that the pipeline executed correctly Pr(f) The probability that the pipeline executed correctly Pr(f) Pr(R 1 > 0 and R 2 = 1) + Pr(R 1 = 0 and R 3 = 1)

28. 28 Systematic Design Framework of SAP Mixed-Integer Programming Formulation s.t. x i – x j ≥ 0, for each edge (v i, v j ) x p = 1, for all v p in PI 1 x p = 0, for all v p in PI 2 x q = 0, for all v q in PO 1 or PO 2 max ∑ i =1 f i · (Pr(f i+1 ) - Pr(f i )) x f = 0, for all v p in V F x i = 0 or x i = 1

29. 29 Systematic Design Framework of SAP Simulated Annealing Solving the MIP Formulation Solution Space X = {x 1, x 2, …, x n, } satisfying the constraints of the MIP formulation Initial Solution x i = 1 iff v i belongs to PI 1 x i = 1 iff v i belongs to PI 1

30. 30 Systematic Design Framework of SAP Simulated Annealing Solving the MIP Formulation Solution Perturbation M 0 (x j, X) (0 to 1 toggle): i) keep the value of x i for each i != j ; ii) change the value of x j from 0 to 1 if 1) x j = 0 and x i = 1 for change the value of x j from 0 to 1 if 1) x j = 0 and x i = 1 for each edge (v i, v j ), and 2) x j does not belong to V F each edge (v i, v j ), and 2) x j does not belong to V F

31. 31 Systematic Design Framework of SAP Simulated Annealing Solving the MIP Formulation Solution Perturbation M 0 (x j, X) (0 to 1 toggle): i) keep the value of x i for each i != j ; ii) change the value of x j from 0 to 1 if 1) x j = 0 and x i = 1 for change the value of x j from 0 to 1 if 1) x j = 0 and x i = 1 for each edge (v i, v j ), and 2) x j does not belong to V F each edge (v i, v j ), and 2) x j does not belong to V F

32. 32 Systematic Design Framework of SAP Simulated Annealing Solving the MIP Formulation Solution Perturbation M 1 (x i, X) (1 to 0 toggle): i) keep the value of x j for each j != i ; ii) change the value of x j from 1 to 0 if 1) x j = 0 and x i = 1 for change the value of x j from 1 to 0 if 1) x j = 0 and x i = 1 for each edge (v i, v j ) each edge (v i, v j )

33. 33 Systematic Design Framework of SAP Simulated Annealing Solving the MIP Formulation Solution Perturbation M 1 (x i, X) (1 to 0 toggle): i) keep the value of x j for each j != i ; ii) change the value of x j from 1 to 0 if 1) x j = 0 and x i = 1 for change the value of x j from 1 to 0 if 1) x j = 0 and x i = 1 for each edge (v i, v j ) each edge (v i, v j )

34. 34 Experiment: Microprocessor Pipeline DEC Alpha-like 6 Stage Pipeline DEC Alpha-like 6 Stage Pipeline –Cache and IF are next to each other Cache has a lot of critical paths, each consisting of small number of gates Cache has a lot of critical paths, each consisting of small number of gates IF has just a few critical paths, each consisting of large number of gates IF has just a few critical paths, each consisting of large number of gates According to FMAX model [Bowman et al, TCAD 2007], the delay of the cache tends to be longer According to FMAX model [Bowman et al, TCAD 2007], the delay of the cache tends to be longer IFMAPIQREGALUCache More Vulnerable Less Vulnerable

35. 35 Experiment: Microprocessor Pipeline DEC Alpha-like 6 Stage Pipeline DEC Alpha-like 6 Stage Pipeline –Cache and IF are next to each other It will be beneficial to create dynamic clock skew when the cache does needs longer execution time It will be beneficial to create dynamic clock skew when the cache does needs longer execution time IFMAPIQREGALUCache More Vulnerable Less Vulnerable

36. 36 Experiment: Microprocessor Pipeline Setup The critical paths of the Cache and IF are extracted from the Verilog code of OpenSPARC processor We assume 45nm technology IFMAPIQREGALUCache More Vulnerable Less Vulnerable

37. 37 Experiment: Microprocessor Pipeline Results: the average frequency increases from 1.989GHz to 2.178GHz (9.5% improvement)

38. 38 Conclusions We identified the challenges in modern VLSI cad tool design We identified the challenges in modern VLSI cad tool design We proposed to leverage Self-Adjusting Pipeline to solve the problems We proposed to leverage Self-Adjusting Pipeline to solve the problems We propose a systematic Design Framework of SAP We propose a systematic Design Framework of SAP Application: Microprocessor Pipeline Application: Microprocessor Pipeline Experimental results illustrates the effectiveness of our approach Experimental results illustrates the effectiveness of our approach

39. 39