UPC Trace-Level Speculative Multithreaded Architecture Carlos Molina Universitat Rovira i Virgili – Tarragona, Spain Antonio González and Jordi Tubella Universitat Politècnica de Catalunya – Barcelona, Spain ICCD´02, Freiburg (Germany) - September 16-18, 2002

Outline Motivation Related Work TSMA Performance Results Conclusions

Motivation Two techniques to avoid serialization caused by data dependences Data Value Speculation Data Value Reuse Speculation predicts values based on past Reuse is posible if has been done in the past Both may be considered at two levels Instruction Level Trace Level

Trace Level Reuse Set of instructions can be skipped in a row These instructions do not need to be fetched Live input test is not easy to handle Dynamic Trace Level Reuse Static

Trace Level Speculation Solves live input test Introduces penalties due to misspeculations Two orthogonal issues microarchitecture support for trace speculation control and data speculation techniques –prediction of initial and final points –prediction of live output values With Live Output Test Trace Level Speculation With Live Input Test

Trace Level Speculation with Live Input Test Live Output Actualization & Trace Speculation NST ST Miss Trace Speculation Detection & Recovery Actions INSTRUCTION EXECUTION NOT EXECUTED LIVE INPUT VALIDATION & INSTRUCTION EXECUTION

BUFFER Trace Level Speculation with Live Output Test Live Output Actualization & Trace Speculation NST ST Miss Trace Speculation Detection & Recovery Actions INSTRUCTION EXECUTION NOT EXECUTED LIVE OUTPUT VALIDATION

Related Work Trace Level Reuse Basic blocks (Huang and Lilja, 99) General traces (González et al, 99) Traces with compiler support (Connors and Hwu, 99) Trace Level Speculation DIVA (Austin, 99) Slipstream processors (Rotenberg et al, 99) Pre-execution (Sohi et al, 01) Precomputation (Shen et al, 01) Nearby and distant ILP (Balasubramonian et al, 01)

TSMA Cache I Engine Fetch Rename Decode & Units Functional Predictor Branch Speculation Trace NST Reorder Buffer ST Reorder Buffer NST Ld/St Queue ST Ld/St Queue NST I Window ST I Window Look Ahead Buffer Engine Verification L1NSDC L2NSDC L1SDC Data Cache Register File NST Arch. Register File ST Arch.

Trace Speculation Engine Two issues may handle to implement a trace level predictor to communicate trace speculation opportunity Trace level predictor PC-indexed table with N entries Each entry contains –live output values –final program counter of trace Trace speculation communication INI_TRACE instruction Additional MOVE instrucions

Look Ahead Buffer First-input first-output queue Stores instructions executed by ST The fields of each entry are: Program Counter Operation Type: indicates memory operation Source register Id 1 & source value 1 Source register Id 2 & source value 2 Destination register Id & destination value Memory address

Verification Engine Validates speculated instructions Mantains the non-speculative state Consumes instructions from LAB Test is performed as follows: testing source values of Is with non-speculative state if matching, destination value of I may be updated memory operations check effective address store instructions update memory, rest update registers Hardware required is minimal

Thread Synchronization Handles trace misspredictions Recovery actions involved are: Instruction execution is stopped ST structures are emptied (IW,LSQ,ROB,LAB) Speculative cache and ST register file are invalidated Two types of synchronization Total (Occurs when NST is not executing instructions) –Penalty due to fill again the pipeline Partial (Occurs when NST is executing instructions) –No penalty –NST takes the role of ST

Mantains memory state speculative non speculative Rules ST store updates values in L1SDC only 1 Traditional memory subsystem is supported Additional and small first level cache is added to mantain memory speculative state ST load get values from L1SDC. If not, get from NS caches 2 NST store updates values and allocate space in NS caches 3 NST loads get values and allocates space in NS caches 4 Line replaced in L1NSDC is copied back to L2NSDC 5 Memory Subsystem L1SDC L1NSDC L2NSDC

Register File Slight modification to permit prompt execution Register map table contains for each entry: Commited Value ROB Tag Counter Counter field is mantained as follows: New ST instruction increases dest. register counter Counter is decreased when ST instruction is commited After trace speculation counter are no longer increased But it is decreased until reaches the value zero.

1 ST Begins Execution 2 Live Output Actualization & Trace Speculation 3 NST Begins Execution 4 VE Validates Instructions 5 NST Executes Speculated Trace 6 NST Executes Some Additional Instructions 7 VE Begins Verification 8 VE Finishes Verification 9 Live Output Actualization & Trace SpeculationNST Execution 10 ST INSTRUCTION EXECUTION NOT EXECUTED LIVE OUTPUT VALIDATION Working Example NST VE

Experimental Framework Simulator Alpha version of the SimpleScalar Toolset Benchmarks Spec95 Maximum Optimization Level DEC C & F77 compilers with -non_shared -O5 Statistics Collected for 125 million instructions Skipping initializations

Base Microarchitecture

TSMA Additional Stuctures

Performance Evaluation Main objective: trace misspeculations cause minor penalties Traces are built following a simple rule from backward branch to backward branch minimum and maximum size of 8 and 64 respectively Simple Trace Predictor is evaluated Stride + Context Value (history of 9) Results provided Percentage of misspeculations Percentage of predicted instructions Speedup

Misspeculations

Predicted Instructions

Speedup

Conclusions TSMA designed to exploit trace-level speculation Special emphasis on minimizing misspeculation penalties Results show: architecture is tolerant to misspeculations speedup of 16% with a predictor that misses 70%

Future Work Agressive trave level predictors bigger traces better value predictors Generalization to multiple threads cascade execution Mixing prediction & execution speculated traces do not need to be fully speculated