1. Phase based adaptive Branch predictor: Seeing the forest for the trees
Karthik Jayaraman
Vivek Shrivastava
Brian Pellin
Martin Hock
Mikko H. Lipasti
University of Wisconsin-Madison

2. Motivation Understanding and exploiting dynamic program behavior
More powerful than static techniques
Program’s flow control is a dynamic behavior
Executes in “phases” or “repeated patterns”

3. Motivation Phase: Programs go through different phases of execution
A period of execution that exhibits relatively stable program characteristics
Programs go through different phases of execution
Phases are often repeated at different times in execution
During each phase hardware is exercised differently
Hardware requirements may vary per phase

4. Motivation
Microprocessors designed to provide good average performance
Inefficient for individual programs
Inefficient for various phase within the same program
Configure Micro architecture features dynamically
Use reconfigurable hardware to take advantage of phase information
Reconfigurable caches
Instruction window size
Dynamic branch predictor

5. Motivation Dynamic reconfiguration algorithms
Detect the current phase of program execution
Tune the reconfigurable hardware for current phase
Portions of configurable units can be turned on/off depending on specific requirements of phase

6. Sample Phase Behavior : gcc

7. Outline Phase Tracking Phase Prediction Phase Based Branch Prediction
Experiments and Results
Conclusions
Future Work

8. Phase Tracking Goal: Track groups of 10 million instructions
Identify program phases with different behavior
Based on “Phase Tracking and Prediction” [Sherwood, Sair, Calder]
Track groups of 10 million instructions
Collect information about instructions and store
Build a phase footprint
After each 10 m instructions, compare footprint with past footprints
If footprint close enough, it is considered a repetition of the phase

9. Accumulator
Branch PC
Hash
# of inst. since branch +

10. Accumulator
Branch PC 2
Hash
# of inst. since branch 20 +
Branch occurs, must increment entry 2 by 20.

11. Accumulator
Branch PC 20 3
Hash
# of inst. since branch 80 +
New branch, increment entry 3 by 80.

12. Accumulator
Branch PC 20 80
Hash
# of inst. since branch +
After a phase completes we need somewhere to store data about previous phases.

13. Past Footprint Table
Accumulator
Branch PC 20 80
Hash
# of inst. since branch +
*At 100 instructions

14. Past Footprint
Past Footprint Table
Accumulator
Branch PC 20 80
Hash
# of inst. since branch +
Accumulator Data is stored in Past Footprint table

15. Past Footprint Table
Past Footprint
Accumulator 90
Branch PC 20 5 80
Hash
# of inst. since branch 5 +
*At 200 instructions
Take the Manhattan distance between accumulator and Past Footprints
= 190

16. Past Footprint Table
Past Footprint
Accumulator 90
Branch PC 20 80 5
Hash
# of inst. since branch 5 +
*At 200 instructions

17. Past Footprint
Past Footprint Table
Accumulator 90
Branch PC 21 20 79 80 5
Hash
# of inst. since branch 5 +
*At 300 instructions
Manhattan distance between this phase and first phase is 2.
This phase is close enough to the first phase to be considered the same
as phase one.

18. Past Footprint
Past Footprint Table
Accumulator
430
Branch PC 21 20 9 10 80
Hash
# of inst. since branch 70 +
*At 30 million instructions
Manhattan distance between this phase and first phase is 2.
This phase is close enough to the first phase to be considered the same
as phase one.

19. Outline Phase Tracking Phase Prediction Phase Based Branch Prediction
Experiments and Results
Conclusions
Future Work

20. Phase prediction To adjust hardware Three strategies
Need to know in advance what phase we will be in
Three strategies
Last seen
Markov with RLE
Perceptron

21. Last seen Predict next phase = last phase
Because last seen is so simple, another predictor would have to beat it significantly to justify the added cost

22. RLE Markov Adapted from Sherwood
Assumes that if we see phase X exactly Y times in a row, followed by phase Z, then if we see phase X exactly Y times again, it will again be followed by Z

23. Perceptron Individual perceptrons work in binary (±1)
Compute S as a function of history
If S ≥ 0, predict “yes”, else predict “no”
Train by adjusting weights for different components of history
But there are many phases, not just 2
Combine perceptrons for multivalue prediction

24. Multivalue perceptron
We have perceptrons P1, P2, …, Pn
Perceptron Pi tries to predict phase i
Train Pi to compute Si only if in phase i
The perceptron with the maximum value above a certain threshold wins

25. Phase prediction results
GCC:
Last phase: 96% accurate
RLE Markov: 94% accurate
Perceptron: much lower

26. Phase prediction comments
Training cost of multiple perceptron means that it does not always adapt quickly
Not worth improving due to the accuracy of last phase

27. Outline Phase Tracking Phase Prediction Phase Based Branch Prediction
Experiments and Results
Conclusions
Future Work

28. Phase Based Dynamic Branch Predictor
Previous research shows the usefulness of adapting branch predictors at run time
“Dynamic history-length fitting: a third level of adaptivity for branch prediction” [Juan, Sanjeevan, Navarro].
“Combining Branch Predictors” [McFarling]
Single branch predictor may not perform well within and across different executions.
“A study of Branch Prediction Strategies” [Smith]
Program behavior almost uniform within a phase -> choose best predictor for each phase

29. Methodology Select a small group of relevant predictors
At the beginning of each new phase, sample all the predictors and choose the best
Save the best for each phase and use it if a phase reoccurs

30. Methodology Select a small group of relevant predictors
At the beginning of each new phase, sample all the predictors and choose the best
Save the best for each phase and use it if a phase reoccurs

31. Methodology Select a small group of relevant predictors
At the beginning of each new phase, sample all the predictors and choose the best
Save the best for each phase and use it if a phase reoccurs

32. Methodology Select a small group of relevant predictors
At the beginning of each new phase, sample all the predictors and choose the best
Save the best for each phase and use it if a phase reoccurs

33. Methodology Select a small group of relevant predictors
At the beginning of each new phase, sample all the predictors and choose the best
Save the best for each phase and use it if a phase reoccurs
Phase 1

34. Methodology Select a small group of relevant predictors
At the beginning of each new phase, sample all the predictors and choose the best
Save the best for each phase and use it if a phase reoccurs
Phase 1
Phase 2

35. Outline Phase Tracking Phase Prediction Phase Based Branch Prediction
Experiments and Results
Conclusions
Future Work

36. Multiple Branch Predictors
Set of predictors
2level [1:1024:8] (Baseline predictor)
Bimodal [1024]
2level [8: 512 :8]
2level [1: 512 :8]
Profiling period
10 million instructions

37. Multiple Branch Predictors
Simulator Used
Simplescalar v3.0d
Set of benchmarks
gcc, vpr, mcf
Selection Criterion
Least Miss Rate
If miss rates of two predictors are within 1 %, select the less expensive (simpler) one

38. Multiple Branch Predictor : Results IPC (gcc)

39. Multiple Branch Predictors: Results Branch Predictor Misses (gcc)

40. Multiple Branch Predictors: Results Branch Predictor Misses (mcf)

41. Multiple Branch Predictors IPC Comparison

42. Multiple Branch Predictors Branch Prediction Misses Comparison

43. Outline Phase Tracking Phase Prediction Phase Based Branch Prediction
Experiments and Results
Summary and Conclusions
Future Work

44. Summary
Significant reduction in branch mispredictions (29.88% %) using phase based branch predictors
Simple predictors beat more complex predictor in many phases
Marginal gains in IPC using multiple branch predictor (2.24% %)

45. Conclusions
Phase based optimizations provides scope for improvements using reconfigurable hardware
Using phase specific branch predictor provides good improvements in mis predictions
A good strategy for saving power because of significant reductions in mispredictions.

46. Outline Phase Tracking Phase Prediction Phase Based Branch Prediction
Experiments and Results
Summary and Conclusions
Future Work

47. Future Work Investigate in detail the impact of design parameters
Phase detection threshold
Impact of hash functions
Investigate the perceptron model in detail
Investigate more benchmarks
Comparing against more complex baseline predictors
McFarling Branch Predictor
Other Hybrid predictors
Direct measurement of power characteristics
Measure power using simulators like WATTCH
Power consumed during switching branch predictors

48. Thank You

49. Questions??

50. Multiple Branch Predictors: Results Branch Predictor Misses (mcf)

51. Multiple Branch Predictor : Results IPC (vpr)

52. Multiple Branch Predictors: Results Branch Predictor Misses (vpr)

53. Phase prediction comments
Sherwood had lower accuracy for last phase (70%), perhaps due to oscillation
Training cost of multiple perceptron means that it does not always adapt quickly
Not worth improving due to the accuracy of last phase