Presentation is loading. Please wait.

Presentation is loading. Please wait.

ASR: Adaptive Selective Replication for CMP Caches

Published byBaldric Jacobs Modified over 6 years ago

Similar presentations

Presentation on theme: "ASR: Adaptive Selective Replication for CMP Caches"— Presentation transcript:

1 ASR: Adaptive Selective Replication for CMP Caches
Brad Beckmann†, Mike Marty, and David Wood Multifacet Project University of Wisconsin-Madison 12/13/06 † currently at Microsoft

2 Introduction: Shared Cache
L1 I $ A Maximize Cache Capacity L2 Bank L2 Bank L1 I $ CPU 3 CPU 4 L1 D $ L1 D $ 40+ Cycles Slow Access Latency L1 I $ L2 Bank L2 Bank L1 I $ CPU 2 CPU 5 L1 D $ L1 D $ L1 I $ L1 I $ L2 Bank L2 Bank CPU 1 CPU 6 L1 D $ L1 D $ L1 I $ L1 I $ L2 Bank L2 Bank CPU 0 CPU 7 L1 D $ L1 D $

3 Introduction: Private Caches
L1 I $ Fast Access Latency L1 I $ CPU 3 Private Private CPU 4 L1 D $ L2 L2 L1 D $ A Lower Effective Capacity L1 I $ L1 I $ CPU 2 Private Private CPU 5 L1 D $ L2 L2 L1 D $ L1 I $ L1 I $ CPU 1 Private Private CPU 6 L1 D $ L2 L2 L1 D $ Desire both Fast Access & High Capacity L1 I $ L1 I $ Private CPU 0 Private CPU 7 L1 D $ L2 L2 L1 D $

4 ASR: Adaptive Selective Replication for CMP Caches
Introduction Previous hybrid proposals Victim Replication, CMP-NuRapid, Cooperative Caching Achieve fast access and high capacity Under certain workloads & system configurations Utilize static rules Non-adaptive Adaptive Selective Replication: ASR Dynamically monitor workload behavior Adapt the L2 cache to workload demand Up to 12% improvement vs. previous proposals Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

5 ASR: Adaptive Selective Replication for CMP Caches
Outline Introduction Understanding L2 Replication Benefit Cost Key Observation Solution ASR: Adaptive Selective Replication Evaluation Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches 5 5

6 Understanding L2 Replication
Three L2 block sharing types Single requestor All requests by a single processor Shared read only Read only requests by multiple processors Shared read-write Read and write requests by multiple processors Profile L2 blocks during their on-chip lifetime 8 processor CMP 16 MB shared L2 cache 64-byte block size Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

7 Understanding L2 Replication
High Locality Low Locality Apache Jbb Oltp Zeus Mid Locality Shared Read-only Shared Read-write Single Requestor Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

8 Understanding L2 Replication: Benefit
L2 Hit Cycles Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

9 Understanding L2 Replication: Cost
L2 Miss Cycles Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

10 Understanding L2 Replication: Key Observation
L2 Hit Cycles Replication Capacity Replicate Frequently Requested Blocks First Top 3% of Shared Read-only blocks satisfy 70% of Shared Read-only requests Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches 10 10

11 Understanding L2 Replication: Solution
Total Cycle Curve Property of Workload Cache Interaction Not Fixed  Must Adapt Optimal Total Cycles Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

12 ASR: Adaptive Selective Replication for CMP Caches
Outline Wires and CMP caches Understanding L2 Replication ASR: Adaptive Selective Replication SPR: Selective Probabilistic Replication Monitoring and adapting to workload behavior Evaluation Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

13 SPR: Selective Probabilistic Replication
Mechanism for Selective Replication Relax L2 inclusion property L2 evictions do not force L1 evictions Non-exclusive cache hierarchy Ring Writebacks L1 Writebacks passed clockwise between private L2 caches Merge with other existing L2 copies Probabilistically choose between Local writeback  allow replication Ring writeback  disallow replication Replicates frequently requested blocks Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

14 SPR: Selective Probabilistic Replication
Private L2 Private L2 L1 I $ D $ CPU 3 CPU 4 L1 D $ L1 I $ Private L2 Private L2 CPU 2 CPU 5 L1 D $ L1 I $ Private L2 Private L2 CPU 1 CPU 6 L1 D $ L1 I $ Private L2 Private L2 CPU 0 CPU 7 L1 D $

15 SPR: Selective Probabilistic Replication
Replication Level 1 2 3 4 5 Prob. of Replication 1/64 1/16 1/4 1/2 Current Level Replication Capacity 1 2 3 4 5 Replication Levels Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

16 Monitoring and Adapting to Workload Behavior
Replication Benefit Curve lower level L2 Hit Cycles higher level current level Replication Capacity Decrease in Replication Benefit Bit marks replicas of the current, but not lower level Increase in Replication Benefit Store 8-bit partial tags of next higher level replications Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

17 Monitoring and Adapting to Workload Behavior
Replication Cost Curve L2 Miss Cycles higher level lower level current level Replication Capacity 3. Decrease in Replication Cost Stores 16-bit partial tags of recently evicted blocks 4. Increase in Replication Cost Way and Set counters track soon-to-be-evicted blocks Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches 17 17

18 ASR: Adaptive Selective Replication for CMP Caches
Outline Wires and CMP caches Understanding L2 Replication ASR: Adaptive Selective Replication Evaluation Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches 18 18

19 ASR: Adaptive Selective Replication for CMP Caches
Methodology Full system simulation Simics Wisconsin’s GEMS Timing Simulator Out-of-order processor Memory system Workloads Commercial apache, jbb, otlp, zeus Scientific (see paper) SpecOMP: apsi & art Splash: barnes & ocean Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

20 Dynamically Scheduled Processor
System Parameters [ 8 core CMP, 45 nm technology ] Memory System Dynamically Scheduled Processor L1 I & D caches 64 KB, 4-way, 3 cycles Clock frequency 5.0 GHz Unified L2 cache 16 MB, 16-way Reorder buffer / scheduler 128 / 64 entries L1 / L2 prefetching Unit & Non-unit strided prefetcher (similar Power4) Pipeline width 4-wide fetch & issue Memory latency 500 cycles Pipeline stages 30 Memory bandwidth 50 GB/s Direct branch predictor 3.5 KB YAGS Memory size 4 GB of DRAM Return address stack 64 entries Outstanding memory request / CPU 16 Indirect branch predictor 256 entries (cascaded) Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

21 Replication Benefit, Cost, & Effectiveness Curves
Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

22 Replication Benefit, Cost, & Effectiveness Curves
Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

23 Comparison of Replication Policies
SPR  multiple possible policies Evaluated 4 shared read-only replication policies VR: Victim Replication Previously proposed [Zhang ISCA 05] Disallow replicas to evict shared owner blocks NR: CMP-NuRapid Previously proposed [Chishti ISCA 05] Replicate upon the second request CC: Cooperative Caching Previously proposed [Chang ISCA 06] Replace replicas first Spill singlets to remote caches Tunable parameter 100%, 70%, 30%, 0% ASR: Adaptive Selective Replication Our proposal Monitor and adjust to workload demand Lack Dynamic Adaptation Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

24 ASR: Adaptive Selective Replication for CMP Caches
ASR: Performance S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

25 ASR: Adaptive Selective Replication for CMP Caches
Conclusions CMP Cache Replication No replications  conservers capacity All replications  reduces on-chip latency Previous hybrid proposals Work well for certain criteria Non-adaptive Adaptive Selective Replication Probabilistic policy favors frequently requested blocks Dynamically monitor replication benefit & cost Replicate benefit > cost Improves performance up to 12% vs. previous schemes Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

26 Backup Slides

27 ASR: Adaptive Selective Replication for CMP Caches
ASR: Memory Cycles S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

28 L2 Cache Requests Breakdown

29 L2 Cache Requests Breakdown: User & OS

30 Shared Read-write Requests Breakdown

31 Shared Read-write Block Breakdown

32 ASR: Decrease-in-replication Benefit
lower level L2 Hit Cycles current level Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

33 ASR: Decrease-in-replication Benefit
Goal Determine replication benefit decrease of the next lower level Mechanism Current Replica Bit Per L2 cache block Set for replications of the current level Not set for replications of lower level Current replica hits would be remote hits with next lower level Overhead 1-bit x 256 K L2 blocks = 32 KB Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

34 ASR: Increase-in-replication Benefit
L2 Hit Cycles current level higher level Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

35 ASR: Increase-in-replication Benefit
Goal Determine replication benefit increase of the next higher level Mechanism Next Level Hit Buffers (NLHBs) 8-bit partial tag buffer Store replicas of the next higher NLHB hits would be local L2 hits with next higher level Overhead 8-bits x 16 K entries x 8 processors = 128 KB Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

36 ASR: Decrease-in-replication Cost
L2 Miss Cycles current level lower level Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

37 ASR: Decrease-in-replication Cost
Goal Determine replication cost decrease of the next lower level Mechanism Victim Tag Buffers (VTBs) 16-bit partial tags Store recently evicted blocks of current replication level VTB hits would be on-chip hits with next lower level Overhead 16-bits x 1 K entry x 8 processors = 16 KB Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

38 ASR: Increase-in-replication Cost
higher level L2 Miss Cycles current level Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

39 ASR: Increase-in-replication Cost
Goal Determine replication cost increase of the next higher level Mechanism Way and Set counters [Suh et al. HPCA 2002] Identify soon-to-be-evicted blocks 16-way pseudo LRU 256 set groups On-chip hits that would be off-chip with next higher level Overhead 255-bit pseudo LRU tree x 8 processors = 255 B Overall storage overhead: 212 KB or 1.2% of total storage Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

40 ASR: Triggering a Cost-Benefit Analysis
Goal Dynamically adapt to workload behavior Avoid unnecessary replication level changes Mechanism Evaluation trigger Local replications or NLHB allocations exceed 1K Replication change Four consecutive evaluations in the same direction Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

41 ASR: Adaptive Algorithm
Decrease in Replication Cost > Increase in Replication Benefit Decrease in Replication Cost < Increase in Replication Benefit Decrease in Replication Benefit > Increase in Replication Cost Go in direction with greater value Increase Replication Decrease in Replication Benefit < Increase in Replication Cost Decrease Do Nothing Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

42 ASR: Adapting to Workload Behavior
Oltp: All CPUs Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

43 ASR: Adapting to Workload Behavior
Apache: All CPUs Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

44 ASR: Adapting to Workload Behavior
Apache: CPU 0 Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

45 ASR: Adapting to Workload Behavior
Apache: CPUs 1-7 Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

46 ASR: Adaptive Selective Replication for CMP Caches
Replication Capacity Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

47 ASR: Adaptive Selective Replication for CMP Caches
Replication Capacity 4 MB 150 Memory Latency In-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

48 Replication Benefit, Cost, & Effectiveness Curves
4 MB 150 Memory Latency In-order processors Benefit Cost Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

49 Replication Benefit, Cost, & Effectiveness Curves
4 MB 150 Memory Latency In-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

50 Replication Benefit, Cost, & Effectiveness Curves
16 MB 500 Memory Latency In-order processors Benefit Cost Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

51 Replication Benefit, Cost, & Effectiveness Curves
16 MB 500 Memory Latency In-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

52 Replication Analytic Model
Utilize workload characterization data Goal: initutition not accuracy Optimal point of replication Sensitive to cache size Sensitive to memory latency Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

53 Replication Model: Selective Replication
Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

54 ASR: Adaptive Selective Replication for CMP Caches
ASR: Memory Cycles S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR 4 MB 150 Memory Latency In-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

55 ASR: Adaptive Selective Replication for CMP Caches
ASR: Performance S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR 4 MB 150 Memory Latency In-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

56 ASR: Adaptive Selective Replication for CMP Caches
ASR: Memory Cycles S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR 16 MB 250 Memory Latency Out-of-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

57 ASR: Adaptive Selective Replication for CMP Caches
ASR: Performance S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR 16 MB 250 Memory Latency Out-of-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

58 ASR: Adaptive Selective Replication for CMP Caches
ASR: Memory Cycles S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR 16 MB 500 Memory Latency Out-of-order processors Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

59 ASR: Adaptive Selective Replication for CMP Caches
ASR: Performance 16 MB 500 Memory Latency Out-of-order processors S: CMP-Shared P: CMP-Private V: SPR-VR N: SPR-NR C: SPR-CC A: SPR-ASR Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

60 ASR: Adaptive Selective Replication for CMP Caches
Token Coherence Proposed for SMPs [Martin 03], CMPs [Marty 05] Provides a simple correctness substrate One token to read All tokens to write Advantages Permits a broadcast protocol on unordered network without acknowledgement messages Supports multiple allocation policies Disadvantages All blocks must be written back (cannot destroy tokens) Token counts at memory Persistent request can be a performance bottleneck Beckmann, Marty, & Wood ASR: Adaptive Selective Replication for CMP Caches

Download ppt "ASR: Adaptive Selective Replication for CMP Caches"

Similar presentations

Virtual Hierarchies to Support Server Consolidation Michael Marty and Mark Hill University of Wisconsin - Madison.

Virtual Hierarchies to Support Server Consolidation Michael Marty and Mark Hill University of Wisconsin - Madison.
A Framework for Coarse-Grain Optimizations in the On-Chip Memory Hierarchy J. Zebchuk, E. Safi, and A. Moshovos.

A Framework for Coarse-Grain Optimizations in the On-Chip Memory Hierarchy J. Zebchuk, E. Safi, and A. Moshovos.
Lecture 19: Cache Basics Today’s topics: Out-of-order execution

Lecture 19: Cache Basics Today’s topics: Out-of-order execution
Coherence Ordering for Ring-based Chip Multiprocessors Mike Marty and Mark D. Hill University of Wisconsin-Madison.

Coherence Ordering for Ring-based Chip Multiprocessors Mike Marty and Mark D. Hill University of Wisconsin-Madison.
Jaewoong Sim Alaa R. Alameldeen Zeshan Chishti Chris Wilkerson Hyesoon Kim MICRO-47 | December 2014.

Jaewoong Sim Alaa R. Alameldeen Zeshan Chishti Chris Wilkerson Hyesoon Kim MICRO-47 | December 2014.
1 A Hybrid Adaptive Feedback Based Prefetcher Santhosh Verma, David Koppelman and Lu Peng Louisiana State University.

1 A Hybrid Adaptive Feedback Based Prefetcher Santhosh Verma, David Koppelman and Lu Peng Louisiana State University.
To Include or Not to Include? Natalie Enright Dana Vantrease.

To Include or Not to Include? Natalie Enright Dana Vantrease.
1 Lecture 17: Large Cache Design Papers: Managing Distributed, Shared L2 Caches through OS-Level Page Allocation, Cho and Jin, MICRO’06 Co-Operative Caching.

1 Lecture 17: Large Cache Design Papers: Managing Distributed, Shared L2 Caches through OS-Level Page Allocation, Cho and Jin, MICRO’06 Co-Operative Caching.
Managing Wire Delay in Large CMP Caches Bradford M. Beckmann David A. Wood Multifacet Project University of Wisconsin-Madison MICRO 2004 12/8/04.

Managing Wire Delay in Large CMP Caches Bradford M. Beckmann David A. Wood Multifacet Project University of Wisconsin-Madison MICRO /8/04.
ACM: An Efficient Approach for Managing Shared Caches in Chip Multiprocessors Mohammad Hammoud, Sangyeun Cho, and Rami Melhem Presenter: Socrates Demetriades.

ACM: An Efficient Approach for Managing Shared Caches in Chip Multiprocessors Mohammad Hammoud, Sangyeun Cho, and Rami Melhem Presenter: Socrates Demetriades.
A KTEC Center of Excellence 1 Cooperative Caching for Chip Multiprocessors Jichuan Chang and Gurindar S. Sohi University of Wisconsin-Madison.

A KTEC Center of Excellence 1 Cooperative Caching for Chip Multiprocessors Jichuan Chang and Gurindar S. Sohi University of Wisconsin-Madison.
1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.

1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.
August 8 th, 2011 Kevan Thompson Creating a Scalable Coherent L2 Cache.

August 8 th, 2011 Kevan Thompson Creating a Scalable Coherent L2 Cache.
High Performing Cache Hierarchies for Server Workloads

High Performing Cache Hierarchies for Server Workloads
CSE 490/590, Spring 2011 CSE 490/590 Computer Architecture Cache III Steve Ko Computer Sciences and Engineering University at Buffalo.

CSE 490/590, Spring 2011 CSE 490/590 Computer Architecture Cache III Steve Ko Computer Sciences and Engineering University at Buffalo.
FLEXclusion: Balancing Cache Capacity and On-chip Bandwidth via Flexible Exclusion Jaewoong Sim Jaekyu Lee Moinuddin K. Qureshi Hyesoon Kim.

FLEXclusion: Balancing Cache Capacity and On-chip Bandwidth via Flexible Exclusion Jaewoong Sim Jaekyu Lee Moinuddin K. Qureshi Hyesoon Kim.
1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.

1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.
The Locality-Aware Adaptive Cache Coherence Protocol George Kurian 1, Omer Khan 2, Srini Devadas 1 1 Massachusetts Institute of Technology 2 University.

The Locality-Aware Adaptive Cache Coherence Protocol George Kurian 1, Omer Khan 2, Srini Devadas 1 1 Massachusetts Institute of Technology 2 University.
Zhongkai Chen 3/25/2010. Jinglei Wang; Yibo Xue; Haixia Wang; Dongsheng Wang Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China This paper.

Zhongkai Chen 3/25/2010. Jinglei Wang; Yibo Xue; Haixia Wang; Dongsheng Wang Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China This paper.
Improving Cache Performance by Exploiting Read-Write Disparity

Improving Cache Performance by Exploiting Read-Write Disparity

Similar presentations

Ads by Google