1 Dynamically Heterogeneous Cores Through 3D Resource Pooling Houman Homayoun Vasileios Kontorinis Amirali Shayan Ta-Wei Lin Dean M. Tullsen Speaker: Houman Homayoun National Science Foundation CI Fellow University of California San Diego

Why Heterogeneity? 2 Existing General Purpose CMP designs use only homogeneous cores A general purpose one-size-fits-all core is not necessarily the most efficient One processor optimized for each application! Core 1 Core 2

Static vs. Dynamic Heterogeneity 3 Prior proposals (e.g., Kumar 2003) propose static heterogeneity. Increases chance of finding an appropriate core Does not guarantee perfect match Others have proposed solutions for dynamic heterogeneity (Core Fusion, TFlex). Due to the difficult of sharing resources at a fine granularity, they enable only coarse- grain sharing. Big (combined) cores or small cores.

4 Outline Resource Pooling Why 3D? Design Solutions Adaptive Policies Results Conclusion

Application Resource Utilization 5

6 ROBLDSQRFIQ Application Resource Utilization

7 Application 1 Application 2 underutilized ROBLDSQRFIQ ROBLDSQRFIQ Application Resource Utilization Dual-Core Machine

Dynamic Heterogeneity Through Resource Pooling 8 Register File ROB Register File ROB Core 2 Core 1

9 Outline Need for Heterogeneity Why 3D? Design Solutions Adaptive Policies Results Conclusion

Why NOT Sharing in 2D? 10 Long wire delay in 2D In 2D, it is not efficient Demanding 500 psec 5 nsec

11 Our Solution: 3D

12 Our Solution: 3D Fast interconnection network As fast as few ps (three order of magnitude smaller than 2D) Minimize the Communication Latency 5 psec 5000 psec A principal advantage No change to the fundamental pipeline design of 2D architectures, yet still exploits the 3D to provide greater energy proportionality and core customization

13 Need for Heterogeneity Why 3D? Design Solutions Adaptive Policies Results Conclusion Outline

Stackable Structures for Resource Pooling Performance bottleneck and power hungry resources Reorder Buffer and Register File (SRAM) Instruction Queue and Load and Store Queue (CAM+SRAM) Our goal: share units across multiple cores with minimal impact on design spec (latency, number of ports and power) Use previously proposed modular design Each partition is a self-standing and independently usable unit Effective in reducing power and access delay 14 Independent partition Part 1 Part 2 Part 3 Part 4 Register File

Example of Resource Sharing 15 Decoder MUX TSV Register File in Core 0 Register File in Core 1 Free Partition Additional logic to decide whether partition is empty Additional logic to route the signal to the right partition

16 Need for Heterogeneity Why 3D? Design Solutions Adaptive Policies Results Conclusion Outline

Adaptive Policies for Resource Pooling Several issues need to be considered Ownership Fast releasing Fast reallocation Cycle by cycle adaptation Prevent starvation A simple adaptive policy specification (MinMax policy) Set limit for the size of resources how much they can grow up to (MAX) or they can shrink down to (MIN) Use free list Use central arbitration 17

18 Arbitration Unit Core 1 Core 2Core 3 Core 4 Free List Application 1Application 2 Application 3Application 4 Register File MinMax Policy Example MIN

19 Arbitration Unit Core 1 Core 2Core 3 Core 4 Free List Application 1Application 2 Application 3Application 4 Register File MinMax Policy Example MIN

20 Arbitration Unit Core 1 Core 2Core 3 Core 4 Free List Application 1Application 2 Application 3Application 4 Register File MinMax Policy Example MIN

21 Arbitration Unit Core 1 Core 2Core 3 Core 4 Free List Application 1Application 2 Application 3Application 4 Register File MinMax Policy Example MIN

22 Need for Heterogeneity Why 3D? Design Solutions Adaptive Policies Results Conclusion Outline

Baseline Architecture 23 Processor Model High-end architecture, four OoO cores with issue width of 4 Medium-end architecture, four OoO cores with issue width of 2 3D Floorplans (different performance, flexibility, and temperature tradeoff) (1) Conventional (Thermal-Optimized Design) (2) Proposed (Performance-Optimized Design) (1)(2)

Evaluation 24 1 Thread 4 Thread 2 Thread Power Performance Temperature Energy-Delay Core 1 Core 2 Core 3 Core 4 Active core Idle core Link

Single Thread Performance 25 Speed Up Standard SPEC2K and SPEC2006 Benchmark Single benchmark (3 out of 4 cores are idle)

Multi-Thread Performance 2Thr: 2 idle cores + underutilized resources in the active cores 4Thr: No idle cores, only underutilized resources 26 Normalized Weighted Speedup (%) gains are dramatic when some cores are idle

Medium-end vs High-end Resource pooling makes the medium core significantly more competitive with the high-end. 27 Normalized Weighted Speedup (%) 28% 14% Only 3%! 0 Idle Core 2 Idle Core3 Idle Core Increase Resource Sharing

28 power (Watt) 3X 4X Pooling pay a small price in power Because of the enhanced throughput. Large speedups on low-IPC threads and high average speedup, but smaller increase in total instruction throughput and thus smaller increase in power Power

29 temperature (Celsius) Temperature Interestingly, the temperature of the medium resource-pooling core is comparable to the high-end core

Efficiency 30 Even still, at equal temperature, the more modest cores have a significant advantage in energy efficiency measured in MIPS 2 /W (MIPS 2 /W is the inverse of energy-delay product) Normalized 2X

Conclusions Homogeneous cores are inherently inefficient for a diverse workload. Cores are typically overprovisioned as a result 3D stacking of cores enables fine-grain sharing (pooling) of resources not possible in 2D designs. Our dynamically heterogeneous 3D architecture allows the processor to construct the right core for each application dynamically, maximizing energy efficiency. Our 3D pooling architecture Leverages our experience in 2D pipeline design, yet still gains significant benefit from 3D Adapts to the specific demands of an application within a few cycles. Reduces reliance on overprovisioned cores, instead grabbing larger resources only when needed. 31

End of presentation