Presentation is loading. Please wait.

Presentation is loading. Please wait.

POWER5 Ewen Cheslack-Postava Case Taintor Jake McPadden.

Similar presentations

Presentation on theme: "POWER5 Ewen Cheslack-Postava Case Taintor Jake McPadden."— Presentation transcript:

1 POWER5 Ewen Cheslack-Postava Case Taintor Jake McPadden

2 POWER5 Lineage IBM 801 – widely considered the first true RISC processor POWER1 – 3 chips wired together (branch, integer, floating point) POWER2 – Improved POWER1 – 2 nd FPU and added cache and 128 bit math POWER3 – moved to 64 bit architecture POWER4…

3 POWER4 Dual-core High-speed connections to up to 3 other pairs of POWER4 CPUs Ability to turn off pair of CPUs to increase throughput Apple G5 uses a single-core derivative of POWER4 (PowerPC 970) POWER5 designed to allow for POWER4 optimizations to carry over

4 Pipeline Requirements Maintain binary compatibility Maintain structural compatibility Optimizations for POWER4 carry forward Improved performance Enhancements for server virtualization Improved reliability, availability, and serviceability at chip and system levels

5 Pipeline Improvements Enhanced thread level parallelism Two threads per processor core a.k.a Simultaneous Multithreading (SMT) 2 threads/core * 2 cores/chip = 4 threads/chip Each thread has independent access to L2 cache Dynamic Power Management Reliability, Availability, and Serviceability


7 POWER5 Chip Stats Copper interconnects Decrease wire resistance and reduce delays in wire dominated chip timing paths 8 levels of metal 389 mm 2

8 POWER5 Chip Stats Silicon on Insulator devices (SOI) Thin layer of silicon (50nm to 100 µm) on insulating substrate, usually sapphire or silicon dioxide (80nm) Reduces electrical charge transistor has to move during switching operation (compared to CMOS) Increased speed (up to 15%) Reduced switching energy (up to 20%) Allows for higher clock frequencies (> 5GHz) SOI chips cost more to produce and are therefore used for high-end applications Reduces soft errors


10 Pipeline Pipeline identical to POWER4 All latencies including branch misprediction penalty and load-to-use latency with L1 data cache hit same as POWER4

11 POWER5 Pipeline IF – instruction fetch, IC – instruction cache, BP – branch predict, Dn – decode stage, Xfer – transfer, GD – group dispatch, MP – mapping, ISS – instruction issue, RF – register file read, EX – execute, EA – compute address, DC – data cache, F6 – six cycle floating point unit, Fmt – data format, WB – write back, CP – group commit

12 Instruction Data Flow LSU – load/store unit, FXU – fixed point execution unit, FPU – floating point unit, BXU – branch execution unit, CRL – condition register logical execution unit

13 Instruction Fetch Fetch up to 8 instructions per cycle from instruction cache Instruction cache and instruction translation shared between threads One thread fetching per cycle

14 Branch Prediction Three branch history tables shared by 2 threads 1 bimodal, 1 path-correlated prediction 1 to predict which of the first 2 is correct Can predict all branches – even if every instruction fetched is a branch

15 Branch Prediction Branch to link register (bclr) and branch to count register targets predicted using return address stack and count cache mechanism Absolute and relative branch targets computed directly in branch scan function Branches entered in branch information queue (BIQ) and deallocated in program order

16 Instruction Grouping Separate instruction buffers for each thread 24 instructions / buffer 5 instructions fetched from 1 thread’s buffer and form instruction group All instructions in a group decoded in parallel

17 Group Dispatch & Register Renaming When all resources necessary for group are available, group is dispatched (GD) D0 – GD: instructions still in program order MP – register renaming, registers mapped to physical registers Register files shared dynamically by two threads In ST mode all registers are available to single thread Placed in shared issue queues

18 Group Tracking Instructions tracked as group to simplify tracking logic Control information placed in global completion table (GCT) at dispatch Entries allocated in program order, but threads may have intermingled entries Entries in GCT deallocated when group is committed

19 Load/Store Reorder Queues Load reorder queue (LRQ) and store reorder queue (SRQ) maintain program order of loads/stores within a thread Allow for checking of address conflicts between loads and stores

20 Instruction Issue No distinction made between instructions for different threads No priority difference between threads Independent of GCT group of instruction Up to 8 instructions can issue per cycle Instructions then flow through execution units and write back stage

21 Group Commit Group commit (CP) happens when all instructions in group have executed without exceptions and the group is the oldest group in its thread One group can commit per cycle from each thread

22 Enhancements to Support SMT Instruction and data caches same size as POWER4 but double to 2 and 4 way associativity respectively IC and DC entries can be fully shared between threads

23 Enhancements to Support SMT Two step address translation Effective address  Virtual Address using 64 entry segment lookaside buffer (SLB) Virtual address  Physical Address using hashed page table, cached in a 1024 entry four way set associative TLB Two first level translation tables (instruction, data) SLB and TLB only used in case of first-level miss

24 Enhancements to Support SMT First Level Data Translation Table – fully associative 128 entry table First Level Instruction Translation Table – 2-way set associative 128 entry table Entries in both tables tagged with thread number and not shared between threads Entries in TLB can be shared between threads


26 Enhancements to Support SMT LRQ and SRQ for each thread, 16 entries But threads can run out of queue space – add 32 virtual entries, 16 per thread Virtual entries – contain enough information to identify the instruction, but not address for load/store Low cost way to extend LRQ/SRQ and not stall instruction dispatch

27 Enhancements to Support SMT Branch Information Queue (BIQ) 16 entries (same as POWER4) Split in half for SMT mode Performance modeling suggested this was a sufficient solution Load Miss Queue (LMQ) 8 entries (same as POWER4) Added thread bit to allow dynamic sharing

28 Enhancements to Support SMT Dynamic Resource Balancing Resource balancing logic monitors resources (e.g. GCT and LMQ) to determine if one thread exceeds threshold Offending thread can be throttled back to allow sibling to continue to progress Methods of throttling Reduce thread priority (using too many GCT entries) Inhibit instruction decoding until congestion clears (incurs too many L2 cache misses) Flush all thread instructions waiting for dispatch and stop thread from decoding instructions until congestion clears (executing instruction that takes a long time to complete)

29 Enhancements to Support SMT Thread priority Supports 8 levels of priority 0  not running 1  lowest, 7  highest Give thread with higher priority additional decode cycles Both threads at lowest priority  power saving mode

30 Single Threaded Mode All rename registers, issue queues, LRQ, and SRQ are available to the active thread Allows higher performance than POWER4 at equivalent frequencies Software can change processor dynamically between single threaded and SMT mode

31 RAS of POWER4 High availability in POWER4 Minimize component failure rates Designed using techniques that permit hard and soft failure detection, recovery, isolation, repair deferral, and component replacement while system is operating Fault tolerant techniques used for array, logic, storage, and I/O systems Fault isolation and recovery

32 RAS of POWER5 Same techniques as POWER4 New emphasis on reducing scheduled outages to further improve system availability Firmware upgrades on running machine ECC on all system interconnects Single bit interconnect failures dynamically corrected Deferred repair scheduled for persistent failures Source of errors can be determined – for non recoverable error, system taken down, book containing fault taken offline, system rebooted – no human intervention Thermal protection sensors

33 Dynamic Power Management Reduce switching power Clock gating Reduce leakage power Minimum low-threshold transistors Low power mode Two stage fix for excess heat Stage 1: alternate stalls and execution until the chip cools Stage 2: clock throttling

34 Effects of dynamic power management with and without simultaneous multithreading enabled. Photographs were taken with a heat-sensitive camera while a prototype POWER5 chip was undergoing tests in the laboratory.

35 Memory Subsystem Memory controller and L3 directory moved on-chip Interfaces with DDR1 or DDR2 memory Error correction/detection handled by ECC Memory scrubbing for “soft errors” Error correction while idle

36 Cache Sizes L1 I-cache64K L1 D-cache32K L21.92M L336M

37 Cache Hierarchy

38 Reads from memory are written into L2 L2 and L3 are shared between cores L3 (36MB) acts as a victim cache for L2 Cache line is reloaded into L2 if there is a hit in L3 Write back to main memory if line in L3 is dirty and evicted


40 Important Notes on Diagram Three buses between the controller and the SMI chips Address/command bus Unidirectional write data bus (8 bytes) Unidirectional read data bus (16 bytes) Each bus operates at twice the DIMM speed

41 Important Notes on Diagram 2 or 4 SMI chips can be used Each SMI can interface with two DIMMs 2 SMI mode – 8-byte read, 2 byte write 4 SMI mode – 4-byte read, 2 byte write

42 Size does matter POWER5Pentium III

43 …compensating?

44 Possible configurations DCM (Dual Chip Module) One POWER5 chip, one L3 chip MCM (Multi Chip Module) Four POWER5 chips, four L3 chips Communication is handled by a Fabric Bus Controller (FBC) “distributed switch”

45 Typical Configurations 2 MCMs to form a “book” 16 way symmetric multi-processor (appears as 32 way) DCM books also used

46 Fabric Bus Controller “buffers and sequences operations among the L2/L3, the functional units of the memory subsystem, the fabric buses that interconnect POWER5 chips on the MCM, and the fabric buses that interconnect multiple MCMs” Separate address and data buses to facilitate split transactions Each transaction tagged to allow for out of order replies

47 16-way system built with eight dual-chip modules.

48 Address Bus Addresses broadcasted from MCM to MCM using ring structure Each chip forwards address down the ring and to the other chip in MCM Forwarding ends when originating chip receives address

49 Response Bus Includes coherency information gleaned from memory subsystem “snooping” One chip in MCM combines other three chips’ snoop responses with the previous MCM snoop response and forwards it on

50 Response Bus When originating chip receives the responses, transmits a “combined response” which details actions to be taken Early combined response mechanism Each MCM determines whether to send a cache line from L2/L3 depending on previous snoop responses Reduces cache-to-cache latency

51 Data Bus Services all data-only transfers (such as cache interventions) Services data-portion of address ops Cache eviction (“cast-out”) Snoop pushes DMA writes

52 eFuse Technology IBM has created a method of morphing processors to increase efficiency Chips physically alter their design prior to or while functioning Electromigration, previously a serious liability, is detected and used to determine the best way to improve the chip

53 The Method The idea is similar to traffic control on busy highways A lane can be used for the direction with the most traffic. So fuses and software algorithm are used to detect circuits that are experiencing various levels of traffic

54 Method Cont. The chip contains millions of micro- fuses The fuses act autonomously to reduce voltage on underused circuits, and share the load of overused circuits Furthermore, the Chip can “repair” itself in the case of design flaws or physical circuit failures.

55 Method Cont. The idea is not new It has been attempted before, however previous fuses have affected or damaged the processor

56 Cons of eFuse Being able to allocate circuitry throughout a processor implies significant redundancy Production costs are increased because extra circuitry is required Over-clocking is countered by the processor automatically detecting a flaw

57 Pros of eFuse Significant savings in optimization cost The need to counter electromigration is no longer as important The same processor architecture can be altered in the factory, for a more specialized task. Self-repair and self-upgrading

58 Benchmarks: SPEC Results

59 Benchmarks

60 Notes on Benchmarks IBM’s 64 processor system beat the previous leader (HP’s 64 processor system) by 3x tpmC IBM’s 32 processor system beat HP’s 64 processor system by about 1.5x Both of IBM’s systems maintain lower price/tpmC


62 POWER5 Hypervisor A hypervisor is the virtualization of a machine allowing multiple operating systems to run on a single system While the HV is not new, IBM has made many important improvements to the design of HV with the POWER5

63 Hypervisor Cont. The main purpose is to abstract the hardware, and divide it logically between tasks Hardware considerations: Processor time Memory I/O

64 Hypervisor – Processor The POWER5 hypervisor virtualizes processors A virtual processor is given to each LPAR The virtual processor is given a percentage of processing time on one or more processors Processor time is determined by preset importance values and excess time

65 HV Processor Cont. Processor time is defined by processing units, which = 1/100 of a CPU. A LPAR is given an entitlement in processing units and a weight Weight is a number between 1 and 256 that implies the importance of the LPAR and its priority in receiving excess PU’s

66 HV Processor Cont. Dynamic micro-partitioning The resources distribution can be further changed in DLPAR These LPARs are put in the control of PLM that can actively change their entitlement In place of a weight, DLPARs are given a “share” value, that indicates its value If a partition is very busy it will be allocated more PU’s, and vice versa.

67 Hypervisor – I/O IBM chose not to give the HV direct control over I/O devices Instead the HV delegates responsibility of I/O devices to specific LPAR A LPAR partition serves as a virtual device for the other LPARs on the system The HV can also give a LPAR sole control over an I/O device.

68 HV – I/O Cont.

69 Hypervisor – SCSI Storage devices are virtualized using SCSI The hypervisor acts as a messenger from between partitions. A queue is implemented and the respective partitions is informed of increases in its queue length The OS running on the controlling partition is in complete control of SCSI execution

70 Hypervisor – Network The HV operates a VLAN within its controlled partitions The VLAN is simplified, secure, and efficient Each Partition is made aware of a virtual network adaptor Each partition is labeled with a virtual MAC address and the HV acts as the switch

71 HV – Network Cont. However, if the MAC address is unknown to the HV it will deliver the packet to a partition with a physical network adaptor This partition will deal with external LAN issues such as address translation, synchronization, and packet size

72 Hypervisor – Console Virtual Consoles are handled in much the same way. The Console can also be assigned by the controlling partition as a monitor on an extended network. The controlling partition can completely simulate the monitors presence through the HV for mundane requirements.

Download ppt "POWER5 Ewen Cheslack-Postava Case Taintor Jake McPadden."

Similar presentations

Ads by Google