1. 1 Research Accelerator for MultiProcessing Dave Patterson, UC Berkeley January 2006 + RAMP collaborators: Arvind (MIT), Krste Asanovíc (MIT), Derek Chiou (Texas), James Hoe (CMU), Christos Kozyrakis (Stanford), Shih-Lien Lu (Intel), Mark Oskin (Washington), David Patterson (Berkeley), Jan Rabaey (Berkeley), and John Wawrzynek (Berkeley)

2. 2 Old: Multiplies are slow, Memory access is fast New: “Memory wall” Memory slow, multiplies fast (200 clocks to DRAM memory, 4 clocks for multiply) Old: Power is free, Transistors expensive New: “Power wall” Power expensive, Xtors free (Can put more on chip than can afford to turn on) Old: Uniprocessor performance 2X / 1.5 yrs New: Power Wall + Memory Wall = Brick Wall  Uniprocessor performance only 2X / 5 yrs Sea change in chip design: multiple “cores” (2X processors per chip / ~ 2 years)  More instances of simpler processors are more power efficient Conventional Wisdom in Computer Architecture

3. 3 Uniprocessor Performance (SPECint) VAX : 25%/year 1978 to 1986 RISC + x86: 52%/year 1986 to 2002 RISC + x86: 20%/year 2002 to present From Hennessy and Patterson, Computer Architecture: A Quantitative Approach, 4th edition, 2006

4. 4 Sea Change in Chip Design Intel 4004 (1971): 4-bit processor, 2312 transistors, 0.4 MHz, 10 micron PMOS, 11 mm 2 chip Processor is the new transistor? RISC II (1983): 32-bit, 5 stage pipeline, 40,760 transistors, 3 MHz, 3 micron NMOS, 60 mm 2 chip 125 mm 2 chip, 0.065 micron CMOS = 2312 RISC II+FPU+Icache+Dcache   RISC II shrinks to ~ 0.02 mm 2 at 65 nm   Caches via DRAM or 1 transistor SRAM (www.t-ram.com ) ?www.t-ram.com   Proximity Communication via capacitive coupling at > 1 TB/s ? (Ivan Sutherland @ Sun / Berkeley)

5. 5 Déjà vu all over again? “… today’s processors … are nearing an impasse as technologies approach the speed of light..” David Mitchell, The Transputer: The Time Is Now (1989) Transputer had bad timing  Custom multiprocessors strove to lead uniprocessors  Procrastination rewarded: 2X seq. perf. / 1.5 years “We are dedicating all of our future product development to multicore designs. … This is a sea change in computing” Paul Otellini, President, Intel (2004) All microprocessor companies switch to MP  Procrastination penalized: 2X sequential perf. / 5 yrs  Biggest programming challenge: 1 to 2 CPUs

6. 6 Problems with Sea Change 1. Algorithms, Programming Languages, Compilers, Operating Systems, Architectures, Libraries, … not ready for 1000 CPUs / chip 2. Software people don’t start working hard until hardware arrives 3 months after HW arrives, SW people list everything that must be fixed, then we all wait 4 years for next iteration of HW/SW 3. How do research in timely fashion on 1000 CPU systems in algorithms, compilers, OS, architectures, … without waiting years between HW generations?

7. 7 Characteristics of Ideal Academic CS Research Supercomputer? Scale – Hard problems at 1000 CPUs Cheap – 2006 funding of academic research Cheap to operate, Small, Low Power – $ again Community – share SW, training, ideas, … Simplifies debugging – high SW churn rate Reconfigurable – test many parameters, imitate many ISAs, many organizations, … Credible – results translate to real computers Performance – run real OS and full apps, results overnight

8. 8 Build Academic SC from FPGAs As ~ 25 CPUs will fit in Field Programmable Gate Array (FPGA), 1000-CPU system from ~ 40 FPGAs? 16 32-bit simple “soft core” RISC at 150MHz in 2004 (Virtex-II) FPGA generations every 1.5 yrs; ~2X CPUs, ~1.2X clock rate HW research community does logic design (“gate shareware”) to create out-of-the-box, MPP that runs standard binaries of OS and applications  Gateware: Processors, Caches, Coherency, Ethernet Interfaces, Switches, Routers, … (some free from open source hardware)  E.g., 1000 processor, standard ISA binary-compatible, 64-bit, cache- coherent supercomputer @ 200 MHz/CPU in 2007

9. 9 Why RAMP Good for Research MPP? SMPClusterSimulate RAMP Scalability (1k CPUs) CAAA Cost (1k CPUs) F ($40M) C ($2-3M) A+ ($0M) A ($0.1-0.2M) Cost of ownershipADAA Power/Space (kilowatts, racks) D (120 kw, 12 racks) A+ (.1 kw, 0.1 racks) A (1.5 kw, 0.3 racks) CommunityDAAA ObservabilityDCA+ ReproducibilityBDA+ ReconfigurabilityDCA+ CredibilityA+ FA Perform. (clock) A (2 GHz) A (3 GHz) F (0 GHz) C (0.1-.2 GHz) GPACB-BA-

10. 10 Completed Dec. 2004 (14x17 inch 22-layer PCB) Module:   5 Virtex II FPGAs, 18 banks DDR2- 400 memory, 20 10GigE conn.   Administration/ maintenance ports: 10/100 Enet HDMI/DVI USB   ~$4K in Bill of Materials (w/o FPGAs or DRAM) RAMP 1 Hardware BEE2: Berkeley Emulation Engine 2

11. 11 Multiple Module RAMP 1 Systems 8 compute modules (plus power supplies) in 8U rack mount chassis 2U single module tray for developers Many topologies possible Disk storage: via disk emulator + Network Attached Storage

12. 12 Quick Sanity Check BEE2 uses old FPGAs (Virtex II), 4 banks DDR2-400/cpu 16 32-bit Microblazes per Virtex II FPGA, 0.75 MB memory for caches  32 KB direct mapped Icache, 16 KB direct mapped Dcache Assume 150 MHz, CPI is 1.5 (4-stage pipe)  I$ Miss rate is 0.5% for SPECint2000  D$ Miss rate is 2.8% for SPECint2000, 40% Loads/stores BW need/CPU = 150/1.5*4B*(0.5% + 40%*2.8%) = 6.4 MB/sec BW need/FPGA = 16*6.4 = 100 MB/s Memory BW/FPGA = 4*200 MHz*2*8B = 12,800 MB/s Plenty of room for tracing, …

13. 13 RAMP FAQ on ISAs Which ISA will you pick?  Goal is replacible ISA/CPU L1 cache, rest infrastructure unchanged (L2 cache, router, memory controller, …) What do you want from a CPU?  Standard ISA (binaries, libraries, …), simple (area), 64-bit (coherency), DP Fl.Pt. (apps)  Multihreading? As an option, but want to get to 1000 independent CPUs When do you need it? 3Q06 RAMP people port my ISA, fix my ISA?  Our plates are full already Type A vs. Type B gateware Router, Memory controller, Cache coherency, L2 cache, Disk module, protocol for each Integration, testing

14. 14 Handicapping ISAs Got it: Power 405 (32b), SPARC v8 (32b), Xilinx Microblaze (32b) Very Likely: SPARC v9 (64b), Likely: IBM Power 64b Probably (haven’t asked): MIPS32, MIPS64 Not likely: x86 Even less likely: x86-64 We’ll sue you: ARM

15. 15 RAMP Development Plan 1. Distribute systems internally for RAMP 1 development  Xilinx agreed to pay for production of a set of modules for initial contributing developers and first full RAMP system  Others could be available if can recover costs 2. Release publicly available out-of-the-box MPP emulator  Based on standard ISA (IBM Power, Sun SPARC, …) for binary compatibility  Complete OS/libraries  Locally modify RAMP as desired 3. Design next generation platform for RAMP 2  Base on 65nm FPGAs (2 generations later than Virtex-II)  Pending results from RAMP 1, Xilinx will cover hardware costs for initial set of RAMP 2 machines  Find 3rd party to build and distribute systems (at near-cost), open source RAMP gateware and software  Hope RAMP 3, 4, … self-sustaining NSF/CRI proposal pending to help support effort  2 full-time staff (one HW/gateware, one OS/software)  Look for grad student support at 6 RAMP universities from industrial donations

16. 16 RAMP Milestones 2006 NameGoalTargetCPUsDetails Red (SU) A Start1Q068 32b Power hard cores Transactional memory SMP Blue (Cal) Scale3Q061024 32b Microblaze soft cores Cluster, MPI White 1.0 2.0 3.0 4.0 Features 2Q06 3Q06 4Q06 1Q07 64 hard PPC 128? soft 32b 64? soft 64b Multiple ISAs Cache coherent, shared address, deterministic, debug/monitor, commercial ISA

17. 17 the stone soup of architecture research platforms I/O Patterson Monitoring Kozyrakis Net Switch Oskin Coherence Hoe Cache Asanovic PPC Arvind x86 Lu Glue-support Chiou Hardware Wawrzynek

18. 18 Gateware Design Framework Insight: almost every large building block fits inside FPGA today  what doesn’t is between chips in real design Supports both cycle-accurate emulation of detailed parameterized machine models and rapid functional-only emulations Carefully counts for Target Clock Cycles Units in any hardware design language (will work with Verilog, VHDL, BlueSpec, C,...) RAMP Design Language (RDL) to describe plumbing to connect units in

19. 19 Gateware Design Framework Design composed of units that send messages over channels via ports Units (10,000 + gates)  CPU + L1 cache, DRAM controller…. Channels (~ FIFO)  Lossless, point-to-point, unidirectional, in-order message delivery…

20. 20 Status Submitted NSF proposal August 2005 Biweekly teleconferences (since June 05) IBM, Sun donating commercial ISA, simple, industrial-strength, CPU + FPU Technical report, RDL document RAMP 1/RDL short course/board distribution in Berkeley for 40 people @ 6 schools Jan 06 FPGA workshop @ HPCA 2/06, @ ISCA 6/06 ramp.eecs.berkeley.edu

21. 21 RAMP uses (internal) Internet-in-a-Box Patterson TCC Kozyrakis Dataflow Oskin Reliable MP Hoe 1M-way MT Asanovic BlueSpec Arvind x86 Lu Net-uP Chiou Wawrzynek BEE

22. 22 Multiprocessing Watering Hole Killer app: All CS Research, Ind. Advanced Development RAMP attracts many communities to shared artifact  Cross-disciplinary interactions  Accelerate innovation in multiprocessing RAMP as next Standard Research Platform? (e.g., VAX/BSD Unix in 1980s) Parallel file system Thread scheduling Multiprocessor switch design Fault insertion to check dependability Data center in a box Internet in a box Dataflow language/computer Security enhancements Router designCompile to FPGA Parallel languages RAMP

23. 23 Supporters (wrote letters to NSF) Gordon Bell (Microsoft) Ivo Bolsens (Xilinx CTO) Norm Jouppi (HP Labs) Bill Kramer (NERSC/LBL) Craig Mundie (MS CTO) G. Papadopoulos (Sun CTO) Justin Rattner (Intel CTO) Ivan Sutherland (Sun Fellow) Chuck Thacker (Microsoft) Kees Vissers (Xilinx) Doug Burger (Texas) Bill Dally (Stanford) Carl Ebeling (Washington) Susan Eggers (Washington) Steve Keckler (Texas) Greg Morrisett (Harvard) Scott Shenker (Berkeley) Ion Stoica (Berkeley) Kathy Yelick (Berkeley) RAMP Participants: Arvind (MIT), Krste Asanovíc (MIT), Derek Chiou (Texas), James Hoe (CMU), Christos Kozyrakis (Stanford), Shih-Lien Lu (Intel), Mark Oskin (Washington), David Patterson (Berkeley), Jan Rabaey (Berkeley), and John Wawrzynek (Berkeley)

24. 24 RAMP as system-level time machine: preview computers of future to accelerate HW/SW generations  Trace anything, Reproduce everything, Tape out every day  FTP new supercomputer overnight and boot in morning  Clone to check results (as fast in Berkeley as in Boston?)  Emulate Massive Multiprocessor, Data Center, or Distributed Computer Carpe Diem  Systems researchers (HW & SW) need the capability  FPGA technology is ready today, and getting better every year  Stand on shoulders vs. toes: standardize on design framework, multi-year Berkeley effort on FPGA platforms (Berkeley Emulation Engine BEE2)  Architecture researchers get opportunity to immediately aid colleagues via gateware (as SW researchers have done in past) “Multiprocessor Research Watering Hole” accelerate research in multiprocessing via standard research platform  hasten sea change from sequential to parallel computing Conclusions

25. 25 Backup Slides

26. 26 UT FAST 1MHz to 100MHz, cycle-accurate, full-system, multiprocessor simulator  Well, not quite that fast right now, but we are using embedded 300MHz PowerPC 405 to simplify X86, boots Linux, Windows, targeting 80486 to Pentium M-like designs  Heavily modified Bochs, supports instruction trace and rollback Working on “superscalar” model  Have straight pipeline 486 model with TLBs and caches Statistics gathered in hardware  Very little if any probe effect Work started on tools to semi-automate micro- architectural and ISA level exploration  Orthogonality of models makes both simpler Derek Chiou, UTexas

27. 27 Example: Transactional Memory Processors/memory hierarchy that support transactional memory Hardware/software infrastructure for performance monitoring and profiling  Will be general for any type of event Transactional coherence protocol Christos Kozyrakis, Stanford

28. 28 Example: PROTOFLEX Hardware/Software Cosimulation/test methodology Based on FLEXUS C++ full-system multiprocessor simulator  Can swap out individual components to hardware Used to create and test a non-block MSI invalidation-based protocol engine in hardware James Hoe, CMU

29. 29 Example: Wavescalar Infrastructure Dynamic Routing Switch Directory-based coherency scheme and engine Mark Oskin, U Washington

30. 30 Example RAMP App: “Internet in a Box” Building blocks also  Distributed Computing RAMP vs. Clusters (Emulab, PlanetLab)  Scale: RAMP O(1000) vs. Clusters O(100)  Private use: $100k  Every group has one  Develop/Debug: Reproducibility, Observability  Flexibility: Modify modules (SMP, OS)  Heterogeneity: Connect to diverse, real routers Explore via repeatable experiments as vary parameters, configurations vs. observations on single (aging) cluster that is often idiosyncratic David Patterson, UC Berkeley

31. 31 Why RAMP Attractive? 1a. Cost of purchase 1b. Cost of ownership (staff to administer it) 1c. Scalability (1000 much better than 100 CPUs) 4. Power/Space (machine room cooling, number of racks) 5. Community synergy (share code, …) 6. Observability (non-obtrusively measure, trace everything) 7. Reproducibility (to debug, run experiments) 8. Flexibility (change for different experiments) 9. Credibility (Faithfully predicts real hardware behavior) 10. Performance (As long as experiments not too slow) Priorities for Research Parallel Computers Insight – Commercial priorities radically different from research

32. 32 Related Approaches (1) Quickturn, Axis, IKOS, Thara:  FPGA- or special-processor based gate-level hardware emulators  Synthesizable HDL is mapped to array for cycle and bit-accurate netlist emulation  RAMP’s emphasis is on emulating high-level architecture behaviors Hardware and supporting software provides architecture-level abstractions for modeling and analysis Targets architecture and software research Provides a spectrum of tradeoffs between speed and accuracy/precision of emulation RPM at USC in early 1990’s:  Up to only 8 processors  Only the memory controller implemented with configurable logic

33. 33 Related Approaches (2) Software Simulators Clusters (standard microprocessors) PlanetLab (distributed environment) Wisconsin Wind Tunnel (used CM-5 to simulate shared memory) All suffer from some combination of:  Slowness, inaccuracy, scalability, unbalanced computation/communication, target inflexibility