1. 1 Digital Space Anant Agarwal MIT and Tilera Corporation

2. 2 Arecibo

3. 3 Stages of Reality mem 1996 CPU Mem 1997 2002 2007 p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m s p m s p m s p m s p m s p m s p m s p m s p m p m s p m p m p m p m p m p m p m p m p m p m 2014 100B transistors 2018 1B transistors 2007

4. 4 Virtual reality Simulator reality Prototype reality Product reality Virtual reality Simulator reality Prototype reality Simulator reality Product reality Prototype reality Simulator reality Virtual Reality

5. 5 The Opportunity 20MIPS cpu in 1987 1996… Few thousand gates

6. 6 The Opportunity The billion transistor chip of 2007

7. 7 How to Fritter Away Opportunity the x1786? does not scale 100 ported RegFil and RR Caches Control More resolution buffers, control “1/10 ns”

8. 8 Lots of ALUs, lots of registers, lots of local memories – huge on-chip parallelism – but with a slower clock Custom-routed, short wires optimized for specific applications Fast, low power, area efficient But not programmable mem Take Inspiration from ASICs

9. 9 Our Early Raw Proposal CPU Mem E.g., 100-way unrolled loop, running on 100 ALUs, 1000 regs, 100 memory banks But how to build programmable, yet custom, wires? Got parallelism?

10. 10 A digital wire Ctrl Software orchestrate it! Customize to application and maximize utilization Multiplex it! Improve utilization Pipeline it! Fast clock (10GHz in 2010) Uh! What were we smoking! A dynamic router! Replace custom wires with routed on-chip networks

11. 11 Static Router Application Compiler Switch Code Switch Code Switch Code Switch Code Switch Code A static router!

12. 12 Replace Wires with Routed Networks Ctrl

13. 13 50-Ported Register File  Distributed Registers Gigantic 50 ported register file

14. 14 Gigantic 50 ported register file 50-Ported Register File  Distributed Registers

15. 15 Distributed Registers + Routed Network Distributed register file R Called NURA [ASPLOS 1998]

16. 16 16-Way ALU Clump  Distributed ALUs ALU Bypass Net RF

17. 17 Distributed ALUs, Routed Bypass Network ALU R Scalar Operand Network (SON) [TPDS 2005]

18. 18 Mongo Cache  Distributed Cache Gigantic 10 ported cache

19. 19 Distributing the Cache

20. 20 Distributed Shared Cache ALU Like DSM (distributed shared memory), cache is distributed; But, unlike NUCA, caches are local to processors, not far away R $ [ISCA 1999]

21. 21 Tiled Multicore Architecture ALU R $

22. 22 E.g., Operand Routing in 16-way Superscalar ALU Bypass Net RF >> + Source: [Taylor ISCA 2004]

23. 23 Operand Routing in a Tiled Architecture >> ALU >> + R $

24. 24 Tiled Multicore Scales to large numbers of cores Modular – design, layout and verify 1 tile Power efficient [ MIT-CSAIL-TR-2008-066] – Short wires C V 2 f – Chandrakasan effect C V 2 f – Dynamic and compiler scheduled routing Processor Core = TileCore+ Switch S

25. 25 A Prototype Tiled Architecture: The Raw Microprocessor The Raw Chip Tile Disk stream Video1 DRAM Packet stream A Raw Tile SMEM SWITCH PC DMEM IMEM REG PC FPU ALU Raw Switch PC SMEM [Billion transistor IEEE Computer Issue ’97] www.cag.csail.mit.edu/raw Scalar operand network (SON): Capable of low latency transport of small (or large) packets [IEEE TPDS 2005]

26. 26 Virtual reality Simulator reality Prototype reality Product reality

27. 27 Scalar Operand Transport in Raw fmul r24, r3, r4 software controlled crossbar software controlled crossbar fadd r5, r3, r24 route P->E, N->S route W->P, S->N Goal: flow controlled, in order delivery of operands

28. 28 RawCC: Distributed ILP Compilation (DILP) tmp0 = (seed*3+2)/2 tmp1 = seed*v1+2 tmp2 = seed*v2 + 2 tmp3 = (seed*6+2)/3 v2 = (tmp1 - tmp3)*5 v1 = (tmp1 + tmp2)*3 v0 = tmp0 - v1 v3 = tmp3 - v2 pval5=seed.0*6.0 pval4=pval5+2.0 tmp3.6=pval4/3.0 tmp3=tmp3.6 v3.10=tmp3.6-v2.7 v3=v3.10 v2.4=v2 pval3=seed.o*v2.4 tmp2.5=pval3+2.0 tmp2=tmp2.5 pval6=tmp1.3-tmp2.5 v2.7=pval6*5.0 v2=v2.7 seed.0=seed pval1=seed.0*3.0 pval0=pval1+2.0 tmp0.1=pval0/2.0 tmp0=tmp0.1 v1.2=v1 pval2=seed.0*v1.2 tmp1.3=pval2+2.0 tmp1=tmp1.3 pval7=tmp1.3+tmp2.5 v1.8=pval7*3.0 v1=v1.8 v0.9=tmp0.1-v1.8 v0=v0.9 pval5=seed.0*6.0 pval4=pval5+2.0 tmp3.6=pval4/3.0 tmp3=tmp3.6 v3.10=tmp3.6-v2.7 v3=v3.10 v2.4=v2 pval3=seed.o*v2.4 tmp2.5=pval3+2.0 tmp2=tmp2.5 pval6=tmp1.3-tmp2.5 v2.7=pval6*5.0 v2=v2.7 seed.0=seed pval1=seed.0*3.0 pval0=pval1+2.0 tmp0.1=pval0/2.0 tmp0=tmp0.1 v1.2=v1 pval2=seed.0*v1.2 tmp1.3=pval2+2.0 tmp1=tmp1.3 pval7=tmp1.3+tmp2.5 v1.8=pval7*3.0 v1=v1.8 v0.9=tmp0.1-v1.8 v0=v0.9 Black arrows = Operand Communication over SON [ASPLOS 1998] Partitioning Place, Route, Schedule C

29. 29 Virtual reality Simulator reality Prototype reality Product reality

30. 30 A Tiled Processor Architecture Prototype: the Raw Microprocessor October 02 Michael Taylor Walter Lee Jason Miller David Wentzlaff Ian Bratt Ben Greenwald Henry Hoffmann Paul Johnson Jason Kim James Psota Arvind Saraf Nathan Shnidman Volker Strumpen Matt Frank Rajeev Barua Elliot Waingold Jonathan Babb Sri Devabhaktuni Saman Amarasinghe Anant Agarwal

31. 31 Raw Die Photo IBM.18 micron process, 16 tiles, 425MHz, 18 Watts (vpenta) [ISCA 2004]

32. 32 Raw Motherboard

33. 33 Raw Ideas and Decisions: What Worked, What Did Not Build a complete prototype system Simple processor with single issue cores FPGA logic block in each tile Distributed ILP and static network Static network for streaming Multiple types of computation – ILP, streams, TLP, server PC in every tile

34. 34 Why Build? Compiler (Amarasinghe), OS and runtimes (ISI), apps (ISI, Lincoln Labs, Durand) folks will not work with you unless you are serious about building hardware Need motivaion to build software tools -- compilers, runtimes, debugging, visualization – many challenges here Run large data sets (simulation takes forever even with 100 servers!) Many hard problems show up or are better understood after you begin building (how to maintain ordering for distributed ILP, slack for streaming codes) Have to solve hard problems – no magic! The more radical the idea, the more important it is to build – World will only trust end-to-end results since it is too hard to dive into details and understand all assumptions – Would you believe this: “Prof. John Bull has demonstrated a simulation prototype of a 64-way issue out-of-order superscalar” Cycle simulator became cycle accurate simulator only after HW got precisely defined Don’t bother to commercialize unless you have a working prototype Total network power few percent for real apps [Aug 2003 ISLPED, Kim et al. Energy characterization of a tiled architecture processor with on-chip networks] [MIT- CSAIL - TR -2008-066 Energy scalability of on-chip interconnection networks in multicore architecures ] – Network power is few percent in Raw for real apps; however, it is 36% only for a highly contrived synthetic sequence meant to toggle every network wire

35. 35 Raw Ideas and Decisions: What Worked, What Did Not Build a complete prototype system Simple processor, single issue FPGA logic block in each tile Distributed ILP Static network for streaming Multiple types of computation – ILP, streams, TLP, server PC in every tile Yes 1GHz, 2-way, inorder in 2016 No Yes ‘02, No ‘06, Yes ‘14 Yes

36. 36 software controlled crossbar software controlled crossbar route P->E, N->S route W->P, S->N Raw Ideas and Decisions: Streaming – Interconnect Support Forced synchronization in static network

37. 37 sub r5, r3, r55 Dynamic Switch add r55, r3, r4 Dynamic Switch Catch all Streaming in Tilera’s Tile Processor TAG Streaming done over dynamic interconnect with stream demuxing (AsTrO SDS) Automatic demultiplexing of streams into registers Number of streams is virtualized

38. 38 Virtual reality Simulator reality Prototype reality Product reality

39. 39 Why Do We Care? Markets Demanding More Performance Wireless Networks - Demand for high thruput – more channels - Fast moving standards LTE, services Networking market - Demand for high performance – 10Gbps - Demand for more services, intelligence Digital Multimedia market - Demand for high performance – H.264 HD - Demand for more services – VoD, transcode Cable & Broadcast Video Conferencing Switches Security Appliances Routers … and with power efficiency and programming ease GGSN Base Station 39

40. 40 Tilera’s TILEPro64™ Processor Power per tile (depending on app)170 – 300 mW Core power for h.264 encode (64 tiles) 12W Clock speed Up to 866 MHz I/O bandwidth40 Gbps Main Memory bandwidth200 Gbps Multicore Performance (90nm) Number of tiles64 Cache-coherent distributed cache5 MB Operations @ 750MHz (32, 16, 8 bit)144-192-384 BOPS Bisection bandwidth2 Terabits per second Power Efficiency I/O and Memory Bandwidth Programming ANSI standard C SMP Linux programming Stream programming Product reality [Tile64, Hotchips 2007] [Tile64, Microprocessor Report Nov 2007]

41. 41 PCIe 1 MAC PHY PCIe 1 MAC PHY PCIe 0 MAC PHY PCIe 0 MAC PHY Serdes Flexible IO GbE 0 GbE 1 Flexible IO UART, HPI JTAG, I2C, SPI UART, HPI JTAG, I2C, SPI DDR2 Memory Controller 3 DDR2 Memory Controller 0 DDR2 Memory Controller 2 DDR2 Memory Controller 1 XAUI MAC PHY 0 XAUI MAC PHY 0 Serdes XAUI MAC PHY 1 XAUI MAC PHY 1 Serdes Tile Processor Block Diagram A Complete System on a Chip PROCESSOR P2 Reg File P1P0 CACHE L2 CACHE L1IL1D ITLBDTLB 2D DMA STN MDN TDN UDN IDN SWITCH

42. 42 Tile Processor NoC 5 independent non-blocking networks – 64 switches per network – 1 Terabit/sec per Tile Each network switch directly and independently connected to tiles One hop per clock on all networks I/O write example Memory write example Tile to Tile access example All accesses can be performed simultaneously on non-blocking networks UDN STN IDN MDN Tiles TDN VDN [IEEE Micro Sep 2007]

43. 43 Multicore Hardwall Implementation Or Protection and Interconnects OS1/APP1 OS1/APP3 OS2/APP2 data valid Switch data valid Switch HARDWALL_ENABLE

44. 44 Product Reality Differences Market forces – Need crisper answer to “who cares” – SMP Linux programming with pthreads – fully cache coherent – C + API approach to streaming vs new language Streamit in Raw – Special instructions for video, networking – Floating point needed in research project, but not in product for embedded market Lessons from Raw – E.g., Dynamic network for streams – HW instruction cache – Protected interconnects More substantial engineering – 3-way VLIW CPU, subword arithmetic – Engineering for clock speed and power efficiency – Completeness – I/O interfaces on chip – complete system chip. Just add DRAM for system – Support for virtual memory, 2D DMA – Runs SMP Linux (can run multiple OSes simultaneously)

45. 45 Virtual reality Simulator reality Prototype reality Product reality

46. 46 What Does the Future Look Like? Corollary of Moore’s law: Number of cores will double every 18 months ‘05‘08‘11‘14 6425610244096 ‘02 16 Research Industry 16642561024 4 (Cores minimally big enough to run a self respecting OS!) 1K cores by 2014! Are we ready?

47. 47 Vision for the Future The ‘core’ is the logic gate of the 21 st century p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s p m s

48. 48 Research Challenges for 1K Cores 4-16 cores not interesting. Industry is there. University must focus on “1K cores”; Everything will change! Can we use 4 cores to get 2X through DILP? Remember cores will be 1GHz and simple! What is the interconnect? How should we program 1K cores? Can interconnect help with programming? Locality and reliability WILL matter for 1K cores. Spatial view of multicore? Can we add architectural support for programming ease? E.g., suppose I told you cores are free. Can you discover mechanisms to make programming easier? What is the right grain size for a core? How must our computational models change in the face of small memories per core? How to “feed the beast”? I/O and external memory bandwidth Can we assume perfect reliability any longer?

49. 49 ATAC Architecture p switch m p m p m p m p m p m p m p m p m p m p m p m p m p m p m p m Optical Broadcast WDM Interconnect Electrical Mesh Interconnect (EMesh) [ Proc. BARC Jan 2007, MIT- CSAIL - TR -2009-018 ]

50. 50 Research Challenges for 1K Cores 4-16 cores not interesting. Industry is there. University must focus on “1K cores”; Everything will change! Can we use 4 cores to get 2X through DILP? What is the interconnect? How should we program 1K cores? Can interconnect help with programming? Locality and reliability WILL matter for 1K cores. Spatial view of multicore? Can we add architectural support for programming ease? E.g., suppose I told you cores are free. Can you discover mechanisms to make programming easier? What is the right grain size for a core? How must our computational models change in the face of small memories per core? How to “feed the beast”? I/O and external memory bandwidth Can we assume perfect reliability any longer?

51. 51 FOS – Factored Operating System Today: User app and OS kernel thrash each other in a core’s cache User/OS time sharing is inefficient Angstrom: OS assumes abstracted space model. OS services bound to distinct cores, separate from user cores. OS service cores collaborate to achieve best resource management User/OS space sharing is efficient The key idea: space sharing replaces time sharing OS File System FS User App FS OS cores collaborate, inspired by distributed internet services model Need new page I/O FS [OS Review 2008]

52. 52 Research Challenges for 1K Cores 4-16 cores not interesting. Industry is there. University must focus on “1K cores”; Everything will change! Can we use 4 cores to get 2X through DILP? What is the interconnect? How should we program 1K cores? Can interconnect help with programming? Locality and reliability WILL matter for 1K cores. Spatial view of multicore? Can we add architectural support for programming ease? E.g., suppose I told you cores are free. Can you discover mechanisms to make programming easier? What is the right grain size for a core? How must our computational models change in the face of small memories per core? How to “feed the beast”? I/O and external memory bandwidth Can we assume perfect reliability any longer?

53. 53 The following are trademarks of Tilera Corporation: Tilera, the Tilera Logo, Tile Processor, TILE64, Embedding Multicore, Multicore Development Environment, Gentle Slope Programming, iLib, iMesh and Multicore Hardwall. All other trademarks and/or registered trademarks are the property of their respective owners.