1. Enabling Technologies for Reconfigurable Computing Enabling Technologies for Reconfigurable Computing and Software / Configware Co-Design Part 2: Data-Stream-based Computing - Reiner Hartenstein University of Kaiserslautern July 8, 2002, ENST, Paris, France

2. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 2 Schedule timeslot 10.00 – 11.00 Reconfigurable Computing (RC) 11.00 – 11.30 coffee break 11.30 – 12.30 Data-Stream-based Computing 12.30 – 14.00 lunch break 14.00 – 15.00 Resources for RC and Data-Stream-based Computing 15.00 – 15.30 Recent developments 15.30 – 16.00 Discussion

3. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 3 Opportunities by new patent laws ? to clever guys being keen on patents: don‘t file for patent following details ! everything shown in this presentation has been published years ago

4. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 4 >> EDA revolution EDA revolution Dead Supercomputer Data-Stream-based Computing Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://kressarray.de

5. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 5 Makimoto’s 3rd wave [Hartenstein] The next EDA Industry Revolution 1978 Transistor entry: Applicon, Calma, CV... 1992 Synthesis: Cadence, Synopsys... 1985 Schematics entry: Daisy, Mentor, Valid... [Keutzer / Newton] EDA industry paradigm switching every 7 years 1999 (Co-) Compilation Data-Stream-based DPU arrays 2006

6. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 6 [Richard Newton]

7. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 7 Biggest Mistake in History

8. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 8 © 2001, reiner@hartenstein.de http://KressArray.de University of Kaiserslautern missing the next revolution Ignoring Reconfigurable Computing and Data-stream-based Computing by teaching computing fundamentals within our CS curricula causing the waste billions of dollars. is one of the biggest mistakes in the history of information technology application [Hartenstein]

9. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 9 >> Dead Supercomputer EDA revolution Dead Supercomputer Data-Stream-based Computing Stream-based Memory Architecture Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

10. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 10 Dead Supercomputer Society ACRI Alliant American Supercomputer Ametek Applied Dynamics Astronautics BBN CDC Convex Cray Computer Cray Research Culler-Harris Culler Scientific Cydrome Dana/Ardent/ Stellar/Stardent DAPP Denelcor Elexsi ETA Systems Evans and Sutherland Computer Floating Point Systems Galaxy YH-1 Goodyear Aerospace MPP Gould NPL Guiltech ICL Intel Scientific Computers International Parallel Machines Kendall Square Research Key Computer Laboratories [Gordon Bell, keynote at ISCA 2000]. MasPar Meiko Multiflow Myrias Numerix Prisma Tera Thinking Machines Saxpy Scientific Computer Systems (SCS) Soviet Supercomputers Supertek Supercomputer Systems Suprenum Vitesse Electronics

11. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 11 Dying Parallel Computing Society

12. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 12 >> Stream-based Computing EDA revolution Dead Supercomputer Data-Stream-based Computing Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

13. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 13 anti particles 1956: anti neutron created on Bevatron 1928: Paul Dirac: „there should be an anti electron having positive charge“ (Nobel price 1933) 1932: Carl David Anderson detected this „positron“ in cosmic radiation (Nobel price 1936) 1955 Owen Chamberlain et al. create anti proton on Bevatron 1954: new accelerators: cyclotron, like Berkeley‘s Bevatron 1965: creation of a deuterium anti nucleus at CERN hydrogen anti hydrogen 1995: hydrogen anti atom created at CERN – by forcing positron and anti proton to merge by very low energy..... but there are asymmetries” “in the universe should be regions of anti matter …

14. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 14 Matter & Antimatter: Atom and Anti Atom For the World of Matter The machine paradigm: the Atom For Anti Matter the machine paradigm: Anti Atom + + Electron spinning - - - Positron spinning +

15. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 15 Matter & Antimatter of Informatics : Anti Machine paradigm instruction stream spinning Machine and Anti Machine + CPU - 1936 1 st electronic computer (Konrad Zuse) Machine paradigm: „von Neumann“ 1946 v. N. machine paradigm 1971 1 st microprocessor (Ted Hoff) data stream spinning 1979 „data streams“ ( systolic array: Kung / Leiserson) 1995 rDPA / DPSS ( supersystolic: Rainer Kress) data-procedural - DPU + 1990 anti machine paradigm published

16. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 16 RAM-based + CPU Data Path instruction sequencer RAM + simple machine paradigm + scalability + relocatability + compatibility = secret of success of software industry CPU:

17. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 17 Nasty Matter + CPU Data Path instruction sequencer RAM Address Computation Overhead Instruction Fetch Overhead central von Neumann bottleneck extremely power hungry and area inefficient performance problems reconfigurable? the wrong machine paradigm alw. new instruction sequencer needed

18. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 18 Parallelism by Concurrency + - + - - + - + + - - + - + independent instruction streams

19. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 19 Concurrent Computing.... Bus (es) or switch box Data Path instruction sequencer Data Path instruction sequencer Data Path instruction sequencer Data Path instruction sequencer Data Path instruction sequencer Data Path instruction sequencer extremely inefficient CPU massive switching activity at runtime may affect far beyond Amdahl‘s law

20. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 20 Coarse Grain Reconfigurable Arrays vs. Parallel Processes I-Seq ALU Parallelism at Process Level Parallelism at Datapath Level hardwired no instruction sequencing ! Data Sequencer rDPA reconfigurable

21. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 21 Some differences: CPU versus DPU + CPU Data Path instruction sequencer transport- triggered - DPU Data Path Unit DPU data streams external signal, or nothing central no vN bottleneck: multiple ports instruction fetch not at run time: no overhead data streams scheduled elsewhere RAM data sequencer RAM data sequencer RAM data sequencer … instruction stream routed here

22. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 22 machine paradigm: some differences + CPU - - DPA + + + - DPU + matter antimatter no. of streams = 1 no. of streams  1

23. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 23 DPA = DPU array - DPA - DPU - - - - - - - - - DPA + + + + + + + + + coherent data streams spinning around

24. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 24 >>> extremely high efficiency avoiding address computation overhead avoiding instruction fetch and interpretation overhead high parallelism, massively multiple deep pipelines much less configuration memory no routing areas to configure functions from CLBs

25. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 25 computing in space Computing in space and time data streams y 1 0  y 2 0 y 3 0 - - - y 1 y 2 y 3 - - - x 1 x 2 x 3 - - - computing in time a 12 a 11 a 21 a 32 a 31 a 23 a 33 a 22 a 13 placement systolic arrays etc. and other transformations migration by re-timing this dichotomy is completely ignored by our CS curricula

26. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 26 2 General Stream-based Computing System heterogenous Array of rDPUs (reconf. data path units) Scheduler Mapper expression tree DPU architectures y + * x a 1 simultaneous placement & routing 3 + ++ + * * * sh * xf - - data streams 4 The same mapper for both: Reconfigurable, or hardwired Kress DPSS [1995] simulated annealing free form pipe network time space

27. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 27 Super Pipe Networks The key is mapping, rather than architecture * *) KressArray [1995]

28. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 28 >> Design Space Explorers EDA revolution Dead Supercomputer Data-Stream-based Computing Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

29. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 29 domain-specific Reconfigurable Platforms will be suitable to cope with the 2 nd Design Crisis just as the general purpose massively parallel computer system general purpose is unrealistic an Illusion... KressArray Explorer... fully general purpose reconfigurable sometimes is....

30. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 30 Universal RAs: is it feasible?... such as obviously also the Universal Massively Parallel Computer Architecture... counter-example: Application Domain of Image Processing The General Purpose (coarse grain) Reconfigurable Array appears to be an Illusion... Motivation

31. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 31 -> Design Space Exploration Exploration: –Design Space Explorer (DSEs) –Platform Space Explorers (PSEs) –Compiler / PSE symbiosis –Parallel computing vs. reconfigurable Design Space Explorers: –For VLSI design in general –for parallel Computer Systems –Xplorer the only one f. reconfigurable platforms

32. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 32 Design Space Exploration Systems

33. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 33 >> KressArray Xplorer EDA revolution Dead Supercomputer Data-Stream-based Computing Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

34. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 34 Architecture & Mapping Editor Statistics KressArray DPSS Datastream Generator HDL Generator Simulator Datapath Generator Delay & Power Estimator Improvement Proposal Generator User DPSS Source Input KressArray (Design Space) Platform Space Explorer http://kressarray.de Xplorer Application Set accessible by internet: http://kressarray.de runs best with Netscape 4.6.1

35. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 35 >> Machine paradigms EDA revolution Dead Supercomputer Data-Stream-based Computing Stream-based Memory Architecture Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

36. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 36 © 2001, reiner@hartenstein.de University of Kaiserslautern instructions program counter : state register Compiler RAM Datapath hardwired Sequencer Computer tightly coupled by compact instruction code “von Neumann” does not support soft data paths does not support soft data paths Datapath reconfigurable Computer: the wrong Machine Paradigm “von Neumann”

37. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 37 © 2001, reiner@hartenstein.de University of Kaiserslautern Xputer Scheduler Compiler RAM (multiple) sequencer Datapath Array “instructions” University of Kaiserslautern loosely coupled by decision data bits only Xputer: The Soft Machine Paradigm reconfigurable also for hardwired Computer: the wrong Machine Paradigm “von Neumann” data stream spec there are some differences s data counter (anti machine)

38. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 38 Machine Paradigms

39. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 39 All Fundamental Concepts available Data Sequencer Methodology Data-procedural Languages (Duality w. v. N.)... supporting memory bandwidth optimization Soft Data Path Synthesis Algorithms Parallelizing Loop Transformation Methods Compilers supporting Soft Machines SW / CW Partitioning Co-Compilers Part 3

40. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 40 >> Co-Compilation EDA revolution Dead Supercomputer Data-Stream-based Computing Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

41. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 41 Changing Models of Computing “von Neumann” downloading RAM downloading data path instruction sequencer I / O (procedural) Software contemporary host hardwired downloading accelerator(s) CAD RAM reconfigurable computing host re- downloading conf. accelerator(s) RAM Software Configware both done at customer site Hardware designer needed done at vendor site ASIC s

42. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 42 Co-Compilation partitioning compiler high level programming language source  Processor Reconfigurable Accelerators interface Anti Machine Paradigm: Reconfigurable Architecture (rDPA) no CAD ! Compilation instead ! Hardware / Software Co-Design turns to Configware / Software Co-Design We introduce: Co-Compilation Computer Machine Paradigm Software running on “Soft” Anti Machine Configware running on

43. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 43 Jürgen Becker’s Co-DE-X Co-Compiler Analyzer / Profiler Host (vN) GNU C compiler paradigm Computer machine DPSS KressArray (rDPA) X-C compiler Anti machine paradigm Partitioner Loop Transfor- mations X-C is C language extended by MoPL X-C Resource Parameters supporting different platforms supporting platform-based design

44. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 44 Loop Transformation Examples loop 1-8 body endloop loop 1-8 body endloop loop 9-16 body endloop fork join strip mining loop 1-4 trigger endloop loop 1-2 trigger endloop loop 1-8 trigger endloop reconf.array: host: loop 1-16 body endloop sequential processes: resource parameter driven Co-Compilation loop unrolling

45. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 45 Future Coarse Grain RA Development It is indispensable to operate within the Convergence Area of Compilers, Co-Compilers, Architecture and full- custom-style VLSI Design (array cells). It is a must, that Products come with a Development Platform which encourages users,especially also those with a limited Hardware Background.

46. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 46 >> Design Space Explorers EDA revolution Dead Supercomputer Stream-based Computing Stream-based Memory Architecture Design Space Explorers KressArray Xplorer Machine paradigms Co-Compilation http://www.uni-kl.de

47. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 47 END

48. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 48 data counter instructions program counter : state register Compiler Memory Datapath hardwired Sequencer Computer tightly coupled by compact instruction code “von Neumann” does not support soft data paths does not support soft data paths Datapath reconfigurable Xputer Scheduler Compiler Memory multiple sequencer Datapath Array “instructions” University of Kaiserslautern loosely coupled by decision data bits only Xputer: The Soft Machine Paradigm reconfigurable also for hardwired Computer: the wrong Machine Paradigm “von Neumann”

49. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 49 Anti Machine Paradigm Xputer Parallel Xputer reconfigurable Scheduler Compiler Memory Sequencer Datapath “instructions” data counter Scheduler Compiler Sequencer Datapath Sequencer “instructions” data counters reconfigurable memory multiple Decision data only; i, e, loose coupling

50. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 50  Processor Co-Compilation partitioning compiler Computer Machine Paradigm Software running on Xputer “Soft” Machine Paradigm Configware running on GNU C compiler Analyzer / Profiler Hardware / Software Co-Design turns to Configware / Software Co-Design supporting different platforms Resource Parameters interface X-C compiler Reconfigurable Accelerators KressArray DPSS high level programming language source X-C Partitioner Jürgen Becker’s Co-DE-X Co-Compiler [ASP-DAC’95]

51. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 51 Computer: the wrong Machine Paradigm Compiler Memory Sequencer Decoder Datapath instructions program counter hardwired tightly coupled by a compact instruction code “von Neumann” does not support soft data paths: does not support soft data paths: “von Neumann” at run time: no instruction fetch : Instruction Sequencer Datapath reconfigurable

52. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 52 KressArray DPSS Application Set DPSS published at ASP-DAC 1995 Architecture Editor Mapping Editor statist. Data Delay Estim. Analyzer Architecture Estimator interm. form 2 expr. tree ALE-X Compiler Power Estimator Power Data VHDL Verilog HDL Generator Simulator User ALEX Code Improvement Proposal Generator Suggestion Selection User Interface interm. form 3 Mapper Design Rules Datapath Generator Kress rDPU Layout data stream Schedule Scheduler KressArray Xplorer (Platform Design Space Explorer) Xplorer Inference Engine (FOX) Sug- gest- ion KressArray family parameters Compiler Mapper Scheduler

53. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 53 Changing Models of Computation contemporary host hardwired Compiler accelerator(s) CAD RAM reconfigurable computing host re- Co-Compiler conf. accelerator(s) RAM Software Configware Machine paradigm EDA tools needed* ASIC s *) even 80% hardware people hate their tools both done at customer site done at vendor site no hardware experts needed

54. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 54 Machine Paradigms

55. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 55 KressArray Design Space Xplorer DPSS-N Data Path Systhesis System Analyser HDL Generator HDL Description.v Module Generator.krs Kress IP Library other IP Editor / User Interface Architecture Estimation Intermediate Format.map ALE-X Compiler ALE-X Code.alex User Mapper Interm. Format.map including configware code Technology Mapping Scheduler Data.seq Sequencing Code Kress rDPU.krs Layout Placement & Routing M a p p i n g Statistical Data.stat to Synthesis Environment

56. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 56 FPGA-Style Mapping for coarse grain reconfigurable arrays Compiler Mapper Scheduler specifies and assembles the data streams from / to array DPSS KressArray DPSS (Datapath Synthesis System)

57. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 57 Design Flow of Domain-specific Architecture Optimization Nageldinger’s KressArray Design Space Xplorer: including a Fuzzy Logic Improvement Proposal Generator accessible by internet: http://kressarray.de runs best with Netscape 4.6.1

58. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 58 History of Loop Transformations David Loveman, 1977, Allen and Kennedy, et al. Loop Unrolling, Loop Fusion, Strip Mining.... (Parameter-driven) Time to Time/Space Partitioning 1995/97 [Karin Schmidt / Jürgen Becker] : downto Datapath Level: e. g.: Transformation from Sequential Process to Super-systolic Multi-dimensional Loop Unrolling / Storage Scheme Optimization supporting burst-mode & parallel Memory Banks 2000 [Michael Herz] : optimized RA to Memory Communication Bandwidth: 70ies - 80ies: at Process Level: Sequential to Parallel Processes, incl. Vectorization

59. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 59 History of Loop Transformations For Sequential Programs on Parallel Computers: David Loveman, 1977, Allen and Kennedy, etc.: Loop Unrolling, Loop Fusion, Strip Mining.... For memory communication: Michael Herz (2000): Multi-Level Loop Unrolling to reduce Memory Cycles needed to create RA Data Streams For parallel Datapaths: Jürgen Becker (1997): to Sequential to Super-Systolic Transformation Optimize Throughput of Reconfigurable Arrays (RAs) Instruction Code vs. Reconfiguration Code

60. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 60 Paradigm Shift Mainstream Tornado Development of Hypergrowth Markets Harper Business 1995

61. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 61 EDA: where Electronics begins [Richard Newton] 1k Dataquest Initiative New book NASDAQ index EDA index

62. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 62 What is next after VHDL ? Motivations HDL-savvy designers needed New Business Model Co-Design never ending HDLs ? Extended HDLs – how far ? Automatic Partitioning

63. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 63 Hot Research Topic: Memory Architectures High Performance Embedded Memory Architectures High Performance Memory Communication Architectures [Herz] Custom Memory Management Methodology [Cathoor] Data Reuse Transformations [Kougia et al.] Data Reuse Exploration [Soudris, Wuytak]

64. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 64 RAs: Cache does not help the memory bandwidth problem is often more dramatic then for microprocessors interleaving is not practicable, since based on sequential instruction streams classical caches do not help, since instruction sequencing is not used the problem: throughput of parallel data streams, not instruction streams super pipe networks, no parallel computers ! Stream-based arrays are a memory bandwidth problem

65. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 65 Memory Communication Architecture hot research topic in embedded systems storage context transformations [Herz, others] for low power for high performance startups provide memory IP or generators

66. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 66 Stream-based Soft Machine Scheduler Memory (data memory) memory bank... “instructions” rDPA Compiler Sequencers (data stream generator)

67. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 67 Stream-based Computing DPU driven by data stream from / to memory or, from / to peripheral interface transport-triggered execution no instruction sequencer inside !

68. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 68 Stream-based Computing: (r) DPU array for both, reconfigurable, and, hardwired DPU driven by data streams

69. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 69 Systolic Stream-based Computing System Systolic Array [ H. T. Kung, 1980 ] : an array of DPUs (Data Path Units) DPU architecture y + * x a data streams equations placement linear projection or algebraic mapping The Mathematician’s Synthesis Method linear pipelines and uniform arrays only no routing!

70. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 70 Converging Design Flows this synthesis method is a generalization of systolic array synthesis: super systolic synthesis and DPA [Broderson, 2000]: terms: DPU: datpath unit DPA: data path array rDPU: reconfigurable DPU rDPA: reconfigurable DPA the same synthesis method may be used for mapping an algorithm onto both: rDPA [Kress, 1995],

71. © 2002, reiner@hartenstein.de http://kressarray.de University of Kaiserslautern 71 Innovation Stalled ? [Richard Newton] What is next after VHDL ?