IPDPS 2004 Software or Configware? About the Digital Divide of Parallel Computing Reiner Hartenstein TU Kaiserslautern Santa Fe, NM, April , 2004

© 2004, TU Kaiserslautern 2 Preface The White House, Sept 2000: Bill Clinton condemns the Digital Divide in America: World Economic Forum 2002: The Global Digital Divide, disparity between the "haves" and "have nots“ The Digital Divide of Parallel Computing: Access to Configware (CW) Solutions access to the internet

© 2004, TU Kaiserslautern 3 The „havenots“ Configware methodology to move data around more efficiently: Configware engineering as a qualification for programming embedded systems * : „ havenots “ are found in the HPC community The „ havenots “ are our typical CS graduates Reconfigurable HPC is torpedoed by deficits in education: curricular revisions are overdue *) also HPC !

© 2004, TU Kaiserslautern 4 Software to Configware Migration this talk will illustrate the performance benfit which may be obtained from Reconfigurable Computing stressing coarse grain Reconfigurable Computing (RC), point of view, this talk hardly mentions FPGAs (But coarse grain may be always mapped onto FPGAs) Software to Configware Migration is the most important source of speed-up Hardware is just frozen Configware

© 2004, TU Kaiserslautern 5 >> HPC << HPC Embedded Computing The wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 6 Earth Simulator 5120 Processors, 5000 pins each ES 20: TFLOPS Crossbar weight: 220 t, 3000 km of cable, moving data around inside the

© 2004, TU Kaiserslautern 7 data are moved around by software (slower than CPU clock by 2 orders of magnitude) i.e. by memory-cycle-hungry instruction streams which fully hit the memory wall extremely unbalanced stolen from Bob Colwell CPU

© 2004, TU Kaiserslautern 8 path of least resistance * : avoiding a paradigm shift Many researchers seem never to stop working on sophisticated solutions for marginal improvements continously ignoring methodologies promising speed-ups by orders of magnitude.... blinders to ignore the impact of morphware... continue to bang their heads against the memory wall instead of *) [Michel Dubois]

© 2004, TU Kaiserslautern 9 the data-stream-based approach has no von Neumann bottle- neck … understand only this parallelism solution: the instruction-stream-based approach von Neumann bottle- necks... cannot cope with this one

© 2004, TU Kaiserslautern 10 >> Embedded Computing << HPC Embedded Computing The wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 11 History of Machine Models mainframe age main frame. compile procedural mind set: instruction-stream-based (coordinates by Makimtos wave) computer age (PC age) accel. µ Proc. compile users: RIKEN institute, ARI, Heidelberg, etc. MD-GRAPE-2 PCI board [1997] 4 chips for N-body simulation converts a PC to 64 GFlops scientific computing example: molecular dynamics, astrophysics, plasma physics, hydrodynamics:

© 2004, TU Kaiserslautern 12 History of Machine Models mainframe age main frame. compile procedural mind set: instruction-stream-based (coordinates by Makimtos wave) computer age (PC age) accel. µ Proc. compile structural mind set: data-stream-based by hardware guys design

© 2004, TU Kaiserslautern 13 the hardware / Software Chasm: typical programmers don‘t understand function evaluation without machine mechanisms (counters, state registers) It‘s the gap between procedural (instruction-stream- based) and structural (datastream-based) mind set accelerators µ processor

© 2004, TU Kaiserslautern 14 Growth Rate of Embedded Software months factor *) Department of Trade and Industry, London (1.4/year) [Moore ’ s law] >10 times more programmers will write embedded applications than computer software by 2010 Embedded software [DTI*] (~2.5/yr) already to-day, more than 98% of all microprocessors are used within embedded systems

© 2004, TU Kaiserslautern 15 typical CS graduates: the „havenots“ To-day, „ typical “ CS graduates are unqualified for this labor market … cannot cope with Hardware / Configware / Software partitioning issues … cannot implement Configware

© 2004, TU Kaiserslautern 16 the current CS mind set is based on the Submarine Model Hardware invisible: under the surface Hardware Algorithm Assembly Language procedural high level Programming Language Software This model does not suport Hardware / Configware / Software partitioning

© 2004, TU Kaiserslautern 17 Hardware / Configware / Software Partitioning skills urgently needed Algorithm partitioning HW CW SW to cope with each of it: SW, CW, HW. SW / HW SW / CW / HW SW / CW CW / HW or: to cope with any combination of co-design. Software to Configware Migration is the most important source of speed-up Hardware is just frozen Configware

© 2004, TU Kaiserslautern 18 By the way... International Conference on Field-Programmable Logic and Applications (FPL) Aug. 20 – Sept 1, 2004, Antwerp, Belgium 288 submissions !... the oldest and largest conference in the field: accel. µ Proc.... going into every type of application they all work on high performance

© 2004, TU Kaiserslautern 19 Dominance of the Submarine Model... Hardware... indicates, that our CS education system produces zillions of mentally disabled CS graduates (procedural) structurally disabled … disabled to cope with solutions other than instruction-stream-based

© 2004, TU Kaiserslautern 20 CS Education procedural have not You cannot * teach Hardware to a Programmer *) efficiently But to a Hardware Guy you always can teach Programming structural have natural

© 2004, TU Kaiserslautern 21 >> the wrong Roadmap << HPC Embedded Computing the wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 22 Completely wrong roadmap beef up old architectural principles by new technology? growth factor µm 0.1 performance area efficiency „Pollack‘s Law“ (simplified) [intel]... the CPU is a methusela, the steam engine of the silicon age

© 2004, TU Kaiserslautern 23 Completely wrong mind set The key problem, the memory wall, cannot be solved by new CPU technology We need a 2 nd machine paradigm (a 2 nd mind set...) The vN paradigm is not a communication paradigm Its monopoly creates a completely wrong mind set We need an architectural communication paradigm But we need both paradigms: a dichotomy

© 2004, TU Kaiserslautern 24 3 rd machine model became mainstream computer age (PC age) accel. design µ Proc. compile (Makimtos wave) mainframe age main frame compile instruction- stream-based DPA r r µ Proc. programmable most CS curricula & HPC are still here morphware age

© 2004, TU Kaiserslautern 25 >> Configware Engineering << Supercomputing (HPC) Embedded Computing The wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 26 de facto Duality of RAM-based platforms traditionalnew RAM-based platformCPUmorphware (FPGA, rDPA..) „running“ on it: softwareconfigware machine paradigmvon Neumann etc.: instruction-stream-based anti machine: data-stream-based 2 nd paradigm We now have 2 types of programmable platforms soft hardware morphware [DARPA] hardware viewed as frozen configware: Just earlier binding

© 2004, TU Kaiserslautern 27 [Gordon Bell]... going into every type of application [Gordon Bell].... the brain hurts CW has become mainstream... Others experienced, that the brain hurts, when trying the paradigm shift The HPC scene believed to be smart, when smiling about us CW guys morphware: fastest growing sector of the IC market

© 2004, TU Kaiserslautern 28 DPA morphware age r r From Software to Configware Industry structural personalization: RAM-based Repeat Success Story by a 2 nd Machine Paradigm ! Growing Configware Industry computer age (PC age) µ Proc. compile Procedural personalization via RAM-based. Machine Paradigm Software Industry 1) 2) Software Industry’s Secret of Success anti machine

© 2004, TU Kaiserslautern 29 benefit from RAM-based & 2 nd paradigm RAM-based platform needed for: flexibility, programmability avoiding the need of specific silicon mask cost: currently 2 mio $ - rapidly growing 1) simple 2nd machine paradigm needed as a common model: to avoid the need of circuit expertize needed to to educate zillions of programmers 2) (vN relocatability is based on the von Neumann bottleneck) high price By the way: relocatability is more difficult, but not hopeless

© 2004, TU Kaiserslautern 30 configware resources: variable Nick Tredennick’s Paradigm Shifts explain the differences 2 programming sources needed flowware algorithm: variable Configware Engineering Software Engineering 1 programming source needed algorithm: variable resources: fixed software CPU

© 2004, TU Kaiserslautern 31 Compilation: Software vs. Configware source program software compiler software code Software Engineering configware code mapper configware compiler scheduler flowware code source „ program “ Configware Engineering placement & routing data

© 2004, TU Kaiserslautern 32 DPA x x x x x x x x x | || xx x x x x xx x -- - input data streams xx x x x x xx x x x x x x x x x x | | | | | | | | | | | | | | output data streams „ data streams “ time port # time port # time port # Flowware defines:... which data item at which time at which port Flowware programs data streams

© 2004, TU Kaiserslautern 33 Flowware: not new computer age (PC age) accel. design µ Proc. compile (Makimtos wave) mainframe age main frame compile DPA r r µ Proc. morphware age *) no confusion, please: no „ dataflow machine “ !!! data stream*... Flowware: around 1980

© 2004, TU Kaiserslautern 34 data streams * : not new 1980: data streams (Kung, Leiserson: systolic arrays) 1989: data-stream-based Xputer architecture 1990: rDPU (Rabaey) 1994: Flowware Language MoPL (Becker et al.) 1995: super systolic array (rDPA) + DPSS tool (Kress) 1996+: Stream-C language, SCCC (Los Alamos), SCORE, ASPRC, Bee (UC Berkeley), : streaming languages (Stanford et al.) 1996: configware / software partitioning compiler (Becker) *) please, don ‘ t confuse with „ data flow “

© 2004, TU Kaiserslautern 35 >> Dual Machine Paradigms << HPC Embedded Computing The wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 36 Why a new machine paradigm ??? The anti machine as the 2 nd paradigm is the key to curricular innovation rDPA µ processor... a Troyan horse to introduce the structural domain to the procedural-only mind set of programmers Programming by flowware instead of software is very easy to learn Flowware education: no fully fledged hardware expert needed to program embedded systems (... same language primitives)

© 2004, TU Kaiserslautern 37 von Neumann vs. anti machine progra m counter DPU CPU RAM memory von Neumann bottleneck (r) DPA without sequencer no CPU ! asMA: auto-sequencing Memory Array asM (r) DPA data stream machine (anti machine) data counter memory bank asM asM: auto-sequencing Memory instruction stream machine (von Neumann etc.)

© 2004, TU Kaiserslautern 38 Behavior of the Counter data counter memory bank asM progra m counter DPU CPU programmed by flowware data streams programmed by software (r) DPA programmed by flowware

© 2004, TU Kaiserslautern 39 Counters: the same micro architecture ? data stream machine (anti machine) data counter memory bank asM progra m counter DPU CPU instruction stream machine: (von Neumann etc.) yes, is possible, but for data counters... *) for history of AGUs see Herz et al.: Proc. ICECS 2002, Dubrovnik, Croatia... a much better AGU methodology is available* AGU: address generator unit

© 2004, TU Kaiserslautern 40 commercial rDPA example: PACT XPP - XPU128 XPP128 rDPA Evaluation Board available, and XDS Development Tool with Simulator buses not shown rDPU Full 32 or 24 Bit Design working silicon 2 Configuration Hierarchies © PACT AG, Munich (r) DPA

© 2004, TU Kaiserslautern 41 array size: 10 x 16 = 160 rDPUs mapping algorithms efficently onto rDPA rout thru only not used backbus connect SNN filter on KressArray [Ulrich Nageldinger] by DPSS: based on simulated annealing

© 2004, TU Kaiserslautern 42 symbiosis of machine models computer age (PC age) accel. design µ Proc. compile (Makimtos wave) mainframe age main frame compile morphware age DPA r r µ Proc. replace PC by PS co-compiler symbiosis

© 2004, TU Kaiserslautern 43 Software / Configware Co-Compilation Analyzer / Profiler SW code SW compiler paradigm “vN" machine CW Code CW compiler anti machine paradigm Partitioner Resource Parameters supporting different platforms Jürgen Becker’s CoDe-X, 1996 High level PL source

© 2004, TU Kaiserslautern 44 >> Speed-up Examples << HPC Embedded Computing The wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 45 Better solutions by Configware Memory cycles minimized e.g.: no instruction fetch at run time & other effects Memory access for data: caches do not help anyhow Loop xforms: no intra-stream data memory cycles Complex address computation: no memory cycles No cache misses! instead of software methodologies not new: high level synthesis (1980+) loop transformations (1970+) many other areas

© 2004, TU Kaiserslautern 46 speed-up examples platformapplication examplespeed-up factormethod PACT Xtreme 4-by-4 array [2003] 16 tap FIR filterx16 MOPS/mW straight forward MoM anti machine with DPLA* [1983] grid-based DRC** 1-metal 1-poly nMOS *** 256 reference patterns > x1000 (computation time) multiple aspects *) DPLA: MPC fabr. via E.I.S. multi univ. project key issue: algorithmic cleverness **) Design Rule Check CPU 2 FPGA [FPGA 2004] migrate several simple application exampes x7 – x46 (compute time) hi level synthesis ***) for 10-metal 3-poly cMOS expected: >> x10,000 DSP 2 FPGA [Xilinx ] from fastest DSP: 10 gMACs to 1 teraMAC X 100 (compute time) not spec. 2) Wim Roelandts

© 2004, TU Kaiserslautern 47 Software to Configware Migration: ( RAW’99 at Orlando) question by a highly respected industrial senior researcher: Ulrich Nageldinger‘s talk about KressArray Xplorer: „But you can‘t implement decisions!“ (symptom of the configware/software chasm)

© 2004, TU Kaiserslautern 48 hypothetical branching example to illustrate time-to-space migration *) if no intermed. storage in register file C = 1 simple conservative CPU example memory cycles nano seconds if C then read A read instruction1100 instruction decoding read operand*1100 operate & reg. transfers if not C then read B read instruction1100 instruction decoding add & store read instruction1100 instruction decoding operate & reg. transfers store result1100 total 5500 S = R + (if C then A else B endif); S + ABR C clock 200 MHz (5 nanosec) =1 section of a major pipe network on rDPA no memory cycles: speed-up factor = 100

© 2004, TU Kaiserslautern 49 rDPA (coarse grain) vs. FPGA (fine grain) commodity roughly: area efficiency (trans/chip, o ‘ o ‘ magnitude) hardwired 4 FPGA 2 µProc 0 rDPA 4 roughly: performanc e (MOPS/mW, o ‘ o ‘ magnitude) hardwired 3 FPGA 2 µProc 0 rDPA 3 DSP 1 Status: ~1998

© 2004, TU Kaiserslautern 50 Why the speed-up although FPGA is clock slower by x 3 or even more (most know-how from „ high level synthesis “ discipline) moving operator to the data stream (before run time) support operations: no clock nor memory cycle decisions without memory cycles nor clock cycles most „ data fetch “ without memory cycle

© 2004, TU Kaiserslautern 51 >> Final Remarks << HPC Embedded Computing The wrong Roadmap Configware Engineering Dual Machine Paradigms Speed-up Examples Final Remarks

© 2004, TU Kaiserslautern 52 First Indications of Change 10th RAW at IPDPS, Nice, France, April 2003: after a decade of non-overlap: first IPDPS people coming HPC Asia th Int‘l Conference on High Performance Computing, July 20-22, 2004 Omiya Sonic City, Tokyo Area, Japan: Workshop on Reconfigurable Systems f. HPC (RHPC) + keynote address * HPCA-11, 11th International Symposium on High-Performance Computer Architecture, San Francisco, Febr , 2005: topic area explicitely: Embedded and reconfigurable architectures SBAC-PAD th Symposium on Computer Architecture and High Performance Computing, Foz do Iguacu, PR, Brazil, October 27-29, 2004: topic area explicitely: Reconfigurable Systems *) keynote speaker: PARS & Speed-up, Basel, Switzerland, March 2003: keynote address * IPDPS, Santa Fe, NM, USA, April 2004: keynote address * PDP’04, La Coruna, Spain, Febr. 2004: keynote address *

© 2004, TU Kaiserslautern 53 HPC experts coming... ARI, Astrononisches Rechen-Institut, founded 1700 in Berlin, moved 1945 to Heidelberg by August Kopff Gottfried Kirch Simulation of Star Clusters: x10 speed-up by supercomputer-to-morphware migration (also molecular biology et al.) Rainer Spurzem, University of Heidelberg Reinhard M ä nner, University of Mannheim HPC pioneer since 1976 (Physics Dept Heidelberg) Configware by Astrophysics by example: N-body problem went configware paper already at FPL

© 2004, TU Kaiserslautern 54 Conclusions We need an academic grass roots movement, for.... RC has become mainstream in all kinds of applications... by a merger with the embedded systems mind set CS education deficits: a curricular revision is overdue...free material & tools for undergraduate lab courses to program and emulate small SW/CW/HW examples all know-how needed readily available: get involved !

© 2004, TU Kaiserslautern 55 END