1. 1 On MDE to implement a Component Assembly Compiler for HPC Journées IDM, June the 7th 2011 Julien Bigot*, Christian Pérez* Laboratoire de l’Informatique du Parallelisme (LIP) ENS Lyon

2. 2 Context: High Performance Computing Scientific applications Domain specialist code Long life cycle (~30 years) Computation intensive Hardware Heterogeneous –Supercomputers –Computing clusters –Computing grids –Clouds Parallel Short life cycle (~3 years) Programming model Low level, fine-tuning of code Hand made … and remade Thermal Optics Dynamics Structural Mechanics SAN Cluster LAN WAN Supercomputer Grid Supercomputer Cluster Satellite modeling ? programming models

3. 3 Outline Context: High Performance Computing (HPC) High Level Component Model (HLCM) Modeling and Implementation Feedback on the Model Driven Approach Conclusion

4. 4 Programming model requirements Ease of development Domain specialist: Domain code / support code separation Long life cycle: clean code behavior and interface specification High Performance Minimal overhead Support for parallelism Hardware evolution Algorithm optimization Interaction optimization –E.g. distributed memory vs. shared memory Software Components

5. 5 Software components Components black boxes Interactions through well defined interaction points Application Set of interconnected instances High level view of architecture Separation of concern Domain code: components Support: component model Eased reuse Well defined interface & behavior CmpACmpB a b c d c1 c2

6. 6 Software Component Models Industrial Component Models (CORBA Component Model, Enterprise JavaBeans, …) Transparent distribution support (CORBA, JavaRMI, …) Overhead for same host components Research Component Models (Fractal, …) Interesting features (Hierarchy, Connectors, …) Features handled at runtime, additional costs High Performance Component Models (CCA, …) Acceptable overheads Low level of abstraction => Hardware adaptation complex

7. 7 Programming model requirements Ease of development Domain specialist: Domain code / support code separation Long life cycle: clean code behavior and interface specification High Performance Minimal overhead Support for parallelism Hardware evolution Algorithm optimization Interaction optimization –E.g. distributed memory vs. shared memory Software Components Component implementation choice

8. 8 Multiple alternative implementations Distinction type/implementation Type: name + ports Implementation: behavior Choice Depending on available hardware CmpA I1 I2

9. 9 Programming model requirements Ease of development Domain specialist: Domain code / support code separation Long life cycle: clean code behavior and interface specification High Performance Minimal overhead Support for parallelism Hardware evolution Algorithm optimization Interaction optimization –E.g. distributed memory vs. shared memory Software Components Component implementation choice 1 st class Software Connectors with implementation choice

10. 10 1st class software connectors 1st class entity A Name Set of roles Multiple alternative implementations Hardware optimization possible Interaction code well identified Multiple versions possible More application specific interactions UP CmpACmpB CmpA CmpB ports roles userprovider

11. 11 Programming model requirements Ease of development Domain specialist: Domain code / support code separation Long life cycle: clean code behavior and interface specification High Performance Minimal overhead Support for parallelism Hardware evolution Algorithm optimization Interaction optimization –E.g. distributed memory vs. shared memory Software Components Algorithmic skeletons, generic programming & hierarchy Component implementation choice 1 st class Software Connectors with implementation choice

12. 12 Generic programming & hierarchy: Algorithmic skeletons Disp Coll W W W Farm...... N times IO Farm

13. 13 Programming model requirements Ease of development Domain specialist: Domain code / support code separation Long life cycle: clean code behavior and interface specification High Performance Minimal overhead Support for parallelism Hardware evolution Algorithm optimization Interaction optimization –E.g. distributed memory vs. shared memory Software Components Compilation based approach Algorithmic skeletons, generic programming & hierarchy Component implementation choice 1 st class Software Connectors with implementation choice No need for new components, only a new assembly model

14. 14 HLCM Compilation Reuse an existing component model (L²C) HLCM: Genericity Hierarchy Connectors Multi-Implem L²C: Low Level Primitive components only Various interaction C++ MPI CORBA … Hardware description PIM PSM PDM MDA

15. 15 HCLM (Meta)Modeling Using the Eclipse Modeling Framework (EMF) PIM Main classes: ComponentType, PortType, Connector, … Primitive implementations: abstract Some patterns identified: genericity, implementation choice 130 Ecore metaclasses (490 lines of EMFatic) 2000 utility Java lines PSM Main classes: ComponentInstance, Connexion, … References to the PIM in each instance 41 Ecore metaclasses (160 lines of EMFatic) 800 utility Java lines

16. 16 HCLM Transformation Four step transformation Parsing HLCM: 400 lines Xtext annotated grammar Compilation: 4000 lines plain Java PSM to Backend Backend Model Dump Compilation Choosing implementations (delegated to an external “oracle”) Concretizing generic elements Flattening composites Until convergence: only primitive elements Final dump Often single hand made step Backend specific

17. 17 Feedback (1/2) Modeling tools: huge gain of time Modeling HLCM –Initial attempt with plain Java: ~1 month work –Second attempt with Ecore: ~2 days work Direct concrete syntax to model parsing (Xtext) Ease of comprehension Most concepts intuitive with UML background Various level of modeling confusing (model vs. metamodel) –E.g. connectors –Fixed set of interactions: various connectors in the metamodel –1st class connectors: various connectors in the model –Combination predefined connectors + user defined : ??? –E.g. genericity: manipulation of metamodel elements in the model About same state

18. 18 Feedback (2/2) Model transformation Transformation tools not ready 2 years ago (Kermeta, QVT, ATL) Plain java transformation – API not human friendly –Collection access => whole collection traversal –Omnipresent type casts (constraint not expressible in Ecore) –Error complex to associate with user error Constraint language evaluation required Real use Applications: Parallel method calls, shared memory, Domain decomposition Backends: L²C, Pure Java, CCM, Charm++, … Transformation speed: 1500+ components, ≤ 10secs

19. 19 Conclusion HLCM: a component assembly model for HPC Hierarchy, Generic Programming, Connectors, Alternative Implementations –Efficiency possible on a wide range of hardware A model based implementation –Lot of time gained –Some tools not used due to immaturity Usable in real world Perspectives / future work Re-evaluate modeling tools (ATL, OCL, …) Introduce the time dimension in HLCM –Models@runtime ?

20. 20

21. 21 HLCM Transformation HLCM/L²C files L²C: Low Level Minimal costs Various interaction HLCM: Genericity Hierarchy Connectors Multi-Implem component App { content { composite { decl { Decrement_Composite D; Viewer v; Decrement d1; d1.h_log_port -- v.v_log_port; D.h_log_port -- v.v_log_port; d1.dp_dec_port -- D.du_dec_port; d1.du_dec_port -- D.dp_dec_port; set d1.Number 3; } service run { seq { exec d1.export; exec D; } component App { content { composite { decl { Decrement_Composite D; Viewer v; Decrement d1; d1.h_log_port -- v.v_log_port; D.h_log_port -- v.v_log_port; d1.dp_dec_port -- D.du_dec_port; d1.du_dec_port -- D.dp_dec_port; set d1.Number 3; } service run { seq { exec d1.export; exec D; } ParsingModel TransformationDump A T B C A A B_0 D B B_0 C B_1 D D L²C files HLCM/L²C model DEST model HLCM/L²C metamodel DEST metamodel

22. 22 Modeling HLCM: Genericity Concepts Generic types –Has generic parameters Generic parameters –Used as a type Generic Arguments –Associate value to parameters Algorithm Copy non generic elements For each generic reference –Create a context (parameter  value) –Copy the generic element in that context A T B C A A B_0 D B B_0 C B_1 D D

23. 23 Modeling HLCM: Alternative Implementations Distinguish component type & component implementation Component type –Generic –List of ports Component implementation –Generic –Implemented component type specialization –Content description –Primitive –Composite Choice at deployment time Match used specialization & implemented specialization Choose amongst possible CmpA A2 A1 A3 T=X T=B T=C CmpA A3 T=B component CmpA exposes { provide p; } primitive A1 implements CmpA { …} composite A3 implements CmpA { …}

24. 24 HLCM Overview HLCM: High Level Component Model Actually a component assembly model Primitive components borrowed from an existing “backend” model HLCM Features Hierarchy: composite components Generic programming (C++ templates, Java generics) Software connectors: interactions as 1 st class concept Multiple alternative implementations for components & connectors HLCM Implementation Model based Transformation when hardware is known Generates an assembly of the backend model

25. 25 Modeling HLCM: Genericity (1/2) Example Generic artifact –ComponentType Artifact usable as parameter –PortType Additional modifications Default values for parameters Constraints on parameter values Explicit specializations Meta-programming PortPortTypeParamPortTypeArgComponentTypePortTypeComponentInstance

26. 26 Modeling HLCM: Alternative Implementations (1/2) PIM: Distinguish 3 concepts Type –ComponentType, name & list of ports Implementation –ComponentImplementation primitive / composite InstanceDescriptor –Composite subcomponent –Application root component PSM: 1 type Instance –ComponentInstance primitive / composite TypeImplementationInstanceDescriptorInstance