1. Gary Kumfert Center for Applied Scientific Computing is not a framework… but it makes frameworks better This work was performed under the auspices of the U.S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under Contract No. W-7405-Eng-48. UCRL-PRES-205332 CCA Common Component Architecture

2. 2 Outline Zen of Babel (too short to be an intro) Establish my “street-cred” (uses & users of Babel) Change-Oriented Software (birth of a new buzz-word?)

3. 3 F90/Babel “Hello World” Application program helloclient use greetings_English implicit none type(greetings_English_t) :: obj character (len=80) :: msg character (len=20) :: name name=’World’ call new( obj ) call setName( obj, name ) call sayIt( obj, msg ) call deleteRef( obj ) print *, msg end program helloclient program helloclient use greetings_English implicit none type(greetings_English_t) :: obj character (len=80) :: msg character (len=20) :: name name=’World’ call new( obj ) call setName( obj, name ) call sayIt( obj, msg ) call deleteRef( obj ) print *, msg end program helloclient These subroutines come directly from the SIDL These subroutines come directly from the SIDL Some other subroutines are “built in” to every SIDL class/interface Some other subroutines are “built in” to every SIDL class/interface

4. 4 greetings.sidl: A Sample SIDL File package greetings version 1.0 { interface Hello { void setName( in string name ); string sayIt ( ); } class English implements-all Hello { } } package greetings version 1.0 { interface Hello { void setName( in string name ); string sayIt ( ); } class English implements-all Hello { } } Next several slides will show implementations of this interface in Python & C++. See C & Fortran 90 in our SC’04 Tutorial Slides www.llnl.gov/CASC/componentswww.llnl.gov/CASC/components   

5. 5 A Python Implementation class English: def __init__(self, IORself): self.__IORself = IORself # DO-NOT-DELETE splicer.begin(__init__) self.d_name = ’’ # DO-NOT-DELETE splicer.end(__init__) def sayIt(self): # DO-NOT-DELETE splicer.begin(sayIt) return ’Hello ’ + self.d_name + ’!’ # DO-NOT-DELETE splicer.end(sayIt) def setName(self, name): # DO-NOT-DELETE splicer.begin(sayIt) self.d_name = name # DO-NOT-DELETE splicer.end(sayIt) class English: def __init__(self, IORself): self.__IORself = IORself # DO-NOT-DELETE splicer.begin(__init__) self.d_name = ’’ # DO-NOT-DELETE splicer.end(__init__) def sayIt(self): # DO-NOT-DELETE splicer.begin(sayIt) return ’Hello ’ + self.d_name + ’!’ # DO-NOT-DELETE splicer.end(sayIt) def setName(self, name): # DO-NOT-DELETE splicer.begin(sayIt) self.d_name = name # DO-NOT-DELETE splicer.end(sayIt) greetings/English_Impl.py     

6. 6 A C++ Implementation ::std::string greetings::English_impl::sayIt() throw () { // DO-NOT-DELETE splicer.begin(greetings.English.sayIt) ::std::string msg(”Hello ”); return msg + d_name + ”!”; // DO-NOT-DELETE splicer.end(greetings.English.sayIt) } ::std::string greetings::English_impl::sayIt() throw () { // DO-NOT-DELETE splicer.begin(greetings.English.sayIt) ::std::string msg(”Hello ”); return msg + d_name + ”!”; // DO-NOT-DELETE splicer.end(greetings.English.sayIt) } namespace greetings { class English_impl { private: // DO-NOT-DELETE splicer.begin(greetings.English._impl) ::std::string d_name; // DO-NOT-DELETE splicer.end(greetings.English._impl) namespace greetings { class English_impl { private: // DO-NOT-DELETE splicer.begin(greetings.English._impl) ::std::string d_name; // DO-NOT-DELETE splicer.end(greetings.English._impl) greetings_English_Impl.hh greetings_English_Impl.cc       

7. 7 When I say “Language Interoperability” I mean something very different than from what most applications do. Simulation Framework (C) Solver Library (C++) Numerical Routines (FORTRAN 77) Scripting Driver (Python) Visualization System (Java) Physics Models (FORTRAN 90) Logging and Plotting (Python) Suppose your iterative solver isn’t converging, but oscillating in a curious way. Can you pause the simulation, write a Python script to extend the (C++) convergence check and log the pertinent physics in those regions?

8. 8 When I say “Language Interoperability” I mean complete language transparency Simulation Framework (C) Solver Library (C++) Numerical Routines (FORTRAN 77) Scripting Driver (Python) Visualization System (Java) Physics Models (FORTRAN 90) Adaptive Sampling (Python) Now suppose you have a regime in your physics models that is of interest. Can you extend Fortran90 modules in some scripts to explore new ideas as the simulation progresses?

9. 9 Mixing Of Languages is Commonplace, but too often Inflexible & Fragile Developers are often painfully aware of language boundaries  Usually fixed early on in design process  Frequent source of portability problems. Usually a great reluctance to add new languages  Often teams will rewrite a package in a language they already use, rather have to add a new one.  Particularly true when several languages are already in play. Most tools for language interoperability are point-to-point & and have a strong bias for one or the other

10. 10 When Mixing n Languages, Tool usage can grow O(n 2 ) C C++ Fortran 90 Python Fortran 77 Java cfortran.h Chasm COM CORBA JNI Native Platform-Dependent Siloon SWIG

11. 11 Babel is an n-way Language Interoperability Tool C C++ f77 f90 Python Java Once a library has been “Babelized” it is equally accessible from all supported languages Additional languages added ~1/year

12. 12 Babel Supports a Uniform Model Across All Languages C C++ f77 f90 Python Java Full OOP, Polymorphism, Exception Handling in every language. Can throw an exception from C++, catch it in F77 and have the exception itself be in C

13. 13 Quick Review: Connectivity Strategies File I/O (slowest…even archival)  UNIX Pipes Messaging Layer (fairly slow, data is packed & unpacked, easiest to do portably)  CORBA, WebServices, SOAP, XMLRPC Interpreted Middleware (virtual machine)  EJB,.NET, Pyre In Process (fastest, most complex/arcane, vendor specific)  Microsoft COM  Babel

14. 14 Applying Babel to Legacy Code 1.Write your SIDL interface 2.Generate server side in your native language 3.Edit Implementation (Impls) to dispatch to your code (Do NOT modify the legacy library itself!) 4.Compile & Link into Library/DLL mycode.sidl Babel Compiler Skels Impls IORs Stubs libmycode.so legacy_library.so

15. 15 Example of Babelized Legacy Code: MPI API choices made in this example: Made sends and receives operations on MPI communicator objects Overloaded methods based on scalar type Use Babel arrays instead of buffer and count  “row-major” (or “column-major”) also guarantees non-strided, even for 1-D arrays. package mpi version 2.0 { class Comm { int send[Int]( in array data, in int dest, in int tag );... } package mpi version 2.0 { class Comm { int send[Int]( in array data, in int dest, in int tag );... } mpi.sidl

16. 16 Example of Babelized Legacy Code (MPI): The *Impl.h MPI is a C standard, so implement the Babel wrappers in C. New Communication Objects have state  For C state is kept in a *_data struct.  Remember to observe splicer blocks /* DO-NOT-DELETE splicer.begin(mpi.Comm._includes) */ #include “mpi.h” /* DO-NOT-DELETE splicer.end(mpi.Comm._includes) */... struct mpi_Comm__data { /* DO-NOT-DELETE splicer.begin(mpi.Comm._data) */ MPI_Comm comm; /* DO-NOT-DELETE splicer.end(mpi.Comm._data) */ }; /* DO-NOT-DELETE splicer.begin(mpi.Comm._includes) */ #include “mpi.h” /* DO-NOT-DELETE splicer.end(mpi.Comm._includes) */... struct mpi_Comm__data { /* DO-NOT-DELETE splicer.begin(mpi.Comm._data) */ MPI_Comm comm; /* DO-NOT-DELETE splicer.end(mpi.Comm._data) */ }; mpi_comm_Impl.h

17. 17 Example of Babelized Legacy Code (MPI): The *Impl.c In C, use *_get_data() to extract user-defined state from Babel object. Since array is 1-D and unstrided, use address of first element as buffer int32_t impl_mpi_Comm_sendInt( mpi_Comm self, SIDL_int__array data, int32_t dest, int32_t tag ) { /* DO-NOT-DELETE splicer.begin(mpi.Comm.sendInt) */ struct mpi_Comm__data *dptr = mpi_Comm__get_data( self ); void * buff = (void*) sidl_int__array_first(data); int count = sidl_int__array_length(data, 0); return mpi_send( buff, count, MPI_INT, dest, tag, dptr->comm); /* DO-NOT-DELETE splicer.end(mpi.Comm.sendInt) */ } int32_t impl_mpi_Comm_sendInt( mpi_Comm self, SIDL_int__array data, int32_t dest, int32_t tag ) { /* DO-NOT-DELETE splicer.begin(mpi.Comm.sendInt) */ struct mpi_Comm__data *dptr = mpi_Comm__get_data( self ); void * buff = (void*) sidl_int__array_first(data); int count = sidl_int__array_length(data, 0); return mpi_send( buff, count, MPI_INT, dest, tag, dptr->comm); /* DO-NOT-DELETE splicer.end(mpi.Comm.sendInt) */ } mpi_comm_Impl.c

18. 18 Outline Zen of Babel (too short to be an intro) Establish my “street-cred” (uses & users of Babel) Change-Oriented Software (birth of a new buzz-word?)

19. 19 Use Babel to… Automate Language Wrappings E.g. Hypre (LLNL)  Scalable linear solvers and preconditioners (Mostly C)  Had 4 partial, non-portable Fortran Bindings  None evolved with changes to their core code Also useful for shifting burden & blame “Blame Babel!” --Randy Bramley

20. 20 Babel Background History:  Started 1999 as LDRD  21 quarterly releases since July 2001 Code-Generator 1MB Java Jar file Runtime Library 2MB C library Make check  Requires 1.5 GB of disk  Takes ~6 hours on ASCI Blue node

21. 21 Use Babel to… Get OOP in non-OO languages Don’t wait for F2003 for polymorphism Get basic benefits of OOP, without venturing beyond C (and without hand-coding your own virtual function dispatch like PETSc and X Windows did)

22. 22 Use Babel to… Build Component Frameworks CCA (SciDAC)  Rob Armstrong already talked about PSI (LLNL, LDRD)  Multiscale physics – Material science  ALE3D or ALE-AMR at coarse scale  DD or Crystal Plasticity at meso-scale  Extend Pope’s work on in-situ adaptive sampling  MPMD model – Fork&shedule multiple MPI subtasks on petascale machines DUNE(LLNL)  Granular Flows  Overlord, federated external codes, unifying parallel data store

23. 23 Use Babel to… Establish Domain Standards TSTT (SciDAC ISIC for Meshing) TOPS (SciDAC ISIC for Solvers) CCA (Started with 3 legacy frameworks which were not interoperable) Others that are “tentative” or “in-the- works”  LAPACK  ScaLAPACK  NetSolve

24. 24 Use Babel to… Connect F77 to F77 ! ;) NWChem (PNNL)  Integrate a F77 code with its own F77 code  Both link internally against C code  One assumes a trailing underscore  Another insists on –fno-underscore flag Really use Babel to create a uniform ABI.

25. 25 Outline Zen of Babel (too short to be an intro) Establish my “street-cred” (uses & users of Babel) Change-Oriented Software (birth of a new buzz-word?)

26. 26 How Big is a “Big Code”? (lines of source) Simulator for Major Systems in a Tokamak? Simulator for capsule physics in NIF? Hewlett-Packard Printer Driver? Thanks to Paul F. DuBois, LLNL

27. 27 How Big is a “Big Code”? (lines of source) Simulator for Major Systems in a Tokamak? Simulator for capsule physics in NIF? Hewlett-Packard Printer Driver? Thanks to Paul F. DuBois, LLNL 300,000 500,000 5,000,000

28. 28 How Big is a “Big Code”? (lines of source) Simulator for Major Systems in a Tokamak? Simulator for capsule physics in NIF? Hewlett-Packard Printer Driver? 300,000 500,000 5,000,000 From a software engineering perspective: if the large scale simulations aren’t really that big, why do they seem so difficult?

29. 29 Scientific Computing Software is Dominated by Change Scientific programs are changed much more often than programs of similar size in other fields.  A twenty-year-old LLNL program changed substantively 75 times in one year. It was not a period of major new development or a new machine. The developers are not the only ones who need to change the program – the users do too. Even the application area may change or expand. Thanks to Paul F. DuBois, LLNL

30. 30 Principles of Change-Oriented Software 1.Assume Change 2.Develop an Appropriate Strategy 3.Constantly Reevaluate

31. 31 Change-Oriented Principles: 1. Assume Change Internal  New Understanding or Interests  New Algorithms External  New Partnerships  New Funding Directions  New Hardware  New Software Tools, Languages, etc.

32. 32 Change-Oriented Principles: 2. Appropriate Strategy Throw-Away Internal Only Targeted External Framework Specific Components Community Code + Upfront Cost -+ Planned Longevity -- Pain w/ Unanticp. Change +

33. 33 Change-Oriented Principles: 3. Constantly Reevaluate Changeability is not fairy dust you sprinkle in your code at the end of the project! Occurrence of painful, unexpected change does not necessarily indicate strategy was flawed

34. 34 All These Software Techniques Try to Support Change Absorb change without losing correctness Empower and exploit the creativity of users Reduce dependency entanglement among developers

35. 35 Current Change-Oriented State of Art: Scripting Python is BIG at Livermore SciPy.org:  SWIG and PyFort shrink-wrapped codes  Enthought, Inc. provides the consulting services PyMPI  gives you a interactive session to parallel machine Python HydroFFTGraphics

36. 36 Users Like Scripting Developers aren’t a bottleneck Users share domain-specific expertise with each other. Users are much more productive Users enjoy coding (scripting is fun) If you expose the “main loop”, they can add physics or modify quantities (e.g., adding noise to boundary conditions or energy deposition).

37. 37 Developers Like Scripting Developers get built-in graphical debugger Prototype algorithms in interpreter. Many facilities get out of compiled code for good  If 90% of runtime is spent in 10% of code, why not script the other 90%? Can try new uses/configurations for existing pieces without a lot of investment

38. 38 Downside of Scripting Does your code look like this?

39. 39 Downside of Scripting Does your code look like this? Or like this?

40. 40 Components vs. Scripting Like Babel, Scripting is still imperative programming model using libraries Components have a “composability model”  Components register themselves to runtime system  Reflection and Introspection  Components declare what they provide as well as what the consume

41. 41 Software Components: Commercial vs. Computational Industry developed component technology to  increase reuse  control costs  scale to large software systems Large Scale Simulation needs it for  integration of small systems to large ones  amenability to change  manage correctness in the face of change

42. 42 Ongoing Babel Research Build (converting source to binary) Distributed Babel Nonblocking RMI Semantics in SIDL Automatic Component Extraction Extensibility (Typemaps)

43. 43 More Information Babel  www.llnl.gov/CASC/components www.llnl.gov/CASC/components  components@llnl.gov components@llnl.gov CCA  www.cca-forum.org www.cca-forum.org kumfert@llnl.gov Babel Team  LLNL: Tom Epperly, Tammy Dahlgren, & Jim Leek, alumni, summer students  CCA: Wael Elwasif (ORNL), Boyana Norris (ANL), Steve Parker (Utah), Ben Allan & Rob Armstrong (SNL)  Open Source Community: Adam Powel III (MIT) Debian Maintainer, and others