1. The InterComm-based CCA MxN components Hassan Afzal Alan Sussman University of Maryland

2. Motivation InterComm is a library that enables parallel applications to exchange data efficiently Intent is to run the library as a component within the Ccaffeine framework. This will enable other components running within the framework to use InterComm functionality. The real power of InterComm lies in allowing communication between MxN topologies of components Note that in Ccaffeine MxN problem isn’t trivial as every component runs in every process

3. MCMD model of InterComm Component In order for InterComm to work within CCA (and the Ccaffeine framework implementation) we create a Manager component that starts up from its go port and in turn starts up the application components, which in our simple example are one exporter and one importer component (which can be MPI programs, for example) The Manager uses ccaffeine.ports.MPIService and creates separate groups/comms for each of the application components. The comms created are optionally passed into the app components. The Manager reads in an InterComm XJD (XML job description) file to determine the number of processes a particular app component should run on and then the app component is started up only on that number of processes by the manager component The InterComm library implementation using CCA is encapsulated as three components: the XJD component, the Schedule component and the Communication component

4. MCMD model of InterComm Component One caveat about running InterComm in Ccaffeine is that the underlying library implementation makes PVM calls and PVM is not thread safe. So the app components must run in disjoint processes so that simultaneous exporter and importer calls to the library don’t cause PVM to crash. Therefore the minimum number of processes required must be at least the total required by all components using InterComm It would be OK to run other app components in the same process as InterComm exporter or importer processes provided they don’t make calls to InterComm (or PVM). For InterComm to run correctly it is necessary that each of the app component’s instances get numbered starting from 0. Coupled with the disjoint process requirement mandates that we create a separate MPI_Comm for each of the participating app component. This is achieved using MPI create and/or split comm functions in the manager component

6. Explanation of Ports Exporter & Importer Control Ports: These ports are provided by the exporter and the importer and used by the manager. The manager uses these ports to start up the app components. It is here the MPI comms specific to each component are passed in as arguments to each app component. XJD Port: This is an InterComm-specific port provided by the XJD Component and used by the app component. This is the first port the app component has to call in order to use InterComm. The port initializes and creates PVM groups to connect (parallel) components. Schedule Port: This is also a InterComm specific port and is provided by the Schedule component and used by the XJD component. The core functionality of this port is to compute the communication schedules for subsequent data transfers using the Comm port. Comm Port: This is also a InterComm specific port and is provided by the Communication component and used by the app components. The app components use this port for the exchange of (distributed) data. Note: the order in which the InterComm ports are used by an app component is as follows: –first call to the XJD port which in turn uses the Schedule port –Second (and subsequent) calls to the Comm port to do the data transfers

7. 2x3 case where the exporter runs on two processes & the importer on three

8. Conclusion This work is an example of the MCMD model, whereby the InterComm library can be used by CCA components (within the Ccaffeine framework). Application components not using MPI can also communicate with other application components using InterComm. MxN topolgies for application components can be realized with the InterComm Manager component. A future direction is to allow creating InterComm connections among app components dynamically, by building communication schedules at runtime, eliminating the requirement for every connection to be defined statically in the xjd file.