1. LQCD Workflow Project L. Piccoli October 02, 2006

2. October 2 20062 Triana

3. October 2 20063 Triana public class Job extends Unit { // parameter data type definitions private String queue; private int nodes; /* * Called whenever there is data for the unit to process */ public void process() throws Exception { // Implementation }

4. October 2 20064 Triana Integration with existing system –Control and data dependencies represented –Substitute for current run makefile tool Creation of toolboxes for generic jobs Not as flexible –Workflows can be turned into command line applications –No Web Services or Grid capabilities used

5. October 2 20065 Triana Requirements –Workflow representation XML containing Triana specific tags –Interaction with external scheduler Possible via task-by-task schedule –As dependencies are resolved, the execution request is passed to campaign scheduler Schedule of complete workflow requires significant changes –Campaign scheduler is in charge of resolving dependencies

6. October 2 20066 Triana Requirements –Scheduling (one or more workflows) Not present Modifications: –Submit workflows to an external scheduler –Update workflow progress based on external monitoring information –Fault Management Not present Triana services may be used for adding these features –Need further investigation

7. October 2 20067 Triana Summary LanguageJava GUIYes ScriptYes (Generated through the GUI) WF RepresetationXML (Triana tags) SchedulingNo External SchedulerNo (Triana Services?) Fault ToleranceNo (Triana Services?) Data DependenciesYes Control DependenciesYes GUI & Execution Engine CouplingYes (Plans to decouple) Non-DAG WorkflowsYes (Control loops)

8. October 2 20068 System Architecture

9. October 2 20069 This workflow represents the dependencies found in the charm and bottom 2-pt run file (available at http://whcdf03.fnal.gov/lqcd/RunWorkf lowFramework). http://whcdf03.fnal.gov/lqcd/RunWorkf lowFramework There are pseudo dependencies in this graph (e.g. hQ*923* do not depend on hQ*127*), forcing the system to execute tasks in an order that fits better to the available hardware. This optimization is manual and requires knowledge of the system by the user. The workflow scheduler should be able to handle this situation automatically.