LQCD Workflow Project L. Piccoli October 02, 2006

October Triana

October Triana public class Job extends Unit { // parameter data type definitions private String queue; private int nodes; /* * Called whenever there is data for the unit to process */ public void process() throws Exception { // Implementation }

October Triana Integration with existing system –Control and data dependencies represented –Substitute for current run makefile tool Creation of toolboxes for generic jobs Not as flexible –Workflows can be turned into command line applications –No Web Services or Grid capabilities used

October Triana Requirements –Workflow representation XML containing Triana specific tags –Interaction with external scheduler Possible via task-by-task schedule –As dependencies are resolved, the execution request is passed to campaign scheduler Schedule of complete workflow requires significant changes –Campaign scheduler is in charge of resolving dependencies

October Triana Requirements –Scheduling (one or more workflows) Not present Modifications: –Submit workflows to an external scheduler –Update workflow progress based on external monitoring information –Fault Management Not present Triana services may be used for adding these features –Need further investigation

October Triana Summary LanguageJava GUIYes ScriptYes (Generated through the GUI) WF RepresetationXML (Triana tags) SchedulingNo External SchedulerNo (Triana Services?) Fault ToleranceNo (Triana Services?) Data DependenciesYes Control DependenciesYes GUI & Execution Engine CouplingYes (Plans to decouple) Non-DAG WorkflowsYes (Control loops)

October System Architecture

October This workflow represents the dependencies found in the charm and bottom 2-pt run file (available at lowFramework). lowFramework There are pseudo dependencies in this graph (e.g. hQ*923* do not depend on hQ*127*), forcing the system to execute tasks in an order that fits better to the available hardware. This optimization is manual and requires knowledge of the system by the user. The workflow scheduler should be able to handle this situation automatically.