A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester

15 Dec 2004Workflow in Astrogrid 1 Part One... A Quick Tour of Workflow Architecture

15 Dec 2004Workflow in Astrogrid 1 What is the purpose of Workflow? To accomplish a complex piece of work: It takes more than one step to achieve. The workflow is executed asynchronously. Each step is executed asynchronously.

15 Dec 2004Workflow in Astrogrid 1 A Skeleton Workflow

15 Dec 2004Workflow in Astrogrid 1 Expansion of Step One VOTABLE ivo://...#jl99/query/6dfQuery.xml ivo://...#jl99/votable/6dfQueryResults.vot

15 Dec 2004Workflow in Astrogrid 1 The View from 10,000 Metres A workflows exist at: Design Time Execution Time *** When submitted for execution, a workflow becomes a job. ***

15 Dec 2004Workflow in Astrogrid 1 Design Time – From 5000 Metres A Design: Is held as an XML file in MySpace. The Design process: Is managed by the Workflow Editor in the Portal. Is aided by holding metadata in the Astrogrid Registry.

15 Dec 2004Workflow in Astrogrid 1 Execution Time – From 5000 Metres Job Execution System: Central controlling element. Interprets the workflow design. Manages the flow of control between Steps. Common Execution Architecture: Manages the local environment for each Step executed. Manages the flow of data required by each Step.

15 Dec 2004Workflow in Astrogrid 1 Overview of Workflow Architecture

15 Dec 2004Workflow in Astrogrid 1 Part Two... Workflow as a Programming Language

15 Dec 2004Workflow in Astrogrid 1 Back to Design... Steps and Tools Sequences and Flows. Variables and Scopes. Loops and Conditional Execution. Scripting and Script Objects.

15 Dec 2004Workflow in Astrogrid 1 Sequence – Sequential Execution

15 Dec 2004Workflow in Astrogrid 1 Flow – Parallel Execution

15 Dec 2004Workflow in Astrogrid 1 Setting Variables Setting... Using variables to construct a filename... ${namestub}-${filecount + 1}

15 Dec 2004Workflow in Astrogrid 1 Conditional Execution... 0}">......

15 Dec 2004Workflow in Astrogrid 1 While Loop...

15 Dec 2004Workflow in Astrogrid 1 For Loop...

15 Dec 2004Workflow in Astrogrid 1 Scripting Scripting Expressions Script 0}">... print("Step produced " + ${count} + "votables");

15 Dec 2004Workflow in Astrogrid 1 Scripting Example plus Script Objects... if (results.size() != 1) { jes.error("didn't produce expected number of results"); } else { votable = results.get('votable'); parser = new XmlParser(); nodes = parser.parseText(votable); urlList = nodes.depthFirst().findAll{it.name() == 'STREAM'}.collect{it.value()}.flatten(); print(urlList); }

15 Dec 2004Workflow in Astrogrid 1 Conclusion – Astrogrid Workflow Capabilities Distributed workflow executing on an asynchronous basis Highly programmable Workflows from the simple to the complex Can integrate astronomical tools

15 Dec 2004Workflow in Astrogrid 1 Contacts. Work Group Leader: Jeff Lusted Workflow and JES: Noel Winstanley Common Execution Architecture: Paul Harrison Workflow Editor: Phil Nicolson