Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Design of Distributed Grid Workflow Composition System Marian Bubak, Tomasz Gubała, Maciej Malawski, Katarzyna Rycerz Institute of Computer Science & ACC CYFRONET - AGH Kraków, Poland

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Outline  Motivation - workflow composition problem  First prototype – AFC  Architecture of the Flow Composition Sytem  Distributed registry subsystem  Agent-based composition subsystem  Application of ontology  Summary

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Grid Application Workflow  Simple definition of a Grid application A Grid application is a combination of cooperating Grid resources (application elements) joined together to achieve particular functionality  Workflow of Grid application The distributed computation is performed by application elements using a remote procedure call facility for co- ordination (data flow though means of procedure parameters passing)

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Flow Composition Problem  To execute an application on the Grid the application workflow description is needed  Workflow description should contain: list of application elements (components, services, etc.) list of element connections (control or data links) technology and framework specific deployment data  Problems: lack of important information frequent changes of environment problem not well understood (Grid prototyping)

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH First Prototype - AFC  Application Flow Composer system  Two main modules  Designed for Common Component Architecture (CCA)  Monolithic registry as GT3 service  Implemented in XCAT Framework with Java

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Advantages and Drawbacks of the AFC  Clear, modular design  Portable (XML + Java)  Two Grid technologies used  Successful as a feasibility study  Encouraging efficiency test results  Consolidated, centralized registry  Non fault-tolerant  Simple decision module (no semantic data used)  High communication and registry search overhead

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Conclusions from Prototype  To exploit advantages stay with the clear decomposition model use portable platform and languages use popular Grid development technology (OGSA)  To eliminate weaknesses introduce full decentralization of data and control add fault tolerance to overcome Grid failures apply ontology for semantic service description implement migration mechanism to reduce distant communication overhead

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Proposed Architecture of the Flow Composition System

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Composer and Registry  Composer Agent and Registry Node can be one program entity exposing externally two different behaviors through various interfaces distinct entities with registry independence and agents as registry plug-ins complete separated bodies, communicating through standardized protocol over Internet link

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Stages of Workflow Composition  User defines initial conditions in Initial Workflow Description  Flow Composer divides IWD into smaller subproblems  Each simple problem is mapped to one existing service  Information about services is achieved from Service Registry  Ready mini-workflows are concatenated into one  The workflow is returned to user as Final Workflow Document (FWD)

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Registry Node  Interfaces: management for admin register for service provider lookup for users dedicated lookup for FC  Internals: Service entries table Peer registry nodes table Data replication subsystem Specialization description Query routing table

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Grid Service Information Document  Service identification and localization data  Service category (application of which domain do use this service frequently)  Interface syntactic information (list of methods, signatures, implemented ports)  Semantics description (service capabilities, method results meaning, pre-conditions and post-conditions of method invocation)

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Data Replication  Whole structure consists of many registries connected for inter-registry communication  One registry includes a few nodes  Every node data is replicated partially in every other node  Full recovery in case of single node failure (all the data may be acquired by means of existing nodes cooperation)

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Composer Agent  Interfaces: Interface with registry Flow composition for users Inter-agent interface  Internals: Matchmaker based on ontology inference engine Problem decomposition and workflow composition mechanism Migration subsystem

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Mobility of Agent  Purpose: to reduce the registry querying time by localization of the communication  Agent behaves as a stateless entity but has its internal state - this implies relocation instead simple re-spawn  Migration involves historical data and rules set marshalling and transfer  Problem of widespread platform

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Application of Ontology  Semantic description of services in GSID  Ontology reasoning to provide accurate matchmaking facility  Semantic description of registry specialization  Specification of rules for automatic complex problem decomposition

Cracow Grid Workshop, October 27 – 29, 2003 Institute of Computer Science AGH Summary – Expected Improvements  Portable for many architectures (highly portable languages)  High scalability (evolves with the Grid environment)  Lightweight underlying platform (deploying automation)  Fault tolerance support (data distribution and replication)  Adaptable to changing environment (dynamic topology)  Faster request handling (work distributed to many nodes)  More accurate decisions (self-learning engines, semantic service and registry description)