1. MSF and MAGE: e-Science Middleware for BT Applications Sep 21, 2006 Jaeyoung Choi Soongsil University, Seoul Korea choi@ssu.ac.kr

2. Sep 21, 2006 2 HG2C System Architecture Infrastructure (Cluster, Network, …) Globus Toolkit (GT3, GT4) Portal (Web) Application Research GridServices Global scheduler, Monitoring, Adaptive API, etc. Visualization Steering Workflow Portal toolkit Application SW Service Component Framework Human Genome to Chemicals (HG2C) Infrastructure Low level : Resource Management Mid level : Fundamental Services Application SW Visualization High level: Steering Workflow PSE toolkit MSF 2.0 Streaming Global scheduler Monitoring Adaptive API Text Search Data Mining High level : Application Support Service Component Framework (MAGE)

3. Sep 21, 2006 3 Meta Services  Define a part of a workflow as a new service  Workflow instance can be declared as a workflow unit in the service description  By overriding some attributes of a workflow unit, Pass parameters of a service to the workflow’s attributes Setup service specific information  The new service can be wrapped to a Web service or a Grid service, therefore it can be easily reused  Manage service specific information  Restrict resources to allocate a specific service (user’s preference and/or organization’s policy)  Scheduling jobs with priority

4. Sep 21, 2006 4 Meta Services

5. Sep 21, 2006 5 Meta Services Framework (1)  Meta Services Framework  MSF is a workflow system for Bio Grid portal  Users can compose a DAG-based workflow using legacy applications such as a BLAST  Schedule User’s workflows on Grid environments  Provide reusable workflows using Meta Services  Users can compose services, flows, and tasks using XML  Can be easily installed and configured

6. Sep 21, 2006 6 Meta Services Framework (2)  MSF Workflow model  Workflow divided into three layers - service layer, flow layer, and task layer  Increase reusability of workflow  Service layer is specially designed using Meta services concept  MSF Middleware Architecture  Consist of five modules (SM, RM, EM, AM, OM)  Three agent modules process each layer of the workflow model (SM - service, RM - flow, EM - task)  AM manages authentication and access control  OM maintains XML description

7. Sep 21, 2006 7 Five Agents in the Architecture  SM (Service Manager)  Manage meta services and converts a meta service to a workflow  RM (Resource Manager)  Collect resource information, and allocating resources  EM (Execution Manager)  Launch & monitor (workflow) jobs, detect faults, and collect results and performance data  AM (Access Manager)  User authentication, environment setup, and a job submission service  OM (Ontology Manager)  Manage ontology of service, flow, and task

8. Sep 21, 2006 8 Middleware Architecture

9. Sep 21, 2006 9 Operations defined in MSF

10. Sep 21, 2006 10 MAGE MAGE Features Provide API for easy development of Grid application Provide transparency to end-users and developers Protocol transparency Running location transparency Provide layered architecture for easy replacement Mobility for each agent MAGE Modular & Adaptive Grid Environments

11. Sep 21, 2006 11 MAGE architecture Interpreter Layer

12. Sep 21, 2006 12 Communication Layer To provide easy replacement of the communication protocol without affecting other layers Administrator can select an appropriate protocol component before running an application Tasks and message interpretation does not affected by changing of communication protocol

13. Sep 21, 2006 13 Interpreter Layer Interpret received messages & deliver to the appropriate task agents Two basic query components implemented Monitor Query: use SQL ’ s SELECT-like statement Table Query: use name=value pair table

14. Sep 21, 2006 14 Task Agent Management Layer Control the life cycle of task agents Provide functions for installation from remote Basic functions for mobile agents Advanced functions for suspending a job & resuming it at another node

15. Sep 21, 2006 15 MAGE & MSF Interpreter Layer

16. Sep 21, 2006 16 Advantages of MAGE & MSF  MSF contains essential functionalities for BT Grid portals  workflow, service interface, job distribution, and parameter scheduling  Implementing MSF modules as MAGE agents increases reconfigurability and adaptability  From MSF modules to MSF agents  MAGE environments provides various services to MSF modules  Resource information, System environments,..

17. Sep 21, 2006 17 HG2C Portal

18. Sep 21, 2006 18 Meta Services Framework Demo (1)

19. Sep 21, 2006 19 Meta Services Framework Demo (2)

20. Sep 21, 2006 20 HG2C Pubmap

21. Sep 21, 2006 21  HG2C Homepage  http://www.hg2c.org http://www.hg2c.org  Open sources of MSF & MAGE  http:// www.sourceforge.net/projects/mage4ubi http://www.sourceforge.net/projects/mage4ubi  http:// www.sourceforge.net/projects/msf http://www.sourceforge.net/projects/msf HG2C References

22. Sep 21, 2006 22 Q & A