Grid Workflow within Triana Ian Wang Cardiff University.

Slides:

Advertisements

Similar presentations

LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (

Advertisements

Open Grid Computing Environments Marlon Pierce (IU) & Gopi Kandaswamy (RENCI)

Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.

Generic MPI Job Submission by the P-GRADE Grid Portal Zoltán Farkas MTA SZTAKI.

Under the Hood of a Workflow Manager Matthew Shields, BiodiversityWorld GRID workshop, NeSC, 30 June - 1 July T r a ai n.

Legacy code support for commercial production Grids G.Terstyanszky, T. Kiss, T. Delaitre, S. Winter School of Informatics, University.

Programming Workflow with Triana Services Matthew Shields, SC4DEVO Workshop, July 2004.

C. Grimme, A. Papaspyrou Scheduling in C3-Grid AstroGrid-D Workshop Project: C3-Grid Collaborative Climate Community Data and Processing Grid Scheduling.

GridLab Enabling Applications on the Grid Jarek Nabrzyski et al. Poznań Supercomputing and Networking.

Universität Dortmund Robotics Research Institute Information Technology Section Grid Metaschedulers An Overview and Up-to-date Solutions Christian.

Portals Team GridSphere and the GridLab Project Jason Novotny Michael Russell Oliver Wehrens Albert.

SC 2003 Demo, NCSA booth GridLab Project Funded by the EU (5+ M€), January 2002 – December 2004 Application and Testbed oriented Cactus Code, Triana Workflow,

W4: Grid Portals Building Web-enabled End-User Environments for accessing Grid Services By Michael Paul Russell Dept Computer Science.

GridLab & Cactus Joni Kivi Maarit Lintunen. GridLab  A project funded by the European Commission  The project was started in January 2002  Software.

Kelly Davis GAT: Grid Application Toolkit Kelly Davis AEI-MPG.

Globus Computing Infrustructure Software Globus Toolkit 11-2.

Kelly Davis Architecture of GAT Kelly Davis AEI-MPG.

Programming Scientific and Distributed Workflow with Triana Services Matthew Shields, GGF10 Workflow Workshop, 9th March.

UCoMS: Grid Computing Framework for Petroleum Engineering.

W w w. h p c - e u r o p a. o r g The HPC-Europa project and GridSphere Dawid Szejnfeld Poznan Supercomputing.

EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space Cracow Grid Workshop’10 Kraków, October 11-13,

SUN HPC Consortium, Heidelberg 2004 Grid(Lab) Resource Management System (GRMS) and GridLab Services Krzysztof Kurowski Poznan Supercomputing and Networking.

Triana Dr Matthew Shields, School of Computer Science, Cardiff University A Problem Solving Environment and Grid Workflow Tool.

General Intro to GridLab Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center.

A Web 2.0 Portal for Teragrid Fugang Wang Gregor von Laszewski May 2009.

Holding slide prior to starting show. A Grid-based Problem Solving Environment for GECEM Maria Lin and David Walker Cardiff University Yu Chen and Jason.

GridLab A Grid Application Toolkit and Testbed IST Jarek Nabrzyski GridLab Project Coordinator Poznań.

5th GridLab Review, 8 April 2005, Amsterdam WP12: Access for mobile users What's new in mobile user support Piotr Grabowski, Bartek Lewandowski

CoG Kit Overview Gregor von Laszewski Keith Jackson.

Flexibility and user-friendliness of grid portals: the PROGRESS approach Michal Kosiedowski

WP9 Resource Management Current status and plans for future Juliusz Pukacki Krzysztof Kurowski Poznan Supercomputing.

Triana: Service-Oriented Examples Ian Taylor Cardiff University, and the Center for Computation and Technology LSU.

Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.

GridLab: A Grid Application Toolkit and Testbed Jarosław Nabrzyski GridLab Project Manager Poznań Supercomputing and Networking Center, Poland

GridLab Applications Gabrielle Allen, Zhou Lei, Archit Kulshrestha Center for Computation & Technology, LSU.

GridLab: A Grid Application Toolkit and Testbed

Workflow and Triana Services Matthew Shields, e-Science Workflow Services, 3-5 December T r a ai n.

London e-Science Centre GridSAM Job Submission and Monitoring Web Service William Lee, Stephen McGough.

Resource Brokering in the PROGRESS Project Juliusz Pukacki Grid Resource Management Workshop, October 2003.

PROGRESS: ICCS'2003 GRID SERVICE PROVIDER: How to improve flexibility of grid user interfaces? Michał Kosiedowski.

General Intro to GridLab Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center.

April 1st, The ASC- GridLab Portal Edward Seidel, Michael Russell, Gabrielle Allen, and the rest of the team Max Plank Institut für Gravitationsphysik.

Grid Architecture William E. Johnston Lawrence Berkeley National Lab and NASA Ames Research Center (These slides are available at grid.lbl.gov/~wej/Grids)

Kelly Davis and Tom Goodale Architecture of GAT Kelly Davis and Tom Goodale and

1 Grid Portal for VN-Grid Cu Nguyen Phuong Ha. 2 Outline Some words about portals in principle Overview of OGCE GridPortlets.

Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,

Developing Applications on Today’s Grids Tom Goodale Max Planck Institute for Gravitational Physics

Ian Taylor, Cardiff Work-Flow Application Toolkit Eger Meeting Ian Taylor & Ian Wang Cardiff University, UK.

What is Triana?. GAPGAP Triana Distributed Work-flow Network Action Commands Workflow, e.g. BPEL4WS Triana Engine Triana Controlling Service (TCS) Triana.

OGSA V1 Document (p19). Terminology What is a job? Job made of multiple tasks –Task: Single unit of work –Job: Multiple tasks GT2: DUROC: Link tasks with.

1 P-GRADE Portal: a workflow-oriented generic application development portal Peter Kacsuk MTA SZTAKI, Hungary Univ. of Westminster, UK.

OGSA-UK: Putting the users first Steven Newhouse OMII Deputy Director.

WHIP - Workflow Hosted in Portals Kurt Mueller and Andrew Harrison School of Computer Science, Cardiff And Ian Taylor School of Computer Science, Cardiff.

GridLab WP-2 Cactus GAT (CGAT) Ed Seidel, AEI & LSU Co-chair, GGF Apps RG, Gridstart Apps TWG Gabrielle Allen, Robert Engel, Tom Goodale, *Thomas Radke.

Distributed Computing With Triana A Short Course Matthew Shields, Ian Taylor & Ian Wang.

GridLab Resource Management System (GRMS) Jarek Nabrzyski GridLab Project Coordinator Poznań Supercomputing and.

Hiding Grid Complexity Behind SSH Session Server framework Tomasz Kuczyński (1,2) 1) Poznan Supercomputing and Networking Center.

GridLab Resource Management System (GRMS) Jarek Nabrzyski GridLab Project Coordinator Poznań Supercomputing and.

GridChem Architecture Overview Rion Dooley. Presentation Outline Computational Chemistry Grid (CCG) Current Architectural Overview CCG Future Architectural.

The Gateway Computational Web Portal Marlon Pierce Indiana University March 15, 2002.

EGI Technical Forum Amsterdam, 16 September 2010 Sylvain Reynaud.

© Geodise Project, University of Southampton, Workflow Support for Advanced Grid-Enabled Computing Fenglian Xu *, M.

RENKEI:UGI Takashi Sasaki. Project history The RENKEI project led by Prof. Ken Miura of NII is funded by MEXT during JFY The goal of the project.

ACGT Architecture and Grid Infrastructure Juliusz Pukacki ‏ EGEE Conference Budapest, 4 October 2007.

Data Bridge Solving diverse data access in scientific applications

Grid Portal Services IeSE (the Integrated e-Science Environment)

Standard Portlet Architecture Supports Reusable Components

Gordon Erlebacher Florida State University

Introduction to the SHIWA Simulation Platform EGI User Forum,

portal broker PingER Replica Mgr RFT GridFTP GateKeeper Job Mgr Akenti

Presentation transcript:

Grid Workflow within Triana Ian Wang Cardiff University

Ian Wang, Cardiff University GridLab E.U. Funded 9 Institutions German (x2), Poland, Hungry, Czech, Holland, Greece, Italy, U.K. 13 Work-Packages GAT Resource Broker (GRMS) Data Management Portal (GridSphere) Monitoring (Mercury) Adaptive Components Information Services (iGrid) …

Ian Wang, Cardiff University GridLab GAT Application-level API Abstract ‘Common’ Capabilities required by Grid Applications Resource Discovery Job Submission File Transfer Capabilities extracted through a number of Grid Application Scenarios. Makes life as a Grid programmer easier … Motivation for the GGF SAGA research group (Simple API for Grid Applications)

Ian Wang, Cardiff University GAT Architecture GRMSGRAM Local File Service GridFTP HTTP Local Resource Broker File GridLab GAT API Application (e.g. Triana) Adaptors Services GRMS iGridMercury File Transfer GRAM Grid FTP HTTP

Ian Wang, Cardiff University Using the GAT GATContext context = new GATContext(); Preferences prefs = new Preferences(); prefs.put("ResourceBroker.adaptor.name", "grms"); URI exe = new URI("file:////bin/hostname"); File outFile = GAT.createFile(context, prefs, new URI("file:///out") SoftwareDescription sd = new SoftwareDescription(); sd.setLocation(exe); sd.setStdout(outFile); ResourceDescription rd = new HardwareResourceDescription(); rd.put("hostname", "bouscat.cs.cf.ac.uk"); JobDescription jd = new JobDescription(sd, rd); ResourceBroker broker = GAT.createResourceBroker(context, prefs); Job job = broker.submitJob(jd);

Ian Wang, Cardiff University Visual GAT GAT Primitives represented as Workflow Objects Job Tasks File Tasks Grid Data Flow implicit in Workflow Connections File Task -> File Task = Data Transfer File Task -> Job Task = Pre-Stage File File Task -> Local Task = File Read Grid Operations handled by GAT Adaptors Grid File -> Grid File = GridFTP HTTP File -> Local Task = HTTP Grid File -> GRMS Job = GRMS Pre-Stage

Ian Wang, Cardiff University Grid Workflow in Triana Local Java Unit File (Local/Remote) Temporary File Job (Local/Remote) File WritePre-StagePost-StageFile Read

Ian Wang, Cardiff University Visual GAT Alternative Grid Operations Expressed Explicitly GridFTP Tasks File Write Tasks File Read Tasks Not Intuitive Not Future Proof

Ian Wang, Cardiff University File Input (HTTP) GridLab GAT File CPI GridFTP Adaptor HTTP Adaptor Data Movement Resource Broker CPI Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor www. gridlab. org Triana WaveToy.par

Ian Wang, Cardiff University GridLab GAT File CPI GridFTP Adaptor HTTP Adaptor Data Movement Resource Broker CPI Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor www. gridlab. org Testbed Triana File Staging (GridFTP) WaveToy.par

Ian Wang, Cardiff University Mercury GridLab GAT File CPI GridFTP Adaptor HTTP Adaptor Data Movement Resource Broker CPI Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor www. gridlab. org Testbed GRMS Data Mvment Testbed Triana Delphoi GAS iGrid Job Submission 1 WaveToy.par Cactus

Ian Wang, Cardiff University GridLab GAT File CPI GridFTP Adaptor HTTP Adaptor Data Movement Resource Broker CPI Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor www. gridlab. org Testbed GRMS Data Mvment Portal Triana GAS Job Submission 2 Testbed Cactus Instant Mess MercuryDelphoi iGrid

Ian Wang, Cardiff University GridLab GAT File CPI GridFTP Adaptor HTTP Adaptor Data Movement Resource Broker CPI Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor www. gridlab. org Testbed Portal Triana Job Submission (GRAM) Testbed Cactus Instant Mess GRAM

Ian Wang, Cardiff University GridMonSteer User Side Grid Side WSPeer User Interface Web Service GridMonSteer (gms) cactus phi_xy.jpg phi_yz.jpg submit Application (e.g. Triana) run monitor invoke using SOAP Resource Broker (e.g. GRMS) gms cactus phi_xy.jpgphi_yz.jpg rundir

Ian Wang, Cardiff University Triana Grid Workflow Summary GridLab GAT Application-level API Abstracts Common Capabilities required by Grid Applications such as Triana Visual GAT GAT Primitives represented in Triana as Workflow Objects Grid Data Flow implicit in Workflow Connections GridMonSteer Generic Architecture for Monitoring and Steering Legacy Applications