We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byJamya Pinnell
Modified over 2 years ago
© 2006 Open Grid Forum OGF Interop Project update: IGE, EMI and FutureGrid GIN-CG Steve Crouch (IGE) Ismael Carrion (IGE), Chris Koeritz (GEN/FG), Shahbaz Memon (EMI), Emmanouil Paisios (IGE), Marcin Pospieszny (IGE), Morris Riedel (EMI), Karolina Sarnowska-Upton (GEN/FG)
© 2006 Open Grid Forum Standards, Data, Security & Application Support Standards: HPC Basic Profile v1.0 OGSA BES (Basic Execution Service) v1.0 JSDL (Job Submission Description Language) v1.0 HPC Profile Application Extension v1.0 HPC File Staging Profile Data protocols used is ftp Only suitable common protocol across all platforms! Security: Direct middleware -> certificate CA trust (just import CAs) Application: Physics: Plasma Charge Minimization Total system energy minimization of point charges around the surface of a sphere
© 2006 Open Grid Forum History Initially shown at OGF27, Banff, Canada, Oct 09 SuperComputing, Nov 09, Nov 2010 ETSI Plugtests, FZJ, UK AHM, Dec 09 OGF28, OGF30, OGE34 Demonstrators: David Wallom, Peter Stefan, Morris Riedel, Shahbaz Memon, Steve Crouch Video available at http://www.omii.ac.uk/wiki/Videos http://www.omii.ac.uk/wiki/Videos
© 2006 Open Grid Forum Client/Service Compatibility ServiceBES++ ClientGridSAM Client InvocationApplicationInvocationApplication ARC BES++ -- Genesis II (work.sol.) GridSAM (+GridWay) -- MS Com.Clus. RENKEI (work.sol.) QosCosGrid (work.sol.) UNICORE VENUS-BES (work.sol.)
© 2006 Open Grid Forum Endpoint Preparation 1.Application provision on resource 2.Data resource (file staging) 3.Security 4.Practical JSDL 5.Iterative testing
© 2006 Open Grid Forum How Interop Demo Fits Together Now… BES++ Client BES++ Client HPC-BP 2 HPC-BP 3 HPC-BP 4 FTP Client Job Service Data Service Minem Application minem- interop.pl Minem Security Service 1. Create Minem input files 1 2. Generate JSDLs from template 2 7. Select best result 7 8. Generate/upload image to web server 8 5. Monitor jobs until completion 5 HPC-BP 1 FTP 4. Submit JSDLs across middlewares 4 6. Download output files 6 3. Upload input files 3 Minem GridSAM Client GridSAM Client … FTP
© 2006 Open Grid Forum Goal: IGE & EMI interop IGE/EMI MoU: interop demo for SC2012 Our OGF34 plan for OGF36/SC2012: IGE v3.0 (GridSAM/GridWay) EMI latest (UNICORE/ARC) UMD – EGI packaging of the above! Genesis II
© 2006 Open Grid Forum Goal: IGE & EMI interop IGE/EMI MoU: interop demo for SC2012 Our OGF34 plan for OGF36/SC2012: IGE v3.0 (GridSAM/GridWay) - achieved EMI latest (UNICORE/ARC) - achieved, v2.0 Matterhorn release UMD – EGI packaging of the above! - not yet Genesis II - achieved
© 2006 Open Grid Forum Goal: move towards Prod. Grids Previously done with historic interop endpoints – nearly all testbed OGF 34 plan for OGF36/SC2012: IGE v3.0 EMI Genesis II
© 2006 Open Grid Forum Goal: move towards Prod. Grids Previously done with historic interop endpoints – nearly all testbed OGF 34 plan for OGF36/SC2012: IGE v3.0 - IGE testbed only -> LRZ EMI - 2 instances on FutureGrid/XSEDE Non-EMI security-light provided by Virginia v2.0 Matterhorn, full authorisation provided by FZJ Genesis II - FutureGrid/XSEDE
© 2006 Open Grid Forum GridWay Metascheduler The GridWay Metascheduler enables large-scale, reliable and efficient sharing of computing resources It supports different LRM systems PBS, SGE, LSF, Condor… Within a single organization or across several administrative domains GridWay uses different information, execution and transfer drivers that enable the interoperation with most existing Grid middlewares GridWay provides two local interfaces: a command line interface (CLI) and a DRMAA-based interface. It provides a BES remote interface as well Supported Standards: BES (both client and server) JSDL (POSIX and HPCProfile extensions) DRMAA, DRMAA2 (C, Java, Perl, Ruby and Python bindings)
© 2006 Open Grid Forum GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GridSAM DRMAA CLI BES Cluster GRAM5 GridFtp PBS Cluster SGE 8 cores FORK 1 core FORK 4 cores FORK Cluster PBS Job Submission Interfaces GridWay Metascheduler Computational and Storage Resources UCM TUDO LRZ NIKHEFEPCCUTCNPSNC
© 2006 Open Grid Forum GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GridSAM DRMAA CLI BES Cluster GRAM5 GridFtp PBS Cluster SGE 8 cores FORK 1 core FORK 4 cores FORK Cluster PBS Job Submission Interfaces GridWay Metascheduler Computational and Storage Resources UCM TUDO LRZ NIKHEFEPCCUTCNPSNC Tightly coupled: DRMAA direct to GridWay instance
© 2006 Open Grid Forum Some Results Security Generally, 2/3 of effort Different implementation/deployment, different constraints (e.g. our two UNICORE deployments) Used a duff certificate – no UNICORE! Cert breaking an RFC? Clients need to be flexible Within a grouping of jobs across different endpoints Support different endpoint security profiles? BES/JSDL Generally more convergent WS Addressing Deployment e.g. WorkingDirectory Standards compliance is obviously important but largely not enough – obvious, but extent is not obvious
© 2006 Open Grid Forum In the Future… UNICORE Genesis ARC FTP GridFTP FTP … EMI -> UMD IGE -> UMD GT GridSAM Client GridSAM Client Taverna/Tri ana GridWay GridSAM BES++ Client BES++ Client
© 2006 Open Grid Forum Future Work New Redmine GIN-CG subproject to host stuff Endpoint Wiki (like we did in OGF27/OGF30) Status e.g. last interop check between implementations HPC-BP Implementation Interoperability - Experiences Document Structural draft uploaded to Redmine GIN-CG interop demo subproject Areas: OGSA-BES/JSDL (inc. headers), Security, Data transfer Bring in other endpoint types and prod. grids ARC endpoint (NorduGrid?) IGE -> LRZ Others? CHAIN interoperability demo? SC2012 prep? Video for backup
© 2006 Open Grid Forum Future Work Another application? Data protocols other than FTP! GridFTP, SFTP, FTPS, …? Go beyond lowest common denominator of protocol Beyond submission and monitoring Common problem of duration of grid demo Problem of resource availability -> very long demos! If jobs submitted and monitored, already demod interop Timeout after x demo seconds, work with what weve got Investigate extensibility with Triana/Taverna2 Integrate GridSAM client as a plugin into Triana/Taverna Basic job submission/monitoring in workflow Replicate interop Minem application demo in workflow
GridSAM Overview Grid Job S ubmission A nd M onitoring Service What is GridSAM? Funded by the OMII Managed Programme (Started in Sept, 04) Client Perspective.
Web: Running applications on interoperable Grid infrastructures, focusing on OMII-UK supported software - HPC-BP.
EGI Technical Forum Amsterdam, 16 September 2010 Sylvain Reynaud.
London e-Science Centre GridSAM A Standards Based Approach to Job Submission A. Stephen M C Gough Imperial College London A Standards Based Approach to.
© 2006 Open Grid Forum OGF HPC-Basic Profile Application Interoperability Demonstration GIN-CG.
London e-Science Centre GridSAM Job Submission and Monitoring Web Service William Lee, Stephen McGough.
COMP3019 Coursework: Introduction to GridSAM Steve Crouch School of Electronics and Computer Science.
Distributed Systems Architecture Research Group Universidad Complutense de Madrid EGEE UF4/OGF25 Catania, Italy March 2 nd, 2009 State and Future Plans.
© 2008 Open Grid Forum Independent Software Vendor (ISV) Remote Computing Primer Steven Newhouse.
JRA1/Job Submission and Monitoring Moreno Marzolla on behalf of JRA1/Job Submission Task INFN Sezione di Padova,
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks, An Overview of the GridWay Metascheduler.
1 Overview of the Application Hosting Environment Stefan Zasada University College London.
Current status of grids: the need for standards Mike Mineter TOE-NeSC, Edinburgh.
Web: Towards Grid Interoperability Richard Boardman, Stephen Crouch, Hugo Mills, Steven Newhouse, Juri Papay and.
Connecting OurGrid & GridSAM A Short Overview. Content Goals OurGrid: architecture overview OurGrid: short overview GridSAM: short overview GridSAM: example.
3-2.1 Topics Grid Computing Meta-schedulers –Condor-G –Gridway Distributed Resource Management Application (DRMAA) © 2010 B. Wilkinson/Clayton Ferner.
Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.
March 6 th, 2009 OGF 25 Unicore 6 and IPv6 readiness and IPv6 readiness
CSF4 Meta-Scheduler Name: Zhaohui Ding, Xiaohui Wei
Legacy code support for commercial production Grids G.Terstyanszky, T. Kiss, T. Delaitre, S. Winter School of Informatics, University.
Oct, 26 th, 2010 OGF 29, FVGA-WG: Firewall Virtualization for Grid Applications Firewall Virtualization for Grid Applications - Status update
The NGS Grid Portal David Meredith NGS + Grid Technology Group, e-Science Centre, Daresbury Laboratory, UK
Why do we need PGI? Shahbaz Memon Jülich Supercomputing Centre (JSC)
Web: OMII-UK LiveCD Demonstrations – Providing Access to Computational Resources for Researchers AHM 2009 Steve Crouch,
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
2 nd GADA Workshop / OTM 2005 Conferences Eduardo Huedo Rubén S. Montero Ignacio M. Llorente Advanced Computing Laboratory Center for.
© 2007 Open Grid Forum SAGA: Simple API for Grid Applications Steven Newhouse Application Standards Area Director.
The European KnowARC Project Péter Stefán, NIIF/HUNGARNET/KnowARC TNC2009, June 2009, Malaga.
The HPC Basic Profile Steven Newhouse Microsoft Corporation.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI WS-PGRADE/gUSE Supporting e-Science communities in Europe Zoltan Farkas.
1 AHE Server Deployment and Hosting Applications Stefan Zasada University College London.
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Applications.
Grid Security: Authentication Most Grids rely on a Public Key Infrastructure system for issuing credentials. Users are issued long term public and private.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks S. Natarajan (CSU) C. Martín (UCM) J.L.
Grid Standardization from the NorduGrid/ARC perspective Balázs Kónya, Lund University, Sweden NorduGrid Technical Coordinator ETSI Grid Workshop on Standardization,
Web: OMII-UK: progressing from adopting open specifications to sustainable open development by fostering Asia-Pacific.
OGF Standards Activity Steven Newhouse. © myOGF Part of the Standards Council Area Directror for Applications Working Groups Co-chair: Basic Execution.
GGF10 DRMAA Working Group Hrabri Rajic Intel GGF10 Berlin, Germany March, 2004 GLOBALGRIDFORUM.ORG.
GRAM: Software Provider Forum Stuart Martin Computational Institute, University of Chicago & Argonne National Lab TeraGrid 2007 Madison, WI.
GRAM5 - A sustainable, scalable, reliable GRAM service Stuart Martin - UC/ANL.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
SHIWA Simulation Platform (SSP) Gabor Terstyanszky, University of Westminster EGI Community Forum Munnich March 2012 SHIWA is supported by the FP7.
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Solving inter grid interoperability issues: example of WISDOM and molecular dynamics Jean Salzemann, LPC Clermont-Ferrand, CNRS/IN2P3 Morris Riedel, Jülich.
Condor Birdbath Web Service interface to Condor Clovis Chapman, Charaka Goonatilake, Wolfgang Emmerich, Matthew Farrellee, Todd Tannenbaum, Miron Livny,
© 2008 Open Grid Forum Production Grid Infrastructure (PGI) 101 Morris Riedel, Balazs Konya, Moreno Marzolla OGF PGI Working Group Co-Chairs.
Globus 4 Guy Warner NeSC Training.
CSF4, SGE and Gfarm Integration Zhaohui Ding Jilin University.
© 2017 SlidePlayer.com Inc. All rights reserved.