We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byJamya Pinnell
Modified over 2 years ago
© 2006 Open Grid Forum OGF Interop Project update: IGE, EMI and FutureGrid GIN-CG Steve Crouch (IGE) Ismael Carrion (IGE), Chris Koeritz (GEN/FG), Shahbaz Memon (EMI), Emmanouil Paisios (IGE), Marcin Pospieszny (IGE), Morris Riedel (EMI), Karolina Sarnowska-Upton (GEN/FG)
© 2006 Open Grid Forum Standards, Data, Security & Application Support Standards: HPC Basic Profile v1.0 OGSA BES (Basic Execution Service) v1.0 JSDL (Job Submission Description Language) v1.0 HPC Profile Application Extension v1.0 HPC File Staging Profile Data protocols used is ftp Only suitable common protocol across all platforms! Security: Direct middleware -> certificate CA trust (just import CAs) Application: Physics: Plasma Charge Minimization Total system energy minimization of point charges around the surface of a sphere
© 2006 Open Grid Forum History Initially shown at OGF27, Banff, Canada, Oct 09 SuperComputing, Nov 09, Nov 2010 ETSI Plugtests, FZJ, UK AHM, Dec 09 OGF28, OGF30, OGE34 Demonstrators: David Wallom, Peter Stefan, Morris Riedel, Shahbaz Memon, Steve Crouch Video available at
© 2006 Open Grid Forum Client/Service Compatibility ServiceBES++ ClientGridSAM Client InvocationApplicationInvocationApplication ARC BES++ -- Genesis II (work.sol.) GridSAM (+GridWay) -- MS Com.Clus. RENKEI (work.sol.) QosCosGrid (work.sol.) UNICORE VENUS-BES (work.sol.)
© 2006 Open Grid Forum Endpoint Preparation 1.Application provision on resource 2.Data resource (file staging) 3.Security 4.Practical JSDL 5.Iterative testing
© 2006 Open Grid Forum How Interop Demo Fits Together Now… BES++ Client BES++ Client HPC-BP 2 HPC-BP 3 HPC-BP 4 FTP Client Job Service Data Service Minem Application minem- interop.pl Minem Security Service 1. Create Minem input files 1 2. Generate JSDLs from template 2 7. Select best result 7 8. Generate/upload image to web server 8 5. Monitor jobs until completion 5 HPC-BP 1 FTP 4. Submit JSDLs across middlewares 4 6. Download output files 6 3. Upload input files 3 Minem GridSAM Client GridSAM Client … FTP
© 2006 Open Grid Forum Goal: IGE & EMI interop IGE/EMI MoU: interop demo for SC2012 Our OGF34 plan for OGF36/SC2012: IGE v3.0 (GridSAM/GridWay) EMI latest (UNICORE/ARC) UMD – EGI packaging of the above! Genesis II
© 2006 Open Grid Forum Goal: IGE & EMI interop IGE/EMI MoU: interop demo for SC2012 Our OGF34 plan for OGF36/SC2012: IGE v3.0 (GridSAM/GridWay) - achieved EMI latest (UNICORE/ARC) - achieved, v2.0 Matterhorn release UMD – EGI packaging of the above! - not yet Genesis II - achieved
© 2006 Open Grid Forum Goal: move towards Prod. Grids Previously done with historic interop endpoints – nearly all testbed OGF 34 plan for OGF36/SC2012: IGE v3.0 EMI Genesis II
© 2006 Open Grid Forum Goal: move towards Prod. Grids Previously done with historic interop endpoints – nearly all testbed OGF 34 plan for OGF36/SC2012: IGE v3.0 - IGE testbed only -> LRZ EMI - 2 instances on FutureGrid/XSEDE Non-EMI security-light provided by Virginia v2.0 Matterhorn, full authorisation provided by FZJ Genesis II - FutureGrid/XSEDE
© 2006 Open Grid Forum GridWay Metascheduler The GridWay Metascheduler enables large-scale, reliable and efficient sharing of computing resources It supports different LRM systems PBS, SGE, LSF, Condor… Within a single organization or across several administrative domains GridWay uses different information, execution and transfer drivers that enable the interoperation with most existing Grid middlewares GridWay provides two local interfaces: a command line interface (CLI) and a DRMAA-based interface. It provides a BES remote interface as well Supported Standards: BES (both client and server) JSDL (POSIX and HPCProfile extensions) DRMAA, DRMAA2 (C, Java, Perl, Ruby and Python bindings)
© 2006 Open Grid Forum GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GridSAM DRMAA CLI BES Cluster GRAM5 GridFtp PBS Cluster SGE 8 cores FORK 1 core FORK 4 cores FORK Cluster PBS Job Submission Interfaces GridWay Metascheduler Computational and Storage Resources UCM TUDO LRZ NIKHEFEPCCUTCNPSNC
© 2006 Open Grid Forum GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GRAM5 GridFtp GridSAM DRMAA CLI BES Cluster GRAM5 GridFtp PBS Cluster SGE 8 cores FORK 1 core FORK 4 cores FORK Cluster PBS Job Submission Interfaces GridWay Metascheduler Computational and Storage Resources UCM TUDO LRZ NIKHEFEPCCUTCNPSNC Tightly coupled: DRMAA direct to GridWay instance
© 2006 Open Grid Forum Some Results Security Generally, 2/3 of effort Different implementation/deployment, different constraints (e.g. our two UNICORE deployments) Used a duff certificate – no UNICORE! Cert breaking an RFC? Clients need to be flexible Within a grouping of jobs across different endpoints Support different endpoint security profiles? BES/JSDL Generally more convergent WS Addressing Deployment e.g. WorkingDirectory Standards compliance is obviously important but largely not enough – obvious, but extent is not obvious
© 2006 Open Grid Forum In the Future… UNICORE Genesis ARC FTP GridFTP FTP … EMI -> UMD IGE -> UMD GT GridSAM Client GridSAM Client Taverna/Tri ana GridWay GridSAM BES++ Client BES++ Client
© 2006 Open Grid Forum Future Work New Redmine GIN-CG subproject to host stuff Endpoint Wiki (like we did in OGF27/OGF30) Status e.g. last interop check between implementations HPC-BP Implementation Interoperability - Experiences Document Structural draft uploaded to Redmine GIN-CG interop demo subproject Areas: OGSA-BES/JSDL (inc. headers), Security, Data transfer Bring in other endpoint types and prod. grids ARC endpoint (NorduGrid?) IGE -> LRZ Others? CHAIN interoperability demo? SC2012 prep? Video for backup
© 2006 Open Grid Forum Future Work Another application? Data protocols other than FTP! GridFTP, SFTP, FTPS, …? Go beyond lowest common denominator of protocol Beyond submission and monitoring Common problem of duration of grid demo Problem of resource availability -> very long demos! If jobs submitted and monitored, already demod interop Timeout after x demo seconds, work with what weve got Investigate extensibility with Triana/Taverna2 Integrate GridSAM client as a plugin into Triana/Taverna Basic job submission/monitoring in workflow Replicate interop Minem application demo in workflow
Grid Interoperability Through Standards Dr. Alistair Dunlop Project Manager.
1/22 Distributed Systems Architecture Research Group Universidad Complutense de Madrid Constantino Vázquez Eduardo Huedo Scaling DRMAA codes to the Grid:
GridSAM Overview Grid Job S ubmission A nd M onitoring Service What is GridSAM? Funded by the OMII Managed Programme (Started in Sept, 04) Client Perspective.
Adoption Status of the OGSA-BES interface Morris Riedel, Mohammad Shahbaz Memon … and many others from the UNICORE community Jülich.
RMS and Scheduling for Future Generation Grids Ramin Yahyapour University Dortmund Leader CoreGRID Institute on Resource Management and Scheduling CoreGRID.
2010 FutureGrid User Advisory Meeting Architecture Roadmap Long term vision 10:00-10:45, Monday, August 2, 2010 Pittsburgh, PA Gregor von Laszewski Representing.
Current status of grids: the need for standards Mike Mineter TOE-NeSC, Edinburgh.
Web: OMII-UK: progressing from adopting open specifications to sustainable open development by fostering Asia-Pacific.
Grid Standardization from the NorduGrid/ARC perspective Balázs Kónya, Lund University, Sweden NorduGrid Technical Coordinator ETSI Grid Workshop on Standardization,
Interoperability and Usability of Grid Infrastructures Alistair Dunlop Achim Streit University of SouthamptonForschungszentrum Jülich.
Grids and the Globus Community Dr. Jennifer M. Schopf Argonne National Lab
OGF35 Closing June 19, 2012 © 2006 Open Grid Forum Chaired by Cees de Laat.
Genesis II Open Source, OGSA Implementation Genesis II: Easy-to-use, Standards Based Grid Middleware Andrew Grimshaw Genesis II Team University of Virginia.
Overview of the FutureGrid Software Presented by Gregor von Laszewski Indiana University Workshop on High Performance Applications of Cloud and Grid Tools.
Experiences of the Grid… Gavin McCance University of Glasgow NeSC Meeting, 24 October 2001.
© 2007 Open Grid Forum OGSA and OGF Reference Model OGSA-WG General Session OGF19 OGSA-WG session #2 29 January, :30pm Chapel Hill, NC.
O.Wäldrich CoreGRID Summer School Bonn, July 25, 2006 Resource Orchestration in Grids Wolfgang Ziegler Department of Bioinformatics Fraunhofer.
Page 1 LAITS Laboratory for Advanced Information Technology and Standards Duh 7/10/03 The GMU Geospatial Grid Technology Development and Application Project.
Craig Lee OGF-24 Mark Reichardt Grid-Enabled Geospatial Systems.
CoreGrid Summer SchoolBudapest, Hungary, 3-7 September, Grid Interoperability Issues in Resource Management: Questions and Solutions Attila Kertész.
Nov 5-6, 2005 SEE-GRID Banjaluka Training Session Introduction to Linux Clusters and Grids Design and Basic Services of LCG Grid Middleware SEE-GRID Infrastructure.
Globus GRAM for Developers Stuart Martin, Peter Lane Argonne National Lab.
Promoting and Standardizing Grid Computing Defining the Grid: Open Grid Services Architecture Current and Future Generation Grid Technology Summer School.
Distributed Systems Architecture Research Group Universidad Complutense de Madrid EGEE UF4/OGF25 Catania, Italy March 2 nd, 2009 State and Future Plans.
© 2006 Open Grid Forum Glue-WG Sergio Andreozzi – INFN-CNAF 31 January 2007.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Experiences with using the EGEE grid infrastructure.
The PLANETS Testbed DPE, PLANETS and CASPAR 2nd Annual Conference Lisbon, 5–6 September, 2007 Max Kaiser, Austrian National Library
BES++ - Standards Adoption through Open Source Chris Smith Platform Computing.
Message Lab Monash e-Science and Grid Engineering Laboratory Bridging Grid Islands for Large Scale e-Science Blair Bethwaite, David Abramson, Ashley Buckle.
Oct, 26 th, 2010 OGF 29, FVGA-WG: Firewall Virtualization for Grid Applications Firewall Virtualization for Grid Applications - Status update
© 2016 SlidePlayer.com Inc. All rights reserved.