Www.see-grid-sci.eu SEE-GRID-SCI The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no.

Slides:



Advertisements
Similar presentations
FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.
Advertisements

WMO Competency Standards: Development and Implementation Status
CS487 Software Engineering Omar Aldawud
SEE-GRID-SCI Hands-On Session: Workload Management System (WMS) Installation and Configuration Dusan Vudragovic Institute of Physics.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
WP6: Grid Authorization Service Review meeting in Berlin, March 8 th 2004 Marcin Adamski Michał Chmielewski Sergiusz Fonrobert Jarek Nabrzyski Tomasz Nowocień.
EGI-Engage Recent Experiences in Operational Security: Incident prevention and incident handling in the EGI and WLCG infrastructure.
QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.
INFSO-RI Enabling Grids for E-sciencE FloodGrid application Ladislav Hluchy, Viet D. Tran Institute of Informatics, SAS Slovakia.
SEE-GRID-SCI Applications of the Meteorology VO in the frame of SEE-GRID-SCI project The SEE-GRID-SCI initiative is co-funded by the.
SEE-GRID-2 The SEE-GRID-2 initiative is co-funded by the European Commission under the FP6 Research Infrastructures contract no
Porting applications to EU-IndiaGrid: EGEE Marco Verlato EU-IndiaGrid Workshop April 2007 Bangalore, India.
SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid.
User requirements for and concerns about a European e-Infrastructure Steven Newhouse, Director.
EMI INFSO-RI SA2 - Quality Assurance Alberto Aimar (CERN) SA2 Leader EMI First EC Review 22 June 2011, Brussels.
SEE-GRID-2 The SEE-GRID-2 initiative is co-funded by the European Commission under the FP6 Research Infrastructures contract no
INFSO-RI Enabling Grids for E-sciencE SA1: Cookbook (DSA1.7) Ian Bird CERN 18 January 2006.
Relationships July 9, Producers and Consumers SERI - Relationships Session 1.
SEE-GRID-SCI Regional Grid Infrastructure: Resource for e-Science Regional eInfrastructure development and results IT’10, Zabljak,
Through the development of advanced middleware, Grid computing has evolved to a mature technology in which scientists and researchers can leverage to gain.
Dr. Harald KornmayerISCG 2007 – 28 th of March 2007 g-Eclipse A framework for Grid users, operators and developers Harald Kornmayer (Forschungszentrum.
1 Women Entrepreneurs in Rural Tourism Evaluation Indicators Bristol, November 2010 RG EVANS ASSOCIATES November 2010.
The Grid System Design Liu Xiangrui Beijing Institute of Technology.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Gridification practices in gLite 3.x Vladimir Dimitrov IPP-BAS “gLite middleware.
Responsibilities of ROC and CIC in EGEE infrastructure A.Kryukov, SINP MSU, CIC Manager Yu.Lazin, IHEP, ROC Manager
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Code Applications Tamas Kiss Centre for Parallel.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks NA3 Resources Robin McConnell.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Next steps with EGEE EGEE training community.
June 24-25, 2008 Regional Grid Training, University of Belgrade, Serbia Introduction to gLite gLite Basic Services Antun Balaž SCL, Institute of Physics.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Next steps with EGEE Gergely Sipos
National Center for Supercomputing Applications Barbara S. Minsker, Ph.D. Associate Professor National Center for Supercomputing Applications and Department.
Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
SEE-GRID-SCI The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Application Development on Grids Gergely Sipos MTA SZTAKI Hungarian.
Getting started DIRAC Project. Outline  DIRAC information system  Documentation sources  DIRAC users and groups  Registration with DIRAC  Getting.
EGEE-II INFSO-RI Enabling Grids for E-sciencE The GILDA training infrastructure.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
Grid Execution Management for Legacy Code Applications Grid Enabling Legacy Applications.
Next Steps: becoming users of the NGS Mike Mineter
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
Trusted Virtual Machine Images a step towards Cloud Computing for HEP? Tony Cass on behalf of the HEPiX Virtualisation Working Group October 19 th 2010.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The EGEE User Support Infrastructure Torsten.
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
Federating PL-Grid Computational Resources with the Atmosphere Cloud Platform Piotr Nowakowski, Marek Kasztelnik, Tomasz Bartyński, Tomasz Gubała, Daniel.
Report from the WLCG Operations and Tools TEG Maria Girone / CERN & Jeff Templon / NIKHEF WLCG Workshop, 19 th May 2012.
The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no Workflow repository, user.
GRID ANATOMY Advanced Computing Concepts – Dr. Emmanuel Pilli.
SEE-GRID-SCI REFS application: NOA The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures.
INFSO-RI SA2 ETICS2 first Review Valerio Venturi INFN Bruxelles, 3 April 2009 Infrastructure Support.
SEE-GRID-SCI Meteo-VO: Overview The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures.
INFSO-RI Enabling Grids for E-sciencE gLite Test and Certification Effort Nick Thackray CERN.
SEE-GRID-SCI NA4 Progress PSC06 Istanbul, 7-8 December 2009 The SEE-GRID-SCI initiative is co-funded by the European Commission under.
Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Introduction to Grid Application Development Vangelis Floros Application.
V7 Foundation Series Vignette Education Services.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Overview of gLite, the EGEE middleware Mike Mineter Training Outreach Education National.
ENEA GRID & JPNM WEB PORTAL to create a collaborative development environment Dr. Simonetta Pagnutti JPNM – SP4 Meeting Edinburgh – June 3rd, 2013 Italian.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Bob Jones EGEE Technical Director
Accessing the VI-SEEM infrastructure
Regional Operations Centres Core infrastructure Centres
Ian Bird GDB Meeting CERN 9 September 2003
Introduction to Grid Technology
EGI Webinar - Introduction -
SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION
Course: Module: Lesson # & Name Instructional Material 1 of 32 Lesson Delivery Mode: Lesson Duration: Document Name: 1. Professional Diploma in ERP Systems.
How To Integrate an Application on Grid
Presentation transcript:

SEE-GRID-SCI The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no Grid - korisnicki pristup i razvoj aplikacija Branko Marovic RCUB - UoB Meteo VO Training, Belgrade 24. June 2008

Application Gridification2/26 SEE-GRID-2 Applications Adaptation & Support Application lifecycle defined in “Gridification Guidelines” document. SG Gridification Guide (wiki pages) structured and contributed by developers. Policy: Applications initially supported at the country level. Application Support Group (ASG) created for gridification support. Application Review Committee(ARC) created to assess existing and new candidate applications. Application selection criteria defined by ARC. MoU Between Applications and SEEGRID-2.

Application Gridification3/26 SEE-GRID-2 Strategy for New Applications There is “Gridification Guideline” document to provide initial help to developers. (application life- cycle, policy and methodology to run and support an application on Grid) There is “SG Gridification Guide” on SEE-GRID-2 Wiki to provide comprehensive help to developers (porting an existing application to the Grid) ARC (Application Review Committee) recommendations push applications towards collaborations with other applications and projects. Applications are required to provide update on their established user community and collaborations within their periodic Application Progress Report.

Application Gridification4/26

Application Gridification5/26 SEE-GRID2 Applications

Application Gridification6/26 Support the deployed applications Application support group (ASG) created to provide fast and continuous gridification support to developers.  Experienced developers & admins  ASG team members assigned for each application.  National level application support  SEE-GRID - global level application support SG Gridification Wiki online. SEE-GRID helpdesk available and used for problems.  Useful Helpdesk Tickets for Developers.  SEE-GRID Helpdesk “Applications Support” knowledge base.  GOODs responsible to dump useful tickets of the week. Work in close collaboration with WP5 (training) and WP3 (software requirements, maintenance of performance)

Application Gridification7/26 Application Lifecycle

Application Gridification8/26 SEE-GRID-2 Gridification Guides An initial guide for the developer including the application life-cycle, VO and site policies, gridification methodology:  Gridification Guideline  grid.eu/content/modules/downloads/SEEGRID2- WP4-TR-008-Gridification_Guideline-n doc Detailed wiki pages with illustrations for gridification issues:  SG Gridification Guide  see.org/index.php/SG_Gridification_Guide

Application Gridification9/26 SEE-GRID Gridification Guide Relevant topics for application developers identified trough online questionnaire system Some investigation areas identified as well – candidates for future GG topics Gridification guide will provide information on these topics Wiki as collaboration medium

Application Gridification10/26 SEE-GRID Gridification Guide 1 Job Management 1.1 Running Jobs (T1) 1.2 Software Installation Management (T3) 2 File Management and Access 2.1 Data Management (T2) 2.2 Data Access (T4) 2.3 Logical Files (T6) 2.4 Advanced Replica Management (T2, T6 related investigation area) 3 Inter-Job Communication 3.1 Workflows (T7) 3.2 MPI (T9) 3.3 Interactive & Pilot Jobs (T1 related investigation area) 3.4 Java networking 4 Information Services 4.1 Application-Level Event Logging & Performance (T5) Applications and Systems monitoring tools Benchmarking 4.2 Metadata Catalogues 5 Development Environments and Portals 5.1 Web Authentication and Authorization (T8) 5.2 P-GRADE Portal (T1-T9 related investigation area) 5.3 P-GRADE (T1-T9 related investigation area)

Application Gridification11/26

Application Gridification12/26 VO Resource Allocation Policies Seegrid VO is used for all applications:  Utilization of groups and roles: /seegrid/COUNTRY_CODE/SHORT_APP_NAME/Role=production /seegrid/COUNTRY_CODE/SHORT_APP_NAME/Role=software For gridification testing phase,  no role or group is used, all sites support all applications. For deployed applications,  sites are encouraged to support as many applications as possible. MoU template between SEE-GRID-2 Project and the Application created.

Application Gridification13/26 SEE-GRID-SCI

Application Gridification14/26 Meteorology VO Members  National Observatory of Athens (coordinator)  South Environment and Weather Agency of Serbia  Hydrometeorological Institute of Montenegro  RBI of Zagreb  University of Zagreb  Federal Hydro-Meteorological Institute of Bosnia and Herzegovina  Republic Hydrometeorogical Institute, Banjaluka Two sets of applications for initial deployment:  Regional scale Multi-model, Multi-analysis ensemble forecasting system  Study of the interaction of airflow with complex terrain

Application Gridification15/26 Developer Resources Grid environment is constantly evolving, but  Useful features persist  New are constantly being added  Bugs are being fixed  Gained knowledge remains relevant, must be updated  Applications can be easily migrated to new/updated APIs gLite User Guide  WMS user guides  SEE-GRID Gridification Guide  SEEGRID Wiki  gLite documentation 

Application Gridification16/26 Grids: a foundation for e- Research Enabling a whole-system approach Collaborative research / engineering / public service … sensor nets Shared data archives computers software colleagues instruments Grid Diagram derived from Ian Foster’s slide

Application Gridification17/26 Načini upotrebe grida Komandna linija, JDL, input/output sandbox i/ili ručna manipulacija podacima Primena skriptova u generisanu poslova, obradi i manipulaciji podacima Korišćenje postojećih Grid portala Primena grid API-ja i servisa iz koda koji se izvršava na gridu Pravljenje aplikativno specifičnih pristupnih tačaka-servisa

Application Gridification18/26 Complexities of grid applications 1. Simple jobs – submitted to WMS to run in batch mode 2. Job invokes grid services To read & write files on SE Monitoring For outbound connectivity (interactive jobs) To manage metadata … 3. Complex jobs An environment controls multiple jobs on users’ behalf High-level services Portals with workflow Software written for the VO (or by the user) …

Application Gridification19/26 Invocation of applications From the UI  Command Line Interfaces / Scripts  APIs  Higher level tools From desktop Windows applications  Use Grids without awareness of them!  But gLite not (yet) fully supporting Windows (more info: From portals  For recurring tasks: “core grid services” as well as application layer  Accessible from any browser  Tailored to applications  Different portal solutions, and wide range of capabilities. Second part of this course: P-GRADE Portal

Application Gridification20/26 Characteristics of VOs What is being shared?  resources of storage and/or compute cycles  software and/or data Distinct groups of developers and of users?  Some VOs have distinct groups of developers and users…  Biomedical applications used by clinicians,….  …. Some don’t  Physics application developers who share data but write own analyses  Effect: need to  hide complexity from the 1 st type of VOs  expose functionality to 2 nd type of Vos  many security issues

Application Gridification21/26 Challenges Research software is often  Created for one user: the developer  Familiarity makes it useable  Short-term goals: Used until papers are written and then discarded Grid applications are often used  by a VO  Without support from developer  In new contexts and workflows Need expertise in: software engineering application domain grid computing Grid application developers are  In a research environment  Yet their s/w must have:  Stability  Documentation  Usability  Extendability i.e. Production quality

Application Gridification22/26 Consequences Team work! Engaged in world-wide initiatives – reuse, don’t make your own! Cross disciplines for solutions. From research to production software: ~5 times the effort.  “80% of the time for last 10% of the functionality & reliability” Standardisation is key  For re-use, for dynamic configuration of services,..  Both for middleware and domain specific Need to follow a deliberate development process  Waterfall? Rapid prototyping?  Requirements engineering, design, implementation, validation, deployment  Engaged with the user community

Application Gridification23/26 Basic tasks while Porting applications to the Grid 1. Developing a non-grid application (or inheriting and updating an ancient one); 2. Go/no-Go decision about gridification Is it suitable for the Grid environment? “Cost/profit” analysis / feasibility study Typical Questions Groups:  Current structure of the application  Dependencies of the application  Available resources (manpower, knowledge, etc.)  Requirements for the gridified application  Expected impact of gridification  Requirements for the grid infrastructure More info: Application Description Template

Application Gridification24/26 Basic tasks while Porting applications to the Grid (contd.) 3. Grid environment access Requesting Certificates / VO membership Accessing Grid environment Appropriate VO UI machine account for command line Portal GUI account; 4. Executing, Testing and Debugging the application; Testing the non-grid application (debugging in Grid environment is a hard task), creating use cases for single (non-grid) runs; 5. Constructing the job suite – JDL (Job Description Language) files, executables, auxiliary scripts and input/output data files; 6. Submitting the job to the Grid as small-scale pilot application;

Application Gridification25/26 Basic tasks while Porting applications to the Grid (contd.) 7. Executing, Testing and Debugging the pilot application; 8. IF something goes wrong THEN GOTO 4; 9. IF everything seems to work THEN increase the scale of the application (increase problem size, amount of used resources); 10. Optimizing the grid application;