1. EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space Jacek Kitowski, Kazimierz Wiatr, Marian Bubak Michał Turała, Łukasz Dutka and Zofia Mosurska ACK CYFRONET AGH, Cracow, Poland CMS’09 Computer Methods and Systems Cracow, Nov. 26-27, 2009

2.  National Grid Initiative in Poland in a nutshell  Motivation – e-Science and Science 2.0  Rationales and Foundations  Status  PL-Grid Project – Current Results  SummaryOutline Integration Activity in the framework of European Grid Initiative

3. Demand from Community for e-Science Approach in spite of complexity: Computing, Storage, Infrastructure  E-Science: collaborative research supported by advanced distributed computations  Multi-disciplinary, Multi-Site and Multi-National  Building with and demanding advances in Computing/Computer Sciences  Goal: to enable better research in all disciplines  System-level Science: beyond individual phenomena, components interact and interrelate, experiments in silico  to generate, interpret and analyse rich data resources From experiments, observations and simulations Quality management, preservation and reliable evidence  to develop and explore models and simulations Computation and data at all scales Trustworthy, economic, timely and relevant results  to enable dynamic distributed collaboration Facilitating collaboration with information and resource sharing Security, trust, reliability, accountability, manageability and agility ACK: M. Atkinson, e-Science (...), Grid2006 & 2-nd Int.Conf.e-Social Science 2006, National e-Science Centre UK ACK: I. Foster, System Level Science and System Level Models, Snowmass, August 1-2, 2007

4. Science 2.0 4 ACK: David de Roure, http://www.soton.ac.uk/~dder/talks.html

5. Users’ Requirements 5 ACK: F. Gagliardi  Resources -- Services  Infrastructure  Data  Computational

6. Introducing the Grid 6  Coordinates resources that are NOT subject of centalized control, which can be used as a virtual computer / storage  Using standard, open, general-purpose protocols and interfaces  Delivers nontrival quality of services  Resources are shared by users due to Virtual Organizations

7. EGEE 7 Mariusz Sterzel CGW'08 Kraków, 13 October 2008 7 EGEE Archeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences … >250 sites 48 countries >100,000 CPUs >30 PetaBytes >10,000 users >150 VOs >150,000 jobs/day

8.  The Consortium consists of five High Performance Computing Polish Centres representing Communities (coordinated by ACK CYFRONET) due to:  Participation in International and National Projects and Collaboration ~35 international projects FP5, FP6, FP7 on Grids (50% common) ~15 Polish projects (50% common)  Needs by Polish Scientific Communities ~75% publications in 5 Communities  Computational resources to date Top500 list  European/Worldwide integration Activities EGEE I-III, EGI_DS, EGI, e-IRG, PRACE, DEISA, OMII, EU Unit F3 „Research Infrastructure” Experts  National Network Infrastructure ready (thanks to Pionier National Project) Rationales behind PL-Grid Consortium GEANT2

9. Partners’ Computing Resources TOP500 – November 2008

10. PL-Grid Foundations – Summary  Motivation  E-Science approach to research  EGI initiative ongoing in Europe (http://web.eu-egi.eu)http://web.eu-egi.eu Formation of stable e-intrastucture in Europe Collaboration with NGIs and coordination development activites  PL-Grid Project (2009-2012)  Application in Operational Programme Innovative Economy, Activity 2.3 (in Sept. 2008)  Got funded March 2, 2009 (via European Structural Funds) Polish Infrastructure for Supporting Computational Science in the European Research Space European Grid Initiative (EGI) National Grid Initiative (NGI) PL-Grid Response to the needs of Polish scientists and ongoing Grid activities in Poland, other European countries and all over the world

11. Grid infrastructure (Grid services) PL-Grid Application Clusters High Performance ComputersData repositories National Computer Network PIONIER Domain Grid Advanced Service Platforms Domain Grid  Polish Grid is developing a common base infrastructure – compatible and interoperable with European and Worldwide Grids.  Specialized, domain Grid systems – including services and tools focused on specific types of applications.  Such an approach should enable efficient use of available financial resources.  Plans for HPC and Scalability Computing enabled. Offer for the Users  Computing Power 215 Tflop/s  Storage 2500 TB  Support from PL-Grid staff to using advanced Grid tools  Support for porting legacy codes to Grid environment  Support for designing applications for PL-Grid environment PL-Grid Assumptions

12. Elements and Functionality  PL-Grid software comprises:  user tools (portals, systems for applications management and monitoring, result visualization and other purposes, compatible with the lower-layer software used in PL-Grid);  software libraries;  virtual organization systems: certificates, accounting, security, dynamic ;  data management systems: metadata catalogues, replica management, file transfer;  resource management systems: job management, applications, grid services and infrastructure monitoring, license management, local resource management, monitoring. Users National computer network Grid Application Programming Interface Virtual organizations and security systems Basic Grid services Grid services LCG/gLite (EGEE) UNICORE (DEISA) Other Grids systems Grid resources Distributed computational resources Grid portals, development tools Distributed data repositories  Three Grid structures are maintained:  production,  reseach,  development/testing.

13. The PL-Grid Project is split into several workpackages Planned Realization of Aims PLANNING AND DEVELOPMENT OF INFRASTRUCTURE P2 Coordination Structure Operation Rules Dissemination PROJECT MANAGEMENT P1 SECURITY CENTER P6 Training SUPPORT FOR VARIOUS DOMAIN GRIDS P5 P4 GRID SOFTWARE AND USERS TOOLS DEVELOPMENT EGEEDEISA …. OPERATIONS CENTER P3 Main Project Indicators: Peak Perf.: 215 Tflops Disk Storage: 2500 TB

14. Operational Center: Tasks EGI Group EGI Testing Middleware EGI Production Middleware  Coordination of Operation  Management and accounting  EGI and DEISA collaboration (Scalability and HPC Computing)  Users’ requirements analysis for operational issues  Running infrastructure for:  Production  Developers  Research  Future consideration:  Computational Cloud  Data Cloud  Internal and External Clouds  Virtualization aspects

15. Operational Center: Current Results  PL-Grid VO (vo.plgrid.pl) fully operational  Production resources for PL-Grid users in 4 centres (Gdańsk, Kraków, Poznań, Wrocław) ~8kCPUs in total, ~142 TB  Infrastructure monitoring (failure detection) tools based on Nagios & EGEE SAM in place  Support teams, tools and EGEE/EGI-compliant operations procedures aimed at achieving high availability of resources established  Knowledge sharing tool for operations problems  Problem tracking tool to be released in November  Provisional user registration procedure  User registration portal in preparation (1 st prototype to be released end of October)  Accounting integrated with EGEE PCSS Cyfronet WCSS

16. Operational Center: Current Results  PL-Grid VO (vo.plgrid.pl) fully operational  Virtual production resources for PL-Grid users in 4 centres (Gdańsk, Kraków, Poznań, Wrocław)  Infrastructure monitoring (failure detection) tools based on Nagios & EGEE SAM in place  Support teams, tools and EGEE/EGI-compliant operations procedures aimed at achieving high availability of resources established  Knowledge sharing tool for operations problems  Problem tracking tool to be released in November  Provisional user registration procedure  User registration portal in preparation (1 st prototype released in October)  Accounting integrated with EGEE PCSS Cyfronet WCSS

17. Operational Center: Next Plans User Registration – access to any person registered as researcher in Poland - www.opi.org.pl – portal being already developed (to be released end of October'09) – access to resources based on grants – giving rights to use X CPUhours and TB of disk space access services – way of accessing resources in particular computing centre PL-Grid Research Infrastructure – “resources on demand” allows to request machines for different purposes – install a grid middleware for testing purposes – for developers to build a testing environment – any other research requiring grid resources – procedures are being defined Enable access to UNICORE resources – small amount of dedicated UNICORE machines – integrate UNICORE and gLite – make use of the same resources

18. 18 Software and Tools Development: Current Results  Establishment of eight groups of research developers, 20+ people from five Polish supercomputing centers, collaborating together to improve the quality of existing European grid infrastructure services, tools and applications  Research and development testbed available based on dedicated physical and virtual machines from supercomputing centers  Initial evaluation of new user tools, environments and grid applications developed and supported in Poland  Basic extensions and improvement procedures proposed to various functionalities provided by gLite, Unicore and QosCosGrid  Software reengineering, development and testing of useful user tools, such as Migrating Desktop, gEclipse, ViroLab/GridSpace, FIVO, Vine Toolkit, QCG-OpenMPI, QCG- ProActive, etc.  Software quality management for new services and tools to be deployed and maintained in the second phase of Pl-Grid  Active links to new user communities in Poland interested in advanced computing technologies in order to collect additional requirements and scenarios: www.plgrid.pl/ankieta

19. 19 Software and Tools Development: in numbers Research developers: 20+ Main research teams: 8 Supercomputing centers involved : 5 Dedicated physical / virtual machines: 30+ Tested infrastructures: EGEE, DEISA Software reengineering and tools testing: QosCosGrid, GridSpace, Migrating Desktop, gEclipse, Virolab/GridSpace, FiVO, Vine, QCG-OpenMPI/ProActive, etc. Software quality management : for new services and tools Current applications under integration and testing procedures: 10+ Current user tools under integration and testing procedures: 5+ Active links to new user communities in Poland: www.plgrid.pl/ankieta

20. 20 Software and Tools Development: Next Plans Software and Tools Development: Next Plans  Closer collaboration with user and developer communities representing various scientific domains (based on feedback received from our questionnaire)  Adaptation of virtualization techniques improving usability, fault tolerance, checkpointing and cluster management  Fancy web GUIs based on Liferay and Vine Toolkit accessing existing and extended e-Infrastructure services  Performance tests of various large scale cross-cluster parallel applications using co- allocation and advance reservation techniques  More analysis of grid management, monitoring and security solutions  Common software repository and deployment rules  Example services, tools and applications available as virtual boxes with different configurations  Main checkpoints and timeline: IX 09VI 09IX 10IV 11I 12 We are here

21. Users’ Support and Training  General task:  Running Help-Desk and users’ training and support tasks  Supporting how to effectively exploit developed infrastructure including selection of right applications for users’ problems and seamless running computation on the grid.  Pl-grid is an user oriented infrastructure, support and training team will be close to the users.  At the moment:  Support Team is organized  Internal training for support team already begun  Selection of licensed application required by the users’ community to be installed on the whole PL-Grid infrastructure  First training for the users delivered.  Plans:  Many, many trainings for beginners and advanced grid users

22. PL-Grid Security Group: Current Results  Defined security guidelines for architecture planning  Defined security policy for local sites installations  Incident reporting systems review to choose best solution for PL- Grid  GGUS - Global Grid User Support  RTIR - Request Tracking for Incident Response  DIHS - Distributed Incident Handling System  Architecture and prototype version of on-line user mapping and credentials distribution system.

23. PL-Grid Security Group: Next Plans  Establish NGI-wide CERT  Introduce security policies into local sites  Develop and deploy local sites security policy conformity monitoring system  check if software (kernel, services) is up to date  check listening ports, firewall rules  monitor suid/sgid lists and integrity  Develop distributed alert correlation system build on top of network and host sensors.

24. How to become a User ?  Each Polish scientist can make use of PL+Grid resources  Just to register  For registration and other information see: http://www.plgrid.pl  Ask questions: m.sterzel@cyfronet.pl  For registration send an e-mail to: email: plgrid-rejestracja@rt.grid.cyfronet.plplgrid-rejestracja@rt.grid.cyfronet.pl 24 ACK: M. Sterzel, Cyfronet

25. Examples 25

26. Example: Virtual Laboratory GridSpace  Use of distributed computational resources and data repositories  High-level tools offered for the user for in-silico experiments ACK: EU Project Virolab, http://virolab.cyfronet.pl 26

27. Sample experiment in ViroLab Environment  Patient’s data  Medical examination  HIV genetic sequence put into database  Experiment in-silico  Collect HIV genetic sequences from database  Perform sequence matching dopasowanie  Calculate virus resistance 27 http://virolab.cyfronet.pl http://www.virolab.org

28. t.szepieniec @cyfronet.pl Example: Biotechnology in Grid ACK: Irena Roterman, Jagiellonian University, Tomasz Szepieniec, Cyfronet Never Born Protein Folding

29. Contract-based Dynamic Virtual Organizations FiVO 29 FiVO Goal:  Allow end users to defined their requirements for the Virtual Organization on a high level of abstraction  Semantic description of domain Results:  Automatic deployment of the VO  Security and monitoring  Optimization of data access  Replication and migration Overall system architecture

30. Summary – Activities  Achieved  Establish PL-Grid VO using Partners’ local and EGEE resources  In operation for Users’ Communities  Provide resources for covering operational costs  Provide resources for keeping international collaboration ongoing  Long term  Software and tools implementation  Users’ support and traning  Provide, keep and extend the necessary infrastructure  Work on approaching paradigms and integration development HPCaaS and Scalable Computing (Capability and Capacity Computing) Clouds Computing (internal-external, computing clouds, data clouds) SOA paradigm, knowledge usage … „Future Internet” as defined by EC in Workprogramme

31. Acknowledgements  ACK Cyfronet AGH  Tomasz Szepieniec  Marcin Radecki  Mariusz Sterzel  Karol Krawentek  Agnieszka Szymańska  Andrzej Oziębło  Tadeusz Szymocha  Alex Kusznir  Teresa Ozga  Aleksandra Mazur  ICM  Piotr Bała  Maciej Filocha  PCSS  Norbert Meyer  Krzysztof Kurowski  Mirosław Kupczyk  WCSS  Józef Janyszek  Bartłomiej Balcerek  Paweł Dziekoński  TASK  Mścisław Nakonieczny  Jarosław Rybicki  Rafał Tylman 31 W ORK PERFORMED IN THE F RAMEWORK OF P ROJECT POIG.02.03.00-00-007/08

32. http://plgrid.pl