1. Building a computational cluster managed by ARC NorduGrid Alexey N. Makarov MakarovAlexey@gmail.com Saint-Petersburg State University, Department of Physics March – 2008 - Saint-Petersburg - JASS2008

2. Outline Computational clusters – necessity of integration What is the Grid? An aim of the Grid Existing Grid projects Introduction to ARC NorduGrid Creating a computational cluster Installation and configuration of ARC server Practical using of created system

3. Computational clusters Necessity of integration High performance clusters deployed to improve performance and availability over that provided by a single computer and used to run programs for time-intensive computations. Nowadays scientists propose such tasks that no one cluster can solve them for available time. Such tasks may be done on set of clusters. Often clusters as well as storage elements, particularized software, scientific equipment, etc. located in some university wants to use uniformly by scientist from another university, city or even country.

4. What is a Grid? Grid is a system, that coordinates resources that are not subject to centralized control… …using standard, open, general-purpose protocols and interfaces… … to deliver non trivial quality of service I.Foster, “What is the grid?”(2002) Grid is a type of parallel and distributed system that enables the sharing, selection, and aggregation of geographically distributed "autonomous" resources dynamically at runtime depending on their availability, capability, performance, cost, and users quality-of-service requirements. Rajkumar Buyya “Convergence Characteristics for Clusters, Grids, and P2P networks “, Panel at the P2P conference, Linkoping, Sweden

5. An aim of the Grid's Provide reliable, stable, universal, inexpensive access to resources Distribute resources inside dynamic virtual organizations Aggregate geographically distributed autonomous resources Provide single sign on and subsequent using of all available resources Provide easy way for administration tasks

6. Are WEB and P2P some kind of Grids? The answer is NO. WEB is support access to distributed resources but NOT coordinate usage of this resources. P2P is NOT provide necessary QoS. Grid is NOT a big cluster because of it management system.

7. What kind of applications require Grid? Computation intensive (large-scale simulation) Data intensive (experimental data analysis) Distributed collaboration (online instrumentation, specialized software)

8. Existing Grids

9. Why ARC? Stability working and good testing Supporting and continuous development from NorduGrid collaboration Easy integration with cluster tools and application software Lightweight client's part

10. Introduction to ARC NorduGrid ARC middleware implements fundamental Grid services The usual grid security: single sign on, Grid ACLs (GACL), VOs (VOMS) Job submission: direct or via matchmaking and brokering Information services: resource aggregation, representation, discovery and monitoring Implements core data management functionality  Automated seamless input/output data movement  Data Indexing (RLS), client-side data movement Job monitoring and management Logging service Build upon standard open source solutions and protocols Globus Toolkit® pre-WS API and libraries (no services!) OpenLDAP, OpenSSL, SASL, SOAP, GridFTP, GSI

11. Introduction to ARC NorduGrid architecture ARC based on independent services interacting with overs Grid manager - handles job management upon client request, interfaces to LRMS GridFTP – server performs most data movement Grid Infosys - publishes resource and job information via LDAP Httpsd – https server for Smart Storage Element, performs secure data movement Monitor - Web interface to the NorduGrid Information System Client - a lightweight User Interface with the built-in Resource Broker Authorization via certification scheme Certificate Authorities centres

12. ARC components Picture from “NorduGrid, the middleware and related projects”, NDGF/Lund University GRID06 - June 26, 2006 – Dubna, Oxana Smirnova

13. Creating a computational cluster Tuning operation system ( sshd, NFS, firewall ) Installation and configuration of PBS ( TORQUE-2.1.6 ) to front-end server and nodes Installation software supporting MPI2 and OpenMP parallel programming ( MPICH-2.0 and gcc-4.2.0 ) Testing productivity of computational cluster High Performance Linpack Benchmark – system for testing productivity of computational clusters. Top500 list creates using results of HPL. This is a list of the most performance systems in the world. On our cluster it’s 5.136e+01 Gflops

14. Mounting equipment GigEthernet LAN: CiscoCatalyst 2960G Front-end server (ap8.gridzone.ru): Intel Pentium 4 Dual Core 3.2GHz 2 x 1024MB DDR2 ECC Computational nodes (w3,w4,w7,w8): 2 x Intel Xeon Dual Core 3.0GHz 2 x 2048MB DDR ECC REG OS: ScientificLinux 4.4

15. Site configuration ap8.gridzone.ru Grid Manager Grid Infosys GridFTP SSE Grid Monitor LocalCA External Dependences ARC Middleware SimpleCA GPT Globus Toolkit® packages VOMS Python, MySQL, libxml2 libraries PBS Server PBS Scheduler Ganglia Cluster 32 CPU PBS Client MPICH-2.0 Gcc-4.2.0 Ganglia client Local Resource Management System and Cluster Tools installing independently from ARC. ARC Middleware installing on front-end server only.

16. Local Certificate Authority Centre User certificates for local resources Host certificates grant abilities for creating local grid infrastructure Simple manipulation with users certificates

17. Testing ARC front-end server Information system ( LdapBrowser, ldapsearch, grid-monitor, ngtest ) Data management system ( ngls, ngcp, ngrm, ngcat, ngtest ) Job management system ( ngsub, ngstat, ngget, ngresub, ngkill, ngrenew, ngtest )

18. Monitoring systems Grid Monitor is a Web interface to the NorduGrid Information System Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters Nagios is an enterprise-class monitoring solutions for hosts, services, and networks Logger is a independently developed system for gathering information about Grid jobs

19. General Grid Monitor

20. Data from information system via Ldap Browser

21. Ganglia

22. Resource utilization Computations done for project «Internet Mathematics 2007» supporting by Yandex® Using in «Grid technology» course for students education Grid community scientists investigations i.e. Olav Syljuasen, Pavel Lihatov, Antti Hyvarinen and others.

23. References 1. I. Foster, C. Kesselmann, S. Tuecke, «The Anatomy of the Grid», 2000 2. I. Foster, C. Kesselmann, S. Tuecke, J.M.Nick «The Physiology of the Grid», 2002 3. I.Foster «What is the Grid? A Three Points Checklist», 2002 4. Rajkumar Buyya «Convergence Characteristics for Clusters, Grids, and P2P networks», Panel at the P2P conference, Linkoping, Sweden 5. Oxana Smirnova, «NorduGrid, the middleware and related projects», NDGF/Lund University GRID06 - June 26, 2006 – Dubna 6. M. Ellert, A.Konstantinov, B. Kónya, O.Smirnova, A.Wäänänen. Architecture Proposal, NORDUGRID-TECH-1, 2002. 7. http://www.globus.org 8. http://www.nordugrid.org 9. http://www.gridcomputing.com 10. http://www.gridclub.ru

24. Thank you for your attention!