1. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 1 Activities of IHEP in LCG/EGEE Projects Vadim Petukhov (IHEP, Protvino, Russia) E-mail: Vadim.Petoukhov@ihep.ru

2. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 2 Outline IHEP within the RDIG Consortium; EGEE activities in IHEP; Existing resources (farms, storage); Usage of resources (monitoring) Problems with communication link.

3. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 3 Protvino and IHEP from the Sky

4. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 4 IHEP within RDIG RDIG Consortium (Russian Data Intensive GRID) has been formed with a view to maintenance of full-scale participation of Russia in EGEE project. It is also planned to involve Russian organizations from various areas of science and education in the EGEE project. Consortium RDIG, according to the structure accepted in project EGEE, is included into the project as regional federation "Russia " on a level with CERN which is carrying out coordinating functions over the project. Institute for High Energy Physics (IHEP) is one of the institutes consisting in the consortium. It is largest particle accelerator center in Russia. 2 000 Personals. 12 specialists are involved in EGEE project with overall activities near 5 FTE.

5. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 5 IHEP Activity in EGEE Project NA3 – User Training and Induction (5 specialists) SA1 - European Grid Operations, Support and Management ( 10 specialists) Activity NA3SA1 Total FTE 1.50 3.254.75

6. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 6 Fig.1 47 WNs x 2 PIV, 2,6 Ghz, 1 GB ( 50 KSI2K)

7. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 7 General view of cluster is presented on fig.1. It operates under management of operating system Scientific Linux SL, release 3.0.5 and GRID middleware LCG 2.7.0. Cluster is used as a multipurpose computing facility and consists of 47 working nodes. Peculiar properties of this cluster are: an administration is doing with the help of Quattor; there is an opportunity for cluster usage by grid users as well as by others IHEP users (without of grid certificates) through network file system AFS; use of Castor system for long-term keeping of huge files of data on magnetic tapes.

8. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 8 Fig.2 NFS UI №1,2 Gateway QUATTOR MON CE LFC DPM SE DPM AFS Drives DISK Internet TAPE Router 5 TB

9. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 9 The Cluster scheme is shown on the fig.2. From this chart it is possible to see that users can use set of working nodes for specific calculations, connected with different experimental groups of researchers. The total amount of disk space at present is near 5 TB. For installation of software and configuration of cluster nodes system Quattor is used (http://quattor.web.cern.ch/quattor /). Quattor is powerful tool for installation, configuration and support of big clusters under Unix. It was adapted in IHEP to fit all features of our Cluster.http://quattor.web.cern.ch/quattor Server nodes have two network interfaces: one is used for communication with working nodes, other – for Internet connection.

10. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 10 Working nodes are placed in virtual subnet, but they also have Internet connection through special network gate, that translates virtual IP addresses of nodes in one real address. For keeping of home catalogues of users, application Software and part of an experimental data AFS is used. AFS is distributed file system and gives for users reliability, security and opportunity to use the home directories from foreign scientific organizations. On IHEP Cluster OpenAFS is used, which is open for use with international community. Token of AFS system has limited lifetime, and for processing of a batch job with greater time, the life time of the token should be prolonged. Extension life time of tokens as required is realized through the system’s exterior module "Afs _ token" (Module Extension to Openpbs, http://www.openpbs.org/). This module was adapted in IHEP for the usage with Torque system. (http://www.clusterresources.com/pages/products/torqu e-resource-manager.php).http://www.clusterresources.com/pages/products/torqu e-resource-manager.php

11. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 11 For the long time data storage system Castor is used (http://castor.web.cern.ch/castor /). On fig.3 logical scheme of this system is presented, master components of which are disk pool manager, name server, database, client and magnetic tapes with overall capacity 8.3 TB and disk pool of 2.2 TB, RAID5.http://castor.web.cern.ch/castor It is obvious that general quantity of magnetic tapes can be easy increased. Cluster IHEP is actively used by GRID and domestic users. Example of cluster loading during last month (August 2007) is presented by fig.4. And on figs.5 – 7 other diagrams are presented for period of one year.

12. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 12 Fig.3

13. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 13 Fig.4

14. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 14 Total number of jobs run per VO in IHEP Fig.5

15. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 15 Fig.6

16. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 16 Fig.7

17. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 17

18. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 18 CPUs: HP ProLiant DL140G3, 4GB x20 (200 KSI2K) DISKs: Intel XEON 3.0 GHz 2MB FCBGA 4 х 1GB DDR400 PC3200 ECC REG 12 х Seagate 400Gb SATA-2 AIC RMC2E2-XPSS, 3WARE 9550 SATA II Raid5 KIT

19. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 19

20. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 20 It is was an agreement with Gaztelecom to organize a channel using gigabit Ethernet technology and to lease out so-called "dark" optical fibres for the extension of the IHEP optical cable to the international communication node in Moscow (M-IX).

21. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 21

22. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 22

23. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 23 Summery parameters of the Russian dTier2 Cluster Summery parameters of the Russian dTier2 Cluster

24. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 24 47 WNs UI № 1,2

25. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 25

26. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 26

27. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 27

28. Institute for High Energy Physics ( http://www.ihep.su )http://www.ihep.su NEC’2007 Varna, Bulgaria, 10-17 September 2007 28