1. IHEP(Beijing LCG2) Site Report Fazhi.Qi, Gang Chen Computing Center,IHEP

2. Outline Infrastructure Infrastructure Local Cluster Status Local Cluster Status LCG Tire2 Site Statistics LCG Tire2 Site Statistics Management & Operation Management & Operation Summary Summary Chen Gang/CC/IHEP 2015-12-21 - 2

3. Infrastructure Serving more than 1000 users Serving more than 1000 users Power supply capacity: 1800Kw Power supply capacity: 1800Kw Shi,Jingyan/CC/IHEP 2015-12-21 3 Cooling: water cooling rack for the blade servers Cooling: water cooling rack for the blade servers

4. Water cooling rack Water cooling rack Inter-row air conditioning Inter-row air conditioning Cooling capacity per rack: 28kw Cooling capacity per rack: 28kw Infrastructure Upgrade Shi,Jingyan CC--IHEP 2015-12-21 - 4 Power Capacity: 1800kw Cooling System

5. Chen Gang/CC/IHEP 2015-12-21 5 Local Cluster --Computing Nodes Most for BES,YBJ,DYB,Atlas,CMS experiments Most for BES,YBJ,DYB,Atlas,CMS experiments Some small projects added Some small projects added Blade system IBM/HP/Dell Blade system IBM/HP/Dell Blade links with GigE/IBBlade links with GigE/IB Chassis links to central switch with 10GigEChassis links to central switch with 10GigE 886 computing nodes: 7082 CPU-cores 886 computing nodes: 7082 CPU-cores Most running SL5.5 (64 bit)Most running SL5.5 (64 bit) Intend to migrate to SL5.8Intend to migrate to SL5.8 A small part stayes in running SL4.5 (32 bit)A small part stayes in running SL4.5 (32 bit)

6. Torque: 2.5.5 Torque: 2.5.5 Maui: 3.2.6 Maui: 3.2.6 Intend to upgrade to 3.4.4 or higher to support MPI jobsIntend to upgrade to 3.4.4 or higher to support MPI jobs Tools developed to monitor the resources usage, queue status etc. Tools developed to monitor the resources usage, queue status etc. Accounting tool developed Accounting tool developed Chen Gang/CC/IHEP 2015-12-21 - 6 Local Cluster -- Scheduler

7. Scheduler 50 queues to fit various requests Besides serial jobs, MPI, GPU jobs are also supported Testbed Integration of Torque and openstack Managing and scheduling VM nodes in batch-cloud Chen.Gang/CC/IHEP 2015-12-21 7

8. Local Cluster -- Storage Gluster system installed Gluster system installed Storage provided less than 4 months Storage provided less than 4 months Keeps optimizing performance Keeps optimizing performance Adjust to deal with the new bugs Adjust to deal with the new bugs Total space: 153TB, Used space: 145TB Total space: 153TB, Used space: 145TB Chen Gang/CC--IHEP 2015-12-21 - 8

9. Beijing LCG Tier II Site For CMS, ATLAS experiments For CMS, ATLAS experiments 1000+ Job slots 1000+ Job slots Storage: Storage: 320TB dCache320TB dCache 320TB dpm320TB dpm 1T disks were replaced by 2T disks1T disks were replaced by 2T disks Chen Gang/CC--IHEP 2015-12-21 - 9

10. Beijing LCG Tier II Site CPU Time CPU Time

11. Orient + Network Connection Daya Bay Beijing CSTNet Hong Kong IHEP USA GLORIAD 10G ASGC IPv4 10G IPv6 Beijing Tsinghua YBJ EUR. 2.5G 155M 10G Others EDU.CN 10G Chen Gang/CC--IHEP 2015-12-21 - 11

12. Perfsonar @IHEP Two hosts for perfsonar Two hosts for perfsonar Perfsonar.ihep.ac.cn for Bandwidth testPerfsonar.ihep.ac.cn for Bandwidth test Perfsonar2.ihep.ac.cn for Latency testPerfsonar2.ihep.ac.cn for Latency test Network performance tuning is in progress between IHEP and EU. Sites Network performance tuning is in progress between IHEP and EU. Sites http://twiki.ihep.ac.cn/twiki/bin /view/InternationalConnectivity/I HEP-CCIN2P3http://twiki.ihep.ac.cn/twiki/bin /view/InternationalConnectivity/I HEP-CCIN2P3 Chen Gang/CC--IHEP 2015-12-21 - 12

13. Network Research (SDN@IHEP) Goal Goal A flexible, reliable and high performance HEP data transfer network (virtual and private) and system platform in ChinaA flexible, reliable and high performance HEP data transfer network (virtual and private) and system platform in China IPv4 and IPv6 supportedIPv4 and IPv6 supported The traffic can be switched between IPv4 and IPv6 infrastructure and physical path automatically or manually based the network performance and applicationsThe traffic can be switched between IPv4 and IPv6 infrastructure and physical path automatically or manually based the network performance and applications SDN@IHEP  IHEPDTN SDN@IHEP  IHEPDTN End user networkEnd user network Backbone network （ IPv6 & IPv4 ）Backbone network （ IPv6 & IPv4 ） SDN Switch (L2VPN gateway & Openflow supported)SDN Switch (L2VPN gateway & Openflow supported) Control center (API to Application)Control center (API to Application) Applications(FTS/NMS/…….)Applications(FTS/NMS/…….) Members Members IHEP/SJU/SDU/TsingHua/……IHEP/SJU/SDU/TsingHua/…… Network Vendor ： Ruijie NetworksNetwork Vendor ： Ruijie Networks

14. SDN@IHEP model

15. Most part of computing environment running well Most part of computing environment running well New gLuster system is in production New gLuster system is in production Network performance between IHEP-Eur. got an clear improvement Network performance between IHEP-Eur. got an clear improvement New Management and Operation system will be deployed to improve the efficiency New Management and Operation system will be deployed to improve the efficiency Summary Chen Gang/CC/IHEP 2015-12-21 - 15

16. Thank you! Questions? Chen Gang/CC/IHEP 2015-12-21 - 16