1. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University SCOTGRID: A PROTOTYPE TIER-2 CENTRE Steve Thorn Authors: A. Earl, P. Clark, S. Thorn University of Edinburgh

2. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Overview What is ScotGrid? ScotGrid resources LCG2 experiences (at Edinburgh) –System administrator –Particle physics user Future plans Conclusions

3. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University What is ScotGrid? One of four Tier-2 sites in the UK Durham, Edinburgh & Glasgow Member of GridPP collaboration Over 100 users Supports: ATLAS, BaBar, Bioinformatics, CDF, Computing Science, Engineering, LHCb, Medipix, MICE, PhenoGrid, UKQCD, ZEUS First UK Tier-2 Over 200 CPUs ~40 TB disk Initially funded by SHEFC (Scottish Higher Education Funding Council) www.scotgrid.ac.uk

4. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Hardware overview

5. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Edinburgh University Front Ends: CE & SE: IBM x205: P-IV 1.8 GHz, 256 MB RAM LCFG: IBM x340: dual P-III 1 GHz, 2 GB RAM Storage (backend): 1 × IBM x440: 8 × Xeon 1.9 GHz, 32 GB RAM 2 × IBM FAStT900 providing 24 TB RAID 10 TB of 155 TB SAN Worker Nodes: 4 × dual Xeon 2.8 GHz, 2 GB RAM, 200 GB disk Housed at the multi-million pound Advanced Computing Facility. 20 Gbit/s link to campus.

6. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Benchmarking IBM x440: 8 × Intel Xeon 1.9 GHz, 32 GB RAM High Performance Linpack benchmark using ATLAS libraries. Matrix calculations to test processor and memory performance. Achieved ~75% of theoretical max performance with un-optimised system Full results at www.ph.ed.ac.uk/~aearl/hpl/www.ph.ed.ac.uk/~aearl/hpl/

7. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Glasgow University Front Ends (CE,SE,LCFG,UI): 4 × Transtec 1001L: P-IV 2.8 GHz, 512 MB RAM Storage: 1 × IBM x370: P-III Xeon 700 MHz, 16 GB RAM 1 × IBM FAStT500 providing 5 TB RAID Worker Nodes: 59 × IBM x330: dual P-III, 1 GHz, 2 GB RAM 34 × IBM blades: dual Xeon 2.4 GHz, 1.5 GB RAM 5 × IBM x340: dual P-III 1 GHz, 2 GB RAM

8. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Usage statistics Glasgow CPU farm 1498570 CPU hours 4471 kilo SPECint2000 weeks (23 September 2004) ATLAS LHCb BaBar CDF Grid Data Management Bioinformatics ZEUS UKQCD Medipix Information Retrieval Device Modelling MICE

9. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Durham University New ScotGrid member since September 2004 Worker Nodes: 40 × dual P-IV 2.2 GHz, 2 GB RAM, 30 GB disk Based at the Institute for Particle Physics Phenomenology Using Sun Grid Engine

10. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University LCG2 experience: sys admin Storage emphasis – too few Front End machines Installed with LCFGng Storage limited to 1 TB until SRM based deployment Hardware moved to Advanced Computing Facility – successful Worker Node 15 miles (24 km) from Front Ends Participating in LCG2 Test Zone:

11. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Installation problems Multiple network interfaces –CE(2) and SE(3) –eth0 (external) is automatically configured by LCFGng –Others require manual config. of driver, network parameters (ip, netmask etc.) and routing –SE eth0 not used so eth1 & eth2 manually defined SCSI and RAID controllers –SE only –Custom boot kernel –Full configuration of /etc/modules.conf required /* Second network interface eth1 */ EXTRA(hardware.modlist) eth1 +hardware.mod_eth1 alias eth1 tg3 EXTRA(network.interfaces) eth1 network.ipaddr_eth1 192.168.1.6 network.netmask_eth1 255.255.255.0 network.hostname_eth1 glenlivet network.network_eth1 192.168.1.0 network.broadcast_eth1 192.168.1.255 REPLACE(install.installmethods,network,network route) install.imethod_route route install/root

12. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University LCG2 Experience: user Aim: to get a physicist with no grid experience to use LCG2 1.Applied for Grid certificate Well documented but time consuming 2.Joined LHCb VO 3.Install UI software on user’s desktop Not possible due to OS mismatch and firewall 4.Used Glasgow’s UI to successfully run jobs

13. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Future plans Edinburgh –Deploy new Worker Nodes: 4 × dual processor from GridPP –Storage software development – SRM to SRB interface –Increase LCG storage to full capacity including SAN Glasgow –Upgrade older Worker Nodes –Migrate to Scientific Linux for Front Ends Durham –Deploy new Front End systems: 4 × dual processor from GridPP –Join LCG2 Test Zone Seeking funding –to increase CPU at Edinburgh and storage at Glasgow –Research and support staff Other institutes?

14. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Conclusions Successfully proven Tier-2 concept Reached million CPU hour landmark Over 100 users within 13 research groups/collaborations Gained valuable LCG2 knowledge (installation and maintenance) Edinburgh & Glasgow participating in LCG2 Test Zone Next steps: prototype → production –Encourage more particle physics analysis –Continue with data challenges –More resources and funding

15. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University Questions?