1. 1 Deployment of an LCG Infrastructure in Australia How-To Setup the LCG Grid Middleware – A beginner's perspective Marco La Rosa mlarosa@physics.unimelb.edu.au eScience 2005 Workshop: Deploying Production Grids – Beyond the Hype

2. 2 Deployment of an LCG Infrastructure in Australia ● Australian Government provides funding for a National High Performance Computing facility. Australian Partnership for Advanced Computing (APAC) ● APAC has partner computing facilities in most of the states ac3, ANU, CSIRO, iVEC, QPSF, SAPAC, TPAC, VPAC ● State facilities support researchers from their member institutions All working together on the Australian National Grid Program

3. 3 Deployment of an LCG Infrastructure in Australia APAC provides funding for application level projects: High Energy Physics Application area within the National Grid Program ● Deploy an LCG infrastructure within Australia ● Develop expertise in the deployment and use of the LCG middleware ● Work with the KEK Computing Facility (Japan) and members of the Belle experiment in the federation of resources ● Deploy the Belle Monte Carlo Analysis on the Grid

4. 4 Deployment of an LCG Infrastructure in Australia Current state of Deployment: ● Compute Element deployed as a Xen Virtual Machine at the Victorian Partnership of Advanced Computing (VPAC) Supported by: ● User Interface, Resource Broker, Global-BDII, Storage Element and VOMS at the School of Physics, The University of Melbourne ● Deployed as Xen Virtual Machines Large tax on resources to get started in Grid - easier to get started if using Virtual Machine technology. National Grid Program is using 'gateway' machines which host different Grid interfaces to resources based on application area requirements

5. 5 Deployment of an LCG Infrastructure in Australia Beyond the Hype ● Grid tools not available on the cluster compute nodes (Worker Nodes) Policy: Compute Nodes should not need access to the outside world ● User accounts: one account – grid-belle ● Shared facility: will the whole facility belong to LCG? ● We have an excellent relationship with VPAC – support our application area, provide computing and human resources ● Use the Compute Element as a Globus interface LCG CE easier to setup than Globus gatekeeper! LCG CE supported by ALL of CERN and members!

6. 6 Deployment of an LCG Infrastructure in Australia Operating System Issues (RedHat, Suse, Scientific Linux 3) ● Athena is supported on Scientific Linux 3 (it's not about the middleware) ● We cannot deploy the Athena toolkit at VPAC because of the OSes they have on their clusters ● We do not have the human resources to compile, test and support Athena on our own whilst it is in development and releases are occurring so frequently We do have portable distribution of the Belle Analysis Software Framework (BASF) ● meta-scheduler developed by Dr. Lyle Winton we can still use the resources

7. 7 Deployment of an LCG Infrastructure in Australia Planned / Potential Deployments: ● ac3, SAPAC, APAC National Facility Expected Concerns: ● Grid tools on cluster compute nodes and connectivity to the outside world ● support issues – limited computing and human resources National Facility ● highly specialised – not a stock standard beowulf ● highly customised PBS – openPBS further developed in- house ● strict security considerations

8. 8 Deployment of an LCG Infrastructure in Australia Advanced Research Computing, The University of Melbourne ● Nordugrid deployment ● Participated in ATLAS Data Challenges 1 and 2 ● Expertise in Grid – prior experience with CERN / LCG Have provided a cluster for use as a prototype Tier 2 ● Service Challenge 4 / ATLAS Data Challenge 3 ● allow us to develop the expertise required to support a Grid site

9. 9 Deployment of an LCG Infrastructure in Australia Beyond the Hype ● Human resources I am the systems and site administrator, security contact This is not my main wage earning occupation! ● Computing resources ● Have had to use Xen Virtual Machines because of a lack of physical computing resources ● small site: 26 cpu, 2Tb disk Minimum commitment to join the Grid – 3 computers (CE, SE, MON) ATLAS Data Challenge requirement – VO-BOX make that 4 computers!

10. 10 Deployment of an LCG Infrastructure in Australia Summary and Conclusions ● Distributed Management ● LCG Australia Site administrator and support ● Systems Managers at each Site ● Site Policy – Very Important! ● The PAC's are not members of LCG / EGEE ● do not necessarily agree with LCG / EGEE security agreements – may change with time ● Other people's resources – shared facilities Good working relationship essential – trust!

11. 11 How-To Setup the LCG Grid Middleware – A beginner's perspective LCG Technical Workshop – KEK, Japan, November 17 – 18, 2005 ● Installation and Configuration of an LCG Grid – Get to know your middleware ● Beginner's introduction to the Grid Range of Experience ● very experienced: Manage a Tier 2 ● no experience: Grid – huh?

12. 12 How-To Setup the LCG Grid Middleware – A beginner's perspective Format of the workshop ● Prepare the node ● Is APT installed? ● Set up APT ● Update the system to Sci Linux 3.0.5 ● Configure NTP ● Download the installer and prepare for installation ● Configure site info files ● Install and configure the middleware component ● Common tasks – certificates, preparing for installation Time Required: One full day ● Testing – does it work? Time Required: One full day

13. 13 How-To Setup the LCG Grid Middleware – A beginner's perspective ● YAIM is a great installer ● experience shows us that the combination of a maturing middleware and intelligent installation tool results in the correct installation and configuration of the components most of the time ● Generally, errors are a result of connectivity between components ● Side job – post doc hears about Grid and wants to give it a go ● post doc is not a network administrator ● tests the Grid and finds problems ● thinks the middleware is buggy ● doesn't realise that port X is blocked at the firewall ● gives up

14. 14 How-To Setup the LCG Grid Middleware – A beginner's perspective Information ● Lots of it and scattered all over the internet ● No good general introduction and guide – ● experience helps – but how do you gain experience if you don't know where to start? As a new administrator it's not immediately obvious: ● what is needed to get started (minimum number of computers) ● can LCG facilities be used? (R-GMA registry and schema server) ● when you do get started, how do you go about joining the Grid if you've never heard of a ROC?

15. 15 What did we learn? Grid does not “just work™” But - “it can™” With the help of: ● experienced administrators, ● good documentation, ● a lesson on debugging network connectivity issues, ● good general understanding of “The Grid”, ● services and ports, ● firewalls... How-To Setup the LCG Grid Middleware – A beginner's perspective

16. 16 The End... Thankyou http://epp.ph.unimelb.edu.au/EPP mlarosa@physics.unimelb.edu.au