1. UMD TIER-3 EXPERIENCES Malina Kirn October 23, 2008 UMD T3 experiences 1

2. What are your service needs?  Basic cluster:  Submit Condor jobs to cluster  Submit CRAB jobs to grid  Run CMSSW  Download data registered in DBS (PhEDEx & srm client)  Computing element:  Service CRAB production jobs  Storage element:  Service all CRAB jobs October 23, 2008 2 UMD T3 experiences

3. UMD cluster basics October 23, 2008 UMD T3 experiences 3  Configuration Configuration  1 HN, 8 WNs, ~9TB disk array  HN = Rocks HN, CE & SE (obviously not scalable)  WNs = 7 interactive WNs + 1 PhEDEx WN  Disk array RAID-6, xfs, logical volume, network mounted from HN (direct attached storage)  Cluster management: RocksRocks  Free, with software rolls such as Ganglia, Condor  “Clean reinstall” model for WN management  Network  All nodes have internal and external network connections  Scalable, but some view as risky

4. gLite-UI & CRAB  gLite-UI (EDG utils) somewhat necessary for CRAB  CRAB now offers CrabServer, which does not have to be installed at your site (direct users to set server_name=bari in crab.cfg)CrabServer  gLite-UI cannot be installed on a Rocks HN, probably not on OSG CE or SE  gLite-UI configuration is a challenge, work from example (not the template) example  Links:  gLite-UI: 1, 2, 3, 41234  YAIM YAIM  CRAB CRAB October 23, 2008 4 UMD T3 experiences

5. CMSSW  Can have CMSSW versions automatically installed and removed via OSG utilities.  ‘Production releases’ of CMSSW  email Bockjoo KimBockjoo Kim  Alternatively, manually install, create link named /cmssoft/cms & edit /etc/grid3-locations.txt  Frontier DB queries require Squid web proxySquid web proxy  Support for CRAB jobs requires site-local-config.xml & storage.xml (examples)site-local-config.xml & storage.xmlexamples October 23, 2008 5 UMD T3 experiences

6. site planning October 23, 2008 6 UMD T3 experiences

7. BeStMan storage element (SE)  Lightweight, easy to install, configure, and use  Will manage files for you or provide a gateway to your existing file system  OSG also supports BeStMan on top of XrootdFS (requires two additional nodes, minimum)  OSG guide for BeStMan on XrootdFS is coming out, OSG guide for just BeStMan (you will want to set your own configuration options)OSG guide for just BeStMan configuration  Getting to work with FNAL srm-client requires using special tags in calls or editing $SRM_CONFIG  webservice_path=srm/v2/server (.wsdl?)  access_latency=ONLINE  pushmode=true October 23, 2008 7 UMD T3 experiences

8. Monitoring October 23, 2008 UMD T3 experiences 8  Ganglia for cluster monitoring, comes with Rocks  RSV for OSG monitoring, comes with OSG RSV  We don’t use SAM  Tests CMS-specific details, very nice! We use CRAB.  Enables participation in official production  SAM tests for BeStMan SEs under development

9. PhEDEx  You will probably want a “PhEDEx node” in addition to your OSG CE & SE node(s)  Transfer public DBS data to or from your site  To site: does not require SE  To site & shown as host in DBS: requires SE  From site: requires dCache SE or a special PhEDEx client just for you at the receiving site  PhEDEx can run atop gLite-UI  gLite-UI required for advanced protocols  Otherwise uses srm  Also requires storage.xml, which can be different from CMSSW’s storage.xml.storage.xml October 23, 2008 9 UMD T3 experiences

10. Tricks October 23, 2008 UMD T3 experiences 10  Always back up your OSG installation before any upgrade! pacman allow easy rollback of software from backup.  Use cp -p : permissions in OSG directory are important  Use soft links on your first install, then you can move it around for upgrades and fixes  Set shell for grid users to /bin/true  Deter brute-force ssh attacks (we use DenyHosts)DenyHosts  Keep a detailed log  Write a user guideuser guide  Train admin backup  Email OSG CMS Tier-3 hypernews to get help Email OSG CMS Tier-3 hypernews