efi.uchicago.edu ci.uchicago.edu FAX meeting intro and news Rob Gardner Computation and Enrico Fermi Institutes University of Chicago ATLAS Federated Xrootd Integration, Deployment and Operations Meeting November 19, 2012

efi.uchicago.edu ci.uchicago.edu 2 WLCG groups We have an on-going set of meetings with two WLCG working groups – WLCG Federated Data Storage (F. Furano Chair) o Explores issues generally and assesses approach taken by each of the LHC experiments – WLCG Xrootd task force (D. Giordano) o New group forming to coordinate deployment and operations o Will seek to engage Grid infrastructure providing groups (EGI/EMI, OSG) for support Both of these will help bring FAX into normal production operations

efi.uchicago.edu ci.uchicago.edu 3 UK cloud update from Wahid Testing federation at Glasgow: Paul / peter are trying to switch to the latest DDM client from cvmfs in test pilots. I'm sure they will update elsewhere how that is going. Many UK DPM sites are now on DPM which will make it easier for them to deploy Glasgow observing many orphaned ping processes from FAX tests. They block ping - the build up of processes causes problems. Fax door checker today noted problems at Oxford - haven't yet checked if that is related to any other issues or transient. RAL: Shaun is working on enabling N2N and hitting some issues - see his on the topic.

efi.uchicago.edu ci.uchicago.edu 4 From Wei is available. Testing at SLAC. No hurry to push to sites except SL6 sites (for GSI). Will work with SL6 sites (LRZ and MPPMU) to enable GSI. Wuppertal: Guenter and I both have concern that their configuration (xrootd4j) is unique and require extra support. If they still wish to use NFS v4.1 (instead of dCache xroot door), we may want them to use the standard proxy for POSIX, which is at least similar to LRZ and MPPMU, if not identical. Will test f-stream at SLAC before deployment (after the collector is able to handle it) Update to N2N. Ilija and Hiro have an update to N2N. I want to add it along with a few other minor changes (unique to C++ N2N, and too small to be a dedicated update).

efi.uchicago.edu ci.uchicago.edu 5 Recapping again the use cases Initial use cases – Failover from stage-in problems with local SE o Now implemented, in production on several sites – Lets discuss this a bit more today for production queues and more sites (Simone, Paul) – Gain access to more CPUs using WAN direct read access o Allow brokering to Tier 2s with partial datasets o Opportunistic resources without local ATLAS storage – Use as caching mechanism at sites to reduce local data management tasks o Eliminate cataloging, consistency checking, deletion services

efi.uchicago.edu ci.uchicago.edu 6 At-large use of FAX Slides from Ilija

efi.uchicago.edu ci.uchicago.edu 7 How to use it? Part - I Datasets should be registered – All the grid produced datasets are automatically registered independently if these are part of official production or simply result of a user's job. – If files are not registered it is trivial to do so. Very detailed description how to do this is given Have your ATLAS grid certificate – Make a proxy setup DQ2 Make sure your code uses TTreeCache! source /afs/cern.ch/project/gd/LCG-share/current_3.2/etc/profile.d/grid_env.sh voms-proxy-init -voms atlas source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.zsh CVMFS version

efi.uchicago.edu ci.uchicago.edu 8 CVMFS environment setup Setup environment Make a proxy setup DQ2 export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh’ export ALRB_localConfigDir=$HOME/localConfig setupATLAS localSetupGLite voms-proxy-init -voms atlas localSetupDQ2Client

efi.uchicago.edu ci.uchicago.edu 9 How to use it? Part - II Check that datasets exist at one of the federated sites Find gLFN's of input datasets – Find closest redirector to compute site. List is here: – Do – make a file with the list of all the gLFN’s export STORAGEPREFIX=root://closestRedirector:port/ dq2-list-files -p data12_8TeV physics_Muons.recon.DESD_ZMUMU.f437_m716_f437 > my_list_of_gLFNS.txt dq2-ls –r myDataSetName

efi.uchicago.edu ci.uchicago.edu 10 How to use it? Part - III From ROOT From prun Instead of giving --inDS myDataset option, provide it with --pfnList my_list_of_gLFNS.txt copy files locally TFile *f = TFile::Open("root://myRedirector:port//atlas/dq2/user/ilijav/HCtest/user.i lijav.HCtest.1/group.test.hc.NTUP_SMWZ.root"); xrdcp root://xrddc.mwt2.org:1096//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtes t.1/group.test.hc.NTUP_SMWZ.root /tmp/myLocalCopy.root

efi.uchicago.edu ci.uchicago.edu 11 FAX dress rehearsal How this actually works in practice, with active users and real workloads, needs to be tested Would like to make a broad announcement – Yet we risk losing users if problems block progress, create frustration, waste time Need to recruit friendly, early adopters willing to tolerate hiccups (there will be hiccups) Recall long ago exercises to validate ANALY queues – we need something similar IMHO it is the only way to know where we really are Proposal is: – Define an FDR sufficient to cover most anticipated user workloads, including user docs – Define a rehearsal period: ~ week and metrics (and needed monitoring) – Poll for site volunteers and an ad-hoc FAX OPS team – Execute the FDR; gather monitoring statistics and accounting data

efi.uchicago.edu ci.uchicago.edu 12 Tier 1/2/3 Jamboree Monday-Tuesday, December Agenda: – w=standard&confId= w=standard&confId= FAX deployment splinter group – Tentative: Tuesday 9 am, E01 – Sign up if you’ll attend o