1. Future of WAN Access in ATLAS
David Cameron
(thanks to Mario Lassnig for most of the input)

2. Experience WAN: Main use cases so far:
A process reading/writing data from a non-local source
Outside the “site”, let’s say with traffic travelling through a different administrative area
Main use cases so far:
Job failover if local input not available
Overflow of jobs to empty sites or imbalanced close sites (BNL + MWT2)
Object stores for event service
Distributed sites (MWT2, NDGF-T1) – but in general network is managed within the site
5 December 2016
US facilities & ADC workshop, BNL

3. Experience 3rd party transfer In general we transfer too much data
Managed replication to fulfil computing model, pre-placement of job input data and consolidation of output date
FTS3 is used for almost everything
3 servers at CERN, BNL and RAL
Globus Online at US HPCs – not much experience yet
In general we transfer too much data
Most of it is short-lived job input/output
We are using the network like it is “free”
5 December 2016
US facilities & ADC workshop, BNL

4. What is important for ADC
Smooth, efficient and controlled use of networks
Not hiding failures
Reducing site-specific modifications
5 December 2016
US facilities & ADC workshop, BNL

5. Short-term roadmap New pilot movers consolidation
use rucio-clients everywhere, except ultra-specific cases (symlinks-only & LSM)
rucio knows location of all replicas of the experiment (=failover)
provides all locations with all available protocols to job/user
either via rucio native replica API or via metalink
list of replicas can be sorted
(now: uniform-random or geoip, in the future: ddm-network-metrics as used by panda & c3p0 already)
AGIS settings for site influence replica sorting
({read/write/delete_wan/lan: [protocol1, protocol2, ...]})
All client tools used in the backend of rucio-clients are available via cvmfs & alrb (gfal, lcg-utils, xrdclient, aria2)
If source replicas are limited to https in metalink response, can use aria2 for chunked parallel downloads
5 December 2016
US facilities & ADC workshop, BNL

6. Remote access For now, primarily useful for analysis jobs only
Reduce usage of TURLs in TFile::Open()
allow ATLAS namespace (scope:name) on the ROOT level
TFile::Open('
Patch contributed to ROOT I/O by davix devs
will return metalink with all protocols that TNetFile can open
will try in order to open the remote file, or continue (=failover)
Branch caching
according to Axel Naumann, the usage of TTreeCache can be greatly improved
provide the list of potential branches upfront
use accessed branches from first job in task as TTreeCache hint for other jobs
5 December 2016
US facilities & ADC workshop, BNL

7. 3rd Party Transfer Continued evolution of FTS
Globus Online integration with Rucio?
On ATLAS side review and reduce unnecessary transfers
GridFTP will be the standard protocol
Others can be evaluated
EOS->Castor was tested with xrootd
S3 for object stores
5 December 2016
US facilities & ADC workshop, BNL

8. Longer-term A lot depends on storage evolution And network evolution
Fewer, large sites?
Greater use of (a few) object stores?
Will every computing resource have local storage?
And network evolution
Will it grow as fast as before?
Will it always be “free”?
It is clear that storage classes will become more important to exploit
cache vs hot disk (eg ceph) vs cold disk vs tape
5 December 2016
US facilities & ADC workshop, BNL

9. Longer-term
SDNs
in their current state, primarily useful for third party transfers
automatically setup virtual circuits based on file properties (size, link parallelism)
Early talks with ESnet people points to many-minute setup times for virtual circuits, cannot be done for ad-hoc transfers
To discuss if Rucio or FTS should be the one requesting this
5 December 2016
US facilities & ADC workshop, BNL