1. Scalable sync-and-share service with dCache
Paul Millar, on behalf of the dCache team
Workshop on Cloud Services for Synchronisation and Sharing (CS3) Amsterdam, The Netherlands
Direct NFS mount of dCache
Looks like a regular file-system; in use at DESY-Cloud
Direct dCache WebDAV support:
Similar to NFS, but supporting more authn options (e.g., X.509, OpenID-Connect)
Tie QoS to directory; e.g., 'important' folder – multiple disk copies, 'archive' folder – contents written to tape.

2. What is this talk about? Web-client Web-client Sync-client Sync-client
𝑥Cloud
𝑥Cloud
NFS
Filesystem
dCache

3. What is this 'dCache' cloud thing?
Managed distributed (aggregation of heterogeneous) storage.
Mature open-source project, core team based at three organisations.
In production for over 15 years and deployed throughout the world.
Instances with tens-of-Petabytes of data, O(10^8) files.
Different QoS within a single dCache instance:
Support different media types: random-IO (SSD, HDD, CEPH), whole-file copying (HSM/tape, S3, …)
Support different replication policies.
Data placement / caching policies.
Through INDIGO-DataCloud (Horizon-2020) project:
Working within RDA to standardise QoS definitions,
Working within SNIA to standardise QoS access models,
Working within dCache to improve QoS support.

4. Media locality QoS in dCache
Different QoS chosen by directory; e.g.,
Different users have different QoS,
A user chooses which QoS by writing file into different directory.
Single
copy:
Redundant
copies: … + +
All QoS options available in 𝑥Cloud:
Media placement, geographical locality, replication factor, …
Principally admin configured:
User chooses into which directory they write.
Target directory chooses which policy is used:
'tape' directory goes to tape, 'redundant' gets multiple copies
Distinct nodes | racks | buildings | campuses | …
Disk + tape
copy: +

5. QoS media type support: CEPH
dCache can use any POSIX storage
Typically RAID storage with some local filesystem.
Admins wanted to be able to use CEPH.
Supported: 1x storage-node per CEPH pool.
Adding redundant storage-nodes
Also benefits cluster file-systems: luster, GPFS, …

6. Geographic placement QoS, part #1

7. Geographic placement QoS, part #2

8. Geographic placement QoS, part #3

9. Updates in dCache: hardening service
Matching NFS semantics to 𝑥Cloud expectations:
Allow file truncation (for when 𝑥Cloud overwriting existing files),
Better support for multi-open (multiple sync- clients),
Implementing more advanced features from the NFS specification.
Real-life: conditions / operation intervention:
Better support for planned storage-node intervention,
Better user experience after unintended storage-node outage/restart,
More stable IO when network is flaky.
High Availability dCache:
All dCache components may be deployed redundantly,
Load-balancing for core components,
Users unaware of (patch-level) upgrades and (planned) hardware intervention.

10. Looking towards the future

11. dCache vision 𝑥Cloud & dCache duplicated namespace:
Pain to keeping these in sync.
Shared files knowledge only in 𝑥Cloud layer:
Shared files are not visible when interacting directly with dCache

12. Teaching dCache about sharing
Users can share files or directories
Can share what you own + those you've been given permission (e.g., ACL ADMINISTRATE bit).
SHARE object is written in other user(s) 'incoming' directory (e.g., home directory)
Only receiving user sees the SHARE
Can rename, move, delete, cd/read; similar to regular sym-link.
SHARE has share-mode for access control: share can be read-only or read-write. Independent of the SHARE's filesystem permissions.
Renaming source does not affect SHARE.
Unshare a subset of recipients removes their SHAREs.
Removing the source also removes all SHAREs

13. Teaching dCache about sharing
Users can share files or directories
Can share what you own + those you've been given permission (e.g., ACL ADMINISTRATE bit).
SHARE object is written in other user(s) 'incoming' directory (e.g., home directory)
Only receiving user sees the SHARE
Can rename, move, delete, cd/read; similar to regular sym-link.
SHARE has share-mode for access control: share can be read-only or read-write. Independent of the SHARE's filesystem permissions.
Renaming source does not affect SHARE.
Unshare a subset of recipients removes their SHAREs.
Removing the source also removes all SHAREs

14. QoS improvements Standardising QoS description: Increasing choice:
Work coming through the INDIGO-DataCloud EU Horizon-2020 project.
Standardisation (SNIA, RDA),
Improved flexibility: more QoS options,
Moves decision from admin into users.
What it looks like:
RESTful interface to dCache
An 𝑥Cloud application to drive RESTful interface
Examples:
User chooses desired QoS for a directory,
Change QoS later on: this data is actually important (multiple copies, store on tape, etc).
Expose choice to sync-n-share users:

15. Summary NFS hardened from extensive use in real-world conditions
Extending QoS options in dCache.
Plan to:
Expose QoS options in 𝑥Cloud,
Implementing SHARE in dCache,
Reduce dependency on 𝑥Cloud database,
Investigate getDirectDownload for direct- from-dCache client HTTP transfers
Finish sync-client support within dCache.

16. Backup slides

17. Solving them: teaching dCache to share
Users can share files or directories
Can share what you own + those you've been given permission (e.g., ACL ADMINISTRATE bit).
SHARE object is written in other user(s) 'incoming' directory (e.g., home directory)
Only receiving user sees the SHARE
Can rename, move, delete, cd/read; similar to regular sym-link.
SHARE has share-mode for access control: share can be read-only or read-write. Independent of the SHARE's filesystem permissions.
Renaming source does not affect SHARE.
Unshare a subset of recipients removes their SHAREs.
Removing the source also removes all SHAREs