1. DSFS: Decentralized Security for Large Parallel File Systems Zhongying Niu, Hong Jiang, Ke Zhou, Dan Feng, Shuping Zhang, Tianming Yang, Dongliang Lei, Anli Chen Wuhan National Laboratory for Optoelectronics Department of Computer Science and Technology Huazhong University of Science and Technology Division of Data Storage System Huazhong Univ. of Sci. & Tech, China Univ. of Nebraska-Lincoln, USA Beijing Institute of Computer Technology and Application, China

2. Page 2 Wuhan National Laboratory for Optoelectronics Outline I. Introduction II. Related Work III. DSFS Architecture IV. DSFS Design on OBS V. Performance Evaluation VI. Conclusions

3. Page 3 Wuhan National Laboratory for Optoelectronics Securing large-scale and high-performance storage systems is an important challenge because of the highly parallel and dense data accesses from a large number of users and the massive amounts of personal and sensitive data stored on them. The design concept of existing large-scale storage systems makes their security solutions more challenging and urgent. Motivation

4. Page 4 Wuhan National Laboratory for Optoelectronics A simple view on large-scale storage systems Query requests Metadata and capabilities I/O requests & capabilities Data Information update

5. Page 5 Wuhan National Laboratory for Optoelectronics Motivations (con.) However, attaching storage devices to the client-network renders these devices vulnerable to various network attacks. The storage devices must be intelligent to authenticate users and restrict illegal accesses. In the existing security schemes, a client wishing to access storage devices must acquire a capability authorizing the I/O from MDS and then present it to the storage device with the I/O request. When incorporated into large parallel file systems, these solutions based on capabilities either degrade performance or strongly depend on the workload and application environment, thus limiting the scope of these approaches.

6. Page 6 Wuhan National Laboratory for Optoelectronics Our solutions To provide decentralized security for large parallel file systems (DSFS), by pushing access control decisions into storage devices. DSFS stores a global access control list (ACL) in a centralized decision-making server, i.e., MDS, and pushes the servers access control decisions into storage devices. The storage devices can thus use the pre- stored decisions, in the form of local pre- authorization list (PAL), to make authorization decisions.

7. Page 7 Wuhan National Laboratory for Optoelectronics Outline I. Introduction II. Related Work III. DSFS Architecture IV. DSFS Design on OBS V. Performance Evaluation VI. Conclusions

8. Page 8 Wuhan National Laboratory for Optoelectronics Capability-based solutions Fine-grained capabilities authorize access privileges at the granularity of a block or object. This requires the generation of tens or even hundreds of millions of capabilities, which imposes a substantial overhead on the authorization server. Coarse-grained capabilities allow a capability to grant access to a group of objects, but constrain the granularity of access control. Extended capabilities to provide scalable I/O security for lager-scale parallel file systems. However, the strategy to group multiple I/O authorizations into an extended capability is strongly dependent on the workload and application environment, thus limiting the scope of this approach.

9. Page 9 Wuhan National Laboratory for Optoelectronics Decentralized access control SCARED stores ACLs at storage devices along with the object. A client can use an identity key obtained from the administrator to access the SCARED device that then authorizes the request according to the local ACLs stored on it. The SCARED device stores and manages the file system data and metadata. The access control policy and enforcement of the whole file system must be built on SCARED devices. Other traditional distributed file systems, like AFS, SFS and NFS V4, store and manage the whole or portion of the file system data and metadata in file servers. These systems have different security requirements with existing parallel file systems, in which the storage device usually does not know how the object it holds fits into the file system.

10. Page 10 Wuhan National Laboratory for Optoelectronics Outline I. Introduction II. Related Work III. DSFS Architecture IV. DSFS Design on OBS V. Performance Evaluation VI. Conclusions

11. Page 11 Wuhan National Laboratory for Optoelectronics The design concept of parallel file systems To separate metadata from data to reduce the complexity on storage devices. The storage devices can thus have an optimized simple implementation. They can dedicate themselves to busy data moving and are oblivious to how the object it holds fits into the file system and how users are authenticated and ACLs are implemented. Therefore, ideal decentralized security solutions for parallel file systems should keep the existing system architecture and file placement and impose a minimal cost of security check on storage devices.

12. Page 12 Wuhan National Laboratory for Optoelectronics Our goals For DSFS we aim to be able to have many of the benefits of the decentralized access control without having any restrictions on data placement and management in existing parallel file systems. To allow users to set arbitrary access control policies without having to change the security code in their storage devices.

13. Page 13 Wuhan National Laboratory for Optoelectronics DSFS architecture

14. Page 14 Wuhan National Laboratory for Optoelectronics The salient feature of DSFS DSFS allows users to flexibly set any policy for the global ACL or even change the global ACL system but without impacting on the implementation of local PALs on storage devices. DSFS enables a network-attached storage device to immediately authorize I/O, instead of demanding a client to acquire a capability from MDS for each object she wants to access. The client can use an identity key obtained from a centralized authentication server to access any devices, including MDS, she wishes to access.

15. Page 15 Wuhan National Laboratory for Optoelectronics Authentication DSFS goal for authentication in parallel file systems is to make authentication fit into existing security infrastructure while keep the authentication process as simple as possible to ensure a high-performance I/O. DSFS separates authentication services from the file system. The client is authenticated at a dedicated authentication server that then issues her an identity key for the subsequent access to the file system. The authentication server can authenticate clients independently or in collaboration with existing security infrastructure.

16. Page 16 Wuhan National Laboratory for Optoelectronics Identity key Access to a storage device is controlled using a shared K between the storage device and authentication server. An identity key can be generate as: idkey = HMAC K (K data ) K data ={K id, identity, expiration} identity = {U id,R id } where Uid is the user identifier and Rid is the identifier of the role assumed by the user. The HMAC binds identity to idkey. Thus one can validate the clients identity by verifying the idkey.

17. Page 17 Wuhan National Laboratory for Optoelectronics Authentication protocol Only two MAC computation is required to verity the users identity at the storage device.

18. Page 18 Wuhan National Laboratory for Optoelectronics pre-authorization list DSFS stores pre- authorization lists along with each object to enable the storage device to immediately authorize I/O. A pre-authorization list is similar in form to the ACL. Two types of PALs, namely, access PAL and super PAL, are proposed for two different purposes.

19. Page 19 Wuhan National Laboratory for Optoelectronics Pre-authorization model A multiple-level inheritance model at MDS will be interpreted to a single-level sharing model on the storage devices.

20. Page 20 Wuhan National Laboratory for Optoelectronics Outline I. Introduction II. Related Work III. DSFS Architecture IV. DSFS Design on OBS V. Performance Evaluation VI. Conclusions

21. Page 21 Wuhan National Laboratory for Optoelectronics Extended PAL attribute page An advantage of storing PALs in the form of attribute page is that the existing commands of setting attributes defined in the standard can be applied to PALs.

22. Page 22 Wuhan National Laboratory for Optoelectronics PAL collection object Access PAL page can be associated with user objects and provides access control to the latter. Super PAL page can be associated with the extended PAL collection. The user objects referenced by a PAL collection can share the super PAL associated with that PAL collection.

23. Page 23 Wuhan National Laboratory for Optoelectronics OBFS structure OBFS stores relatively fixed attribute pages in the forepart of objects to achieve fast accesses to attributes. The extended attribute region stores the overrunning part of a variable-length attribute that fails to completely fit in a fixed-length attribute page.

24. Page 24 Wuhan National Laboratory for Optoelectronics Object structure and PAL layout OBFS stores a PAL attribute page in an object onode. The overrunning PAEs will be stored in the extended attribute region.

25. Page 25 Wuhan National Laboratory for Optoelectronics Outline I. Introduction II. Related Work III. DSFS Architecture IV. DSFS Design on OBS V. Performance Evaluation VI. Conclusions

26. Page 26 Wuhan National Laboratory for Optoelectronics Evaluation Goals DSFSs security overhead and benefits relative to the T10 OSD-2 security protocol and the state-of-the-art Maat security protocol; System throughput and scalability of different security setups under a high- bandwidth workload.

27. Page 27 Wuhan National Laboratory for Optoelectronics Prototype Implementation DSFS is prototyped in the HUST OSD project. Global ACLs are stored along with other metadata in a Berkeley database. The SET ATTRIBUTES command defined in the T10 standard is used to set the preauthorization lists. The CMDRSP security method defined in the T10 OSD standard is used to protect the OSD command from various network attacks. The prototype supports three security schemes: DSFS, OSD-2 and Maat.

28. Page 28 Wuhan National Laboratory for Optoelectronics Experimental Setup 3 to 17 Linux hosts Connected by a 1000 Mbit/s Ethernet One MDS, various numbers of OSDs and clients, each running on a separate machine CPU Intel XEON 3.0G Memory 512MB DDR DIMM (except MDS with 4GB physical Memory) SATA HBA Highpoint RocketRAId 2240 Disk Seagate, 300GB SATA, 7200RPM Avg. seek time 8.5ms NIC Broadcom 5721 Gigabit Ethernet controller OS RedHat AS3 2.4.21 Host configurations

29. Page 29 Wuhan National Laboratory for Optoelectronics Security Overhead DSFS achieves lower operation latencies than the T10 OSD-2 and Maat security protocols. Both OSD-2 and Maat need to maximize capability cache hits to achieve the same performance.

30. Page 30 Wuhan National Laboratory for Optoelectronics Bandwidth Read Bandwidth Write Bandwidth Both OSD-2 and DSFS setups have comparable bandwidth with the non- secure setup because the benchmark hardly yields overhead on MDS in a single-client to single-OSD system. Compared to the non-secure setup, the read and write performances with both the OSD-2 and DSFS setups decrease by less than 5%.

31. Page 31 Wuhan National Laboratory for Optoelectronics Aggregate Throughput The aggregate read throughput of all clients for the non-secure and DSFS setups scales linearly with the number of clients/OSDs. As the number of clients/OSDs scales up above 6, the aggregate read throughput in the OSD-2 system starts to flatten out. The MDSs idle CPU time of the DSFS system is comparable to that of the non- secure system. The MDSs idle CPU time with the OSD-2 system declines faster than that of the non-secure and DSFS systems because the MDS must prepare a capability for each read or write request with OSD-2. Aggregate read bandwidth with multiple clients/OSDs Average percentage of idle CPU time on MDS

32. Page 32 Wuhan National Laboratory for Optoelectronics Outline I. Introduction II. Related Work III. DSFS Architecture IV. DSFS Design on OBS V. Performance Evaluation VI. Conclusions

33. Page 33 Wuhan National Laboratory for Optoelectronics Conclusions 1. 1. DSFS, a decentralized security system is designed to provide decentralized access control for large parallel file systems. 2. 2. The design of DSFS on an object-based storage system, which is compliant with the current T10 OSD standard, is detailed. 3. 3. We prototyped DSFS in the HUST OSD project. Our implementation requires minimal changes to the current T10 OSD standard but enables the standard to support decentralized access control. 4. 4. Experimental Results show that DSFS achieves lower operation latencies than the OSD-2 and Maat security protocols. Both OSD-2 and Maat need to maximize capability cache hits to achieve the same performance.

34. Page 34 Wuhan National Laboratory for Optoelectronics Thank You!