Milestone 2 Workshop in Information Security – Distributed Databases Project Access Control Security vs. Performance By: Yosi Barad, Ainat Chervin and Ilia Oshmiansky 1 Project web site:

Milestone 2: 2 Our Plan: Examine the Cassandra source code and research different implementation options Write some initial code Test our implementation using YCSB++ with basic configuration Compare the test results to the results of our initial tests

Milestone 2: 3 Our Plan: Try to implement several different solutions to be tested against each other Set up a more advanced configuration of Cassandra Test the performance of Cassandra with and without the added Cell-level Security

Evaluate how to further improve the implementation Run more advanced set-ups of YCSB++ (custom workloads) Create a final version of the implementation based on the test results Milestone 2: 4 Our Plan:

Plan Step 1: 5 Examine the Cassandra source code and research different implementation options Steps: Fully understand the Cassandra data structure Cassandra terminology- - A "Cell" is actually called a "Column" it is the lowest/smallest increment of data and is represented as a tuple (triplet) that contains a name, a value and a timestamp.

Plan Step 1: 6 Examine the Cassandra source code and research different implementation options -Next we have a "Column Family" which is a container for rows, and can be thought of as a table in a relational system. Each row in a column family can be referenced by its key.

Plan Step 1: 7 Examine the Cassandra source code and research different implementation options -Finally, a "Keyspace" is a container for column families. It's roughly the same as a schema or database that is, a logical collection of tables.

Plan Step 1: 8 Examine the Cassandra source code and research different implementation options Understand the code flow of get and set operations. The security logic is separate from the rest of the code and has two main interfaces: 1) Iauthenticator – responsible for authenticating the user that logs in. 2) Iauthority – responsible for authorizing the access of an authenticated user to a specific resource. It is possible to write our own code which implements these Interfaces.

Plan Step 1: 9 Examine the Cassandra source code and research different implementation options The Initial code had the following classes: "AllowAllAuthentication.java“ - implements Iauthenticator. "AllowAllAuthorization.java“ - implements Iauthority. With some research we found: "SimplelAuthentication.java" "SimpleAuthorization.java” That allow some simple user authentication and a basic ACL.

Plan Step 1: 10 Examine the Cassandra source code and research different implementation options Some information about these classes: use two additional configuration files: password.properties – a list of users and their passwords. access.properties – contains a set of permissions.

Plan Step 1: 11 Examine the Cassandra source code and research different implementation options There are several problems with this implementation: Very inefficient Need a lot of maintenance. Despite these issues we still believed we should use this code as a starting point having full intentions to improve it.

12 Write some initial code Plan Step 2: The initial code we wrote: Implemented the RowKey access control. At this point we could limit the access of Read/Write to a specific Row within a ColumnFamily. Keyspace1.Users.key1. =yosi

13 Write some initial code Plan Step 2: Implemented the Column access control. Same as The RowKey access control, this time we went a level lower. Keyspace1.Users.key1.column1 =yosi

14 Write some initial code Plan Step 2: We set a new syntax to the Cassandra client: : [ ] [...]. For example: Set Users['key1']['City'] = 'Haifa:scott,yosi rw:Andrew ro'; This command does the following: 1.creates a new column named "City" with value "Haifa" in column family “Users“. 2.Writes the permissions to the access.control file in the correct format. In the following example we will add the following two lines to the access.control file: Keyspace1.Users.key1.City =scott,yosi Keyspace1.Users.key1. City =Andrew

15 Test our implementation using YCSB++ with basic configuration Plan Step 3: We ran one basic test on it to get an idea of where we stand in terms of performance. The results we got were terrifying (as expected): An average of under 40 op/sec and it got lower the more tests we ran every new entry meant another line in the file that we need to scan. Our next step was to improve the implementation so that it won't rely on configuration files.

16 Try to implement several different solutions to be tested against each other Plan Step 4: Our two implementations were: 1) Writing the permissions to a file. 2) Storing the permissions within the values of the columns in Cassandra.

Plan Step 5: This stage included the following: Parse the value returned from Cassandra. Add a new "get" function to grab and separate the ACL from the actual value of a Column. For example: Yosi wants to run the following command - This command will now work as following: 1)Get the value and check the ACL in the value 2)Perform the validation 3)Perform the actual insert. Storing the permissions within the values of the columns in Cassandra.

18 Try to implement several different solutions to be tested against each other Milestone 2 Story to be told - version 1 יוסי הקפדן איליה הסקרן עינת המגניבה VerySecretValue:Yosi,Ainat rw Won’t be able to access VerySecretValue Access and modify VerySecretValue

19 Try to implement several different solutions to be tested against each other Milestone 2 Story to be told – version 2 יוסי הקפדן איליה הסקרן עינת המגניבה VerySecretValue:Yosi,Ainat rw:Ilia ro Access but not modify VerySecretValue Access and modify VerySecretValue

Milestone 2 20 Demonstration

Plan Step 5: In order to further enhance the performance: We removed the "SimpleAuthority" related functions. We changed the implementation of the "authorize" function in the SimpleAuthority class to read the ACL from the value within the Cassandra DB rather than from the access.Properties file. Storing the permissions within the values of the columns in Cassandra.

22 Run more advanced set-ups of YCSB++ (custom workloads) Plan Step 5: To test our implementation we had to: - Add ACL entries to the values YCSB sent to Cassandra - Get YCSB to login to Cassandra with a user and password This included the following steps: -Compile YCSB (this was a challenge since this code has no documentation anywhere) -Edit the YCSB code to connect to Cassandra with our user. -Change the way YCSB generated values to fit our custom format ( : : ). -Recompile it with different number of ACL entries for our "increasing ACL" test. Now we got much better results:

Workload A: Update heavy workload - mix of 50/50 reads and writes. Plan Step 5: 23

Workload B: Read mostly workload – This workload has a 95/5 reads/write mix Plan Step 5: 24

Workload C: Read only - This workload is 100% read. Plan Step 5: 25

Workload D: Read latest workload - In this workload, new records are inserted, and the most recently inserted records are the most popular. Plan Step 5: 26

Workload F: Read-modify-write - In this workload, the client will read a record, modify it, and write back the changes. Plan Step 5: 27

28 Compare the test results to the results of our initial tests Plan Step 5: The current results aren't satisfying as they do not sit well with what was expected (we expected the throughput to decrease with each additional entry).

29 Set up a more advanced configuration of Cassandra (consisting of several nodes) Plan Step 6: We realized that we rather wait for a local HD allocation for running several Cassandra nodes because: - A shared hard drive would be the bottleneck and won't increase the performance - It is very hard to benchmark this remote storage. - It is time consuming to set-up the clusters and if we'll get the local HD allocation we might spend more time on building this configuration again. We sent a request to the system admin for local HD allocations so we could install Cassandra and test performance running on a local HD.

30 Test the performance of the advanced configuration of Cassandra with and without the added Cell-level Security Plan Step 7: Once we will finish the more advanced set-up, we will be able to run both Cassandra implementations (with and without the added security) and thus get the desired results.

31 Create a final version of the implementation based on the test results Plan Step 10: We would like to further analyze the code and find ways to improve it (see plans for ahead). Furthermore, we cannot rely on the tests we ran so far as they do not accurately assess the performance.

Progress Compared to Plan: Milestone 2 32 Plan StepStatus Examine the Cassandra source code and research different implementation options Write some initial code Test our implementation using YCSB++ with basic configuration Compare the test results to the results of our initial tests Try to implement several different solutions to be tested against each other Set up a more advanced configuration of Cassandra (consisting of several nodes) Test the performance of the advanced configuration of Cassandra with and without the added Cell-level Security Evaluate how to further improve the implementation Run more advanced set-ups of YCSB++ (custom workloads) Create a final version of the implementation based on the test results

We completed the goal of implementing cell-level ACL security, but there is still some work to be done in the performance testing and perhaps the code can be further improved. Milestone 2 33 Overall

Milestone 2 34 Plans for ahead Expand the Cassandra setup – Upon receiving the local HD allocation we requested we can continue with the more advanced testing and create a setup consisting of several nodes/clusters. Expand the tests, search for limiting factors- We plan on expending our tests in several directions.

Milestone 2 35 Plans for ahead Evaluate how to further improve the implementation – At this point we do not see any major issues in our implementation. Also, we will have to run better tests to understand the actual performance penalty of our implementation and perhaps need some guidance to see where we can improve. Start analyzing security holes due to inconsistencies – We need to figure out how to measure the inconsistencies using YCSB++ and assess the security threats that might arise from these inconsistencies.

Milestone 2 36 Questions?