1. Executing SQL over Encrypted Data in Database-Service-Provider Model Hakan Hacigumus University of California, Irvine Bala Iyer IBM Silicon Valley Lab. Chen Li University of California, Irvine Sharad Mehrotra University of California, Irvine SIGMOD 2002, Madison, Wisconsin, USA

2. 2 What do we want to do? We want to store the data on “a server” User Encrypted User Database Server User Data But the problem is we do not trust “the server” for sensitive information! encrypt the data and store it but still be able to run queries over the encrypted data do most of the work at the server If the server is trusted, ICDE 2002 Distrusted

3. 3 Why is it important anyway? Application Service Provider (ASP) Model for Database DB management transferred to service provider for backup, administration, restoration, space management, upgrades etc. use the database “as a service” provided by an ASP use SW, HW, human resources of ASP, instead of your own User Encrypted User Database (Distrusted) Application Service Provider User Data Distrusted Server

4. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

5. 5 Service Provider Architecture Encrypted User Database Query Translator Server Site Temporary Results Query Executer Metadata Original Query Server Side Query Encrypted Results Actual Results Service Provider User Client Site Client Side Query ? ? ?

6. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

7. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

8. 8 Relational Encryption NAMESALARYPID John500002 Marry1100002 James950003 Lisa1050004 etupleN_IDS_IDP_ID fErf!$Q!!vddf>></|50110 F%3w&%gfErf!$65210 &%gfsdf$%343v<l50220 %33w&%gfs##!65220 Server Site Store an encrypted string – etuple – for each tuple in the original table This is called “row level encryption” Any kind of encryption technique can be used Blowfish encryption algorithm is used for this work Create an index for each (or selected) attribute(s) in the original table

9. 9 Building the Index: Partition and Identification Functions Partition function divides domain values into partitions (buckets) Partition (R.A) = { [0,200], (200,400], (400,600], (600,800], (800,1000] } partitioning function has an impact on performance as well as privacy 20004006008001000 27514 Domain Values Partition (Bucket) ids Identification function assigns a partition id to each partition of attribute A e.g.ident R.A ( (200,400] ) = 7 Any function can be use as identification function, e.g., hash functions

10. 10 Mapping Functions Mapping function maps a value v in the domain of attribute A to the id of the partition which value v belongs to e.g.Map R.A ( 250 ) = 7, Map R.A ( 620 ) = 1 20004006008001000 27514 Domain Values Partition (Bucket) ids

11. 11 Storing Encrypted Data R=  R S = etuple = encrypt ( A | B | C ) A_id = Map R.A ( A ), B_id = Map R.B ( B ), C_id = Map R.C ( C ) NAMESALARYPID John500002 Marry1100002 James950003 Lisa1050004 EtupleN_IDS_IDP_ID fErf!$Q!!vddf>></|50110 F%3w&%gfErf!$65210 &%gfsdf$%343v<l50220 %33w&%gfs##!65220 Table: EMPLOYEE Table: EMPLOYEE S

12. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

13. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

14. 14 Mapping Conditions Q: SELECT name, pname FROM emp, proj WHERE emp.pid=proj.pid AND salary > 100k Server stores attribute indices determined by mapping functions Client stores metadata and utilizes that to translate the query Conditions: Condition  Attribute op Value Condition  Attribute op Attribute Condition  (Condition  Condition) | (Condition  Condition) | (not Condition)

15. 15 Mapping Conditions (2) Example: Attribute = Value Map cond ( A = v )  A S = Map A ( v ) Map cond ( A = 250 )  A S = 7 20004006008001000 27514 Domain Values Partition Ids

16. 16 Mapping Conditions (3) Attribute1 = Attribute2 Map cond ( A = B )   N (A S = ident A ( p k )  B S = ident B ( p l )) where N is p k  partition (A), p l  partition (B), p k  p l   PartitionsA_id [0,100]2 (100,200]4 (200,300]3 PartitionsB_id [0,200]9 (200,400]8 C : A = B  C’ : (A S = 2  B S = 9)  (A S = 4  B S = 9)  (A S = 3  B S = 8)

17. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

18. 18 Relational Operators over Encrypted Relations Partition the computation of the operators across client and server Compute (possibly) superset of answers at the server Filter the answers at the client Objective : minimize the work at the client and process the answers as soon as they arrive without requiring storage at the client Operators studied: Selection Join Grouping and Aggregation Sorting Duplicate Elimination Set Difference Union Projection

19. 19 Selection Operator  A=250 TABLE 20004006008001000 27514 Example:  A=250 D E_TABLE  A_id = 7 Client Query Server Query  c ( R ) =  c ( D (  S Mapcond(c) ( R S ) )

20. 20 Join Operator C EMPPROJ C : A = B  C’ :(A_id = 2  B_id = 9)  (A_id = 4  B_id = 9)  (A_id = 3  B_id = 8) PartitionsA_id [0,100]2 (100,200]4 (200,300]3 PartitionsB_id [0,200]9 (200,400]8 R c T =  c ( D ( R S S Mapcond(c) T S ) Example: C’ E_EMPE_PROJ  A=B D Client Query Server Query

21. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

22. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

23. 23 Query Decomposition Client Query Q: SELECT name, pname FROM emp, proj WHERE emp.pid=proj.pid AND salary > 100k Server Query Encrypted (EMP) Encrypted (PROJ)  salary >100k  name,pname D D e.pid = p.pid EMP PROJ  salary >100k  name,pname e.pid = p.pid

24. 24 Query Decomposition (2) E_EMP E_PROJ  salary >100k D D E_EMP E_PROJ  salary >100k D D  s_id = 1 v s_id = 2 e.pid = p.pid  name,pname Client Query Server Query Client Query Server Query

25. 25 Query Decomposition (3) e.p_id = p.p_id E_EMP E_PROJ  salary >100k  e.pid = p.pid D  s_id = 1 v s_id = 2 e.pid = p.pid E_EMP E_PROJ  salary >100k D D  s_id = 1 v s_id = 2  name,pname Client Query Server Query

26. 26 Query Decomposition (4) Q: SELECT name, pname FROM emp, proj WHERE emp.pid=proj.pid AND salary > 100k Q S : SELECT e_emp.etuple, e_proj.etuple FROM e_emp, e_proj WHERE e.p_id=p.p_id AND s_id = 1 OR s_id = 2 Q C : SELECT name, pname FROM temp WHERE emp.pid=proj.pid AND salary > 100k e.p_id = p.p_id E_EMP E_PROJ  salary >100k  e.pid = p.pid D  s_id = 1 v s_id = 2  name,pname Client Query Server Query

27. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

28. Talk Outline Service Provider Architecture How to create Metadata: Relational Encryption and Storage Model Query Decomposition and Relational Operators Query Decomposition – Examples Experimental Results Conclusion

29. 29 Experimental Evaluation Data TPC-H database, scale factor 0.1 Queries TPC-H Queries, versions of Q#6 and Q#3 Partitioning Strategy Equi-depth histograms for the first set of experiments Equi-width histograms for the second set of experiments

30. 30 Effect of Number of Buckets in Non-Join Query Client and communications costs decreases with increasing number of buckets due to better filtering at the server Server cost doesn’t decrease as much, table scan remains best choice in the optimizer

31. 31 Effect of Number of Buckets in Non-Join Query Single Server: Server is trusted and performs all operations including decryption on site Shows that proposed query execution protocol doesn’t introduce significant overhead

32. 32 Effect of Number of Buckets in Join Query Sharp decrease in query response time with increase in the number of buckets due to better filtering at the server Client side query response time is greater than server side query response time due to dominant decryption cost on the query (second graph)

33. 33 Effect of Number of Buckets in Join Query Single Server: Server is trusted and performs all operations including decryption on site Consistent with the previous results showing proposed communication protocol doesn’t introduce significant overhead

34. 34 Conclusion ASP model is a promising solution for enterprise computing in Internet era We studied data privacy problem in the context of ASP model when the ASP is not trusted Proposed solution encrypts data, creates “coarse indexes” and stores the data at ASP allows only data owner to decrypt the data With query decomposition most of query execution performed at ASP client only performs filtering and continues to benefit from ASP model