Download presentation

Presentation is loading. Please wait.

Published byDestini Cumming Modified over 2 years ago

1
Secure Multiparty Regression Based on Homomorphic Encryption Rob Hall Joint work with Yuval Nardi (Technion) and Steve Fienberg 1

2
Structure Setting and motivation. Basic tools of cryptography. Prior work Techniques for regression. Logistic regression 2 Well known Our contribution

3
Multiple parties with private data: e.g., is this vaccine causing hepatitis? Long term vaccine safety surveillance (c.f., the FDAs sentinel initiative) Setting 3 Health insurance agency Hospital

4
Secure Multiparty Regression 4 Party 1 Party 2 Each party has a private (partial) data matrix Additional variables may be present

5
Secure Multiparty Regression 5 Full data Goal is regression on full data Assumptions: Complete and properly joined

6
Secure Multiparty Regression 6 Data are private e.g., HIPAA

7
Alternate Settings 7 Fictional scenario based on discussion with CyLab corporate partners: Records of transactions Records of commercial views StoreTV Network Regression of advertising effect

8
Two Types of Privacy Breach Information leakage via the computation itself: – Focus of this talk. – Dealt with via cryptographic protocols. Information leakage via the output: – Not in this talk. – Assume the parties have deemed that the regression is safe to compute. – Otherwise may use e.g., Differential Privacy. 8

9
The Ideal Scenario vs. Real Life 9 Data submitted to trusted 3 rd party. Ideal: Parties see their own data and the output.

10
The Ideal Scenario vs. Real Life 10 Data submitted to trusted 3 rd party. Trusted party computes regression, sends coefficients back to each party. Ideal: Parties see their own data and the output.

11
The Ideal Scenario vs. Real Life 11 Data submitted to trusted 3 rd party. Trusted party computes regression, sends coefficients back to each party. Ideal: Parties see their own data and the output. Real: Parties also see intermediate messages. Parties exchange messages and perform local computation according to a protocol

12
The Ideal Scenario vs. Real Life 12 Data submitted to trusted 3 rd party. Trusted party computes regression, sends coefficients back to each party. Ideal: Parties see their own data and the output. Real: Parties also see intermediate messages. Parties exchange messages and perform local computation according to a protocol Protocol is secure if intermediate messages dont reveal any information beyond whatever is contained in the output.

13
Security by Simulation 13 Consider the messages to party 1: Depends on others private inputs A distribution, since the protocol is randomized.

14
Security by Simulation 14 Consider the messages to party 1: Depends on what's available in ideal case Depends on others private inputs Suppose we construct a simulator: A distribution, since the protocol is randomized.

15
Security by Simulation 15 Consider the messages to party 1: Try to decide which one a particular transcript is from: Depends on what's available in ideal case Depends on others private inputs A poly-time algorithm Suppose we construct a simulator: A distribution, since the protocol is randomized.

16
Security by Simulation 16 Consider the messages to party 1: Try to decide which one a particular transcript is from: Depends on what's available in ideal case Depends on others private inputs A poly-time algorithm Suppose we construct a simulator: Cant decide messages reveal no more than input/output. A distribution, since the protocol is randomized.

17
Computational Indistinguishability 17 Negligible function of a security parameter k Probability over transcripts and coin tosses of A Probability that decision is correct 0.5

18
Computational Indistinguishability 18 Negligible function of a security parameter k Probability over transcripts and coin tosses of A Probability that decision is correct 0.5 A proper relaxation of statistical closeness: Polynomially (in k) many secure sub-protocols may be composed.

19
Basic Tools 19 Uniformly distributed among all solutions. Hide intermediate values as random shares: Intermediate value One share per party Sums may be computed locally

20
Basic Tools 20 Use a sub-protocol for computing products of shares: Uniformly distributed among all solutions. Hide intermediate values as random shares: Intermediate value One share per party

21
Basic Tools 21 Use a sub-protocol for computing products of shares: Uniformly distributed among all solutions. Random shares easy to simulate. Sub protocols compose yielding secure protocol. Uniformly distributed among all solutions. Hide intermediate values as random shares: Intermediate value One share per party

22
Basic Tools 22 Homomorphic encryption (e.g., Paillier 99) Public key (like e.g., RSA) Ciphertexts are indistinguishable. Allows math operations on encrypted values: (note, on ring mod n) Allows construction of the product sub-protocol… n 2 k Security parameter Public key

23
23 Secure Products (Integer) Party 1 (has private key) Party 2 Data held by party 2 Data held by party 1

24
24 Secure Products (Integer) Party 1 (has private key) Party 2 Encrypt values and send them.

25
25 Secure Products (Integer) Party 1 (has private key) Party 2 Draw r uniformly at random

26
26 Secure Products (Integer) Party 1 (has private key) Party 2 Decrypt, add local product

27
27 Secure Products (Integer) Party 1 (has private key) Party 2 Share of product

28
28 Secure Products (Integer) Party 1 (has private key) Party 2 Share of product Encrypted Uniform random variable

29
Yaos Construction In principle may now evaluate any circuit: 29 xor, and for binary a,b

30
Yaos Construction In principle may now evaluate any circuit: 30 This is essentially a theoretical construction (nevertheless it is implemented in practice c.f., fairplay). To accomplish even a floating point addition would take many encryptions. xor, and for binary a,b

31
Prior Work in Secure Multiparty Regression 31 Inner products Matrix inversion Inner products Linear regression is sums and products (with tricks) Chris Clifton et. al: Inner product protocols for a weak definition of secure. Alan Karr et. al: Compute, share them. This work: A secure protocol which reveals only the output All reveal some info in addition to the estimate

32
Input Data Setup We suppose the data obey the following: Subsumes all data partitioning schemes. Leads to a general protocol for all situations. – Although, specialized protocols may be faster. 32 X data of party iFull data

33
Our Protocol Yaos approach: very clean but inefficient. Our approach: messy but fast(er)… – Fixed precision arithmetic. 33 Mostly sums and products. Sadly: real numbers not integers

34
Secure Products (Real Approx) Approximate reals with integers: 34 The real numberInteger representation

35
Secure Products (Real Approx) Approximate reals with integers: Using the previous method is wrong: Need to divide off 35 The real numberInteger representation Decimal point is pushed left

36
Secure Products (Real Approx) Approximate reals with integers: Using the previous method is wrong: Cant just correct shares locally: 36 The real numberInteger representation Extra term due to mod in definition of RS

37
Secure Products (Real Approx) Approximate reals with integers: Using the previous method is wrong: Cant just correct shares locally: 37 The real numberInteger representation Extra term due to mod in definition of RS Proposed solution: Assume bound on magnitude of product (mild assumption) Restrict domain of noise to ensure that c = 1 Correct the results of locally dividing shares. Shares remain C.I. from uniform distribution

38
Our Protocol We can do sums and products on reals and everything composes nicely! 38 Matrix inversion is all we need

39
Inversion by Sums and Products 39 Computing the reciprocal of a The zero of this function is x = a -1

40
Inversion by Sums and Products 40 f(x) = a -1 Computing the reciprocal of a Use Newtons method Convergence is quadratic if 0 < x 0 < a -1

41
Inversion by Sums and Products 41 f(x) = a -1 Use Newtons method Convergence is quadratic if 0 < x 0 < a -1 Inverting the matrix A Sums and products Number of iterations required depends on condition of A Computing the reciprocal of a

42
Putting it Together 42 Step 1: Compute (shares of) X T X, X T y Easy to parallelize by slicing X horizontally Step 2: Compute shares of inverse Step 3: Multiply shares of inverse with shares of X T y Use reciprocal of trace as starting point. Step 4: Pool final shares and construct output.

43
CPS - Experimental Verification Survey data with samples, 22 covariates. Artificially split into 3 parties holding 10,8,4 covariates respectively (for all cases). Using 1024 bit long keys. Computation of X T X, X T y parallelized on 9 CPUs, takes roughly 1.5 days. Matrix inversion takes 1 hour. 43

44
Logistic Regression Iteratively Re-weighted Least Squares: A non-linear thing to compute: Repeated matrix inversion 44 Similar to linear regression….except:

45
Logistic Regression 45 Think of these as variables to update

46
Logistic Regression 46 Use Eulers method to integrate the gradient Multiple steps, per iteration Introduces some error

47
Logistic Regression 47 Multiple steps, per iteration Introduces some error Gradient only involves sums and products. Use Eulers method to integrate the gradient

48
Logistic Regression Avoid repeated matrix inversion: 48 Invert only once (see e.g., Tom Minka)

49
Logistic Regression Avoid repeated matrix inversion: Algorithm converges and has following property: 49 Invert only once (see e.g., Tom Minka) Distance between optimizer of approximation and IRLS Data dependent constant Number of steps of Eulers

50
Logistic Regression 50

51
Summary Intro to cryptographic protocols. Secure product protocol. Our linear regression protocol: – Approximation of real math with integer math. – Reduction of matrix inverse to sums and products. Our logistic regression protocol: – Approximation of logistic function by sums and products. 51

52
Ongoing Work Record linkage Implementation (R bindings?) Regression variants – LARS, Lasso etc. Privacy implications of regression coefficients. 52

53
Thanks 53

54
Privacy Implications 54 The (2 party) protocol computes the estimate: At the end, party 1 may conclude that the data of party 2 falls into the set: e.g., invertible implies total privacy invasion

55
Privacy Implications (Vertical) 55 Consider the partitioning scheme: The OLS estimate may be written as:

56
Privacy Implications (Vertical) 56 Consider the partitioning scheme: The OLS estimate may be written as: We may express M in terms of its projection onto X 1

57
Privacy Implications (Vertical) 57 Consider the partitioning scheme: The OLS estimate may be written as: We may express M in terms of its projection onto X 1 Grinding out the maths gives:

58
Privacy Implications (Vertical) 58 Express M 2 in terms of the new variables: q = 1 means A is revealed

59
Ongoing Work Logistic Regression (done but slow). Lasso, LARs etc. Record linkage (assumed here). Imputation of missing data. Secure computation of goodness-of-fit statistics. 59

60
Questions For the technical details and code please see: 60

61
Logistic Regression (IRLS) Newton-Raphson iterates: Approximate sigmoid by the empirical CDF: Secure computation of greater than is well known. Approximation error decreases with. 61

62
CPS - Experimental Verification 62 No. in Household

63
CPS - Experimental Verification 63 Age(3)

64
Alternative Approaches 64 Parties sanitize data Release Sanitized Data i.e., transform, the data into something they are willing to release

65
Alternative Approaches 65 Sanitization scheme may affect estimator Parties sanitize data Release Sanitized Data Data are pooled

66
Alternative Approaches 66 ? Sanitization scheme may affect estimator Output the correct result Distributed computation that ensures privacy Parties sanitize data Secure Multiparty Computation Release Sanitized Data Data are pooled

67
Yaos Protocol Theoretically can now compute anything! How: – Compose sums and products in mod 2. – Corresponds to xor and and. – Sufficient to compute any circuit. 67 Theoretically, were done already … but

68
Yaos Protocol Theoretically can now compute anything! How: – Compose sums and products in mod 2. – Corresponds to xor and and. – Sufficient to compute any circuit. 68 Theoretically, were done already … but Leads to very slow protocols!

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google