1. Helping Kinsey Compute Cynthia Dwork Microsoft Research Cynthia Dwork Microsoft Research

2. The Problem Exploit Data, eg, Medical Insurance Database — Does smoking contribute to heart disease? — Was there a rise in asthma emergency room cases this month? — What fraction of the admissions during 2004 were men 25-35? …while preserving privacy of individuals

3. Holistic Statistics Is the dataset well clustered? What is the single best predictor for risk of stroke? How are attributes X and Y correlated; what is the cov(X,Y)? Are the data inherently low-dimensional?

4. Statistical Database Query (f,S) f: row  [0,1] S µ [n] Exact Answer  f(row r) Database (D 1, … D n ) (D 1, … D n ) f f f f  + noise

5. Statistical Database f f f f  + noise Under control of interlocutor: Noise generation Number of queries T permitted

6. Why Bother With Noise? Limiting interface to queries about large sets is insufficient: A = {1, …, n} and B = {2, …, n}  a2 A f(row a) -  b2 B f(row b) = f(row 1)

7. Previous (Modern) Work in this Model Dinur, Nissim [2003] Single binary attribute (query function f = identity) Non-privacy: whp adversary guesses 1-  rows — Theorem: Polytime non-privacy if whp |noise| is o(√n) — Theorem: Privacy with o(√n) noise if #queries is << n Privacy “for free” ! Rows » samples from underlying distribution: Pr[row i = 1] = p E[# 1’s] = pn, Var =  (n) Acutal #1’s » pn §  (√n) |Privacy-preserving noise| is o(sampling error)

8. Real Power in this Model Dwork, Nissim [2004] Multiple binary attributes q=(S,f), f:{0,1} d ! {0,1} — Definition of privacy appropriate to enriched query set — Theorem: Privacy with o(√n) noise if #queries is << n — Coined term SuLQ Vertically Partitioned Databases — Learn joint statistics from independently operated SuLQ databases: Given SulQ A, SuLQ B learn if A implies B in probability Eg, heart disease risk increases with smoking Enables learning statistics for all Boolean fns of attributes

9. Still More Power [Blum, Dwork, McSherry, Nissim 05] Extend Privacy Proofs — Real-valued functions f: [0,1] d ! [0,1] — Per row analysis: drop dependence on n! How many queries has THIS row participated in? Our Data, Ourselves Holistic Statistics: A Calculus of Noisy Computation — Beyond statistics: (not too) noisy versions of k-means, perceptron, ID3 algs (not too) noisy optimal projections SVD, PCA All of STAT learning

10. Towards Defining Privacy: “Facts of Life” vs Privacy Breach Diabetes is more likely in obese persons — Does not imply THIS obese person has or will have diabetes Sneaker color preference is correlated with political party — Does not imply THIS person in red sneakers is a Republican Half of all marriages result in divorce — Does not imply Pr [ THIS marriage will fail ] = ½

11. ( , T)-Privacy Power of adversary: Phase 0: Specify a goal function g: row  {0,1} Actually, a polynomial number of functions; Adversary will try to learn this information about someone Phase 1: Adaptively make T queries Phase 2: Choose a row i to attack; get entire database except for row i Privacy Breach: Occurs if adversary’s “confidence” in g( row i ) changes by  Notes: Adversary chooses goal My privacy is preserved even if everybody else tells their secrets to the adversary

12. Flavor of Privacy Proofs Define confidence in value of g( row i ) — c 0 = log [p 0 /(1-p 0 )] — 0 when p = ½, skyrockets as p moves toward 0 or 1 Model evolution of confidence as a martingale — Argue expected difference at each step is small — Compute absolute upper bound on difference — Plug these two parameters into Azuma’s inequality Obtain probabilistic statement regarding change in confidence, equivalently, change from prior to posterior probabilities about value of g( row i ) c0c0

13. Remainder of This Talk Description of SuLQ Algorithm + Statement of Main Theorem Examples — k means — SVD, PCA — Perceptron — STAT learning Vertically Partitioned Data — Determining if  )  in probability: Pr[  |  ] ¸ Pr[  ]+  when  and  are in different SuLQ databases Summary

14. Azuma’s Inequality Let s 1, … s T be i.i.d. such that E[s j ] ·  and |s j | · . Then Pr[|  i s i | > (  +  ) T 1/2 + T  ] · 2e - /2 We will take  = 1/2R and  = (2 log (T/  )/R) 1/2 + 1/2R 2

15. The SuLQ Algorithm Algorithm: — Input: query (S µ [n], f: [0,1] d ! [0,1]) — Output:  i 2 S f( row i ) + N(0, R) Theorem: 8 , with probability at least 1- , choosing R > 32 log(2/  ) log (T/  )T/  2 ensures that for each (target, predicate) pair, after T queries the probability that the confidence has increased by more than  is at most . R is independent of n. Bigger n means better stats.

16. k Means Clustering physics, OR, machine learning, data mining, etc.

17. SuLQ k Means Estimate size of each cluster Estimate average of points in cluster — Estimate their sum; and — Divide estimated sum by estimated average

18. Side by Side: k Means and SuLQ k-Means Basic step: Input: data points p 1,…,p n and k ‘centers’ c 1,…,c k in [0,1] d S j = points for which c j is the closest center Output: c’ j = average of points in S j, j=1, … k Basic step: Input: data points p 1,…,p n and k ‘centers’ c 1,…,c k in [0,1] d s j = SuLQ( f(d i ) := 1 if j = arg min j ||c j – d i || 0 otherwise)  ’ j = SuLQ( f(d i ) := d i if j = arg min j ||c j - d i || 0 otherwise) / s j k(1+d) queries total

19. Small Error! For each 1 · j · k, if |S j | >> R 1/2 then with high probability ||  ’ j – c’ j || is O( (||  j || + d 1/2 ) R 1/2 /|S j |). Inaccuracies: — Estimating |S j | — Summing points in S j Even with just the first: (1/s j - 1/|S j |)  I 2 S j d i = (1/s j - 1/|S j |) (  j |S j |) = ((|S j | - s j )/s j )  j ¼ (noise/size)  j

20. Reducing Dimensionality Reduce Dimensionality in a dataset while retaining those characteristics that contribute most to its variance Find Optimal Linear Projections — Latent semantic indexing, spectral clustering, etc., employ best rank k approximations to A Singular Value Decomposition uses top k eigenvectors of A T A Principal Component Analysis uses top k eigenvectors of cov(A) Approach — Approximate A T A and cov(A) using SuLQ, then compute eigenvectors

21. Optimal Projections A T A =  i d i T d i  = (  i d i )/n cov(A) =  i (d i -  ) T (d i -  ) SuLQ (f(i) = d i T d i ) = A T A + N(0,R) d £ d  ’ = SuLQ(f(i)=d i )/n SuLQ( f(i) = (d i -  ’) T (d i -  ’) ) d 2 and d 2 +d queries, respectively

22. Perceptron [Rosenblatt 57] Input: n points p 1,…,p n in [-1,1] d, and labels b 1,…,b n in {-1,1} — Assumed linearly separable, with a plane through the origin Initialize w randomly h w, p i b > 0 iff label b agrees with sign of h w, p i While 9 labeled point (p i,b i ) s.t. h w i, p i i b i · 0, s et w = w + p i ·b i Output: w pipipipi w w

23. SuLQ Perceptron Initialize w 0 = 0 d and s 0 = n. For j = 0, 1, 2, …, repeating so long as s j >> R 1/2 Count the misclassified rows (1 query): s j = SuLQ(f(d i ) := 1 if h d i, w j i b i · 0 and 0 ow) Synthesize a misclassified vector (d queries): v j = SuLQ(f(d i ) := b i d i if h d i, w j i ¢ b i · 0 and 0 ow) / s j. Update w: Set w j+1 = w j + v j. Return the final value of w.

24. SuLQ Perceptron Initialize w = 0 d and s= n. Repeat while s >> R 1/2 Count the misclassified rows (1 query) : s = SuLQ(f(d i ) := 1 if h d i, w i b i · 0 and 0 ow) Synthesize a misclassified vector (d queries) : v = SuLQ(f(d i ) := b i d i if h d i, w i ¢ b i · 0 and 0 ow) / s Update w: Set w = w + v Return the final value of w.

25. How Many Rounds? Theorem: If there exists a unit vector w’ and scalar  such that for all i hw',d i i b i ¸  and for all j,  >> (dR) 1/2 /|S j | then with high probability the algorithm terminates in at most 32 max i |d i | 2 /  rounds. |S j | = number of misclassified vectors at iteration j In each round j, hw', wi increases by more than |w| does. Since hw', wi · |w'| ¢ |w| = |w|, this must stop. Otherwise hw', wi would overtake |w|.

26. The Statistical Queries Learning Model [Kearns93] Concept c: {0,1} d  {0,1} Distribution D on {0,1} d STAT(c,D) Oracle — Query: (p,  ) where p:{0,1} d+1  {0,1} and  =1/poly(d) — Answer: Pr x  D [p(x,c(x))] +  for |  |  

27. Capturing STAT Each row contains a labeled example (x, c(x)) Input: predicate p and accuracy  Initialize tally = 0. Reduce variance: Repeat t ¸ R/  n 2 times tally = tally + SuLQ(f(d i ) := p(d i )) Output: tally / tn

28. Capturing STAT Theorem: For any algorithm that  -learns a class C using at most q statistical queries of accuracy {  1, …,  q }, the adapted algorithm can  -learn C on a SuLQ database of n elements, provided that n 2 ¸ R log(q /  )}/(T-q) £  j · q 1/  j

29. Probabilistic Implication: Two SuLQ Databases  implies  in probability: Pr[  |  ] ≥ Pr[  ]+  Construct a tester for distinguishing   2 (for constants  1 <  2 ) — Estimate  by binary search In the analysis we consider deviations from an expected value, of magnitude  (√n) — As perturbation << √n, it does not mask out these deviations Results generalize to functions  and  of attributes in two distinct SuLQ databases

30. Key Insight: Test for Pr[  |  ] ≥ Pr[  ]+  Assume T chosen so that noise = o(√n). 1. Find a “heavy” set S for  : a subset of rows that have more than |S| a +[a(1-a) |S] 1/2 ones in  database. Here, a = Pr[  ] and |S| =  (n). Find S s.t. a S,  > |S| a  + √ [|S|( a (1- a ))]. Let excess  = a S,  - |S|  a. Note that excess is  (n 1/2 ). 2. Query the SuLQ database for , on S If a S,  ¸ |S| Pr[  ] + excess  (  / (1 - a )) then return 1 else return 0 If  is constant then noise is too small to hide the correlation.

31. Probabilistic Implication – The Tester Pr[  |  ] ≥ Pr[  ]+  Distinguishing   2 : — Find a `heavy’ query (S,  ) s.t. a S,  > |S|  p  + √(n a (1- a )) Let bias  = a S,  - |S|  p  — Issue query (S,  ) If a S,  > threshold(bias , p ,  1 ) output 1 random S <1<1<1<1 >2>2>2>2 (2-1)(n)(2-1)(√n)(2-1)(n)(2-1)(√n) 10 Pr[a S,  ] a S, 

32. Summary SuLQ framework for privacy-preserving statistical databases — real-valued query functions — Variance for noise depends (roughly linearly) on number of queries, not size of database Examples of power of SuLQ calculus Vertically Partitioned Databases

33. Sources C. Dwork and K. Nissim, Privacy-Preserving Datamining on Vertically Partitioned Databases A. Blum, C. Dwork, F. McSherry, and K. Nissim, Practical Privacy: The SuLQ Framework See http://research.microsoft.com/research/sv/DabasePrivacy