Online Auditing Kobbi Nissim Microsoft Based on a position paper with Nina Mishra

2 The Setting Dataset: {d 1,…,d n } –Entries d i : Real, Integer, Boolean Query: q = (f,i 1,…,i k ) –f : Min, Max, Median, Sum, Average, Count… Some users are bad… Statistical database f (d i1,…,d ik ) q = (f,i 1,…,i k )

3 Auditing Statistical database Query log q 1,…,q i Here’s a new query: q i+1 Here’s the answer Query denied (as the answer would cause privacy loss) OR Auditor

4 Auditing [Adam, Wortmann 89] classify auditing as a query restriction method –Such methods limit the queries users may post, usually imposing some structure (e.g. combinatorial, algebraic) –“Auditing of an SDB involves keeping up-to-date logs of all queries made by each user (not the data involved) and constantly checking for possible compromise whenever a new query is issued” Partial motivation: May allow for more queries to be posed, if no privacy threat occurs Early work: Hofmann 1977, Schlorer 1976, Chin, Ozsoyoglu 1981, 1986 Recent interest: Kleinberg, Papadimitriou, Raghavan 2000, Li, Wang, Wang, Jajodia 2002, Jonsson, Krokhin 2003

5 Design choices in Prior Work Out of the scope for this talk (but important): –Very weak privacy guarantee: Privacy breached (only) when a database entry may be uniquely deduced –Exact answers given Important for this talk: –Data taken into account in decision procedure Answers to q 1,…,q i and q i+1 taken into account Denials ignored

6 Some Prior Work on Auditors DataQueriesBreachComplexity Sum/Max [Chin] realSum/maxd i learnedNP-hard Boolean [KPR00] 0/1Sum--”--NP-hard Max [KPR00]RealMax--”--PTIME Interval based [LWWJ02] d i  [a,b] sumd i within accuracy . PTIME Generalized results [JK03] NP-hard / PTIME

7 Example 1: Sum/Max auditing Oh well… q1 = sum(d1,d2,d3) sum(d1,d2,d3) = 15 q2 = max(d1,d2,d3) Denied (the answer would cause privacy loss) q2 is denied iff d1=d2=d3 = 5 I win! d i real, sum/max queries Auditor

8 Example 2: Interval Based Auditing q1 = sum(d1,d2) Sorry, denied q2 = sum(d2,d3) sum(d2,d3) = 50 d1,d2  [0,1] d3  [49,50] d i  [0,100], sum queries,  =1 (PTIME) Auditor

9 Sounds Familiar? On the advice of my counsel I respectfully and regretfully decline to answer the question based on my constitutional rights. Colonel Oliver North, on the Iran-Contra Arms Deal: Mr. Chairman, I would like to answer the committee's questions, but on the advice of my counsel I respectfully decline to answer the question based on the protection afforded me under the Constitution of the United States. David Duncan, Former auditor for Enron and partner in Andersen:

10 What about Max Auditing? q1 = max(d1,d2,d3,d4) M 1234 d i real M 123 / denied If denied: d4=M 1234 M 12 / denied If denied: d3=M 123 Recover 1/8 of the database! Auditor q2 = max(d1,d2,d3) q2 = max(d1,d2) d1d2d4d6d3d5d7d8…dndn d n-1

11 What about Boolean Auditing? 1 / denied q i denied iff d i = d i+1  learn database/complement Auditor … 1 / 2 Recover the entire database! Let d i,d j,d k not all equal, where q i-1, q i, q j-1, q j, q k-1, q k all denied d i Boolean d1d2d4d6d3d5d7d8…dndn d n-1 q1 = sum(d1,d2) q2=sum(d2,d3) q2=sum(d i,d j,d k )

12 What are the Problems? Obvious problem: denied queries ignored –Algorithmic problem: not clear how to incorporate denials in the deicion Subtle problem: –Query denials leak (potentially sensitive) information Users cannot decide denials by themselves Possible assignments to {d 1,…,d n } Assignments consistent with (q 1,…q i ) q i+1 denied

13 Sum/Max, Interval based, Boolean, Max Cell suppression k-anonimity q 1,…,q i, q i+1 a 1,…,a i, a i+1 q 1,…,q i, q i+1 a 1,…,a i A Spectrum of Auditors Size overlap restriction Algebraic structure q 1,…,q i, q i+1 ExamplesDecision data “safe” “unsafe” *Note: can work in “unsafe” region, but need to prove denials do not leak crucial information

14 Simulatable Auditing* An auditor is simulatable if a simulator exists s.t.: Auditor q i+1  Deny/answer Simulator Simulation  denials do not leak information * `self auditors’ in [DN03] q 1,…,q i a 1,…,a i Statistical database q 1,…,q i

15 Summary Subtleties in current definition of auditors allow for information leakage, and potentially, privacy breaches –Denials are not taken into account –Auditor uses information not available to user Simulatable auditors provably don’t leak information in decision –New starting point for research on auditors

16 A Spectrum of Auditors Sum/Max, Interval based, Boolean, Max q 1,…,q i, q i+1 a 1,…,a i, a i+1 Size overlap restriction Algebraic structure q 1,…,q i, q i+1 ExamplesDecision data Cell Suppression k-anonimity

17 Sounds Familiar? On the advice of my counsel I respectfully and regretfully decline to answer the question based on my constitutional rights. Colonel Oliver North, on the Iran-Contra Arms Deal: I would like to answer the committee's questions, but on the advice of my counsel, I respectfully decline to answer the questions based on the protection afforded me under the Constitution of the United States. Andrew Fastow, CFO, Enron Corporation: