1. KNOWLEDGE-ORIENTED MULTIPARTY COMPUTATION Piotr (Peter) Mardziel, Michael Hicks, Jonathan Katz, Mudhakar Srivatsa (IBM TJ Watson)

2. Secure multi-party computation Multiple parties have secrets to protect. Want to compute some function over their secrets without revealing them. 2 x1x1 x2x2 Q 1 (x 1,x 2 ) True / False Q 1 = if x 1 ≥ x 2 then out := True else out := False

3. T Secure multi-party computation Use trusted third party. 3 x1x1 x2x2 Q 1 (x 1,x 2 ) True Q 1 = if x 1 ≥ x 2 then out := True else out := False

4. Secure multi-party computation SMC lets the participants compute this without a trusted third party. 4 T x1x1 x2x2 Q 1 (x 1,x 2 ) True Q 1 = if x 1 ≥ x 2 then out := True else out := False

5. Secure multi-party computation Nothing is learned beyond what is implied by the query output. Assume it is publicly known that 10 ≤ x 1,x 2 ≤ 100 5 implies x 2 = 10 x1x1 x2x2 Q 1 (10,x 2 ) True Q 1 = if x 1 ≥ x 2 then out := True else out := False

6. Our goal Make sure what is implied is not too much. Model knowledge. Model inference. 6 Q 1 = if x 1 ≥ x 2 then out := True else out := False x1x1 x2x2 Q 1 (x 1,x 2 ) True

7. This talk Secure multiparty computation. Knowledge-based security For a simpler setting For SMC Evaluation 7

8. Knowledge in a simpler setting 8

9. Only one party, A 2, has a secret to protect. No need for SMC. 9 x 1 =80 x 2 =60 A2A2 A1A1 Q 1 (x 1,x 2 ) True Q 1 = if x 1 ≥ x 2 then out := True else out := False

10. Knowledge in a simpler setting A 2 imposes a limit on knowledge about x 2. 10 x 1 =80 A1A1 δ 1 : 10 ≤ x 2 ≤ 100 δ’ 1 : 10 ≤ x 2 ≤ 80 out = True A2A2 x 2 =60 Q 1 = if x 1 ≥ x 2 then out := True else out := False “(prior) belief” “revised belief” “revision” δ 1 | (out = True)

11. Knowledge in a simpler setting A 2 imposes a limit on knowledge about x 2. 11 δ’ 1 : 10 ≤ x 2 ≤ 80 “Knowledge-based” policy: | δ’ 1 | = 71 ≥ t x 1 =80 A1A1 A2A2 x 2 =60 Q 1 = if x 1 ≥ x 2 then out := True else out := False

12. Knowledge in a simpler setting Non-deterministic queries. 12 Q’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := True x 1 =80 A1A1 A2A2 x 2 =60

13. Knowledge in a simpler setting Non-deterministic queries. 13 Q’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := True δ 1 (x 2 ) = 1/91 for 10 ≤ x 2 ≤ 100 out = True δ’ 1 (x 2 ) = 2/162 for 10 ≤ x 2 ≤ 80 δ’ 1 (x 2 ) = 1/162 for 81 ≤ x 2 ≤ 100 x 1 =80 A1A1 A2A2 x 2 =60

14. Q’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := True x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting Policy 14 Q’ 1 (80,60) = True δ 1 | (out = True) = δ’ 1 δ’ 1 (x 2 ) = 2/162 for 10 ≤ x 2 ≤ 80 δ’ 1 (x 2 ) = 1/162 for 81 ≤ x 2 ≤ 100 Policy?: δ’ 1 (x 2 ) ≤ t 2 for every x 2 “belief threshold”

15. Q’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := True x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting Policy 15 Q’ 1 (80,60) = True δ 1 | (out = True) = δ’ 1 δ’ 1 (x 2 ) = 2/162 for 10 ≤ x 2 ≤ 80 δ’ 1 (x 2 ) = 1/162 for 81 ≤ x 2 ≤ 100 Policy?: δ’ 1 (x 2 ) ≤ t 2 for every x 2 “belief threshold” o ∀ o in range Q’ 1 (80,  )

16. x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting Policy. 16 δ 1 | ( out = True ) δ 1 | ( out = False ) “max belief” = max δ’,x { δ’(x) } where δ’ = δ 1 | (out = o) for some o Policy: P(Q’ 1,x 1 =80,δ 1,t) = max belief ≤ t “(max) belief threshold” Q’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := True If successful Q’ 1 (80,60) = True Track δ 1 | ( out = True )

17. x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting 17 Q1Q1 δ  δ’ out = True δ’  δ’’ NOPE Q2Q2 TIME A 2 maintains a representation of A 1 ’s belief. Assumption: initial belief is accurate.

18. PL? Theory of Clarkson et al. Model knowledge as a probability distribution δ Assumption: δ is agent’s actual knowledge Model rational agent learning from query outputs. Probabilistic program semantics and revision. δ’ = ( [[S]] δ ) | (out = True) 18 P’ = ( [[ S ]] P ) | (out = True) Sound: δ ∊ γ(P)  δ’ ∊ γ(P’) Implementation Resistant to state-space size Ex. | support(δ) | > 2 * 10 13 ∊ γ(P), an abstraction actual knowledge ∊ γ(P) (Abstract) Policy to limit knowledge: max-belief ≤ t Sound: max-belief(P) ≤ t  max-belief(δ) ≤ t

19. Knowledge in the SMC setting 19

20. Knowledge in the SMC setting All parties want to protect their secret. 20 x 1 =80 x 2 =60 A2A2 A1A1 Q 1 (x 1,x 2 )

21. Knowledge in the SMC setting All parties want to protect their secret. 21 x 1 =80 x 2 =60 A2A2 A1A1 Q 1 (x 1,x 2 ) True

22. Knowledge in the SMC setting Assumption: common knowledge/belief. 22 x 1 =80 A1A1 x 2 =60 A2A2 δ(x 1,x 2 ) = 1/91 2 10 ≤ x 1,x 2 ≤ 100

23. Knowledge in the SMC setting Assumption: initial belief is derived from common knowledge, revised by secret value. 23 x 1 =80 A1A1 x 2 =60 A2A2 δ | (x 1 = 80) = δ 1 80 (x 2 ) = 1/91 10 ≤ x 2 ≤ 100 δ | (x 2 = 60) = δ 2 60 (x 1 ) = 1/91 10 ≤ x 1 ≤ 100

24. Belief sets A 2 considers all possible values of x 1 24 x 2 =60 A2A2 δ 1 10 = δ | (x 1 = 10) x 1 =10 A1A1 10 ≤ x 1 ≤ 100 x 1 =11x 1 =100 δ 1 11 = δ | (x 1 = 11) δ 1 100 = δ | (x 1 = 100) …

25. Belief sets A 2 considers all possible values of x 1 25 x 2 =60 A2A2 A1A1 Δ = { δ 1 x }

26. Belief sets A 2 conservatively enforces max belief threshold. 26 x 2 =60 A2A2 δ 1 10  δ’ 1 10 x 1 =10 A1A1 x 1 =11 … x 1 =80 A1A1 Q δ 1 11  δ’ 1 11 max belief ≤ t

27. Belief sets A 2 maintains belief set. A 1 does similarly. 27 x 2 =60 A2A2 10 ≤ x 1 ≤ 100 A1A1 Δ 1 = { δ 1 x } x A1A1 policy P 2 A1A1 Δ’ 1 = { δ 1 x | (out = True) } x policy P 1 T Q 1 (x 1,x 2 ) True x1x1 x2x2 Δ2Δ2 Δ’ 2 TIME

28. Belief sets Very conservative. 28 x 1 =80 A1A1 δ 1 80 (x 2 ) = 1/91 10 ≤ x 2 ≤ 100 out = True δ’ 1 80 (x 2 ) = 1/71 10 ≤ x 2 ≤ 80 x 1 =10 A1A1 δ 1 10 (x 2 ) = 1/91 10 ≤ x 2 ≤ 100 out = True δ’ 1 10 (x 2 ) = 1 10 ≤ x 2 ≤ 10 Q 1 = if x 1 ≥ x 2 then out := True else out := False

29. Belief sets Expensive in computation and representation. Abstraction might help. Have: γ(P) = { δ } Can do: γ(P) ⊇ { δ | (x 1 = v) } 10 ≤ v ≤ 100 Would also like: γ(P) ≈ { δ | (x 1 = v) } 10 ≤ v ≤ 100 29 A1A1 Δ = { δ | (x 1 = v) }

30. Different approach: Knowledge tracking via SMC 30

31. T Knowledge tracking via SMC SMC: “trusted third party”. 31 Q 1 (x 1,x 2 ) True Q 1 = if x 1 ≥ x 2 then out := True else out := False x1x1 A1A1 x2x2 A2A2

32. T policy Q 1 (x 1,x 2 ) δ1δ1 Knowledge tracking via SMC Use trusted third party for knowledge tracking and policy checking. Policy check on actual belief, instead conservatively over all plausible beliefs. 32 δ2δ2 x 1 =80 A1A1 x 2 =60 A2A2 δ δ | (x 1 = 80)δ | (x 2 = 60) True TIME δ’ 1 δ’ 2 P 1 (δ 2, … ) ∧ P 2 (δ 1, … )

33. T policy Q 1 (x 1,x 2 ) δ1δ1 Knowledge tracking via SMC Problem 2: policy decision leaks information. 33 δ2δ2 x 1 =80 A1A1 x 2 =60 A2A2 δ δ | (x 1 = 80) δ | (x 2 = 60) Reject TIME δ1δ1 δ2δ2 P 1 (δ 2, … ) ∧ P 2 (δ 1, … )

34. T policy Q 1 (x 1,x 2 ) δ1δ1 Knowledge tracking via SMC Agents trust the “trusted third party” to enforce their policies. 34 δ2δ2 x 1 =80 A1A1 x 2 =60 A2A2 δ δ | (x 1 = 80) δ | (x 2 = 60) Reject Accept TIME δ1δ1 δ’ 2 P 1 (δ 2, … ) P 2 (δ 1, … ) True

35. T policy Q 1 (x 1,x 2 ) δ1δ1 Knowledge tracking via SMC Knowledge tracking within SMC More permissive than belief sets. Unsatisfying uncertainty about one’s own policy decisions. “SMC is 1000 times slower than normal computation” Active research area (getting better). 35 δ2δ2 x 2 =60 δ | (x 2 = 60) Reject Accept δ1δ1 δ’ 2 P 1 (δ 2, … ) P 2 (δ 1, … ) True

36. Comparison and Examples 36

37. Millionaires 37 x 1 =? A1A1 Q 1 = if x 1 ≥ x 2 && x 1 ≥ x 3 then out := True else out := False max belief x 2 =? A2A2 x 3 =? A3A3 δ1δ1 δ 1 x2 δ 1 x3

38. Reduce precision 38 similar w = avg := (x 1 + x 2 + x 3 )/3 if | x 1 – avg | ≤ w && | x 2 – avg | ≤ w && | x 3 – avg | ≤ w then out := True else out := False max belief x 1 =? A1A1 x 2 =? A2A2 x 3 =? A3A3

39. Introduce noise 39 richest p = out := 0 if x 1 > x 2 && x 1 > x 3 then out := 1 if x 2 > x 1 && x 2 > x 3 then out := 2 if x 3 > x 1 && x 3 > x 2 then out := 3 if rand() < p then out := uniform(0,1,2,3) max belief x 1 =? A1A1 x 2 =? A2A2 x 3 =? A3A3

40. Summary+conclusions 40

41. Knowledge-Oriented Multiparty computation SMC: agents do not learn beyond what is implied by query. Our work: agents limit what can be inferred. Two approaches with differing (dis)advantages. Ongoing work in PL and crypto for tractability. 41 x1x1 x2x2 Q 1 (x 1,x 2 ) True

42. Knowledge in the SMC setting Each other’s secret is unknown, but in some initial known set. 42 x 1 =80 A1A1 x 2 =60 A2A2 x 2 =? A2A2 x 1 =? A1A1 10 ≤ x 2 ≤ 100 10 ≤ x 1 ≤ 100

43. δ 1 =δ | (x 1 =80) δ 2 =δ | (x 2 =60) T policy P 1 (δ 2, … ) ∧ P 2 (δ 1, … ) Q 1 (80,60) Knowledge tracking via SMC Use trusted third party for knowledge tracking and policy checking. Policy check on actual belief, instead conservatively over all plausible beliefs. 43 True δ 2 | (out=True) x 1 =80 A1A1 x 2 =60 A2A2 δ(x 1,x 2 ) = 1/91 2 10 ≤ x 1,x 2 ≤ 100

44. T policy P 1 (δ 2, … ) ∧ P 2 (δ 1, … ) Q 1 (80,60) Knowledge tracking via SMC Problem 1: agents cannot be trusted to provide their true beliefs. They cannot be trusted to look at each other’s beliefs either. 44 True x 1 =80 A1A1 x 2 =60 A2A2 δ 1 =δ | (x 1 =80) δ 2 =δ | (x 2 =60) δ 2 | (out=True)

45. Knowledge in a simpler setting Simulatable policy. 45 Q’’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := x 2 x 1 =80 A1A1 A2A2 x 2 =60

46. x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting δ 1 (x 2 ) = 1/91 for 10 ≤ x 2 ≤ 100 out = True δ’ 1 (x 2 ) = 1/71 for 10 ≤ x 2 ≤ 80 Simulatable policy. 46 Q’’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := x 2

47. x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting δ 1 (x 2 ) = 1/91 for 10 ≤ x 2 ≤ 100 out = 60 δ’ 1 (x 2 ) = 1 for x 2 = 60 Simulatable policy. 47 Q’’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := x 2

48. x 1 =80 A1A1 A2A2 x 2 =60 Knowledge in a simpler setting Simulatable policy. 48 Q’’ 1 = if x 1 ≥ x 2 then out := True else out := False if rand() < 0.5 then out := x 2 δ 1 | ( out = True ) δ 1 | ( out = False ) “max belief” = max δ’,x { δ’(x) } Policy: 1 = max belief ≤ t δ 1 | ( out = 60 )

49. Belief sets What A 1 learns depends on x 1. 49 x 1 =? A1A1 Q 1 = if x 1 ≥ x 2 && x 1 ≥ x 3 then out := True else out := False max belief threshold

50. Belief sets Conservative policy check approach can still allow non- trivial thresholds for some queries. 50 x 1 =? A1A1 max belief threshold

51. Knowledge tracking via SMC Agents cannot be trusted to provide their true beliefs. Cannot let A 1 ’s belief be tracked/known by A 2 or vice versa. 51 x 1 =? A1A1 δ 1 (x 2 ) = 1/91 10 ≤ x 2 ≤ 100 out = True δ’ 1 (x 2 ) = 1/71 10 ≤ x 2 ≤ 80 Q 1 = if x 1 ≥ x 2 then out := True else out := False 80

52. Knowledge tracking via SMC Policy decision leaks information. 52 x 2 =60 t 2 =0.5 A2A2 x 1 =? A1A1 δ 1 (x 2 ) = 1/91 for 10 ≤ x 2 ≤ 100 δ’ 1 (x 2 ) > 0.5 for some x 2 (reject) Q 1 = if x 1 ≥ x 2 then out := True else out := False 10