1. Belief in Information Flow Michael Clarkson, Andrew Myers, Fred B. Schneider Cornell University 18 th IEEE Computer Security Foundations Workshop June 20, 2005

2. Clarkson et al.: Belief in Information Flow 2 Password Checker PWC: if p = g then a := 1 else a := 0 Some programs require leakage of information p : stored password g : guessed password a : authentication flag

3. Clarkson et al.: Belief in Information Flow 3 Password Checker PWC: if p = g then a := 1 else a := 0  a depends on p : Noninterference is too strong But intuitively secure because PWC leaks little information about p …

4. Clarkson et al.: Belief in Information Flow 4 Quantitative Information Flow Quantitative security policies: –Expected rate of flow is at most k bits per second –At most k bits leak in any execution Enforcing these requires a model for quantitative information flow (QIF) This work: A model for QIF

5. Clarkson et al.: Belief in Information Flow 5 Traditional Model for QIF Flow = Uncertainty(H in ) – Uncertainty(H in | L out ) Uncertainty measured with some variant of entropy [Denning 82; McIver and Morgan 03; Clark, Hunt, and Malacaria 05] S H in L in H out L out probability distributions Information flows when uncertainty is decreased

6. Clarkson et al.: Belief in Information Flow 6 Adding Beliefs to Model Model attacker’s uncertainty about H inputs as a probability distribution We call this distribution a belief S H in L in H out L out

7. Clarkson et al.: Belief in Information Flow 7 Analyzing PWC PWC: if p = g then a := 1 else a := 0 Attacker guesses A : g = A Attacker believes: p = A 0.98 B 0.01 C 0.01 After observing a = 0, attacker believes: p = A 0 B 0.5 C 0.5 (Password is really C)

8. Clarkson et al.: Belief in Information Flow 8 Analyzing PWC p = A 0.98 B 0.01 C 0.01 p = A 0 B 0.5 C 0.5 more uncertaintya little uncertainty PrebeliefPostbelief Information flows when uncertainty is decreased  Traditional metric: Uncertainty = closeness to uniform distribution

9. Clarkson et al.: Belief in Information Flow 9 Why Uncertainty Fails Uncertainty-based approach addresses objective probabilities on system but not subjective probabilities of attacker

10. Clarkson et al.: Belief in Information Flow 10 Metric for Belief p = A 0.98 B 0.01 C 0.01 prebelief p = A 0 B 0.5 C 0.5 postbelief p = A 0 B 0 C 1 reality d 1 > d 2 : postbelief closer to reality because of observation of program d1d1 d2d2

11. Clarkson et al.: Belief in Information Flow 11Accuracy Accuracy: Distance from a belief to reality Certainty Accuracy Accuracy is the correct metric for QIF p = A 0 B 0.5 C 0.5 postbelief p = A 0.98 B 0.01 C 0.01 prebelief

12. Clarkson et al.: Belief in Information Flow 12 Belief in Information Flow Experiment protocol Describes how attackers revise beliefs from observation of program execution Accuracy metric How change in accuracy of belief can be used to measure the amount of information flow Extensions Repeated experiments, other metrics, and misinformation

13. Clarkson et al.: Belief in Information Flow 13Experiments p = A 0.98 B 0.01 C 0.01 p = A 0 B 0.5 C 0.5 PrebeliefPostbelief Experiment: How an attacker revises his belief

14. Clarkson et al.: Belief in Information Flow 14 Experiment Protocol S H in L in H out L out preBpostB

15. Clarkson et al.: Belief in Information Flow 15 Experiment Protocol 1.Attacker chooses prebelief S H in L in H out L out preBpostB preB

16. Clarkson et al.: Belief in Information Flow 16 Experiment Protocol 1.Attacker chooses prebelief 2.System and attacker choose inputs H in, L in S H in L in H out L out preBpostB preB

17. Clarkson et al.: Belief in Information Flow 17 Experiment Protocol 1.Attacker chooses prebelief 2.System and attacker choose inputs H in, L in 3.System executes S and produces observation Execution modeled as a distribution transformer semantics: « S ¬ : Dist ! Dist S H in L in H out L out preBpostB preB observation

18. Clarkson et al.: Belief in Information Flow 18 Experiment Protocol 1.Attacker chooses prebelief 2.System and attacker choose inputs H in, L in 3.System executes S and produces observation 4.Attacker conducts thought-experiment to obtain prediction preBpostB preB S L in H in L in H out L out observation

19. Clarkson et al.: Belief in Information Flow 19 H in Experiment Protocol 4.Attacker conducts thought-experiment to obtain prediction S L in H’ out L’ out prediction H in preB

20. Clarkson et al.: Belief in Information Flow 20 Experiment Protocol 1.Attacker chooses prebelief 2.System and attacker choose inputs H in, L in 3.System executes S and produces observation 4.Attacker conducts thought-experiment to obtain prediction 5.Attacker infers postbelief: preBpostB preB S observation H in L in S preB L in prediction postB prediction = | observation Bayesian inference

21. Clarkson et al.: Belief in Information Flow 21 Belief Revision in PWC postB = prediction | observation p,a = A,0 0 A,1 0.98 B,0 0.01 B,1 0 C,0 0.01 C,1 0 a = 0 | = ´ p = A 0 B 0.5 C 0.5 p = A 0.98 B 0.01 C 0.01 preB =

22. Clarkson et al.: Belief in Information Flow 22 Accuracy Metric Amount of flow Q is improvement in accuracy of belief i.e. (initial error) – (final error) Q = D(preB ! H in ) – D(postB ! H in ) preBpostB H in D(preB ! H in ) D(postB ! H in )

23. Clarkson et al.: Belief in Information Flow 23 Belief Distance Unit is (information theoretic) bits Relative entropy: When D is defined as relative entropy, Q is the amount of information that the attacker’s observation contains about the high input.

24. Clarkson et al.: Belief in Information Flow 24 Amount of Flow from PWC Q = D( .98,.01,.01  !  0, 0, 1  ) – D(  0,.5,.5  !  0, 0, 1  ) = 5.6 bits Information is in the eye of the beholder Max leakage of lg 3 bits implies uniform prebelief: p = A,B,C 1/3 each  0, 0, 1  1/3, 1/3, 1/3  1.6 bits .98,.01,.01  5 bits  0,.5,.5 .6 bits

25. Clarkson et al.: Belief in Information Flow 25 Repeated Experiments Experiment protocol is compositional B Exp B’ Exp B’’ Information flow is also compositional The amount of information flow over a series of experiments is equal to the sum of the amount of information flow in each individual experiment.

26. Clarkson et al.: Belief in Information Flow 26 Extensions of Metric So far: exact flow for a single execution Extend to: {Expected, maximum} amount of information flow {for a given experiment, over all experiments} Language for quantitative flow policies

27. Clarkson et al.: Belief in Information Flow 27 Non-probabilistic programs cannot create misinformation.Misinformation Certainty Accuracy ? FPWC: if p = g then a := 1 else a := 0; if random () <.1 then a := !a

28. Clarkson et al.: Belief in Information Flow 28Summary Attackers have beliefs Quantifying information flow with beliefs requires accuracy –Traditional uncertainty model is inappropriate Presented more expressive, fine-grained model of quantitative information flow –Compositional experiment protocol –Probabilistic language semantics –Accuracy-based metric

29. Clarkson et al.: Belief in Information Flow 29 Related Work Information theory in information flow –[Denning 82], [Millen 87], [Wittbold, Johnson 90], [Gray 91] Quantitative information flow –Using uncertainty: [Lowe 02], [McIver, Morgan 03], [Clark, Hunt, Malacaria 01 - 05] –Using sampling theory: [Di Pierro, Hankin, Wiklicky 00 - 05] Database privacy –Using relative entropy: [Evfimievski, Gehrke, Srikant 03]

30. Clarkson et al.: Belief in Information Flow 30 Future Work Extended programming language Lattice of security levels Static analysis Quantitative security policies

31. Belief in Information Flow Michael Clarkson, Andrew Myers, Fred B. Schneider Cornell University 18 th IEEE Computer Security Foundations Workshop June 20, 2005

32. Clarkson et al.: Belief in Information Flow 32 Extra Slides

33. Clarkson et al.: Belief in Information Flow 33 Beliefs as Distributions Other choices: Dempster-Shafer belief functions, plausibility measures, etc. Probability distributions are: –Quantitative –Axiomatically justifiable –Straightforward –Familiar Abstract operations –Product: combine disjoint beliefs –Update: condition belief to include new information –Distance: quantification of difference between two beliefs

34. Clarkson et al.: Belief in Information Flow 34 Program Semantics

35. Clarkson et al.: Belief in Information Flow 35 Postbeliefs are Bayesian Standard techniques for inference in applied statistics –Bayesian inference –Sampling/frequentist theory Bayesian inference: –Formalization of scientific method –Consistent with principles of rationality in a betting game May be subjective but is not arbitrary

36. Clarkson et al.: Belief in Information Flow 36 Interpreting Flow Quantities Uncertainty (entropy) interpreted as: –Improvement of expected codeword length Accuracy (relative entropy) interpreted as: –Improvement in efficiency of optimal code