1. Kevin H. Knuth Department of Physics University at Albany Clearing up the Mysteries: The Fundamentals of Probability Theory

2. 212 July 2006MaxEnt 2006 Paris, France Clearing up the Mysteries

3. Cox

4. 412 July 2006MaxEnt 2006 Paris, France Richard Threlkeld Cox Cox R.T. Probability, frequency, and reasonable expectation. Am. J. Physics, Vol. 14, pp. 1–13, 1946. Cox, R.T. The algebra of probable inference. Baltimore:Johns Hopkins Press, 1961.

5. 512 July 2006MaxEnt 2006 Paris, France Generalize Implication to Degrees Cox generalized the notion of Boolean implication to real numbers called degrees is generalized to

6. 612 July 2006MaxEnt 2006 Paris, France Generalize Implication to Degrees Cox and Jaynes attributed meaning to these degrees, and referred to them as degrees of belief or plausibility. Cox generalized the notion of Boolean implication to real numbers called degrees is generalized to

7. 712 July 2006MaxEnt 2006 Paris, France Calculus from Consistency The values that Cox’s degrees take must be consistent with Boolean algebra. The must be related to and Associativity requires that There are two ways to compute this. Consistency requires that they give the same answer.

8. 812 July 2006MaxEnt 2006 Paris, France Calculus from Consistency The values that Cox’s degrees take must be consistent with Boolean algebra. Associativity leads to a Product Rule… Complementation leads to a Sum Rule… Commutivity leads to Bayes Theorem…

9. 912 July 2006MaxEnt 2006 Paris, France The Problem of Meaning Many probabilities have to be assigned. How do you know that the numbers you calculate from the numbers you assign have the meaning you intend?

10. 1012 July 2006MaxEnt 2006 Paris, France The Problem of Meaning Many probabilities have to be assigned. How do you know that the numbers you calculate from the numbers you assign have the meaning you intend? This is not immediately clear given Cox’s derivation In fact, in Dempster-Shafer Theory, they also assign degrees, which they call plausibility. However, in that theory it is not true that How do we KNOW that Cox’s plausibility is the right plausibility? is related to

11. Order

12. 1212 July 2006MaxEnt 2006 Paris, France Posets A partially ordered set (poset) is a set of elements together with a binary ordering relation. includes covers

13. 1312 July 2006MaxEnt 2006 Paris, France Lattices A lattice is a poset P where every pair of elements x and y has a least upper bound called the join a greatest lower bound called the meet Similarly The green elements are upper bounds of the blue circled pair. The green circled element is their least upper bound or their join.

14. 1412 July 2006MaxEnt 2006 Paris, France The Lattice Identities L1. Idempotent L2. Commutative L3. Associative L4. Absorption If the meet and join follow the Consistency Relations C1. ( x is the greatest lower bound of x and y ) C2. ( y is the least upper bound of x and y ) Lattice Identities

15. 1512 July 2006MaxEnt 2006 Paris, France Lattices are Algebras Structural Viewpoint Operational Viewpoint

16. 1612 July 2006MaxEnt 2006 Paris, France Lattices are Algebras Structural Viewpoint Operational Viewpoint Assertions, Implies Sets, Is a subset of Positive Integers, Divides Integers, Is less than or equal to

17. Hypothesis Space

18. 1812 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space The atoms are mutually exclusive and exhaustive logical statements a = ‘It is an apple!’ b = ‘It is a banana!’ c = ‘It is a citrus fruit!’

19. 1912 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc The atoms are mutually exclusive and exhaustive logical statements a = ‘It is an apple!’ b = ‘It is a banana!’ c = ‘It is a citrus fruit!’

20. 2012 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc  The meet of any two atoms is the absurdity: a  b =  We do not allow our state of knowledge to include: ‘The fruit is an apple AND a banana!’

21. 2112 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc 

22. 2212 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc  a  b The join of any two elements represents a logical OR: a  b

23. 2312 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc  a  ba  cb  c The join of any two elements represents a logical OR: a  b The result is a less-definitive, but more probable statement

24. 2412 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc  a  ba  cb  c The final join gives us the TOP element, which is the TRUISM  = a  b  c 

25. 2512 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc  a  ba  cb  c The lattice is ordered according to the ordering relation “implies”.  implies

26. 2612 July 2006MaxEnt 2006 Paris, France The Boolean Hypothesis Space abc  a  ba  cb  c The lattice is ordered according to the ordering relation “implies”.  implies Meaning is Explicit

27. 2712 July 2006MaxEnt 2006 Paris, France The Hypothesis Space This is a HYPOTHESIS SPACE!!! It consists of all the statements that can be constructed from a set of mutually exclusive exhaustive statements. The space is ordered by the ordering relation “implies” We allow concepts like: ‘The fruit is an apple OR a banana!’ while we disallow concepts like: ‘The fruit is an apple AND a banana!’ abc  a  ba  cb  c 

28. 2812 July 2006MaxEnt 2006 Paris, France Superpositions of States??? Consider our piece of fruit… abc  a  ba  cb  c 

29. 2912 July 2006MaxEnt 2006 Paris, France Superpositions of States??? Consider our piece of fruit… abc  a  ba  cb  c 

30. 3012 July 2006MaxEnt 2006 Paris, France Superpositions of States??? Physically, we might agree that the fruit can only be in one of the states represented by the atomic statements. But each of these logical statements can describe my possible state of knowledge about the fruit. WHAT DOES IT MEAN??? abc  a  ba  cb  c 

31. 3112 July 2006MaxEnt 2006 Paris, France What is your state of Knowledge? 

32. 3212 July 2006MaxEnt 2006 Paris, France It is NOT a Banana! 

33. 3312 July 2006MaxEnt 2006 Paris, France It is NOT a Banana! 

34. 3412 July 2006MaxEnt 2006 Paris, France It is NOT a Banana! 

35. 3512 July 2006MaxEnt 2006 Paris, France It is NOT a Banana! 

36. 3612 July 2006MaxEnt 2006 Paris, France It is NOT a Banana! 

37. 3712 July 2006MaxEnt 2006 Paris, France It is NOT a Banana! 

38. 3812 July 2006MaxEnt 2006 Paris, France The New Hypothesis Space 

39. 3912 July 2006MaxEnt 2006 Paris, France It is an Orange! 

40. 4012 July 2006MaxEnt 2006 Paris, France It is an Orange! 

41. 4112 July 2006MaxEnt 2006 Paris, France It is an Orange! 

42. 4212 July 2006MaxEnt 2006 Paris, France It is an Orange! 

43. 4312 July 2006MaxEnt 2006 Paris, France It is an Orange! 

44. 4412 July 2006MaxEnt 2006 Paris, France It is an Orange! 

45. 4512 July 2006MaxEnt 2006 Paris, France It is an Orange! 

46. 4612 July 2006MaxEnt 2006 Paris, France It is an Orange! 

47. 4712 July 2006MaxEnt 2006 Paris, France The Final Hypothesis Space 

48. 4812 July 2006MaxEnt 2006 Paris, France The State of the Fruit  This was your initial state of knowledge. The fruit was never in this state! This was the state of the fruit.

49. 4912 July 2006MaxEnt 2006 Paris, France Microstates Consider a system with 3 microstates  i each of energy E. 11 22 33

50. 5012 July 2006MaxEnt 2006 Paris, France Microstates Consider a system with N microstates  i each of energy E. We can’t know which microstate the system is in. We only know that it is in one of the 3 microstates. 11 22 33  1  21  2 1  31  3 2  32  3 1  2  31  2  3

51. 5112 July 2006MaxEnt 2006 Paris, France Macrostates This is OUR STATE OF KNOWLEDGE! 11 22 33  1  21  2 1  31  3 2  32  3 1  2  31  2  3

52. 5212 July 2006MaxEnt 2006 Paris, France Macrostates This is what we call the MACROSTATE But if we believe that microstates are real states of the system, then we must admit that the system is never in the macrostate. 11 22 33  1  21  2 1  31  3 2  32  3 1  2  31  2  3

53. 5312 July 2006MaxEnt 2006 Paris, France Macrostates This is what we call the MACROSTATE But if we believe that microstates are real states of the system, then we must admit that the system is never in the macrostate. 11 22 33  1  21  2 1  31  3 2  32  3 1  2  31  2  3 Statistical Mechanics is a theory about OUR STATE OF KNOWLEDGE

54. 5412 July 2006MaxEnt 2006 Paris, France The Mind Projection Fallacy

55. 5512 July 2006MaxEnt 2006 Paris, France Mind Projection Fallacy abc  a  ba  cb  c Jaynes’ Mind Projection Fallacy warns that it is fallacious to view our state of knowledge about a system as a property possessed by the system. Which begs the question: Since we work with hypothesis spaces, is there such a thing as a STATE SPACE? 

56. 5612 July 2006MaxEnt 2006 Paris, France Mind Projection Fallacy abc  a  ba  cb  c Jaynes’ Mind Projection Fallacy warns that it is fallacious to view our state of knowledge about a system as a property possessed by the system. Which begs the question: Since we work with hypothesis spaces, is there such a thing as a STATE SPACE? ARE THE LAWS OF PHYSICS ACTUALLY RULES OF INFERENCE? 

57. 5712 July 2006MaxEnt 2006 Paris, France Mind Projection Fallacy abc  a  ba  cb  c Jaynes’ Mind Projection Fallacy warns that it is fallacious to view our state of knowledge about a system as a property possessed by the system. Which begs the question: Since we work with hypothesis spaces, is there such a thing as a STATE SPACE? ARE THE LAWS OF PHYSICS ACTUALLY RULES OF INFERENCE? As Carlos Rodriguez once asked: “ARE WE CRUISING A HYPOTHESIS SPACE?” 

58. 5812 July 2006MaxEnt 2006 Paris, France Clearing Up Another Mystery abc  a  ba  cb  c Jaynes’ once asked ‘Why Sample Spaces?’ With the order-theoretic formulation, we can now answer that. Cox’s algebraic constraints do not live in a vacuum. The algebraic operations AND, OR and NOT actually represent relationships among logical statements in a hypothesis (sample) space. The set of statements and their ordering is dual to the algebra. 

59. 5912 July 2006MaxEnt 2006 Paris, France Key Concepts The lattice of logical statements ordered by implication is the HYPOTHESIS SPACE The HYPOTHESIS SPACE and the BOOLEAN ALGEBRA of a set of logical statements are one and the same We work with HYPOTHESIS SPACES not STATE SPACES

60. Zeta Functions to Probability

61. 6112 July 2006MaxEnt 2006 Paris, France Inclusion and the Zeta Function The Zeta function encodes inclusion on the lattice. abc  a  ba  cb  c 

62. 6212 July 2006MaxEnt 2006 Paris, France Inclusion and the Zeta Function abc  a  ba  cb  c  since The Zeta function encodes inclusion on the lattice.

63. 6312 July 2006MaxEnt 2006 Paris, France Inclusion and the Zeta Function The Zeta function encodes inclusion on the lattice. We can define its dual by flipping around the ordering relation

64. 6412 July 2006MaxEnt 2006 Paris, France Degrees of Inclusion and Z We generalize the dual of the Zeta function to the function z

65. 6512 July 2006MaxEnt 2006 Paris, France Z The function z Continues to encode inclusion, but has generalized the concept to degrees of inclusion. In the lattice of logical statements ordered by implies, this function describes degrees of implication.

66. 6612 July 2006MaxEnt 2006 Paris, France Probability Changing notation The MEANING of p(x|y) is made explicit via the Zeta function. These are degrees of implication! Let us do away with linguistic traps, such as plausibility and degrees of belief

67. 6712 July 2006MaxEnt 2006 Paris, France Calculus from Consistency The values that this function z takes must be consistent with the structure of the poset. Following the inspiration of R.T. Cox and results from A. Caticha (1998), I showed Associativity leads to a Sum Rule… Distributivity leads to a Product Rule… Commutivity leads to Bayes Theorem…

68. 6812 July 2006MaxEnt 2006 Paris, France Distributive Lattices Thus a function z can be associated with all distributive lattices, which then follows a Sum Rule, a Product Rule, and a Bayes Theorem. This function is what Gian-Carlo Rota called a bi-valuation, or a valuation for short. Any lattice with elements that are sets, and are ordered by ‘is a subset of’ is a distributive lattice. Thus we can generalize their ordering relation (inclusion) and compute degrees of inclusion.

69. 6912 July 2006MaxEnt 2006 Paris, France Inclusion-Exclusion For Distributive lattices, the Sum Rule leads to the inclusion- exclusion principle. This is intimately related to the Möbius function for the lattice, which is related to the Zeta function.

70. 7012 July 2006MaxEnt 2006 Paris, France How to Derive a Measure Knuth K.H. 2003. Deriving laws from ordering relations. In: G.J. Erickson, Y. Zhai (eds.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Jackson Hole WY 2003, AIP Conference Proceedings 707, American Institute of Physics, Melville NY, pp. 204-235. 1.Define the objects that you want to measure 2.Select the appropriate ordering relation 3.Determine the algebra from the lattice/poset structure 4.Use the constraints from the algebra to derive a calculus Skipping steps by relying on intuition will most likely result in errors and waste time and effort!!!

71. 7112 July 2006MaxEnt 2006 Paris, France Key Concepts The Zeta function of a lattice can be generalized to describe degrees of inclusion For Distributive algebras, these degrees follow a Sum Rule, a Product Rule, and a Bayes’ Theorem

72. Two Kinds of AND

73. 7312 July 2006MaxEnt 2006 Paris, France Hypothesis Space of Fruit Consider this space of Fruit It is an apple! It is a banana!  ba a  b F

74. 7412 July 2006MaxEnt 2006 Paris, France Hypothesis Space of Quality And this space of Quality It is ripe!It is spoiled! It is an apple! It is a banana!  ba a  b F  sr r  s Q

75. 7512 July 2006MaxEnt 2006 Paris, France Focus on the Atomic Statements A(F)A(F) A(Q)A(Q) It is ripe!It is spoiled! It is an apple! It is a banana! basr

76. 7612 July 2006MaxEnt 2006 Paris, France Focus on the Atomic Statements ab A(F)  A ( Q )

77. 7712 July 2006MaxEnt 2006 Paris, France Take the Lattice Product a, rb, s A(F)  A ( Q ) a, s b, r

78. 7812 July 2006MaxEnt 2006 Paris, France Take the Lattice Product a, rb, s A(F)  A ( Q ) a, s b, r

79. 7912 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s 

80. 8012 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s

81. 8112 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,sa,r  b,r

82. 8212 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,sa,r  b,r a,s  b,r

83. 8312 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,sa,r  b,r a,r  b,s a,s  b,r

84. 8412 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,sa,r  b,ra,s  b,s a,r  b,s a,s  b,r

85. 8512 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s b,r  b, s a,r  b,ra,s  b,s a,r  b,s a,s  b,r

86. 8612 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s b,r  b, s a,r  b,ra,s  b,s a,r  b,s a,s  b,r a,r  a,s  b,r

87. 8712 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s b,r  b, s a,r  b,ra,s  b,s a,r  b,s a,s  b,r a,r  a,s  b,r a,r  a,s  b,s

88. 8812 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s b,r  b, s a,r  b,ra,s  b,s a,r  b,s a,s  b,r a,r  a,s  b,r a,r  a,s  b,s a,r  b,r  b,s

89. 8912 July 2006MaxEnt 2006 Paris, France Take the Powerset P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s b,r  b, s a,r  b,ra,s  b,s a,r  b,s a,s  b,r a,r  a,s  b,r a,s  b,r  b,s a,r  a,s  b,s a,r  b,r  b,s

90. 9012 July 2006MaxEnt 2006 Paris, France The Joint Hypothesis Space P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  a,r  a,s b,r  b, s a,r  b,ra,s  b,s a,r  b,s a,s  b,r a,r  a,s  b,r a,s  b,r  b,s a,r  a,s  b,s a,r  b,r  b,s 

91. 9112 July 2006MaxEnt 2006 Paris, France Simplified Notation 

92. 9212 July 2006MaxEnt 2006 Paris, France Simplified Notation  

93. 9312 July 2006MaxEnt 2006 Paris, France Simplified Notation P ( A(F)  A ( Q ) ) a,ra,sb,rb,s  abrs a,r  b,s a,s  b,r a  b,r a,s  b a  b,s r  b,s 

94. 9412 July 2006MaxEnt 2006 Paris, France Two Different Kinds of AND 1.There is the usual Boolean AND: the Meet a,ra,sb,rb,s  abrs a,r  b,s a,s  b,r a  b,r a,s  b a  b,s r  b,s

95. 9512 July 2006MaxEnt 2006 Paris, France Two kinds of AND 2. There is the AND induced by the Lattice Product a,ra,sb,rb,s  abrs a,r  b,s a,s  b,r a  b,r a,s  b a  b,s r  b,s

96. The Product Rule

97. 9712 July 2006MaxEnt 2006 Paris, France Assigning Probabilities in a Product Space How do we assign ? a,ra,sb,rb,s  abrs a,r  b,s a,s  b,r a  b,r a,s  b a  b,s r  b,s 

98. 9812 July 2006MaxEnt 2006 Paris, France Product Rule First, note that Now apply the Product Rule So that

99. 9912 July 2006MaxEnt 2006 Paris, France Assigning Probabilities in a Product Space a,ra,sb,rb,s  abrs a,r  b,s a,s  b,r a  b,r a,s  b a  b,s r  b,s 

100. 10012 July 2006MaxEnt 2006 Paris, France The Utility of the Product Rule Probability in the Joint Space Probability in the Factor Space

101. 10112 July 2006MaxEnt 2006 Paris, France The Utility of the Product Rule Probability in the Joint Space Probability in the Factor Space

102. 10212 July 2006MaxEnt 2006 Paris, France The Utility of the Product Rule The Product Rule allows us to compute the degree to which any statement implies any other statement. We can write this in terms of the degree to which the truism implies statements.

103. 10312 July 2006MaxEnt 2006 Paris, France But Assignment can be Difficult The Product Rule allows us to compute the degree to which any statement implies any other statement. We can write this in terms of the degree to which the truism implies statements. Difficult to Assign Easy to Assign

104. 10412 July 2006MaxEnt 2006 Paris, France Bayes Theorem Solving for the difficult to assign degree to which a statement in one space implies a statement in another space gives us Bayes Theorem.

105. Observing Data

106. 10612 July 2006MaxEnt 2006 Paris, France Two Spaces Again Hypothesis SpaceData Space  h2h1 h1  h2 H  d2 d1 d1  d2 D

107. 10712 July 2006MaxEnt 2006 Paris, France The Joint Space P ( A(H)  A ( D ) ) h1, d1h1, d2h2, d1h2, d2  h1,d1  h1,d2 h2,d1  h2,d2 h1,d1  h2,d1 h1,d2  h2,d2 h1,d1  h2,d2 h1,d2  h2,d1 h1,d1  h1,d2  h2,d1 h1,d2  h2,d1  h2,d2 h1,d1  h1,d2  h2,d2 h1,d1  h2,d1  h2,d2 

108. 10812 July 2006MaxEnt 2006 Paris, France Bayes Theorem Bayes Theorem allows us to compute all these probabilities before taking data! h1, d1h1, d2h2, d1h2, d2  h1,d1  h1,d2 h2,d1  h2,d2 h1,d1  h2,d1 h1,d2  h2,d2 h1,d1  h2,d2 h1,d2  h2,d1 h1,d1  h1,d2  h2,d1 h1,d2  h2,d1  h2,d2 h1,d1  h1,d2  h2,d2 h1,d1  h2,d1  h2,d2 

109. 10912 July 2006MaxEnt 2006 Paris, France The Collapse Now say we take a measurement and find d1. The lattice collapses, since: This means that

110. 11012 July 2006MaxEnt 2006 Paris, France h1,d1  h2,d1 The Posterior One of the Posterior Probabilities is h1, d1h1, d2h2, d1h2, d2  h1,d1  h1,d2 h2,d1  h2,d2 h1,d2  h2,d2 h1,d1  h2,d2 h1,d2  h2,d1 h1,d1  h1,d2  h2,d1 h1,d2  h2,d1  h2,d2 h1,d1  h1,d2  h2,d2 h1,d1  h2,d1  h2,d2 

111. 11112 July 2006MaxEnt 2006 Paris, France h1,d1  h2,d1 The Posterior One of the Posterior Probabilities is h1, d1h1, d2h2, d1h2, d2  h1,d1  h1,d2 h2,d1  h2,d2 h1,d2  h2,d2 h1,d1  h2,d2 h1,d2  h2,d1 h1,d1  h1,d2  h2,d1 h1,d2  h2,d1  h2,d2 h1,d1  h1,d2  h2,d2 h1,d1  h2,d1  h2,d2 

112. 11212 July 2006MaxEnt 2006 Paris, France h1,d1  h2,d1 How the Lattice Collapses Watch the UPDATE! h1, d1h1, d2h2, d1h2, d2  h1,d1  h1,d2 h2,d1  h2,d2 h1,d2  h2,d2 h1,d1  h2,d2 h1,d2  h2,d1 h1,d1  h1,d2  h2,d1 h1,d2  h2,d1  h2,d2 h1,d1  h1,d2  h2,d2 h1,d1  h2,d1  h2,d2 

113. 11312 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses Equivalence Class of 4 Elements h1, d1h1,  D h2, d1h2,  D  h1,d1  h1,  D h2,d1  h2,  D h1,d1  h2,d1 h1,  D  h2,  D h1,d1  h2,  D h1,  D  h2,d1 h1, d1  h1,  D  h2, d1 h1,  D  h2,d1  h2,  D h1,d1  h1,  D  h2,  D h1,d1  h2,d1  h2,  D  H, d 1

114. 11412 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses h1, d1 h1,  D h2, d1 h2,  D  h1,d1  h1,  D h2,d1  h2,  D h1,d1  h2,d1 h1,  D  h2,  D h1,d1  h2,  D h1,  D  h2,d1 h1, d1  h1,  D  h2, d1 h1,  D  h2,d1  h2,  D h1,d1  h1,  D  h2,  D h1,d1  h2,d1  h2,  D  H, d 1

115. 11512 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses h1, d1 h1,  D h2, d1 h2,  D  h1,d1  h1,  D h2,d1  h2,  D h1,d1  h2,d1 h1,  D  h2,  D h1,d1  h2,  D h1,  D  h2,d1 h1, d1  h1,  D  h2, d1 h1,  D  h2,d1  h2,  D h1,d1  h1,  D  h2,  D h1,d1  h2,d1  h2,  D  H, d 1

116. 11612 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses h1, d1 h1,  D h2, d1 h2,  D  h1,d1  h1,  D h2,d1  h2,  D h1,d1  h2,d1 h1,  D  h2,  D h1,d1  h2,  D h1,  D  h2,d1 h1, d1  h1,  D  h2, d1 h1,  D  h2,d1  h2,  D h1,d1  h1,  D  h2,  D h1,d1  h2,d1  h2,  D  H, d 1

117. 11712 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses h1, d1 h1,  D h2, d1 h2,  D  h1,d1  h1,  D h2,d1  h2,  D h1,d1  h2,d1 h1,  D  h2,  D h1,d1  h2,  D h1,  D  h2,d1 h1, d1  h1,  D  h2, d1 h1,  D  h2,d1  h2,  D h1,d1  h1,  D  h2,  D h1,d1  h2,d1  h2,  D  H, d 1

118. 11812 July 2006MaxEnt 2006 Paris, France  H, d 1 How the Lattice Collapses h1, d1h2, d1 

119. 11912 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses  H, d 1 h2, d1  h1, d1

120. 12012 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses  H, d 1 h2, d1  h1, d1

121. 12112 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses The measurement collapses our hypothesis space  H, d 1 h2, d1  h1, d1

122. 12212 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses The measurement collapses our hypothesis space  H, d 1 h2, d1  h1, d1

123. 12312 July 2006MaxEnt 2006 Paris, France How the Lattice Collapses  H,  D h2,  D  h1,  D has become our new ‘prior’ Our ‘posterior’

124. On Notation

125. 12512 July 2006MaxEnt 2006 Paris, France P is for Probability Probability is a function that takes two logical statements as arguments: These logical statements are elements in the hypothesis space.

126. 12612 July 2006MaxEnt 2006 Paris, France P is for Probability The truism is the TOP element of the hypothesis space. While they are convenient abuses, the following notations are not meaningful in this framework: Often the second argument is the truism:

127. 12712 July 2006MaxEnt 2006 Paris, France P is for Probability Since probability can take any two elements of the lattice, this function is the same function regardless of which arguments in the lattice are used. Therefore in the following: PriorLikelihoodPosterior p is the same function!

128. Assigning Probabilities

129. 12912 July 2006MaxEnt 2006 Paris, France How Do We Assign Probabilities? abc  a  ba  cb  c  We must assign p ( x | y ) for each pair x and y. There are 2 3 = 8 elements. There are 8^2 = 64 probabilities to assign.

130. 13012 July 2006MaxEnt 2006 Paris, France How Do We Assign Probabilities? abc  a  ba  cb  c  First, let’s do the easy ones: If then 30 = 8 + 3*5 + 3*2 + 1 13 = 3*3 + 3*1 + 1 64 Total

131. 13112 July 2006MaxEnt 2006 Paris, France How Do We Assign Probabilities? abc  a  ba  cb  c  First, let’s do the easy ones: If then 30 = 8 + 3*5 + 3*2 + 1 13 = 3*3 + 3*1 + 1 We now assign: 3 64 Total

132. 13212 July 2006MaxEnt 2006 Paris, France How Do We Assign Probabilities? abc  a  ba  cb  c  First, let’s do the easy ones: If then 30 = 8 + 3*5 + 3*2 + 1 13 = 3*3 + 3*1 + 1 We now assign: 3 Now use the Sum Rule to get: 3 64 Total

133. 13312 July 2006MaxEnt 2006 Paris, France How Do We Assign Probabilities? abc  a  ba  cb  c  First, let’s do the easy ones: If then 30 = 8 + 3*5 + 3*2 + 1 13 = 3*3 + 3*1 + 1 We now assign: 3 Now use the Sum Rule to get: 3 Last use Bayes Theorem eg: 15 = 3*5 64 Total

134. Summary

135. 13512 July 2006MaxEnt 2006 Paris, France Summary 1.The lattice of logical statements ordered by implication is the hypothesis space 2.The laws of science refer to our state of knowledge about a system---not necessarily the system itself. 3.The Zeta function of a lattice can be generalized to describe degrees of inclusion 4.For distributive lattices, these degrees follow a Sum Rule, a Product Rule, and a Bayes’ Theorem 5.The Sum Rule, Product Rule and Bayes Theorem enable us to compute all the probabilities on the lattice. 6.There are two kinds of AND: the meet and that induced by the lattice product. 7.The joint space of hypotheses and data collapses when a datum point is measured. This collapse is the update. 8.The probabilities of the join-irreducible elements given the truism must be assigned. These assignments can be arbitrary!

136. Special Thanks to: John Skilling Ariel Caticha Philip Goyal Steve Gull Carlos Rodriguez for many insightful discussions and comments