A Note on Foundations of Bayesianism Stefan Arnborg, KTH Gunnar Sjödin, SICS

BAYES RULES! -in finite models (ECCAI2000) -but not in infinite! (MaxEnt2000) Stefan Arnborg, KTH Gunnar Sjödin, SICS

Normative claim of Bayesianism 4 EVERY type of uncertainty should be treated as probability 4 Aristotle, Sun Zi(300BC), Bayes(1763), Laplace, de Finetti, Jeffreys, Keynes, Ramsey, Adams, Lindley, Cheeseman, Jaynes,… 4 This claim is controversial and not universally accepted: Fisher(1922), Cramér, Zadeh, Dempster, Shafer, Walley(1999) …

Fundamental Justifications 4 Consistent Betting Paradigm: de Finetti, Savage(1950), Lindley(1982), … Snow(1999) 4 Information Based: Cox(1946), Aczél(1966), Jaynes(1994) Criticized by Paris(1994), Halpern(1999)

Main Result: 4 Cox information based justification can be derived with weak common sense assumptions. Difference between finite and infinite models. 4 Assumptions are: Refinability Information independence Strict monotonicity Infinite case: Model is closed

Caveats 4 Classical Bayesianism only for finite model 4 And for closed infinite model embeddable in the real line 4 Ordered closed infinite model: Extended probability (with infinitesimals) also permissible (non-monotonic and default logics).

Jaynes’s Desiderata on Uncertainty Management 4 Uncertainty is measured by real number, dependent on information subject possesses: A|C : plausibility of A given C. 4 Consistency. 4 Common sense.

Real Numbered uncertainties 4 Given set of statements (possible world sets) A, B, C, … 4 Plausibility A|C: plausibility of A given that C is known to be true - a real number 4 Conjunction: AB 4 Disjunction A+B, Difference A-B 4 AB|C=F(A|BC, B|C) 4 A+B|C=G(A|C,B-A|C) 4 not A|C=S(A|C)

RESCALABILITY THEOREMS 4 Under suitable assumptions there is a strictly monotone function w(x) such that 4 w(F(x,y))=w(x)w(y) 4 w(G(x,y))=w(x)+w(y) 4 I.E., by rescaling the plausibility measure by w, model becomes a probability model 4 I.E., if you accept the assumptions, then Bayes Rules!

Invariance under rescaling 4 * and + are strictly monotone, symmetric, associative and jointly distributive 4 These properties are invariant under strictly monotone rescaling 4 If F and G violate the properties, rescaling is impossible.

Consistency 4 AB|C==BA|C, thus F(A|BC,B|C)=F(B|AC,A|C) 4 A+B|C==B+A|C 4 (AB)C|D==A(BC)|D 4 (A+B)C|D==AC+BC|D 4 Does this mean that F,G must be associative, symmetric and jointly distributive??? 4 No, not without additional assumptions!

OUR common sense assumptions 4 REFINABILITY: Assume B’|B=c was defined; It should be possible to refine another event A by A’ so that A’|A=c (cf Tribus, Jimison, Heckerman) 4 INFORMATION INDEPENDENCE: New events obtained by refinement of same event can be postulated independent: A|BC=A|C and B|AC=B|C ’Knowledge of one has no effect on plausibility of the other’

Halpern’s Example: 4 Worlds A B C D E G H I J K L M D|E=H|J B|C = L|M A|C = I|JE|G = A|B H|J≈K|M D|G = K|LM

Example: F(F(x,y),z)≈F(x,F(y,z))   C D E G H I J K L M D|E=H|J=x B|C = L|M=z A|C = I|JE|G = A|B=y H|J≈K|M D|G = K|LM

Refine:A’|A=D|E: INCONSISTE   C D E G H I J K L M D|E=H|J=x B|C = L|M=z A|C = I|JE|G = A|B=y H|J≈K|M D|G = K|LM A’ H|J=A’AB|C=K|M !!!!!!!!!!!!!

OBSERVATION 4 The functions F and G must be symmetric and associative if refinability and information independence accepted 4 F, G must likewise be jointly distributive F(G(x,y),z)=G(F(x,z),F(y,z)) 4 But only on the finite range of definition 4 Not enough for rescalability

Rescalability is solvability of LP L4+L4-La=0 L3+L5-La=0 L2+L4-Lb=0 L1+L5-Lb=0 L4+L6-Lc=0 L3+L7-Lc=0 L2+L6-Ld=0 L1+L8-Ld=0 L1<L2<… L8 F(x4,x4)=F(x3,x5)=a F(x2,x4)=F(x1,x5)=b F(x4,x6)=F(x3,x7)=c F(x2,x6)=F(x1,x8)=d x1<x2<…<x8 Find L such that ML=0 and DL>0 Unfortunately, the equalities imply L7=L8 !!!!

Theorem 4, finite case: 4 Rescalability follows for finite models from weak common sense assumptions: refinability and information independence using finite-dimensional duality theory 4 Conjecture: Savages and Lindleys consistent betting behavior analyses can be similarly strengthened

Probability model Counterexample, x^(i+1)=F(x^i,x) Log probability i INFINITE CASE: NON-SEPARABILITY x y

Extended Probability 4 Probability values are taken from ordered field. 4 An ordered field is generated by rationals, reals and infinitesimals (Conway)  Previous example explained by x=0.5, y=0.5+ . 4 Extended probability has been shown equivalent to non-monotonic reasoning schemes (Benferhat, Dubois, Prade, 1997).

Ordered rings and fields Integers Modular Rational Gaussian Algebraic reals Complex Reals Ordered infinitesimals Conway’s No No: a universal ordered field, extension of any ordered field.

Infinite Models 4 Theorem 6:Model is rescalable iff all plausibilities are separable !! - but separability somewhat contrived. 4 Assume plausibility model can be closed: F(x,y) defined on D^2 G(x,y) defined on D^2 if y<S(x) Range of F in D Range of G in D.

Theorems, Infinite Closed Models: 4 Theorem 9: Every plausibility measure is equivalent to extended probability. 4 Corollary 10:Every plausibility measure that can be embedded in the reals is equivalent to standard probability.

Proof Sketch 4 Assume plausibility measure closed in D F G S(x) Introduce subtraction (a,b) (a,b)≈(c,d) if G(a,d)=G(c,b) or G(S(a),S(d))=G(S(c),S(b)) D DxD/≈

Extend from -infty to +infty … (i+d)+(j+e)=i+j+d+e (i+d)(j+e)=ij+ie+jd+de We now have an ordered ring which is also an integral domain * and + are extensions of F and G D ZxD

Create an ordered field 4 Standard quotient construction for ordered integral domain gives an ordered field (MacLane-Birkhoff) 4 This field is a subfield of the universal ordered field No of Conway

Proof Sketch, continued 4 Ordered fields are generated by reals and infinitesimals, non-zero values smaller than any real number, i.e., are models of extended probability

Proof Sketch, continued... 4 If the closed model is required to consist of real numbers, the model is equivalent to standard probability: All subfields of No with lowest upper bounds are embeddable in the real number field, and if field is real, then D was also real (the embedding process does not introduce infinitesimals in separable model).

SUMMARY 4 With assumptions of refinability, independence and strict monotonicity, finite ordered plausibility models are equivalent to probability models 4 With further assumption of closability, (infinite) plausibility models are equivalent to extended probability models 4 And closed plausibility models embeddable in the reals are equivalent to probability models.

OPEN PROBLEMS 4 NONE? (for well-defined statements) 4 However, Bayesian analysis of imprecise statements interesting alternative to fuzzy/rough/possibilistic logics (cf Wittgenstein (1956) word games). 4 Meaning of A fuzzy -> meaning depends on context -> modelled as conditional meaning.