I The meaning of chance Axiomatization

E Plurbus Unum

Fair coin? How do you check whether a coin is fair? Rephrase: how many times do you need to toss a coin in order to be “confident” that it is fair?

Classical definition of probability Pierre Simon Laplace. “Théorie analytique des probabilités” The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

Frequentist view … defines an event's probability as the limit of its relative frequency in a large number of trials. The development of the frequentist account was motivated by the problems and paradoxes of the previously dominant viewpoint, the classical interpretation. Frequentists: Venn (pictured), Fisher, von Mises

Frequentists talk about probabilities only when dealing with well-defined random experiments The set of all possible outcomes of a random experiment is called the sample space of the experiment. An event is defined as a particular subset of the sample space that you want to consider. For any event only one of two possibilities can happen; it occurs or it does not occur. The relative frequency of occurrence of an event, in a number of repetitions of the experiment, is a measure of the probability of that event.

Monte Hall Problem Monte's dilemma “ Suppose you’re on a game show, and you’re given a choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say number 1, and the host, who knows what’s behind the doors, opens another door, say number. 3, which has a goat. He says to you, ‘Do you want to pick door number 2?’ Is it to your advantage to switch your choice of doors?”

Bayesian probability Bayesian approach treats “probability” as 'a measure of a state of knowledge' -- not as a frequency. objectivist school : rules of Bayesian statistics justified by desiderata of rationality and consistency ; extension of Aristotelian logic. subjectivist school : the state of knowledge corresponds to a 'personal belief' [3]. “Machine learning” methods are based on objectivist Bayesian principles. A probability can be assigned to a hypothesis, -- not possible under the frequentist view -- a hypothesis can only be accepted or rejected. Aristotelian logic : every statement is either true or false Bayesian reasoning: incorporates uncertainty.

Bayes’ rule

Cox’s axioms Cox wanted his system to satisfy the following conditions: 1. Divisibility and comparability - The plausibility of a statement is a real number and is dependent on information we have related to the statement. 2. Common sense - Plausibilities should vary sensibly with the assessment of plausibilities in the model. 3. Consistency - If the plausibility of a statement can be derived in many ways, all the results must be equal.

P(A) is the prior probability or marginal probability of A. It is "prior" in the sense that it does not take into account any information about B. P(A|B) is the conditional probability of A, given B. It is also called the posterior probability-- it is derived from or depends upon the specified value of B. P(B|A) is the conditional probability of B given A. P(B) is the prior or marginal probability of B, and acts as a normalizing constant.

Bayesian probability interprets the concept of probability as 'a measure of a state of knowledge The frequentist view of probability overshadowed the Bayesian view during the first half of the 20th century due to prominent figures such as Ronald Fisher, JerzyNeyman and Egon Pearson. The word Bayesian appeared in the 1950s, and by the 1960s it became the term preferred by people who sought to escape the limitations and inconsistencies of the frequentist approach to probability theory

Bertrand's paradox

Questions What is the main difference between the classical definition vs frequentist definition of probability? What is the main difference between frequentist probability and Bayesian probability?