1. Graduate School of Information Sciences, Tohoku University
Physical Fluctuomatics 2nd Mathematical Preparations (1): Probability and statistics
Kazuyuki Tanaka
Graduate School of Information Sciences, Tohoku University
Physics Fluctuomatics (Tohoku University)

2. Physics Fluctuomatics (Tohoku University)
Probability
Event and Probability
Joint Probability and Conditional Probability
Bayes Formula, Prior Probability and Posterior Probability
Discrete Random Variable and Probability Distribution
Continuous Random Variable and Probability Density Function
Average, Variance and Covariance
Uniform Distribution
Gauss Distribution
Physics Fluctuomatics (Tohoku University)

3. Event, Sample Space and Event
Experiment: Experiments in probability theory means that outcomes are not predictable in advance. However, while the outcome will not be known in advance, the set of all possible outcomes is known
Sample Point: Each possible outcome in the experiments.
Sample Space：The set of all the possible sample points in the experiments
Event：Subset of the sample space
Elementary Event：Event consisting of one sample point
Empty Event：Event consisting of no sample point
Physics Fluctuomatics (Tohoku University)

4. Physics Fluctuomatics (Tohoku University)
Various Events
Whole Events Ω:Events consisting of all sample points of the sample space.
Complementary Event of Event A: Ac=Ω╲A
Defference of Events A and B: A╲B
Union of Events A and B: A∪B
Intersection of Events A and B: A∩B
Events A and B are exclusive of each other: A∩B=Ф
Events A, B and C are exclusive of each other:
[A∩B=Ф]Λ[B∩C=Ф]Λ[C∩A=Ф]
Physics Fluctuomatics (Tohoku University)

5. Empirically Definition of Probability
Definition by Laplace: Let us suppose that the total number of all the sample points is N and they can occur equally Likely. Probability of an event A with N sample points is defined by p=n/N.
Statistical Definition: Let us suppose that an event A occur r times when the same experiment are repeated R times. If the ratio r/R tends to a constant value p as the number of times of the experiments R go to infinity, we define the value p as probability of event A.
Physics Fluctuomatics (Tohoku University)

6. Definition of Probability
Definition of Kolmogorov: Probability Pr{A} for every event A in the specified sample space Ω satisfies the following three axioms:
Axion 1:
Axion 2:
Axion 3: For every events A, B that are exclusive of each other, it is always valid that
Physics Fluctuomatics (Tohoku University)

7. Joint Probability and Conditional Probability
Probability of Event A
Joint Probability of Events A and B
Conditional Probability of Event A
when Event B has happened. A B
Physics Fluctuomatics (Tohoku University)

8. Joint Probability and Independency of Events
Events A and B are independent of each other
In this case, the conditional probability can be expressed as A B A B
Physics Fluctuomatics (Tohoku University)

9. Physics Fluctuomatics (Tohoku University)
Marginal Probability
Let us suppose that the sample space W is expressed by Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj is exclusive of each other. Ai B
Marginal Probability of Event B for Joint Probability Pr{Ai,B}
Marginalize A B
Simplified Notation
Summation over all the possible events in which every pair of events are exclusive of each other.
Physics Fluctuomatics (Tohoku University)

10. Four Dimensional Point Probability and Marginal Probability
Marginal Probability of Event B A B C D
Marginalize
Physics Fluctuomatics (Tohoku University)

11. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

12. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

13. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

14. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

15. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

16. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

17. Derivation of Bayes Formulas
Physics Fluctuomatics (Tohoku University)

18. Physics Fluctuomatics (Tohoku University)
Bayes Formula
Prior Probability A B
Posterior Probability
It is often referred to as Bayes Rule.
Bayesian Network
Physics Fluctuomatics (Tohoku University)

19. Probability and Random Variable
We introduce a one to one mapping X(A) from every events A to a mutual different real number. The mapping X(A) is referred to as Random Variable of A. The random variable X(A) is often denoted by just the notation X.
Probability of the event X=x that the random variable X takes a real number x is denoted by Pr{X=x}. Here, x is referred to as the state of the random variable X．The set of all the possible states is referred to as State Space.
If events X=x and X=x’ are exclusive of each other, the states x and x’ are excusive of each other.
Physics Fluctuomatics (Tohoku University)

20. Discrete Random Variable and Continuous Random Variable
　Random Variable in Discrete State Space
　　　　　　　　　　Example:{x1,x2,…,xM}
Continuous Random Variable:
　Random Variable in Continuous State Space
　　　　　　　　　　Example：(−∞,+∞)
Physics Fluctuomatics (Tohoku University)

21. Discrete Random Variable and Probability Distribution
Let us suppose that the sample W is expressed by Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj are exclusive of each other.
We introduce a one to one mapping X:Ai xi (i=1,2,…,M).
If all the probabilities for the events X=x1, X=x2,…, X=xM are expressed in terms of a function P(x) as follows:
Random Variable
State Variable
State
the function P(x) and the variable x are referred to as Probability Distribution and State Variable, respectively.
Physics Fluctuomatics (Tohoku University)

22. Discrete Random Variable and Probability Distribution
Probability distributions have the following properties:
Normalization Condition
Physics Fluctuomatics (Tohoku University)

23. Physics Fluctuomatics (Tohoku University)
Average and Variance
Average of Random Variable X : μ
Variance of Random Variable X: σ2
s：Standard Deviation
Physics Fluctuomatics (Tohoku University)

24. Discrete Random Variable and Joint Probability Distribution
If the joint probability
Pr{(X=x)∩(Y=y)}= Pr{X=x,Y=y}
is expressed in terms of a function P(x,y) as follows:
P(x,y) is referred to as Joint Probability Distribution.
Probability Vector
State Vector
Physics Fluctuomatics (Tohoku University)

25. Discrete Random Variable and Marginal Probability Distribution
Let us suppose that the sample W is expressed by Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj are exclusive of each other.
We introduce a one to one mapping X:Ai xi (i=1,2,…,M).
Marginal Probability Distribution
Simplified Notation
Summation over all the possible events in which every pair of events are exclusive of each other.
Normalization Condition
Physics Fluctuomatics (Tohoku University)

26. Discrete Random Variable and Marginal Probability
Marginal Probability of High Dimensional Probability Distribution
Marginal Probability Distribution X Y Z U
Marginalize
Physics Fluctuomatics (Tohoku University)

27. Independency of Discrete Random Variable
If random variables X and Y are independent of each others,
Probability Distrubution of Random Variable Y
Joint Probability Distribution of Random Variables X and Y
Probability Distribution of Random Variable X
Marginal Probability Distribution of Random Varuiable Y
Physics Fluctuomatics (Tohoku University)

28. Covariance of Discrete Random Variables
Covariance of Random Variables X and Y
Covariance Matrix
Physics Fluctuomatics (Tohoku University)

29. Example of Probability Distribution
Physics Fluctuomatics (Tohoku University)

30. Example of Joint Probability Distributions
Cov[X,Y]
Physics Fluctuomatics (Tohoku University)

31. Example of Conditional Probability Distribution
Conditional Probability of Binary Symmetric Channel
Physics Fluctuomatics (Tohoku University)

32. Continuous Random Variable and Probability Density Function
For a random variable X defined in the state space (−∞,+∞), the probability that the state x is in the interval (a,b) in expressed as
Distribution Function
Probability Density Function
Physics Fluctuomatics (Tohoku University)

33. Continuous Random Variable and Probability Density Function
Normalization Condition
Physics Fluctuomatics (Tohoku University)

34. Average and Variance of Continuous Random Variable
Average of Random Variable X
Variance of Random Variable X
Physics Fluctuomatics (Tohoku University)

35. Continuous Random variables and Joint Probability Density Function
For random variables X and Y defined in the state space (−∞,+∞), the probability that the state vector (x,y) is in the region (a,b)(c,d) is expressed as
確率変数 X と Y の状態空間 (−∞,+∞) において状態 x と y が区間 (a,b)×(c,d) にある確率
Joint Probability Density Function
Normalization Condition
Physics Fluctuomatics (Tohoku University)

36. Continuous Random Variables and Marginal Probability Density Function
Marginal Probability Density Function of Random Variable Y
Physics Fluctuomatics (Tohoku University)

37. Independency of Continuous Random Variables
Random variables X and Y are independent of each other.
Probability Density Function of Y
Joint Probability Density Function of X and Y
Probability Density Function of X
Marginal Probability Density Function Y
Physics Fluctuomatics (Tohoku University)

38. Covariance of Continuous Random Variables
Covariance of Random Variables X and Y
Covariance Matrix
Physics Fluctuomatics (Tohoku University)

39. Uniform Distribution U(a,b)
Probability Density Function of Uniform Distribution
p(x) x a b
(b-a)-1
Physics Fluctuomatics (Tohoku University)

40. Gauss Distribution N(μ,σ2)
Probability Density Function of Gauss Distribution with average μ and variance σ2
p(x) μ x
The average and the variance are derived by means of Gauss Integral Formula
Physics Fluctuomatics (Tohoku University)

41. Multi-Dimensional Gauss Distribution
For a positive definite real symmetric matrix C, two-Dimensional Gaussian Distribution is defined by
The covariance matrix is given in terms of the matrix C as follows:
by using the following d -dimentional Gauss integral formula
Physics Fluctuomatics (Tohoku University)

42. Physics Fluctuomatics (Tohoku University)
Law of Large Numbers
Let us suppose that random variables X1,X2,...,Xn are identical and mutual independent random variables with average m. Then we have
Central Limit Theorem
We consider a sequence of independent, identical distributed random variables, {X1,X2,...,Xn}, with average m and variance s2. Then the distribution of the random variable
tends to the Gauss distribution with average m and variance s2/n as n+.
Physics Fluctuomatics (Tohoku University)

43. Physics Fluctuomatics (Tohoku University)
Summary
Event and Probability
Joint Probability and Conditional Probability
Bayes Formula, Prior Probability and Posterior Probability
Discrete Random Variable and Probability Distribution
Continuous Random Variable and Probability Density Function
Average, Variance and Covariance
Uniform Distribution
Gauss Distribution
Last Talk
Present Talk
Physics Fluctuomatics (Tohoku University)

44. Physics Fluctuomatics (Tohoku University)
Practice 2-1
Let us suppose that a random variable X takes binary values 1 and the probability distribution is given by
Derive the expression of average E[X] and variance V[X] and draw their graphs by using your personal computer.
Physics Fluctuomatics (Tohoku University)

45. Physics Fluctuomatics (Tohoku University)
Practice 2-2
Let us suppose that random variables X and Y take binary values 1 and the joint probability distribution is given by
Derive the expressions of Marginal Probability Destribution of X, P(X), and the covariance of X and Y, Cov[X,Y].
Physics Fluctuomatics (Tohoku University)

46. Physics Fluctuomatics (Tohoku University)
Practice 2-3
Let us suppose that random variables X and Y take binary values 1 and the conditional probability distribution is given by
Show that it is rewritten as
Hint
cosh(c) is an even function for any real number c.
Physics Fluctuomatics (Tohoku University)

47. Physics Fluctuomatics (Tohoku University)
Practice 2-4
Prove the Gauss integral formula:
Hint
Physics Fluctuomatics (Tohoku University)

48. Physics Fluctuomatics (Tohoku University)
Practice 2-5
Let us suppose that a continuous random variable X takes any real number and its probability density function is given by
Prove that the average E[X] and the variance V[X] are given by
Draw the graphs of p(x) for μ=0, σ=10, 20, 40 by using your personal computer.
Physics Fluctuomatics (Tohoku University)

49. Physics Fluctuomatics (Tohoku University)
Practice 2-6
Make a program for generating random numbers of uniform distribution U(0,1). Draw histgrams for N generated random numbers for N=10, 20, 50, 100 and 1000.
In the C language, you can use the function rand() that generate one of values 0,1,2,…,randmax, randomly. Here, randmax is the maximum value of outputs of rand().
Physics Fluctuomatics (Tohoku University)

50. Physics Fluctuomatics (Tohoku University)
Practice 2-7
Make a program that generates random numbers of Gauss distribution with average m and variance σ2. Draw histgrams for N generated random numbers for N=10, 20, 50, 100 and 1000.
For n random numbers x1,x2,…,xn generated by any probability distribution, (x1+x2+…+xn )/n tends to the Gauss distribution with average m and variance σ2/n for sufficient large n. [Central Limit Theorem]
Hint:
First we have to generate twelve uniform random numbers x1,x2,…,x12 in the interval [0,1].
Gauss random number with average 0 and variaince 1
σξ+μ generate Gauss random numbers with average μ and variance σ2
Physics Fluctuomatics (Tohoku University)

51. Physics Fluctuomatics (Tohoku University)
Practice 2-8
For any positive integer d and d d positive definite real symmetric matrix C, prove the following d-dimensional Gauss integral formulas:
Hint:
By using eigenvalues λi and their corresponding eigenvectors (i=1,2,…,d) of the matrix C, we have
Physics Fluctuomatics (Tohoku University)

52. Physics Fluctuomatics (Tohoku University)
Practice 2-9
We consider continuous random variables X1,X2,…,,Xd. The joint probability density function is given by
Prove that the average vector is and the covariance matrix is C.
Physics Fluctuomatics (Tohoku University)

53. Physics Fluctuomatics (Tohoku University)
Summary
Event and Probability
Joint Probability and Conditional Probability
Bayes Formulas, Prior Probability and Posterior Probability
Discrete Random Variable and Probability Distribution
Continuous Random Variable and Probability Density Function
Average, Variance and Covariance
Uniform Distribution
Gauss Distribution
Physics Fluctuomatics (Tohoku University)