Lecture 2 Probability and Measurement Error, Part 1

Syllabus Lecture 01Describing Inverse Problems Lecture 02Probability and Measurement Error, Part 1 Lecture 03Probability and Measurement Error, Part 2 Lecture 04The L 2 Norm and Simple Least Squares Lecture 05A Priori Information and Weighted Least Squared Lecture 06Resolution and Generalized Inverses Lecture 07Backus-Gilbert Inverse and the Trade Off of Resolution and Variance Lecture 08The Principle of Maximum Likelihood Lecture 09Inexact Theories Lecture 10Nonuniqueness and Localized Averages Lecture 11Vector Spaces and Singular Value Decomposition Lecture 12Equality and Inequality Constraints Lecture 13L 1, L ∞ Norm Problems and Linear Programming Lecture 14Nonlinear Problems: Grid and Monte Carlo Searches Lecture 15Nonlinear Problems: Newton’s Method Lecture 16Nonlinear Problems: Simulated Annealing and Bootstrap Confidence Intervals Lecture 17Factor Analysis Lecture 18Varimax Factors, Empircal Orthogonal Functions Lecture 19Backus-Gilbert Theory for Continuous Problems; Radon’s Problem Lecture 20Linear Operators and Their Adjoints Lecture 21Fréchet Derivatives Lecture 22 Exemplary Inverse Problems, incl. Filter Design Lecture 23 Exemplary Inverse Problems, incl. Earthquake Location Lecture 24 Exemplary Inverse Problems, incl. Vibrational Problems

Purpose of the Lecture review random variables and their probability density functions introduce correlation and the multivariate Gaussian distribution relate error propagation to functions of random variables

Part 1 random variables and their probability density functions

random variable, d no fixed value until it is realized indeterminate d=1.04 indeterminate d=0.98

random variables have systematics tendency to takes on some values more often than others

N realization of data index, i didi

(C)(B)(A)

p(d) ΔdΔd d probability of variable being between d and d+Δd is p(d) Δd

in general probability is the integral probability that d is between d 1 and d 2

the probability that d has some value is 100% or unity probability that d is between its minimum and maximum bounds, d min and d max

How do these two p.d.f.’s differ? d p(d) d

Summarizing a probability density function typical value “center of the p.d.f.” amount of scatter around the typical value “width of the p.d.f.”

p(d) d d ML Several possibilities for a typical value point beneath the peak or “maximum likelihood point” or “mode”

p(d) d d median point dividing area in half or “median” 50%

p(d) d balancing point or “mean” or “expectation”

can all be different d ML ≠ d median ≠

formula for “mean” or “expected value”

≈ s d dsds ≈ s NsNs N data histogram NsNs dsds p ≈ s P(d s ) probability distribution step 1: usual formula for mean step 2: replace data with its histogram step 3: replace histogram with probability distribution.

If the data are continuous, use analogous formula containing an integral: ≈ s P(d s )

quantifying width

This function grows away from the typical value: q(d) = (d- ) 2 so the function q(d)p(d) is small if most of the area is near, that is, a narrow p(d) large if most of the area is far from, that is, a wide p(d) so quantify width as the area under q(d)p(d)

(C)(B)(A) (F)(E)(D)

variance width is actually square root of variance, that is, σ mean

estimating mean and variance from data

usual formula for “sample mean” usual formula for square of “sample standard deviation”

MabLab scripts for mean and variance d bar = Dd*sum(d.*p); q = (d-dbar).^2; sigma2 = Dd*sum(q.*p); sigma = sqrt(sigma2); from tabulated p.d.f. p from realizations of data dbar = mean(dr); sigma = std(dr); sigma2 = sigma^2;

two important probability density functions: uniform Gaussian (or Normal)

uniform p.d.f. d d min d max p(d) 1/(d max- d min ) probability is the same everywhere in the range of possible values box-shaped function

Large probability near the mean, d. Variance is σ 2. d 2σ2σ Gaussian (or “Normal”) p.d.f. bell-shaped function

probability between ±nσ Gaussian p.d.f.

Part 2 correlated errors

uncorrelated random variables no pattern of between values of one variable and values of another when d 1 is higher than its mean d 2 is higher or lower than its mean with equal probability

d1d1 d2d p joint probability density function uncorrelated case

d1d1 d2d p no tendency for d 2 to be either high or low when d 1 is high

in uncorrelated case joint p.d.f. is just the product of individual p.d.f.’s

d1d1 d2d p 2σ12σ1

d1d1 d2d p 2σ12σ1 tendency for d 2 to be high when d 1 is high

d1d1 d2d p 2σ12σ1 2σ12σ1 2σ22σ2 θ

d1d1 d2d2 2σ12σ1

d1d1 d2d2 d1d1 d2d2 d1d1 d2d2 (A)(B)(C)

formula for covariance + positive correlation high d 1 high d 2 - negative correlation high d 1 low d 2

joint p.d.f. mean is a vector covariance is a symmetric matrix diagonal elements: variances off-diagonal elements: covariances

estimating covariance from a table D of data D ki : realization k of data-type i in MatLab, C=cov(D)

un ivariate p.d.f. formed from joint p.d.f. p(d) → p(d i ) behavior of d i irrespective of the other d s integrate over everything but d i

d1d1 d2d2 d1d1 integrate over d 2 integrate over d 1 d2d2 p(d 1,d 2 ) p(d 2 ) p(d 1 )

multivariate Gaussian (or Normal) p.d.f.

covariance matrix mean

Part 3 functions of random variables

functions of random variables data with measurement error data analysis process inferences with uncertainty

functions of random variables given p(d) with m=f(d) what is p(m) ?

univariate p.d.f. use the chair rule

rule for transforming a univariate p.d.f.

multivariate p.d.f. determinant of matrix with elements ∂d i /∂m j Jacobian determinant

multivariate p.d.f. rule for transforming a multivariate p.d.f. Jacobian determinant

simple example data with measurement error data analysis process inferences with uncertainty one datum, d uniform p.d.f. 0<d<1 m = 2d one model parameter, m

m=2d and d=½m d=0 → m=0 d=1 → m=2 dd/dm = ½ p(d ) =1 p(m ) = ½

p(d) 012 d0 1 p(m) 012 m0 1

another simple example data with measurement error data analysis process inferences with uncertainty one datum, d uniform p.d.f. 0<d<1 m = d 2 one model parameter, m

m=d 2 and d=m ½ d=0 → m=0 d=1 → m=1 dd/dm = ½m -½ p(d ) =1 p(m ) = ½ m -½

A) B) p(d) p(m) d m

multivariate example data with measurement error data analysis process inferences with uncertainty 2 data, d 1, d 2 uniform p.d.f. 0<d<1 for both m 1 = d 1 +d 2 m 2 = d 2 -d 2 2 model parameters, m 1 and m 2

d2d2 d1d1 1 1 m 1 =d 1 +d 1 m 2 =d 1 -d 1 (A)

p(d ) = 1 m=Md with d=M -1 m so ∂d/ ∂ m = M -1 and J= |det(M -1 )|= ½ and |det(M)|=2 p(m ) = ½

d2d2 d1d1 1 1 m 1 =d 1 +d 1 m 2 =d 1 -d 1 m2m2 m1m d1d1 p(d 1 ) 0.5 p(m 1 ) 0 m1m (A) (B) (C) (D) Note that the shape in m is different than the shape in d

d2d2 d1d1 1 1 m 1 =d 1 +d 1 m 2 =d 1 -d 1 m2m2 m1m d1d1 p(d 1 ) 0.5 p(m 1 ) 0 m1m (A) (B) (C) (D) The shape is a square with sides of length √ 2. The amplitude is ½. So the area is ½ ⨉√ 2 ⨉√ 2 = 1

d2d2 d1d1 1 1 m 1 =d 1 +d 1 m 2 =d 1 -d 1 m2m2 m1m d1d1 p(d 1 ) 0.5 p(m 1 ) 0 m1m (A) (B) (C) (D)

moral p(m) can behavior quite differently from p(d)