Download presentation

Presentation is loading. Please wait.

Published byDerek Chadburn Modified over 2 years ago

1
1 Maximum Likelihood Estimates and the EM Algorithms II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University hslu@stat.nctu.edu.tw http://tigpbp.iis.sinica.edu.tw/courses.htm

2
2 Part 1 Computation Tools

3
3 Include Functions in R source( “ file path ” ) Example In MME.R: In R:

4
4 Part 2 Motivation Examples

5
5 Example 1 in Genetics (1) Two linked loci with alleles A and a, and B and b A, B: dominant a, b: recessive A double heterozygote AaBb will produce gametes of four types: AB, Ab, aB, ab F ( Female) 1- r ’ r ’ (female recombination fraction) M (Male) 1-r r (male recombination fraction) A Bb a B A b a a B b A A B b a 5

6
6 Example 1 in Genetics (2) r and r ’ are the recombination rates for male and female Suppose the parental origin of these heterozygote is from the mating of. The problem is to estimate r and r ’ from the offspring of selfed heterozygotes. Fisher, R. A. and Balmukand, B. (1928). The estimation of linkage from the offspring of selfed heterozygotes. Journal of Genetics, 20, 79 – 92. http://en.wikipedia.org/wiki/Genetics http://www2.isye.gatech.edu/~brani/isyebayes/ba nk/handout12.pdf http://en.wikipedia.org/wiki/Genetics http://www2.isye.gatech.edu/~brani/isyebayes/ba nk/handout12.pdf 6

7
7 Example 1 in Genetics (3) MALE AB (1-r)/2 ab (1-r)/2 aB r/2 Ab r/2 FEMALEFEMALE AB (1-r ’ )/2 AABB (1-r) (1-r ’ )/4 aABb (1-r) (1-r ’ )/4 aABB r (1-r ’ )/4 AABb r (1-r ’ )/4 ab (1-r ’ )/2 AaBb (1-r) (1-r ’ )/4 aabb (1-r) (1-r ’ )/4 aaBb r (1-r ’ )/4 Aabb r (1-r ’ )/4 aB r ’ /2 AaBB (1-r) r ’ /4 aabB (1-r) r ’ /4 aaBB r r ’ /4 AabB r r ’ /4 Ab r ’ /2 AABb (1-r) r ’ /4 aAbb (1-r) r ’ /4 aABb r r ’ /4 AAbb r r ’ /4 7

8
8 Example 1 in Genetics (4) Four distinct phenotypes: A*B*, A*b*, a*B* and a*b*. A*: the dominant phenotype from (Aa, AA, aA). a*: the recessive phenotype from aa. B*: the dominant phenotype from (Bb, BB, bB). b* : the recessive phenotype from bb. A*B*: 9 gametic combinations. A*b*: 3 gametic combinations. a*B*: 3 gametic combinations. a*b*: 1 gametic combination. Total: 16 combinations. 8

9
9 Example 1 in Genetics (5) 9

10
10 Example 1 in Genetics (6) Hence, the random sample of n from the offspring of selfed heterozygotes will follow a multinomial distribution: 10

11
11 Example 1 in Genetics (7) Suppose that we observe the data of y = (y1, y2, y3, y4) = (125, 18, 20, 24), which is a random sample from Then the probability mass function is 11

12
12 Maximum Likelihood Estimate (MLE) Likelihood: Maximize likelihood: Solve the score equations, which are setting the first derivates of likelihood to be zeros. Under regular conditions, the MLE is consistent, asymptotic efficient and normal! More: http://en.wikipedia.org/wiki/Maximum_lik elihood 12

13
13 MLE for Example 1 (1) Likelihood MLE: A B C 13

14
14 MLE for Example 1 (2) Checking: (1) (2) (3) 14

15
15 Part 3 Numerical Solutions for the Score Equations of MLEs

16
16 A Banach Space A Banach space B is a vector space over the field K such that 1. 2. 3. 4. 5. Every Cauchy sequence of B converges in B (i.e., B is complete). (http://en.wikipedia.org/wiki/Banach_space)http://en.wikipedia.org/wiki/Banach_space

17
17 Lipschitz Continuous A closed subset and mapping 1. F is Lipschitz continuous on A with if. 2. F is a contraction mapping on A if F is Lipschitz continuous and (http://en.wikipedia.org/wiki/Lipschitz_continuous)http://en.wikipedia.org/wiki/Lipschitz_continuous

18
18 Fixed Point Theorem If F is a contraction mapping on A if F is Lipschitz continuous and 1. F has an unique fixed point such that 2. initial, k=1,2, … 3. ( http://en.wikipedia.org/wiki/Fixed-point_theorem) http://en.wikipedia.org/wiki/Fixed-point_theorem (http://www.math-linux.com/spip.php?article60)http://www.math-linux.com/spip.php?article60

19
19 Applications for MLE (1)

20
20 Applications for MLE (2) Optimal ?

21
21 Parallel Chord Method (1) Parallel chord method is also called simple iteration.

22
22 s Parallel Chord Method (2)

23
23 Plot the Parallel Chord Method by R

24
24 Define Functions for Example 1 in R We will define some functions and variables for finding the MLE in Example 1 by R

25
25 Parallel Chord Method by R (1)

26
26 Parallel Chord Method by R (2)

27
27 Parallel Chord Method by C/C++

28
28 http://math.fullerton.edu/mathews/n2003 /Newton'sMethodMod.html http://math.fullerton.edu/mathews/n2003 /Newton'sMethodMod.html http://en.wikipedia.org/wiki/Newton%27s _method http://en.wikipedia.org/wiki/Newton%27s _method Newton-Raphson Method (1)

29
29 s Newton-Raphson Method (2)

30
30 Plot the Newton-Raphson Method by R

31
31 Newton-Raphson Method by R (1)

32
32 Newton-Raphson Method by R (2)

33
33 Newton-Raphson Method by C/C++

34
34 Halley ’ s Method The Newton-Raphson iteration function is It is possible to speed up convergence by using more expansion terms than the Newton-Raphson method does when the object function is very smooth, like the method by Edmond Halley (1656-1742): (http://math.fullerton.edu/mathews/n2003/Halley'sMethodMod.html)http://math.fullerton.edu/mathews/n2003/Halley'sMethodMod.html

35
35 Halley ’ s Method by R (1)

36
36 Halley ’ s Method by R (2)

37
37 Halley ’ s Method by C/C++

38
38 Bisection Method (1) Assume that and that there exists a number such that. If and have opposite signs, and represents the sequence of midpoints generated by the bisection process, then and the sequence converges to r. That is,. (http://en.wikipedia.org/wiki/Bisection_method )http://en.wikipedia.org/wiki/Bisection_method

39
39 1 Bisection Method (2)

40
40 Plot the Bisection Method by R

41
41 Bisection Method by R (1) > fix(Bisection)

42
42 Bisection Method by R (2)

43
43 Bisection Method by R (3)

44
44 Bisection Method by C/C++ (1)

45
45 Bisection Method by C/C++ (2)

46
46 Secant Method (http://en.wikipedia.org/wiki/Secant_method )http://en.wikipedia.org/wiki/Secant_method (http://math.fullerton.edu/mathews/n2003/Secant MethodMod.html )http://math.fullerton.edu/mathews/n2003/Secant MethodMod.html

47
47 Secant Method by R (1) >fix(Secant)

48
48 Secant Method by R (2)

49
49 Secant Method by C/C++

50
50 Secant-Bracket Method The secant-bracket method is also called the regular falsi method. S C A B

51
51 Secant-Bracket Method by R (1) >fix(RegularFalsi)

52
52 Secant-Bracket Method by R (2)

53
53 Secant-Bracket Method by R (3)

54
54 Secant-Bracket Method by C/C++ (1)

55
55 Secant-Bracket Method by C/C++ (1)

56
56 Fisher Scoring Method Fisher scoring method replaces by where is the Fisher information matrix when the parameter may be multivariate.

57
57 Fisher Scoring Method by R (1) > fix(Fisher)

58
58 Fisher Scoring Method by R (2)

59
59 Fisher Scoring Method by C/C++

60
60 Order of Convergence Order of convergence is p if and c<1 for p=1. (http://en.wikipedia.org/wiki/Order_of_convergence)http://en.wikipedia.org/wiki/Order_of_convergence Note: Hence, we can use regression to estimate p.

61
61 Theorem for Newton-Raphson Method If, F is a contraction mapping then p=1 and If exists, has a simple zero, then such that of the Newton-Raphson method is a contraction mapping and p=2.

62
62 Find Convergence Order by R (1) R=Newton(y1, y2, y3, y4, initial) #Newton method can be substitute for different method temp=log(abs(R$iteration-R$phi)); y=temp[2:(length(temp)-1)] x=temp[1:(length(temp)-2)] lm(y~x)

63
63 Find Convergence Order by R (2)

64
64 Find Convergence Order by R (3)

65
65 Find Convergence Order by C/C++

66
66 Exercises Write your own programs for those examples presented in this talk. Write programs for those examples mentioned at the following web page: http://en.wikipedia.org/wiki/Maximum_li kelihood Write programs for the other examples that you know. 66

67
67 More Exercises (1) Example 3 in genetics: The observed data are (nO, nA, nB, nAB) = (176, 182, 60, 17) ~ Multinomial(r^2, p^2+2pr, q^2+2qr, 2pq), where p, q, and r fall in [0,1] such that p+q+r = 1. Find the MLEs for p, q, and r. 67

68
68 More Exercises (2) Example 4 in the positron emission tomography (PET): The observed data are n*(d) ~Poisson(λ*(d)), d = 1, 2, …, D, and The values of p(b,d) are known and the unknown parameters are λ(b), b = 1, 2, …, B. Find the MLEs for λ(b), b = 1, 2, …, B.. 68

69
69 More Exercises (3) Example 5 in the normal mixture: The observed data x i, i = 1, 2, …, n, are random samples from the following probability density function: Find the MLEs for the following parameters: 69

Similar presentations

Presentation is loading. Please wait....

OK

Maximum likelihood (ML)

Maximum likelihood (ML)

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on indian defence services Ppt on question tags example 3d holographic display ppt on tv Ppt on abstract art coloring Ppt on indian retail industry Ppt on water pollution download Ppt on indian politics quotes Types of window display ppt on tv Ppt on brand equity Ppt on team building training module