Download presentation

Presentation is loading. Please wait.

Published byDerek Chadburn Modified about 1 year ago

1
1 Maximum Likelihood Estimates and the EM Algorithms II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University

2
2 Part 1 Computation Tools

3
3 Include Functions in R source( “ file path ” ) Example In MME.R: In R:

4
4 Part 2 Motivation Examples

5
5 Example 1 in Genetics (1) Two linked loci with alleles A and a, and B and b A, B: dominant a, b: recessive A double heterozygote AaBb will produce gametes of four types: AB, Ab, aB, ab F ( Female) 1- r ’ r ’ (female recombination fraction) M (Male) 1-r r (male recombination fraction) A Bb a B A b a a B b A A B b a 5

6
6 Example 1 in Genetics (2) r and r ’ are the recombination rates for male and female Suppose the parental origin of these heterozygote is from the mating of. The problem is to estimate r and r ’ from the offspring of selfed heterozygotes. Fisher, R. A. and Balmukand, B. (1928). The estimation of linkage from the offspring of selfed heterozygotes. Journal of Genetics, 20, 79 – 92. nk/handout12.pdf nk/handout12.pdf 6

7
7 Example 1 in Genetics (3) MALE AB (1-r)/2 ab (1-r)/2 aB r/2 Ab r/2 FEMALEFEMALE AB (1-r ’ )/2 AABB (1-r) (1-r ’ )/4 aABb (1-r) (1-r ’ )/4 aABB r (1-r ’ )/4 AABb r (1-r ’ )/4 ab (1-r ’ )/2 AaBb (1-r) (1-r ’ )/4 aabb (1-r) (1-r ’ )/4 aaBb r (1-r ’ )/4 Aabb r (1-r ’ )/4 aB r ’ /2 AaBB (1-r) r ’ /4 aabB (1-r) r ’ /4 aaBB r r ’ /4 AabB r r ’ /4 Ab r ’ /2 AABb (1-r) r ’ /4 aAbb (1-r) r ’ /4 aABb r r ’ /4 AAbb r r ’ /4 7

8
8 Example 1 in Genetics (4) Four distinct phenotypes: A*B*, A*b*, a*B* and a*b*. A*: the dominant phenotype from (Aa, AA, aA). a*: the recessive phenotype from aa. B*: the dominant phenotype from (Bb, BB, bB). b* : the recessive phenotype from bb. A*B*: 9 gametic combinations. A*b*: 3 gametic combinations. a*B*: 3 gametic combinations. a*b*: 1 gametic combination. Total: 16 combinations. 8

9
9 Example 1 in Genetics (5) 9

10
10 Example 1 in Genetics (6) Hence, the random sample of n from the offspring of selfed heterozygotes will follow a multinomial distribution: 10

11
11 Example 1 in Genetics (7) Suppose that we observe the data of y = (y1, y2, y3, y4) = (125, 18, 20, 24), which is a random sample from Then the probability mass function is 11

12
12 Maximum Likelihood Estimate (MLE) Likelihood: Maximize likelihood: Solve the score equations, which are setting the first derivates of likelihood to be zeros. Under regular conditions, the MLE is consistent, asymptotic efficient and normal! More: elihood 12

13
13 MLE for Example 1 (1) Likelihood MLE: A B C 13

14
14 MLE for Example 1 (2) Checking: (1) (2) (3) 14

15
15 Part 3 Numerical Solutions for the Score Equations of MLEs

16
16 A Banach Space A Banach space B is a vector space over the field K such that Every Cauchy sequence of B converges in B (i.e., B is complete). (http://en.wikipedia.org/wiki/Banach_space)http://en.wikipedia.org/wiki/Banach_space

17
17 Lipschitz Continuous A closed subset and mapping 1. F is Lipschitz continuous on A with if. 2. F is a contraction mapping on A if F is Lipschitz continuous and (http://en.wikipedia.org/wiki/Lipschitz_continuous)http://en.wikipedia.org/wiki/Lipschitz_continuous

18
18 Fixed Point Theorem If F is a contraction mapping on A if F is Lipschitz continuous and 1. F has an unique fixed point such that 2. initial, k=1,2, … 3. ( (http://www.math-linux.com/spip.php?article60)http://www.math-linux.com/spip.php?article60

19
19 Applications for MLE (1)

20
20 Applications for MLE (2) Optimal ?

21
21 Parallel Chord Method (1) Parallel chord method is also called simple iteration.

22
22 s Parallel Chord Method (2)

23
23 Plot the Parallel Chord Method by R

24
24 Define Functions for Example 1 in R We will define some functions and variables for finding the MLE in Example 1 by R

25
25 Parallel Chord Method by R (1)

26
26 Parallel Chord Method by R (2)

27
27 Parallel Chord Method by C/C++

28
28 /Newton'sMethodMod.html /Newton'sMethodMod.html _method _method Newton-Raphson Method (1)

29
29 s Newton-Raphson Method (2)

30
30 Plot the Newton-Raphson Method by R

31
31 Newton-Raphson Method by R (1)

32
32 Newton-Raphson Method by R (2)

33
33 Newton-Raphson Method by C/C++

34
34 Halley ’ s Method The Newton-Raphson iteration function is It is possible to speed up convergence by using more expansion terms than the Newton-Raphson method does when the object function is very smooth, like the method by Edmond Halley ( ): (http://math.fullerton.edu/mathews/n2003/Halley'sMethodMod.html)http://math.fullerton.edu/mathews/n2003/Halley'sMethodMod.html

35
35 Halley ’ s Method by R (1)

36
36 Halley ’ s Method by R (2)

37
37 Halley ’ s Method by C/C++

38
38 Bisection Method (1) Assume that and that there exists a number such that. If and have opposite signs, and represents the sequence of midpoints generated by the bisection process, then and the sequence converges to r. That is,. (http://en.wikipedia.org/wiki/Bisection_method )http://en.wikipedia.org/wiki/Bisection_method

39
39 1 Bisection Method (2)

40
40 Plot the Bisection Method by R

41
41 Bisection Method by R (1) > fix(Bisection)

42
42 Bisection Method by R (2)

43
43 Bisection Method by R (3)

44
44 Bisection Method by C/C++ (1)

45
45 Bisection Method by C/C++ (2)

46
46 Secant Method (http://en.wikipedia.org/wiki/Secant_method )http://en.wikipedia.org/wiki/Secant_method (http://math.fullerton.edu/mathews/n2003/Secant MethodMod.html )http://math.fullerton.edu/mathews/n2003/Secant MethodMod.html

47
47 Secant Method by R (1) >fix(Secant)

48
48 Secant Method by R (2)

49
49 Secant Method by C/C++

50
50 Secant-Bracket Method The secant-bracket method is also called the regular falsi method. S C A B

51
51 Secant-Bracket Method by R (1) >fix(RegularFalsi)

52
52 Secant-Bracket Method by R (2)

53
53 Secant-Bracket Method by R (3)

54
54 Secant-Bracket Method by C/C++ (1)

55
55 Secant-Bracket Method by C/C++ (1)

56
56 Fisher Scoring Method Fisher scoring method replaces by where is the Fisher information matrix when the parameter may be multivariate.

57
57 Fisher Scoring Method by R (1) > fix(Fisher)

58
58 Fisher Scoring Method by R (2)

59
59 Fisher Scoring Method by C/C++

60
60 Order of Convergence Order of convergence is p if and c<1 for p=1. (http://en.wikipedia.org/wiki/Order_of_convergence)http://en.wikipedia.org/wiki/Order_of_convergence Note: Hence, we can use regression to estimate p.

61
61 Theorem for Newton-Raphson Method If, F is a contraction mapping then p=1 and If exists, has a simple zero, then such that of the Newton-Raphson method is a contraction mapping and p=2.

62
62 Find Convergence Order by R (1) R=Newton(y1, y2, y3, y4, initial) #Newton method can be substitute for different method temp=log(abs(R$iteration-R$phi)); y=temp[2:(length(temp)-1)] x=temp[1:(length(temp)-2)] lm(y~x)

63
63 Find Convergence Order by R (2)

64
64 Find Convergence Order by R (3)

65
65 Find Convergence Order by C/C++

66
66 Exercises Write your own programs for those examples presented in this talk. Write programs for those examples mentioned at the following web page: kelihood Write programs for the other examples that you know. 66

67
67 More Exercises (1) Example 3 in genetics: The observed data are (nO, nA, nB, nAB) = (176, 182, 60, 17) ~ Multinomial(r^2, p^2+2pr, q^2+2qr, 2pq), where p, q, and r fall in [0,1] such that p+q+r = 1. Find the MLEs for p, q, and r. 67

68
68 More Exercises (2) Example 4 in the positron emission tomography (PET): The observed data are n*(d) ~Poisson(λ*(d)), d = 1, 2, …, D, and The values of p(b,d) are known and the unknown parameters are λ(b), b = 1, 2, …, B. Find the MLEs for λ(b), b = 1, 2, …, B.. 68

69
69 More Exercises (3) Example 5 in the normal mixture: The observed data x i, i = 1, 2, …, n, are random samples from the following probability density function: Find the MLEs for the following parameters: 69

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google