Presentation is loading. Please wait.

Presentation is loading. Please wait.

Distant Kin in the EM Family David A. van Dyk Department of Statistics University of California, Irvine. (Joint work with Xiao Li Meng and Taeyong Park.)

Similar presentations


Presentation on theme: "Distant Kin in the EM Family David A. van Dyk Department of Statistics University of California, Irvine. (Joint work with Xiao Li Meng and Taeyong Park.)"— Presentation transcript:

1

2 Distant Kin in the EM Family David A. van Dyk Department of Statistics University of California, Irvine. (Joint work with Xiao Li Meng and Taeyong Park.)

3 Outline The EM Family Tree The Stochastic Cousins Some Odd Relations oNoNested EM and The Partially Blocked Gibbs Sampler A Newly Found Kinsman oAoA Stochastic ECME/AECM Sampler: The Partially Collapsed Gibbs Sampler

4 The EM Family Tree EM Algorithm Stochastic Simulation Variance Calculations Gauss- Seidel Efficient DA Monte Carlo Integration ECM ECME SEM DA sampler MCEM PXEM Efficient DA EM NEM AECM SECM Algorithms Methods 1977 (Dempster, Laird, & Rubin) 1987 (Tanner & Wong) 1990 (Wie & Tanner) 1998 (Liu, Rubin, & Wu) 1995 (van Dyk, Meng, & Rubin) 1991 (Meng & Rubin) 1993 (Meng & Rubin) 2000 (van Dyk) 1994 (Liu & Rubin) 1997 (Meng & van Dyk)

5 Stochastic Cousins EM Algorithm Gauss- Seidel Efficient DA Monte Carlo Integration ECM ECME MCEM PXEM Efficient DA EM NEM AECM DA Sampler Gibbs Sampler Marginal DA PX-DA Partially Blocked Gibbs Sampler ???

6 The EM and DA Algorithms p(M| ) p( |M) p(M| ) p( |M) EM Algorithm DA Sampler Expectation Step Maximization Step Random Draw BACK

7 An NEM Algorithmwith a Monte Carlo E-step NEM AlgorithmThe Stochastic Version E-Step M-Step Draw p(M 1 | ) p( |M 1 ) p(M| ) p(, M 2 |M 1 ) p(M 1 |M 2, )p(M 2 |M 1, )p(M 1 |M 2, )p(M 2 |M 1, ) p( |M)p(M 2 |M 1, )p( |M)

8 A Partially-Blocked Sampler NEM AlgorithmPartially-Blocked Sampler E-Step M-Step Draw p(M 1 | ) p( |M 1 ) p(M 1 |, M 2 ) p(, M 2 |M 1 ) p(M 1 |M 2, )p(M 2 |M 1, ) p( |M)p(M 2 |M 1, )p( |M) BACK

9 Ordering CM-steps in ECME ECME Algorithm E-Step M-Step Draw p(M| ) p( 2 | 1 )p( 1 |M, 2 ) Monotone Convergence ECME Algorithm p(M| ) p( 1 |M, 2 )p( 2 | 1 ) NO Monotone Convergence Reducing conditioning Speed up convergence! But BE CAREFUL! Step order Matters!

10 A Stochastic Version of ECME ECME Algorithm E-Step M-Step Draw p(M| ) p( 2 | 1 )p( 1 |M, 2 ) p(M| ) p( 2 | 1 )p( 1 |M, 2 ) Incompatible Conditional Distributions What is the stationary distribution of this chain??

11 AECM and Partially Collapsed Samplers E M D p(M| ) p(M, 2 | 1 )p( 1 |M, 2 ) p(M| ) p( 1 |M, 2 )p( 2 | 1 ) p(M, 2 | 1 ) p( 1 |M, 2 ) p(M| ) p( 1 |M, 2 ) p( 2 |M 1, 1 ) p(M| ) p( 2 |M 1, 1 ) p( 1 |M, 2 ) ECME Partially Collapsed Blocked Sampler AECM Partially Collapsed Incompatible draws Stationary distribution must be verified!

12 Completely Collapsed Samplers E M D p(M| ) p(M, 2 | 1 )p(M, 1 | 2 ) p(M| ) p( 1 |M, 2 )p( 2 | 1 ) p( 1 | 2 ) ECME Collapsed Sampler Complete Collapse Blocking (ECME) is a special case of Partial Collapse (AECM). We expect Collapsed Samplers (CM) to perform better than Partially Collapsed Samplers (AECM). And we expect Collapsing (CM) to perform better than Blocking (ECME). Many of these relationships are known, I emphasize the connections between EM-type DA-type algorithms.

13 Reducing Conditioning in Gibbs: The Simplest Example Consider a two-step Gibbs Sampler: The Markov Chain has stationary dist’n With target margins but Without the correlation of the target distribution AND converges quickly! Iteration t Iteration t+1/2 We regain the target distribution with a one-step shifted chain.

14 Heads Up!! Reducing the conditioning within Gibbs involves new challenges: The order of the draws may effect the stationary distribution of the chain. The conditional distributions may no be compatible with any joint distribution. The steps sometimes can be blocked to form an ordinary Gibbs sampler with fewer steps.

15 An Example from Astronomy Parameterized Latent Poisson Process Underlying Poisson intensity is a mixture of a broad feature and several narrow features. The “line location” and mixture indicator are highly correlated. Photon energy Line location Emission Line Spectral Model for Photon Counts

16 An Example from Astronomy Standard sampler simulates Photon energy Line location Emission Line Spectral Model for Photon Counts which may converge very slowly or not at all.

17 An Incompatible Gibbs Sampler

18 Computational Gains

19 Verifying the Stationary Distribution of Sampler 2

20 The General Strategy BACK


Download ppt "Distant Kin in the EM Family David A. van Dyk Department of Statistics University of California, Irvine. (Joint work with Xiao Li Meng and Taeyong Park.)"

Similar presentations


Ads by Google