1. Pharmacometrics research group Department of Pharmaceutical Biosciences Uppsala University Approximations of the population Fisher information matrix- differences and consequences Joakim Nyberg, Sebastian Ueckert, Andrew C. Hooker

2. 2 Background At PODE 2009 all Population Optimal Design (OD) Software should evaluate the same simple Warfarin problem… 1-compartment model, 1st order absorption, oral dose 70 mg Proportional error model (σ 2 =0.01) 32 subjects with 8 measurements at 0.5, 1, 2, 6,24, 36, 72,120 hours (evaluation) ParametersFixed effectsω 2 (IIV, exp) CL/F (L/h)0.150.07 V/F (L)8.00.02 ka (1/h)1.00.6

3. 3 Fisher Information Matrix (FIM) FIM can be calculated in different ways: A* is somewhat modified/updated if full is used, i.e. Assuming var(y) w.r.t. the fixed effects≠0 Assuming var(y) w.r.t. the fixed effects=0 Different between full and reduced

4. 4 Fisher Information Matrix (FIM) The FIM If we have correlation between fixed effects and random effects in like the FULL is the “theoretically correct” method. If not, the Reduced is “correct theoretically” but this is seldom the case in Pharmacometrics

5. 5 Results from last PODE 2009 The “truth” * Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003 * *

6. 6 Results from last PODE 2009 The “truth”

7. 7 Results from last PODE 2009 Possibly issues with the Cramér-Rao inequality

8. 8 Results from last PODE 2009 summary Software gave similar results with similar approximations Reduced superior to Full in terms of predicting the “truth” Even less predictive performance with higher order FOCE-based FIM.

9. 9 Possible reasons – Initial ideas The derivation of Full or Red is wrong - Derive FIM with simulations, i.e. integrate over observed FIM FO-approximation too poor - FOCE is obviously not enough, try high order approximations Asymptotic behavior (FIM -1 ≤COV) - Increase data set x 2 => SE should decrease by 2 (1/2) Numerical instability in Full but not in Red FIM - Using automatic differentiation (AD) to avoid step length issues Estimation software is not true ML-estimator, i.e. efficiency of estimator not accurate - NM hard to know how the parameter search is performed but Monolix well documented

10. 10 Investigations – Reducing the complexity ln-transform model to have additive res-error (avoiding interaction terms) Check that the problem holds for prop IIV structure (FO approximation => proportional IIV = exp IIV) Fix all parameters except fixed effect Ka

11. 11 Results – Reducing the complexity Θ ka RSE(%) [CI] “Truth” NONMEM FOCE SSE (1000)13.59% * [13.06-13.88] PopED Full (Analytic)6.71% PopED Reduced (Analytic)13.90% PopED Full FOCE 10 000 **4.95% PopED Red FOCE 10 000 **6.49% * 100 000 bootstrap samples ln model, add error, exp IIV = prop IIV Issues still remaining => work with simplified model ** Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003

12. 12 Results – Full vs Reduced Asymptotic behavior (FIM -1 ≤COV) - Increase data set x 2 => SE should decrease by 2 (1/2) Numerical instability in Full but not in Red FIM - Using automatic differentiation (AD) to avoid step length issues None of this affected the results (2 down 3 to go)

13. 13 Next things to try... The derivation of Full or Red is wrong - Derive FIM with simulations, i.e. integrate over observed FIM FO-approximation too poor - FOCE is obviously not enough, try high order approximations SO

14. 14 Results – High order approximations & simulation based derivations * 100 000 bootstrap samples Θ ka RSE(%) “Truth” NONMEM FOCE SSE (1000)13.59% [13.06-13.88] * NONMEM Integrated (400) Full FOCE14.00% PopED Full Analytic FO6.71% PopED Integrated (100) Full FO6.80% PopED Full Analytic SO8.94% PopED Full FOCE 10 0004.95% PopED Reduced Analytic FO13.90% PopED Red Analytic SO14.04% PopED Red FOCE 10 0006.49% Similar SO – Closer to truth Simulation based FIM FOCE from NONMEM solve the problem!

15. 15 Results Full vs Red The derivation of Full or Red is wrong Integration FIM ≈ Analytic FIM => Not the answer FO-approximation too poor - SO shrinks the differences but still to poor of an approx. - FOCE is worse but NONMEM integrated FOCE FIM is good?! - Possibly issues with the FOCE method?

16. 16 FOCE FIM – Differences & Improvements NONMEM FOCE assumes linearization around the mode of the distribution => correlation between the individual parameters and the population parameters. Analytic FIM FOCE * does not assume this -To calculate individual mode data is needed Update Analytic FIM FOCE to include the correlation: Calculate Expected Empirical Bayes Estimates (EEBE) EEBE are not data dependent Whenever PopED differentiates pop parameters; differentiate EEBE as well * Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003

17. 17 Results – Updated FIM FOCE Θ ka RSE(%) Old FOCE method Full (1000) (4.46s)4.95% Old FOCE method Red (1000) (2.6s)6.49% New FOCE method (1000) Full (~40s)13.62% New FOCE method (1000) Red (~23.7s)12.5% - 13.8% The new FOCE method solves the problem!

18. 18 The answer The Full FIM does not always work with the FO-approximation

19. 19 When to use which method?

20. 20 Does Full/Red affect the optimal design? Full Reduced 1 2 2 2 1 6 1 1 3 support points 5 support points

21. 21 Does linearization method affect the optimal design? Optimal SOCE SOCE>MC SOCE=MC SOCE<MC Example from PAGE 2009 Nyberg et al Surface of |FIM| for SOCE-MC Optimal MC

22. 22 Always use reduce FIM instead? Reduced |FIM| Results from Nyberg et al PODE 2008 Full |FIM| Similar results with the transit compartment model

23. 23 Surface of RSE(%) – Full FIM only different from reduced in some regions Full Ka RSE(%) Reduced Ka RSE(%) S3-S7: (7.79h) S8: (120h) Full ≈ Red

24. 24 Conclusions The FO-approximation is not always enough for Full FIM Possibly also too poor approx. for Reduced Reduced FIM collapses occasionally High order approximations stabilize differences Different approximations give different optimal designs, e.g. different sampling times and different number of support points

25. 25 Suggestions 1)If runtime allows – Use high order approximations FOCE, SO, SOCE, MC etc. 2)If Red is stable – Use reduced to optimize but evaluate with both 3)If Red is unstable – Optimize with Full but evaluate with Red Beware: - No “golden” solution is presented - The Cramer-Rao inequality does not hold comparing different methods when optimizing / estimating - To get “correct” SE from the FIM either sim/est needs to be performed or high order FIMs need to be evaluated

26. 26 Thank you I would like to acknowledge Sergei Leonov for our interesting emails discussing these issues.