1. © Tripos, L.P. All Rights Reserved Computational Challenges of PK/PD NLME Bob Leary Pharsight Corporation

2. © Tripos, L.P. All Rights Reserved Slide 2 Computational challenge #1 – make execution time reasonable Many PK/PD NLME software packages - NONMEM (with many choices for methods) is by far the most popular, but not necessarily always the most appropriate All methods are to some degree computationally intensive – execution time can be a limiting factor, even for a single run Many types of analyses require multiple runs (bootstrap, covariate search, likelihood profiling, etc. – execution time constraints can be severe).

3. © Tripos, L.P. All Rights Reserved Slide 3 Execution time, contd There are trades-offs between accuracy/statistical quality and speed: FO vs FOCE vs MCPEM/SAEM/NPAG Technology (parallel computing) can help a lot, but algorithmic improvements are at least equally important (SAEM, MCPEM vs. FOCE)

4. © Tripos, L.P. All Rights Reserved Slide 4

5. © Tripos, L.P. All Rights Reserved Slide 5 NPAG Outperforms NPEM CPU HRS MB LOG -LIK NPEM: 2037 10000 -433.1 NPAG: 0.5 6 -425.0

6. © Tripos, L.P. All Rights Reserved Slide 6 Computational Challenges #2 - #4 PK/PD NLME models and data are complex and computationally demanding, probably much more so that most other NLME application areas. Special purpose software is needed. Many of the methods are complex, not well documented, approximate, not easily understood by the user base, and at least somewhat fragile Software is relatively difficult to learn and use

7. © Tripos, L.P. All Rights Reserved Slide 7 A chronology of events in development of NLME 1972 – Sheiner, Rosenberg, Melmon paper (FO) 1977 – NONMEM group established at UCSF (L. Sheiner and S. Beal) 1979 – First NONMEM FO program appears 1986 – First nonparametric method NPML (A. Mallet) 1990 – First FOCE method (Lindstrom/Bates) 1990 – First Bayesian method (Gelfand/Smith – Bugs and PKBugs)

8. © Tripos, L.P. All Rights Reserved Slide 8 Chronology, contd 1991 - NPEM nonparametric method (Schumitzky) 1992 – First PAGE meeting (63 participants, 500+ in 2010) 1993 - First Laplacian method - enables general LL models (Wolfinger) 1999 – FDA Guidance for POP PK 2004 –2005 EM methods (SAEM, MCPEM, PEM), Lyon inter-method comparison exercises, MONOLIX 2007 – EMEA guidelines for POP PK 2009 – NONMEM SAEM/MCPEM/Bayesian, Pharsight PHOENIX

9. © Tripos, L.P. All Rights Reserved Slide 9 Some PK/PD software NONMEM (L. Sheiner and S. Beal, UCSF 1979 – to date) -primarily parametric modeling, although has primitive NP method -classical approximate likelihood methods (FO, FOCE, FOCEI, Laplacian) -new accurate likelihood EM methods (SAEM and MCPEM) (2009) -Bayesian methods (2009) USC*PACK (R. Jelliffe, USC/LAPK et al., 1993-to date) -nonparametric (NPEM, NPAG) (A. Schumitzky, R. Leary) -individual dosing optimization – multiple model control (D. Bayard)

10. © Tripos, L.P. All Rights Reserved Slide 10 PK/PD software, contd Monolix (INSERM, 2005 - to date) - SAEM (Stochastic Approximation Expectation Maximization) Adapt/S-Adapt (USC/BMSR, D. DArgenio, R. Bauer, 1989-to date) MCPEM (Monte Carlo Parametric Expectation Maximization) + Bayesian PHOENIX (Pharsight, 2009 – to date) classical NM methods + AGQ + SAEM + QMCPEM + NPAG + WinNonLin single subject and NCA modeling BUGS, WinBUGS – (1999 to date) – Bayesian S+ NLME, R NLME, SAS PROC-NLMIXED can be used, but not well suited for PK/PD

11. © Tripos, L.P. All Rights Reserved Slide 11 PK/PD Software User Base WinNonLin (Single Subject, NCA): 6000 (3000 academic, 3000 commercial) NONMEM (Population NLME): 1500 Commercial demand for experienced users exceeds supply

12. © Tripos, L.P. All Rights Reserved Slide 12 Population PK analysis is concerned with identifying and quantifying the random [random effects] and nonrandom [covariate effects] variability in the PK behavior of the patient population About 25% of recent submissions at time of writing included a population analysis Magnitude of random variability is particularly important because the safety and efficacy of a drug is affected. Mentions Standard Two Stage and NLME modeling as possible methods FDA Guidance for Industry, 1999

13. © Tripos, L.P. All Rights Reserved Slide 13 EMEA Guidelines 2007 NLME Pop PK analysis appears to be mandatory, or at least expected No mention of STS Extensive specification of model validation diagnostics and validation techniques (CWRES, predictive checks, etc.) Notes FDA Guidance is from 1999 and The FDA guidance should be read bearing in mind that it was written in 1999 and that population pharmacokinetics is an evolving science

14. © Tripos, L.P. All Rights Reserved Slide 14 Obligatory ODE section

15. © Tripos, L.P. All Rights Reserved Slide 15 ODE Considerations Most PK models are dynamical systems that can be described by ordinary differential equations (ODEs) ODEs often need to be solved numerically (many PK/PD software packages use ODEPACK, a library of ODE solvers developed by A. Hindmarsh at LLNL) If system is linear and homogeneous with constant coefficients, the matrix exponential can be used Some special cases (1, 2, and 3-compartment models) are best handled by built-in closed form solutions. Special handling capabilities are built in to the software for lag times, bioavailability, etc.

16. © Tripos, L.P. All Rights Reserved Slide 16 A Simple PK Model as ODE : 1-Compartment IV Bolus

17. © Tripos, L.P. All Rights Reserved Slide 17 IV Bolus closed form solution

18. © Tripos, L.P. All Rights Reserved Slide 18 Multiple Doses: Use superposition if model ODE is linear Covariate models with time varying covariates pose additional complications – suppose K=tvK(1+(coef)(SCR-SCR0))

19. © Tripos, L.P. All Rights Reserved Slide 19 1-Comp first order absorption extra-vascular dosing

20. © Tripos, L.P. All Rights Reserved Slide 20 1-Comp first order absorption extra-vascular dose solution

21. © Tripos, L.P. All Rights Reserved Slide 21 1 compartment 0-order (IV) dosing ODE

22. © Tripos, L.P. All Rights Reserved Slide 22 General N-compartment model : 0 and 1 st order dosing

23. © Tripos, L.P. All Rights Reserved Slide 23 Nonlinear cases must be solved numerically with ODE solvers (ODEPACK)

24. © Tripos, L.P. All Rights Reserved Slide 24 ODE solver order of preference/speed 1.Closed form (1, 2, 3 compartment, 0 and 1 st order dosing) 2.Matrix Exponential (Linear, constant coefficient) 3.Non-stiff numerical ODE solver (Runge-Kutta, Adams) 4.Stiff ODE solver (Gear BDF) Node execs = (Niter_out)(Nfix+Nran)(Nsub)(Niter_in)(Nran)(Ntime) (100)(10)(1000)(20)(5)(10) = 1,000,000,000

25. © Tripos, L.P. All Rights Reserved Slide 25 End of ODE section, Start of methods section

26. © Tripos, L.P. All Rights Reserved Slide 26 Simple (single subject) regression Model PK Model Data Residual Error Model

27. © Tripos, L.P. All Rights Reserved Slide 27 Extended least squares objective function

28. © Tripos, L.P. All Rights Reserved Slide 28 Computational challenge : minimize Nonlinear, nonconvex, But no likelihood approximations are necessary in single subject case Unconstrained (can add bound constraints if desired) No exploitable structure Use general purpose unconstrained quasi-Newton method UNCMIN from TOMS is 99+% reliable, but may encounter problems with multiple minima -

29. © Tripos, L.P. All Rights Reserved Slide 29 Regression model to estimate V and K

30. © Tripos, L.P. All Rights Reserved Slide 30 A simple population PK model: IV Bolus contd

31. © Tripos, L.P. All Rights Reserved Slide 31 Population Likelihood function

32. © Tripos, L.P. All Rights Reserved Slide 32 L i cannot be evaluated analytically – how to proceed? Numerical quadrature - adaptive Gaussian quadrature, Monte Carlo integration, quasi-Monte Carlo integration – very slow, dimensionality problems Laplace approximation – FO, FOCE, Laplace (Y. Wang, 2006) Use a method that does not require integration (SAEM,PEM, MCPEM, Bayesian methods, nonparametric methods)

33. © Tripos, L.P. All Rights Reserved Slide 33 Laplacian Approximation (FO, FOCE, Laplacian)

34. © Tripos, L.P. All Rights Reserved Slide 34 Joint log likelihood J( ) and Laplacian, FOCE, and FO approximations

35. © Tripos, L.P. All Rights Reserved Slide 35 Conditional methods (FOCE, Laplace) require nested optimizations to find mode of J, FO does not Each top level evaluation of requires Nsub mode-finding optimizations of Total number of innter optimizations = (Neval)(Nsub) - can easily reach 100,000 or more, leading to a reliability problem

36. © Tripos, L.P. All Rights Reserved Slide 36 Lyon 2004-2005 bake-off of NLME methods

37. © Tripos, L.P. All Rights Reserved Slide 37

38. © Tripos, L.P. All Rights Reserved Slide 38 STATISTICAL EFFICIENCIES

39. © Tripos, L.P. All Rights Reserved Slide 39 Approximate likelihoods can destroy statistical efficiency

40. © Tripos, L.P. All Rights Reserved Slide 40 SAEM, MCPEM, NPEM/NPAG

41. © Tripos, L.P. All Rights Reserved Slide 41 The ideal case –V i and K i can be observed Parametric estimators Nonparametric histogram

42. © Tripos, L.P. All Rights Reserved Slide 42 The real case: V i and K i are not directly observable We only have time profiles of drug plasma concentrations Measurement and dosing protocols are not uniform over different individuals At best, we can get estimates by solving a regression model

43. © Tripos, L.P. All Rights Reserved Slide 43 Standard Two-Stage Method V i and K i are estimated by simple nonlinear regression methods Parametric estimators Nonparametric histogram

44. © Tripos, L.P. All Rights Reserved Slide 44 MCPEM and SAEM are Monte Carlo versions of STS

45. © Tripos, L.P. All Rights Reserved Slide 45 NPEM and NAG: Many PK/PD populations have sub- populations that would be missed by parametric techniques A - True two-parameter population distribution B – Best normal approximation to population distribution

46. © Tripos, L.P. All Rights Reserved Slide 46 NPEM and NPAG 1.Assign an unknown probability (or probability density value) p j to each grid point 2.Grid the relevant portion of the (V,K) with grid points (Vj,Kj) 3.Estimate probabilities p j by maximizing the (exact) nonparametric log likelihood

47. © Tripos, L.P. All Rights Reserved Slide 47 NPEM vs NPAG NPEM uses a fixed, static grid and and EM algorithm to solve optimization problem (no formal numerical optimization) for the probabilities p j NPAG uses an adaptive grid (multiple iterations) and a convex special purpose primal-dual algorithm to optimize the log likelihood A later extension of NPAG incorporated a d-optimal design criterion based on the dual solution that enables candidate new grid points to be tested very rapidly for potential for improving the likelihood Final optimal nonparametric distribution is discrete with at most Nsub support points.

48. © Tripos, L.P. All Rights Reserved Slide 48 NPAG results format looks like ideal case of direct observation

49. © Tripos, L.P. All Rights Reserved Slide 49 PHX NPAG vs FOCE for bimodal distribution of Ke values