Presentation is loading. Please wait.

Presentation is loading. Please wait.

From Population to Individual Drug Dosing in Chronic Illness Intelligent Control for Management of Renal Anemia Challenges in Dynamic Treatment Regimes.

Similar presentations


Presentation on theme: "From Population to Individual Drug Dosing in Chronic Illness Intelligent Control for Management of Renal Anemia Challenges in Dynamic Treatment Regimes."— Presentation transcript:

1 From Population to Individual Drug Dosing in Chronic Illness Intelligent Control for Management of Renal Anemia Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Adam E Gaweda University of Louisville Department of Medicine

2 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Overview Anemia management Anemia management Dose-response modeling Dose-response modeling Model-based control in drug dosing Model-based control in drug dosing Model-free control in drug dosing Model-free control in drug dosing

3 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Anemia Management Biological vs. clinical rHuEPO

4 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Anemia Management Clinical guidelines Dosing guidelines (NKF – KDOQI) Dosing guidelines (NKF – KDOQI) –Maintain Haemoglobin (Hb) between 11 and 12 g/dL ( Hematocrit (Hct) between 33 – 36 % ). –Titration of EPO: If the increase in Hb after EPO initiation or after a dose increase has been less than 1 g/dL over a 2- to 4-week period, the dose of EPO should be increased by 50%. If the absolute rate of increase of Hb after EPO initiation or after a dose increase exceeds 3 g/dL per month (eg, an increase from a Hgb 7 to 10 g/dL), or if the Hgb exceeds the target, reduce the weekly dose of EPO by 25%. When the weekly EPO dose is being increased or decreased, a change may be made in the amount administered in a given dose and/or the frequency of dosing.

5 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Anemia Management Current state-of-the-art Anemia Management Protocols (AMP) Anemia Management Protocols (AMP) –Frequency of Hb observation: Every 4 weeks if Hb within the target Every 4 weeks if Hb within the target Every 2 weeks if Hb outside of the target Every 2 weeks if Hb outside of the target –EPO dose adjustment: Minimum adjustment amount 10% (of current dose) Minimum adjustment amount 10% (of current dose) Maximum decrease 50% (if Hb > 15 g/dL) Maximum decrease 50% (if Hb > 15 g/dL) Maximum increase 70% (if Hb < 9 g/dL) Maximum increase 70% (if Hb < 9 g/dL) –Problem with AMP Based on average response. Based on average response. Only 1/3 of the patient population achieve the target. Only 1/3 of the patient population achieve the target. Can we improve the outcome of anemia management by making it patient-specific using control theory and machine learning techniques ? Can we improve the outcome of anemia management by making it patient-specific using control theory and machine learning techniques ?

6 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Dose-response modeling Overview In control system design and simulation, a good process model is priceless. In control system design and simulation, a good process model is priceless. Models of erythropoiesis: Models of erythropoiesis: –Physiological model (Uehlinger et al. 1992) –PK / PD model (Brockmöller et al. 1992) –Bayesian network model (Bellazzi et al. 1993) –Artificial Neural Network (ANN) models (Martin Guerrero et al. 2003, Gaweda et al. 2003, Gabutti et al. 2006)

7 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Dose-response modeling Population vs. subpopulation modeling Model 1 Model 2 selection Subpopulation 1 e.g. responders (EPO/Hb < ) Subpopulation 2 e.g. non-responders (EPO/Hb ) dose response data subsets (batch) Model 1 Whole population dose response data set (batch)

8 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Dose-response modeling Example of response prediction

9 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Dose-response modeling Open problems Prediction seems to lag behind the actual value Prediction seems to lag behind the actual value –Do our data allow us to build a model that shows the true effect of EPO on Hb ( Hct ) ? Lets estimate a dynamic linear model Hb(k+1) = f( Hb(k), EPO(k) ) Lets estimate a dynamic linear model Hb(k+1) = f( Hb(k), EPO(k) ) Hb m (k+1) = 0.82 Hb(k) + 0.011 EPO(k) + 1.91 Lets now estimate a model of Δ Hb(k+1) = f( EPO(k) ) Lets now estimate a model of Δ Hb(k+1) = f( EPO(k) ) Δ Hb m (k+1) = 0.015 EPO(k) - 0.23 Both models achieve comparable accuracy. The second model explains the dose effect better.

10 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Dose-response modeling Open problems Our data come from clinical treatment (closed-loop system) Our data come from clinical treatment (closed-loop system) –How does that affect the model ? output distributionabsolute prediction error vs. output Martin Guerrero et al. report the same phenomenon.

11 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-based control Model Predictive Control (MPC) Rationale for using Model Predictive Control Rationale for using Model Predictive Control –There is a delay between EPO administration and Hb response (about 17 days – from EPO manufacturer information). –The relationship between EPO dose and Hb increase is nonlinear (monotonically increasing with saturation – Uehlinger et al. 1992). –The effect of EPO continues throughout the lifetime of red blood cells (up to 120 days). –We plan to include constraints on EPO dose (in the future) (such as minimization of the total dose or minimization of dose changes).

12 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-based control MPC - Schematic diagram MODEL (population) Hb(k+1) = Hb(k) + F NN (EPO(k),EPO(k-1),EPO(k-2)) PATIENT CONTROLLER Hb m Hb EPO* EPO

13 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-based control MPC Clinical trial - setup Trial population: Trial population: –60 patients: 30 controls (dosed by physicians) / 30 treatment (dosed by MPC) 30 controls (dosed by physicians) / 30 treatment (dosed by MPC) 45 African-American / 15 Caucasian 45 African-American / 15 Caucasian 35 males / 25 females 35 males / 25 females Average age 58, min 21, max 84 Average age 58, min 21, max 84 Trial length: Trial length: –8 months 2 months wash-out period / 6 months for outcome analysis 2 months wash-out period / 6 months for outcome analysis Treatment goal: Treatment goal: –maintain Hb at 11.5 g/dL –performance measure: mean absolute deviation from 11.5

14 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-based control MPC - Clinical trial results (thus far) Mean |11.5-Hb| Month

15 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-based control Open problems Simulating MPC Simulating MPC –How do we accurately represent the mismatch between the model and the patient ? –How do we effectively simulate adverse events ? Measuring success Measuring success –We try to individualize the treatment yet we use a mean performance measure – what are the alternatives ? Individual performance measures (e.g. within-subject StDev of Hb ) ???? Individual performance measures (e.g. within-subject StDev of Hb ) ???? –How do we eliminate influence of Hb changes due to adverse events on the performance measure ?

16 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-free control Reinforcement Learning Drug administration in chronic conditions is a trial-and-error control process that resembles reinforcement learning Drug administration in chronic conditions is a trial-and-error control process that resembles reinforcement learning disease symptoms – initial state (s 0 ) (standard) initial dose – action (a 0 ) k = 1 Repeat (infinitely) evaluate patient (remission/progression/side effects) – new state (s k ), reward (r k ) evaluate patient (remission/progression/side effects) – new state (s k ), reward (r k ) adjust dosing strategy – update state-action table/function (Q k ), extract policy ( k ) adjust dosing strategy – update state-action table/function (Q k ), extract policy ( k ) administer new dose – action (a k ) administer new dose – action (a k ) k = k + 1 k = k + 1End

17 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-free control Q-Learning simulation - Schematic diagram Q-LEARNING AGENT PATIENT SIMULATOR (subpopulation model) Hb(k+1) = F( Hb(k), EPO(k), IRON(k) ) POLICY ( ) R i : IF Hb = Hb i THEN EPO = EPO i EPO (a) IRON (disturbance) Hb (s)

18 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-free control Reward function 11.5

19 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-free control Q-table update Dose-response relationship (EPO to Δ Hb) is monotonically increasing with saturation ( Uehlinger et al. 1992 ). Dose-response relationship (EPO to Δ Hb) is monotonically increasing with saturation ( Uehlinger et al. 1992 ). Lets update multiple entries in the Q-table at a time : Lets update multiple entries in the Q-table at a time : –If Hb(k) < 11.5 and Hb(k+1) Hb(k) or Hb(k) = 11.5 and Hb(k+1) < Hb(k) then update Q( s, a ) for all s Hb(k) and all a EPO(k) –If Hb(k) > 11.5 and Hb(k+1) Hb(k) or Hb(k) = 11.5 and Hb(k+1) > Hb(k) then update Q( s, a ) for all s Hb(k) and all a EPO(k)

20 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-free control Q-Learning - Simulated clinical trial Trial population: Trial population: –200 individuals with various degrees of response to EPO –100 distinct responders / 100 distinct non-responders –In the first run, all individuals dosed by AMP –In the second run, all individuals dosed by policy updated on-line by Q-learning Trial length: Trial length: –24 months Treatment goal: Treatment goal: –drive Hb to, and maintain at 11.5 g/dL –performance measure: mean absolute deviation from 11.5

21 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Model-free control Q-Learning - Simulation results Mean |11.5-Hb| Month

22 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Conclusions and open problems We believe that we are on a good path to successfully individualize anemia management using presented techniques. We believe that we are on a good path to successfully individualize anemia management using presented techniques. However, we need to address the following: How do we produce reliable dose-response models that perform well on under-represented data instances ? How do we produce reliable dose-response models that perform well on under-represented data instances ? What performance measure do we need to use in order to adequately evaluate the success of an individualized treatment ? What performance measure do we need to use in order to adequately evaluate the success of an individualized treatment ?

23 June 21, 2007Challenges in Dynamic Treatment Regimes and Multistage Decision-Making Acknowledgments UofL Division of Nephrology UofL Division of Nephrology –George R Aronoff –Michael E Brier –Alfred A Jacobs UofL Dept Electrical and Computer Engineering UofL Dept Electrical and Computer Engineering –Mehmet K Muezzinoglu –Jacek M Zurada Michael E Brier has been sponsored by Department of Veterans Affairs Merit Review Grant. Adam E Gaweda is sponsored by NIDDK (1K25DK072085-01A2).


Download ppt "From Population to Individual Drug Dosing in Chronic Illness Intelligent Control for Management of Renal Anemia Challenges in Dynamic Treatment Regimes."

Similar presentations


Ads by Google