Presentation is loading. Please wait.

Presentation is loading. Please wait.

Optimal Dynamic Treatment Regimes

Published byÏΚαϊνάμ Ρέντης Modified over 5 years ago

Similar presentations

Presentation on theme: "Optimal Dynamic Treatment Regimes"— Presentation transcript:

1 Optimal Dynamic Treatment Regimes
Julia Wang, Kwan Lee, and Xiwu Lin RWE Analytics at Janssen Quantitative Sciences Sept. 7th, 2017

2 What is Optimal DTR? Dynamic Treatment Regimes (DTR) are individually tailored treatment sequences based on subject-level information. A DTR algorithm provides a mapping from the space of subject-level information (including baseline, treatments, and responses) to the treatment action space. Optimal DTRs maximize overall average treatment effects for a population following such regimes.

3 bmiData from r Package iqLearn
> head(bmiData) gender race parent_BMI baseline_BMI month4_BMI month12_BMI A1 A2 CD MR CD MR MR CD CD CD CD CD MR MR

4

5 Baseline Characteristics

6 Baseline Characteristics
Observations On-study Baseline Characteristics

7 Baseline Characteristics
Observations On-study Baseline Characteristics Randomized Groups

8 Two-Stage Randomized Treatment Set-up
MR or CD Stage 2: MR or CD Baseline

9 Subject-level information at Stage 1:
Gender, Race, Baseline BMI, and Parents BMI Subject-level information at Stage 2: Gender, Race, Baseline BMI, Parents BMI and Month 4 BMI Stage 1 Baseline BMI vs. Parent's BMI Stage 2 Month 4 BMI vs. Parent's BMI 50 50 45 45 Baseline BMI Month 4 BMI 40 40 35 35 30 30 25 25 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

10 Randomized Regime Stage 1 Baseline BMI vs. Parent's BMI Stage 2
Month 4 BMI vs. Parent's BMI MR CD 50 50 45 45 Baseline BMI Month 4 BMI 40 40 35 35 30 30 MR CD 25 25 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

11 A partition? Optimized Regime Stage 1 Baseline BMI vs. Parent's BMI
Month 4 BMI vs. Parent's BMI MR CD 50 50 45 45 Baseline BMI Month 4 BMI 40 40 35 35 30 30 MR CD 25 25 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

12 Data Generation Process
Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space Data Generation Process Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD Y Baseline Characteristics This process can be either randomized or observational, how do we estimate the optimal DTR from the data obtained?

13 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 )))

14 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) Maximize by partition the subject-level space defined by S1, S2, S3

15 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) M-EY Maximize by partition the subject-level space defined by S1, S2, S3

16 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) M-EY M-EY Maximize by partition the subject-level space defined by S1, S2, S3

17 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) M-EY

18 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) M-EY Maximize by partition the subject-level space defined by S1 and S2

19 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) MM-EY M-EY Maximize by partition the subject-level space defined by S1 and S2

20 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) MM-EY MM-EY M-EY Maximize by partition the subject-level space defined by S1, and S2

21 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) MM-EY MM-EY M-EY

22 A Conditional Expectation Property
The Goal: Maximize E ( Y ) E ( Y ) = E ( E ( Y | S1 )) = E ( E ( E ( Y | S1 , S2 ))) = E ( E ( E ( E ( Y | S1 , S2, S3 ))) MM-EY MM-EY M-EY Maximize by partition the subject-level space defined by S1

23 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD Y Baseline Characteristics

24 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD Y Baseline Characteristics

25 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD Y Baseline Characteristics

26 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD Y Baseline Characteristics

27 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD M-EY Y Baseline Characteristics

28 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD M-EY Y Baseline Characteristics

29 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD M-EY Baseline Characteristics

30 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD M-EY Baseline Characteristics

31 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD M-EY Baseline Characteristics

32 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD MM-EY M-EY Baseline Characteristics

33 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD MM-EY Baseline Characteristics

34 Estimation of DTR as a Recursive Partitioning of the Subject-level Information Space
History Stage 1 TRT OBS MR CD Stage 2 TRT OBS MR CD Stage 3 TRT OBS MR CD Stage 4 TRT OBS MR CD MMM-EY Baseline Characteristics We need to be able to link the outcome Y with the history space. One such method of optimal DTR estimation is Q-learning

35 Q-learning Recursive Linear Model Fitting, with treatment by covariates interaction

36 Q-learning Recursive Linear Model Fitting, with treatment by covariates interaction

37 Q-learning Recursive Linear Model Fitting, with treatment by covariates interaction MMM-EY MM-EY M-EY

38 r DTR Packages iqLearn DynTxRegime DTRreg DTR DTRlearn

39 Illustration 1 using bmiData and Q-learning Stage 1 only and Parent’s BMI only

40 Month 4 and Month 12 Outcome Variable
y = -100*(Month12_BMI – Baseline_BMI)/Baseline_BMI y4 = -100*(Month14_BMI – Baseline_BMI)/Baseline_BMI

41 Weight Reduction Outcome at Month 4 vs. Parental BMI
MR CD 20 20 Negative Percent Change in BMI at 4 Month Negative Percent Change in BMI at 4 Month 10 10 −10 −10 −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

42 Weight Reduction Outcome at Month 4 vs. Parental BMI
MR CD 20 20 Negative Percent Change in BMI at 4 Month Negative Percent Change in BMI at 4 Month 10 10 −10 −10 −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

43 Treatment by Parent’s BMI interaction
Expected Weight Reduction Outcome at Month 4 vs. Parental BMI Treatment by Parent’s BMI interaction 20 Negative Percent Change in BMI at 4 Month ( , 7.96 ) ● 10 −10 MR CD −20 25 30 35 40 45 50 Parent's BMI

44 ●● Expected Weight Reduction Outcome at Month 4 vs. Parental BMI 20
Negative Percent Change in BMI at 4 Month ●●● ●● ● ●●● ●●●●●● ( , 7.96 ) 10 ●●●●●● ● ●● ●●●●● ● ● ● ●● ● ●● ●●● ●●●●●● ●● ●●●●●●●●●●●●●●●● ●● ●●●● ●●●● ●● ● ● ●●●●●●●●●● ●● ●● ●●● ●●●●●●●● ●●●● ● ● ● ● ●●●●●● ●●●● ●●● ● ● ● ● ●● ●●● ●● ● ● ● ●●● ● ●● ●●●● ● ● −10 ●● ● MR CD −20 25 30 35 40 45 50 Parent's BMI

45 ●●● Expected Weight Reduction Outcome at Month 4 vs. Parental BMI ● MR
CD 20 20 Negative Percent Change in BMI at 4 Month ●● ● Negative Percent Change in BMI at 4 Month ● ● ●● ●●● ●●● 10 ● ●●● 10 ● ●●● ●● ●●● ● ● ● ● ●●●●●●●●●●●● ●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●●●●● ●●●●●● ●●●●●● ●● ●● ● ●● ●● ● ● ●● ● ●● ● ● ●● ●● ● ●● ●● ●● ● ● −10 −10 ● −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

46 ●● Expected Weight Reduction Outcome at Month 4 vs. Parental BMI ● MR
CD 20 20 ●● Negative Percent Change in BMI at 4 Month ●● ● Negative Percent Change in BMI at 4 Month ● ● ● ●● ● ●● ●● ●●● ● ● ●●● ●● 10 ● ●●● 10 ●● ●● ● ●● ●●● ●● ●● ● ●● ●● ● ●● ● ●●●●● ● ● ●●●● ●●●●●●●●●●●●● ●● ●●●●●● ●● ●●●●●●● ●●●● ● ● ● ● ●●●●●● ●●●●● ● ● ● ● ● ●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●● ●● ● ●● ●● ● ● ●● ● ●● ● ● ●● ●● ● ●● ●● ●● ● ● −10 −10 ● −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

47 Optimal Expected Weight Reduction Outcome at Month 4 vs. Parental BMI
MR CD 20 20 ●● Negative Percent Change in BMI at 4 Month ●● ● Negative Percent Change in BMI at 4 Month ● ● ● ●● ● ●● ●● ●●● ● ● ●●● ●● 10 ● ●●● 10 ●● ●● ● ●● ●●● ●● ● ●● ●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●● ●● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●● ●● ● ● ● −10 −10 −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

48 Optimal Expected Weight Reduction Outcome at Month 4 vs. Parental BMI
MR CD 20 20 ●● Negative Percent Change in BMI at 4 Month ●● ● Negative Percent Change in BMI at 4 Month ● ● ● ●● ● ●● ●● ●●● ● ● ●●● ●● 10 ● ●●● 10 ●● ●● ● ●● ●●● ●● ● ●● ●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●● ●● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●● ●● ● ● ● −10 −10 −20 −20 25 35 30 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

49 A Classification Problem
Expected Weight Reduction Outcome at Month 4 vs. Parental BMI MR CD Loss 20 20 ●● Negative Percent Change in BMI at 4 Month ●● ● Negative Percent Change in BMI at 4 Month ● ● ● ●● ● ●● ●● ●●● ● ● ●●● ●● 10 ● ●●● 10 ●● ●● ● ●● ●●● ●● ●● ● ●● ●●●●●● ●● ●●●●●● ●● ●●●●●●● ●●●● ● ● ●● ● ●●●●● ●● ● ● ● ●●●● ●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ● ●●●●●● ●●●●● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ● ●● ● ●● ● ● ●● ●● ● ●● ●● ●● ● ● −10 −10 ● A Classification Problem −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

50 Illustration 2 using bmiData Stage 1 only and Parent’s BMI and Baseline BMI

51 Model Fitted MR (A1=1) and CD (A1=-1)

52 Predicted Stage 1 Optimal Treatment based on Month 4 Outcome MR Group
Demarcation Line MR better CD better 50 45 Baseline BMI 40 35 30 25 30 35 40 45 50 Parent's BMI

53 Predicted Stage 1 Optimal Treatment based on Month 4 Outcome CD Group
Demarcation Line MR better CD better 50 45 Baseline BMI 40 35 30 25 30 35 40 45 50 Parent's BMI

54 Expected Weight Reduction Outcome at Month 4 vs. Parental BMI
Parent’s BMI only 20 Negative Percent Change in BMI at 4 Month ( , 7.96 ) ● 10 −10 MR CD −20 25 30 35 40 45 50 Parent's BMI

55 Predicted Stage 1 Optimal Treatment based on Month 4 Outcome MR Group
Demarcation Line MR better CD better 50 45 Baseline BMI 40 35 30 30.95 25 30 35 40 45 50 Parent's BMI

56 Predicted Stage 1 Optimal Treatment based on Month 4 Outcome MR Group
Demarcation Line MR better CD better 50 45 Baseline BMI 40 35 30 30.95 25 30 35 40 45 50 Parent's BMI

57 ●● Expected Weight Reduction Outcome at Month 4 vs. Parental BMI ● MR
CD 20 20 ●● Negative Percent Change in BMI at 4 Month ●● ● Negative Percent Change in BMI at 4 Month ● ● ● ●● ● ●● ●● ●●● ● ● ●●● ●● 10 ● ●●● 10 ●● ●● ● ●● ●●● ●● ●● ● ●● ●● ● ●● ● ●●●●● ● ● ●●●● ●●●●●●●●●●●●● ●● ●●●●●● ●● ●●●●●●● ●●●● ● ● ● ● ●●●●●● ●●●●● ● ● ● ● ● ●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●● ●● ● ●● ●● ● ● ●● ● ●● ● ● ●● ●● ● ●● ●● ●● ● ● −10 −10 ● −20 −20 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

58 Optimal vs. Predicted Month 4 Outcome
MR CD 18 20 Optimal Predicted Month 4 Outcome Optimal Predicted Month 4 Outcome 16 10 14 12 10 −10 8 −20 6 −20 −10 10 20 6 8 10 12 14 16 18 Predicted Month 4 Outcome Predicted Month 4 Outcome

59 Illustration 3 using bmiData Both Stages Stage 1: Parents BMI and Baseline BMI Stage 2: Parents BMI and Month 4 BMI

60 Models Fitted Stage 2: y ~ gender + parent_BMI + month4_BMI +
A2*(parent_BMI + month4_BMI) A2 coded as 1 and -1 Stage 1: Opt_y~ gender + race + parent_BMI + baseline_BMI A1*(parent_BMI + baseline_BMI) A1 coded as 1 and -1

61 Predicted Stage 2 Optimal Treatment based on Month 12 Outcome MR Group
Demarcation Line MR better CD better 45 40 Month 4 BMI 35 30 25 30 35 40 45 50 Parent's BMI

62 Predicted Stage 2 Optimal Treatment based on Month 12 Outcome CD Group
Demarcation Line MR better CD better 45 40 Month 4 BMI 35 30 25 30 35 40 45 50 Parent's BMI

63 Predicted Stage 1 Optimal Treatment based on Optimal Month 12 Outcome MR Group
Demarcation Line MR better CD better 50 45 Baseline BMI 40 35 30 25 30 35 40 45 50 Parent's BMI

64 Predicted Stage 1 Optimal Treatment based on Optimal Month 12 Outcome CD Group
Demarcation Line MR better CD better 50 45 Baseline BMI 40 35 30 25 30 35 40 45 50 Parent's BMI

65 Models Fitted Stage 2: y ~ gender + parent_BMI + month4_BMI +
A2*(parent_BMI + month4_BMI) A2 coded as 1 and -1 Contrast Term Stage 1: Opt_y~ gender + race + parent_BMI + baseline_BMI A1*(parent_BMI + baseline_BMI) A1 coded as 1 and -1

66 Estimated Contrast Term vs. Parent's BMI in Stage 2
MR (coded as 1) CD (coded as −1) 4 4 2 2 Estimated Contrast Term Estimated Contrast Term −2 −2 −4 −4 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

67 Switch!!! Estimated Contrast Term vs. Parent's BMI in Stage 2
MR (coded as 1) CD (coded as −1) 4 4 2 2 Estimated Contrast Term Estimated Contrast Term Switch!!! −2 −2 −4 −4 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

68 Switch!!! Estimated Contrast Term vs. Parent's BMI in Stage 2
MR (coded as 1) CD (coded as −1) 4 4 2 2 Estimated Contrast Term Estimated Contrast Term Switch!!! −2 −2 −4 −4 25 30 35 40 45 50 25 30 35 40 45 50 Parent's BMI Parent's BMI

69 The Goal of Optima DTR maximize overall average treatment effects for a population following such regimes The outcome of a fixed treatment regime can be estimated from the population of subjects whose observe regime is the same as the optimal regime.

70

71

72 Estimated Mean Y 9.9284

73 7.9175 8.0631 3.5236 6.2016 Estimated Mean Y 9.9284

74 What if the data is observational?
Use Inverse Probability Weighting, at each stage It creates an approximate pseudo-randomized population at each stage Why?

75 IPW creates an approximate pseudo-randomized population
Assuming there are N subjects in the study, and the probability of being treated is p We will have (on average) N*p treated subjects. If we weight each of these subjects by 1/p, then these N*p subjects will represent N*p*(1/p)=N subjects. The same reasoning applies to the control subjects weighted by 1/(1-p), and they will represent on average N*(1-p)*(1/(1-p))=N subjects.

76 An IPW Illustration Assuming there are N=90m (m is an integer multiplier) subjects in the study, evenly spread within a single confounder with 9 subgroups; The probability of being treated varies from p = 0.1, 0.2, to 0.9 within each subgroup of the confounder. Within each subgroup, we will have (on average) n1=10m*p treated and n2=10m*(1-p) controls. This leads to: n1*(1/p) = n2* (1/(1-p)) = 10m Conclusion: IPW balances the distribution of the confounders. However, in practice, these probabilities, also called propensity scores, need to be estimated.

77 Distribution of Confonders before IPW
● ● 0.1 0.2 ● 0.3 ● 0.4 0.5 ● 0.6 ● 0.7 0.8 ● 0.9 Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Subjects Confonder Subgroups

78 ● Distribution of Confonders for Treated Group before IPW Gr. 1 Gr. 2
Subjects Confonder Subgroups

79 Distribution of Confonders for Treated Group before IPW
● 1/ 0.9 ● 1/ 0.8 ● 1/ 0.7 ● 1/ 0.6 ● 1/ 0.5 ● 1/ 0.4 ● 1/ 0.3 ● 1/ 0.2 ● 1/ 0.1 Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Subjects Confonder Subgroups

80 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Distribution of Confonders for Treated Group before IPW ● ● ● ● ● ● ● ● ● ● Subjects ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 1 / 0.1 ● 2 / 0.2 ● 3 / 0.3 ● 4 / 0.4 ● 5 / 0.5 ● 6 / 0.6 ● 7 / 0.7 ● 8 / 0.8 ● 9 / 0.9 Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

81 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Distribution of Confonders for Treated Group before IPW ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Subjects ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

82 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Distribution of Confonders for Treated Group after IPW ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Subjects ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

83 Distribution of Confonders before IPW
● ● 0.1 0.2 ● 0.3 ● 0.4 0.5 ● 0.6 ● 0.7 0.8 ● 0.9 Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Subjects Confonder Subgroups

84 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Distribution of Confonders for Control Group before IPW ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Subjects ● ● ● ● ● ● ● ● ● ● Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

85 Distribution of Confonders for Control Group before IPW
● 1/ 0.9 ● 1/ 0.8 ● 1/ 0.7 ● 1/ 0.6 ● 1/ 0.5 ● 1/ 0.4 ● 1/ 0.3 ● 1/ 0.2 ● 1/ 0.1 ● 1/ 0.2 Subjects Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

86 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Distribution of Confonders for Control Group before IPW ● 9 / 0.9 ● 8 / 0.8 ● 7 / 0.7 ● 6 / 0.6 ● 5 / 0.5 ● 4 / 0.4 ● 3 / 0.3 ● 2 / 0.2 ● 1 / 0.1 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Subjects ● ● ● ● ● ● ● ● ● ● Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

87 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Distribution of Confonders for Control Group before IPW ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Subjects ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5 Gr. 6 Gr. 7 Gr. 8 Gr. 9 Confonder Subgroups

88 ● Distribution of Confonders for Control Group after IPW Gr. 1 Gr. 2
Subjects Confonder Subgroups

89 ● ● Distribution of Confonders after IPW Gr. 1 Gr. 2 Gr. 3 Gr. 4 Gr. 5
Subjects Confonder Subgroups

90 Propensity Scores The more accurately a propensity score can be estimated, the better it is at reducing confounding and bias. How to estimate propensity scores properly is a very big topic in itself.

91

92 7.9175 8.0631 3.5236 6.2016 Estimated Mean Y 9.9284 dwols:

93 Comparing Q-learning vs. dwols using bmiData
Stage 2 dwols Q-learning CD MR 94 1 4 111 Stage 1 dwols Q-learning CD MR 111 1 2 96

94

Download ppt "Optimal Dynamic Treatment Regimes"

Similar presentations

Controlling for Time Dependent Confounding Using Marginal Structural Models in the Case of a Continuous Treatment O Wang 1, T McMullan 2 1 Amgen, Thousand.

Controlling for Time Dependent Confounding Using Marginal Structural Models in the Case of a Continuous Treatment O Wang 1, T McMullan 2 1 Amgen, Thousand.
A workshop introducing doubly robust estimation of treatment effects

A workshop introducing doubly robust estimation of treatment effects
2 – In previous chapters: – We could design an optimal classifier if we knew the prior probabilities P(wi) and the class- conditional probabilities P(x|wi)

2 – In previous chapters: – We could design an optimal classifier if we knew the prior probabilities P(wi) and the class- conditional probabilities P(x|wi)
Lecture 4 Linear random coefficients models. Rats example 30 young rats, weights measured weekly for five weeks Dependent variable (Y ij ) is weight for.

Lecture 4 Linear random coefficients models. Rats example 30 young rats, weights measured weekly for five weeks Dependent variable (Y ij ) is weight for.
Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy LSU ---- Geaux Tigers! April 2009.

Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy LSU ---- Geaux Tigers! April 2009.
Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy ENAR March 2009.

Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy ENAR March 2009.
Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy Psychiatric Biostatistics Symposium May 2009.

Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy Psychiatric Biostatistics Symposium May 2009.
Statistical Methods For Engineers ChE 477 (UO Lab) Larry Baxter & Stan Harding Brigham Young University.

Statistical Methods For Engineers ChE 477 (UO Lab) Larry Baxter & Stan Harding Brigham Young University.
Advanced Statistics for Interventional Cardiologists.

Advanced Statistics for Interventional Cardiologists.
Poisson Random Variable Provides model for data that represent the number of occurrences of a specified event in a given unit of time X represents the.

Poisson Random Variable Provides model for data that represent the number of occurrences of a specified event in a given unit of time X represents the.
Biostatistics Case Studies 2005 Peter D. Christenson Biostatistician Session 5: Classification Trees: An Alternative to Logistic.

Biostatistics Case Studies 2005 Peter D. Christenson Biostatistician Session 5: Classification Trees: An Alternative to Logistic.
Review of Chapters 1- 5 We review some important themes from the first 5 chapters 1.Introduction Statistics- Set of methods for collecting/analyzing data.

Review of Chapters 1- 5 We review some important themes from the first 5 chapters 1.Introduction Statistics- Set of methods for collecting/analyzing data.
Chapter 6 Lecture 3 Sections: 6.4 – 6.5.

Chapter 6 Lecture 3 Sections: 6.4 – 6.5.
Propensity Score Matching for Causal Inference: Possibilities, Limitations, and an Example sean f. reardon MAPSS colloquium March 6, 2007.

Propensity Score Matching for Causal Inference: Possibilities, Limitations, and an Example sean f. reardon MAPSS colloquium March 6, 2007.
Lecture 12: Cox Proportional Hazards Model

Lecture 12: Cox Proportional Hazards Model
Single-Factor Studies KNNL – Chapter 16. Single-Factor Models Independent Variable can be qualitative or quantitative If Quantitative, we typically assume.

Single-Factor Studies KNNL – Chapter 16. Single-Factor Models Independent Variable can be qualitative or quantitative If Quantitative, we typically assume.
Categorical Independent Variables STA302 Fall 2013.

Categorical Independent Variables STA302 Fall 2013.
Chapter 6 Lecture 3 Sections: 6.4 – 6.5. Sampling Distributions and Estimators What we want to do is find out the sampling distribution of a statistic.

Chapter 6 Lecture 3 Sections: 6.4 – 6.5. Sampling Distributions and Estimators What we want to do is find out the sampling distribution of a statistic.
1 Hester van Eeren Erasmus Medical Centre, Rotterdam Halsteren, August 23, 2010.

1 Hester van Eeren Erasmus Medical Centre, Rotterdam Halsteren, August 23, 2010.
© 2010 Pearson Prentice Hall. All rights reserved 7-1.

© 2010 Pearson Prentice Hall. All rights reserved 7-1.

Similar presentations

Ads by Google