DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network July 18th on 2018

Overview Goals: Contributions: Show that deep learning to survival analysis performs good in predicting risk. Demonstrate that deep neural network can be used as a personalized treatment recommender system for further medical research. Contributions: The model performs good than other survival analysis method. The model can be used on treatment recommendations which could increase the median survival time.

Survival analysis A study on the relationship between duration of time and survival probability of individual. death in biological organisms failure in mechanical systems. Can be generalized to the method to study the correlation between events and its happen time.

Survival analysis COX proportional hazards model (CPH) Censored data A survival analysis model to analyze how multi-variables affect survival time. Advantage over logistics regression: dealing with censored data. Censored data Termination of the study A person withdraws from the study or lost to follow-up during the study period

Survival analysis Survival data Survival function: Hazard function: x: a patient’s baseline data T: a failure event time E: event indicator (1: event was observed 0: response was censored) Survival function: Hazard function:

Linear survival model---CPH Hazard function: Estimation of regression parameter COX partial likelihood: Maximize log-likelihood to get estimator Only non-censored individual has partial likelihood (E=1) Summary: The CPH is a semiparametric model that calculates the effects of observed covariates on the risk of an event occurring. The model assumes that a patient’s log-risk of failure is a linear combination of the patient’s covariates. Baseline hazard function Risk score

Non-linear survival model---DeepSurv DeepSurv: combination of CPH & neural network. Neural network learns complex non-linear relationship between prognostic features and individual’s risk of failure. Input: a patient’s baseline data x Output: a single node with a linear activation which estimates the log-risk function in CPH Objective function: average negative log partial likelihood Training method: gradient decent

Treatment recommender system Recommender function: Log of hazard ratio to calculate the personal risk-ratio of prescribing one treatment option over another. Eg. Positive: I leads to higher risk to death than j Advantage over CPH Calculates the recommender function without an a priori specification of treatment interaction terms. CPH model computes a constant recommender function unless treatment interaction terms are added to the model.

Treatment recommender system Advantage over CPH Calculates the recommender function without an a priori specification of treatment interaction terms. CPH model computes a constant recommender function unless treatment interaction terms are added to the model.

Journal of Biomedical Informatics 68 (2017) 50–57 Dynamic strategy for personalized medicine: An application to metastatic breast cancer Journal of Biomedical Informatics 68 (2017) 50–57 Xi Chen a,⇑, Ross D. Shachter a, Allison W. Kurian b, Daniel L. Rubin c a Department of Management Science & Engineering, Stanford University, Stanford, CA, USA b Department of Medicine, Stanford University, Stanford, CA, USA c Department of Radiology, Stanford University, Stanford, CA, USA July 18th on 2018

Challenging question Is the current therapy effective? Tumor size will not change significantly between visits, hardly observable. Tumors develop resistance to therapies. When to switch to another therapy? Effectiveness of current therapy is not directly observable. Traditional signal based on progression might not be the best signal. Use the past treatment and observation history to update our belief about the current therapy effectiveness.

POMDPs introduction POMDPs describe the uncertainty about actions outcome.

Belief state Probability distributions over states of the underlying MDP. The agent keeps an internal belief state, b, that summarizes its experience.

Belief state probability for the environment being in state The agent uses a state estimator, SE, for updating the belief state b’ Since b’ contains all the information of observation & model, B in POMDP can be regarded as S in MDP. b ′ 包含了观察和模型的全部信息： 1 上一时刻的全部信息 b; 2 模型信息 T; 3 观察信息 O(s ′ , a, o), s ′ ∈ S

POMDP as Belief-MDP The policy of a POMDP maps the current belief state into an action. As the belief state holds all relevant information about the past, the optimal policy of the POMDP is the the solution of belief MDP. The Bellman equation for this belief MDP: In general case: Very Hard to solve continuous space MDPs. Unfortunately, DP updates cannot be carried out because there are uncountably many of belief states. One cannot enumerate every equation of value function. To conduct DP steps implicitly, we need to discuss the properties of value functions.

POMDP-Belief MDP Action: A,B,C,D Reward: State: 2 hormone therapies：Tamoxifen and Fulvestrant 2 chemotherapies: Docetaxel and Capecitabine One of the four therapies can be chosen each period. Reward: Tumor response: negative log of the tumor size Disutility due to chemotherapy side effects State: Current probabilities for effectiveness of the different therapies What action to take given current probability distribution rather than given current state. current tumor response joint probability of effectiveness for all therapies

Inputs tumor response observation probability matrix the initial probability of effectiveness within-therapy decline rate 𝛼 between-therapy decline rate 𝛽

Compute belief states Kalman Filter Q-Learning Methods (KFQL) Q-value: Approximates the optimal net present value of the future rewards. Kalman Filter is used to upgrade the weights r. Approximate Kalman Filter Q-Learning Methods (AKFQL) Ignores the dependence among basis functions reduces the computation complexity Successive Approximations of the Reachable Space under Optimal Policies (SARSOP)

Results Comparison of policies policies from KFQL and AKFQL are the most effective at limiting tumor growth. all three computed policies apply chemotherapy in fewer periods than the standard therapy strategy. Converge to policies with similar average reward. AKFQL converges faster

Results Comparison of policies