1. Designing JITAI for Skylar
Maria Mayorga
Associate Professor
North Carolina State University
NSF International Workshop on Dynamic Modeling of Health Behavior Change and Maintenance,
Sept 8-9, 2015, London, UK

2. Baseline Data Physical information
Weight, measurements, % body fat, BP
Food consumption (food diary)
Calories, nutrients, when
Exercise/physical activity (diary, pedometer, wearable such as fitbit, heart rate monitor)
Water intake
Supplements or cigarette use?
Fitness test

3. Baseline Data Reinforcements Sleep patterns Location (GPS)
Stress (potentially through a wearable)
Social media interactions
Type of event/activity engaged in (e.g. dinner for work)
Dieting history

4. Baseline Data Results from validated survey instrument on
Self-confidence, body image
Mood (depression)
Belief regarding potential success
Time should allow one to
Detect patterns (day-to-day, weekly, monthly)
Observe reaction to stressful events
My thought is 1-3 months, but with very detailed data this may be too much
Potentially repeated week long collections

5. Prediction Model (Long term)
Several layers, or hybrid
Agent-based model would allow one to capture interactions, especially if a cohort of individuals with connections
Depending on the level of detail, some sort of “natural history” or biological model for calorie balance
At a slightly higher level a Markov or partially observable Markov model
Requires other models to extract information from data (potentially machine learning)

6. External factor or triggers, can be modeled as events
Biological model
Input- calorie intake, physical activity
Output-expected weight loss/gain
Self-efficacy
Input- state in theoretical model
Output- extract transition probabilities
“Health Status” model
Can we categorize skylar’s current behavior into some trajectory, e.g. (low physical activity but eating well and confident) and extract transitions between states?
L,L
H,L
L,H
H,H
Agent-based or discrete event Simulation model

7. Stochastic model of Skylar’s progression
To model skylar’s progression, we define a “state” as skylar’s condition (e.g. Moderately active and eating well) and a “state transition” is a change in the his condition.
The “state” focuses on the features of the Skylar’s condition, while the “state transition” provides the mechanism for structuring the patient's condition’s dynamic behavior.
We can model the patient’s health using a continuous time semi-Markov process where we define health-state conditional Markov transition model as shown before.
Transitions can depend on external factors, such as level of stress or confidence, at a population level, could depend on age, gender, race, etc.

8. Identifying Dynamic Interventions – Notification Setting
A Markov Decision Process (MDP) model provides a framework for decision making under uncertainty defined by <S, A, P, R>: a finite set of states S, a finite set of actions A, the transition probability, P, characterized by a Markov chain defining movement between states as a function of the current state and action, and an immediate reward, R(s,a), associated with performing the action a in state s.
In an MDP, a decision maker can influence the behavior of a probabilistic system through his/her actions as the system evolves through time
The decision maker’s goal is to choose a sequence of actions which allow the system of interest to perform optimally by predetermined success criteria, such as minimizing the expected time for Skylar to reach his goal.
An SMDP is an extension of an MDP in which the state transitions depend not only on the current state and action but also on how long the system has been in the current state

9. Side Notes

10. Types of Models/Questions to Answer
Pre-intervention:
Statistical models – what causes skylar to binge eat? Or what factors are associated with binge eating?
Post-intervention:
Need to test hypotheses, to build a model for the effect of the intervention on sever

11. Questions
Are we interested in Skylar’s weight loss and other physical measurements or in his “journey”
What time scale are we interested in?
Can we gain from using information from several individuals like skylar, to incorporate variability uncertainty