Issues in Treatment Study Design John Whyte, MD, PhD Neuro-Cognitive Rehabilitation Research Network Moss Rehabilitation Research Institute

Topics to Address Developmental trajectory of rehabilitation treatment research (analogous to pharmaceutical phases) Treatment theory Implications for subject selection, choice of outcome measures, overall study design

Treatment Development

Pharmaceutical Model Treatment is well defined, only dosage is in question Phase I: Primarily safety, dose finding Phase II: Small, often single center studies to establish preliminary evidence of efficacy Phase III: Large, multicenter test of efficacy/effectiveness against primary outcome (Phase IV): Post-marketing surveillance of actual use

Rehabilitation Model Formalize a treatment protocol that is repeatable* Assess safety of the treatment* Establish a treatment effect Establish parameters for optimal “ dose ” and schedule of treatment Examine probable mechanisms of treatment effect* Identify or develop outcome measures that capture both change and impact*

Rehab Model (cont.) Identify the treatment ’ s efficacy in a population and in subgroups of that population Determine the treatment ’ s generalizability Determine the impact of the treatment on broader, real-world outcomes Determine the effectiveness of the treatment as delivered in real-world healthcare delivery systems to diverse patients§ Assess cost effectiveness of the treatment

Treatment Theory A proposed specification of the “active ingredients” of treatment A proposed specification of the mechanism of action of those active ingredients

Framing Treatment Theory Related to Electrical Stimulation Acceleration of natural recovery mechanisms Direct reorganization of specific neural networks Alteration of the balance among competing networks Temporary facilitation/inhibition that alters learning potential (i.e., must be coupled with experience) General theory applicable to heterogeneous neural systems, or specific theories related to specific neural/functional domains?

Treatment Theory Informs Other Aspects of Study Design Selection of appropriate study participants Selection of appropriate outcome measures Selection of the optimal experimental design

Study Participants Given the proposed mechanism of action, who can realistically respond? (early phase – rehabilitation research) – Presence of intact homologous system? – Presence of some residual capacity in injured system (assessed neurophysiologically or behaviorally)? – Ability to engage in experiential learning?

Study Participants (cont.) Of those able to respond to the treatment mechanistically, who will show meaningful functional gains (later phase, enablement research)? – Functional domains likely to show improvement in principle – Relevant coexisting impairments – Selection of individuals with “pure” impairments, measurement of confounding impairments, or multifaceted treatments?

Characterizing the Participants Once selected, one must ask whether the participants in different treatment groups are “comparable” – comparable in what way(s)? – Balance achieved between randomized groups – Statistical adjustment of differences in observational studies – Need prior data on prognostic factors

Comparability & Adjustment Comparability and adjustment in terms of prognosis on the outcome measures chosen Comparability and adjustment in terms of the characteristics that predict responsiveness to the treatment under study

Outcome Measures Given the proposed mechanism of action of the treatment, where would you expect to see treatment impact? – The most “proximal” impact (as evidence for the proposed mechanism) – A more “distal” impact (as evidence that the change achieved has clinical meaning, ecological validity) What other factors are likely to modify the chosen outcomes?

Outcome Measures (cont.) The same treatment may be judged effective or ineffective, depending on the outcomes chosen: – Oral antispasticity medications – Memory remediation treatments

Overall Study Design Does the proposed mechanism of action suggest a permanent effect of intervention? (parallel group vs. crossover design) – What about “temporary facilitation” of “long-term learning”? Does the proposed mechanism of action suggest “localized” or “generalized” treatment impacts? (feasibility of multiple baseline across behaviors design; utility of multiple outcomes)

Study Design (cont.) Does the proposed treatment capitalize on neurologic recovery (early vs. late enrollment)? Are there important covariates that might affect treatment response that should be measured? Can preliminary evidence of treatment impact be gathered without an untreated control group?

Discussion Issues

General Issues What “phase(s)” have been reached in the studies to date? Does/can a single investigative group take a treatment from beginning to end, or is there a need to “hand off” or partner with different types of investigators as research progresses?

Treatment Theory How have researchers, to date, framed hypotheses about treatment theory, regarding such issues as: – Restitution/compensation? – Passive vs. active (and relationship to experience)? – Duration of action? Optimizing treatment prior to large-scale implementation: – Dose/schedule – Generality/specificity – Experiential component

Subject Issues In studies to date, have subjects been selected on: – Neuroanatomic and/or behavioral grounds? (implications for focal vs. diffuse injuries) – Acuity/chronicity? – Status of coexisting impairments?

Outcome Issues Optimal ways of confirming basic mechanism, e.g.: – Changes on functional imaging – Changes in threshold for stimulation – Changes in a tightly linked behavioral domain (Later) Optimal ways of demonstrating functional impact

Overall Design Issues Have studies to date used: – Pre-post treatment designs? – Single subject designs? – Cross-over designs? – Parallel group designs?