Pilot and Feasibility Studies NIHR Research Design Service Sam Norton, Liz Steed, Lauren Bell

Overview Definitions and appropriate NIHR programmes Designing a study (task) Things to include / avoid in an application Summary & resources

What are they? Feasibility study and pilot study used interchangeably Both describe activities to prepare for a larger study – small studies used to plan larger studies Definitions emphasise different things –Feasibility studies focus on deciding whether a full-scale study could be delivered –Pilot studies focus on improving the quality of the future study? Typically NIHR follows definition agreed by EME, PHR, HTA & RfPB

NIHR definitions Feasibility study ‘Research done before a main study in order to answer the question “Can this study be done?”.’ Do not evaluate the primary outcome for efficacy Objectives typically focused on –feasibility of recruitment, follow-up, outcomes May or may not be randomised Sample size not based on a power calculation, but on precision of estimates for the parameters of interest

NIHR definition Pilot study Miniature version of the main study to ‘test whether the components of the main study can all work together’ Focused on the processes of the main study ‘to ensure recruitment, randomisation, treatment, and follow-up assessments all run smoothly’ Two types –Internal pilot: data potentially included in final analysis –External pilot: data analysed but not used in the final analysis of the main study

Schemes FeasibilityPilotLimit% Research for Patient Benefit  £250k>50% Programme Development Grant £100k~33%* Programmer Grants for Applied Research <£800k~50% Efficacy & Mechanism Evaluation None~20%* Health Technology Assessment None~20%* Health Service & Delivery Research None~15%* Public Health Research None~33% Fellowship None~20% *

Exercise Research Question Can a training intervention directed at community pharmacy workers reduce obesity rates? Planned design for main trial is a pragmatic cluster RCT What should the overall aim be? What should the design of this study be? What are the objectives? What might progression criteria be?

Objectives TO TEST KEY UNCERTAINTIES IN STUDY Trial Uncertainties Integrity of study protocol Recruitment and consent rates Selecting primary outcome measure Randomisation procedure Data Collection Intervention Uncertainties –Acceptability of intervention –Implementation and delivery –Training of staff

Things to include Rationale for why feasibility/pilot needed Clear objectives which study will meet Pre-specified feasibility criteria for full scale trial to be feasible –60% of patients must consent to participation in study –75% of patients must complete >4 of 6 exercise sessions How proposed pilot/feasibility study relates to full study and what the trajectory is Patient & public involvement activities

Trajectory: aims & objectives Example “The purpose of the future trial will be to investigate the clinical and cost effectiveness of intervention Y for the treatment of condition X. “ The aim of this present proposal is to pave the way for that future trial by resolving key uncertainties in its planning. Specific objectives are to:- - assess acceptability of the intervention defined as… - assess likely recruitment and retention rates - check suitability of the potential primary outcome

Trajectory: dissemination and outputs Example “Findings from the intended large future trial will be disseminated to maximise potential influence on clinical practice. They will be sent to relevant patient groups, presented at international conferences and published in high impact peer-reviewed journals, etc…. “Results from the present pilot/feasibility study will be presented at national conferences in order to stimulate enthusiasm for centres to participate in the intended future trial….”

Things to avoid Often there aren’t compelling reasons given for why a pilot or feasibility study is needed Justification for the intended follow-on trial is often weak Objectives are not always clear in proposals Likely to be underpowered by design, do not confuse with a phase II study and focus on efficacy Papers based on findings from pilot studies often don’t acknowledge they were pilot studies

Summary Feasibility and pilot study are related but (according to NIHR definition) distinct types of study Appropriate for all NIHR research programmes Need carefully thought out (and justified) aims and objectives Progression criteria useful for judging success Resources –Research Design Service –NIHR databases of funded research –BMC Journal: Feasibility & Pilot Studies –CONSORT guideline extension in preparation

Finally …… THANK YOU GOOD LUCK!!