1. Understanding methods for the assessment of patient adherence: a case study in removable devices for the improvement of trial outcome data
Zoe Tolkien, Shelley Potter, Daniel Yeomans, Abhilash Jain, James Henderson, Jane M Blazeby
School of Social and Community Medicine, Bristol

2. Background
Patient adherence is a key determinant of outcomes for healthcare interventions
Non adherence to drugs therapy is well evidenced
Patient adherence is a key determinant of outcomes for healthcare interventions. If unmonitored, introduces uncertainty into the efficacy of the intervention and validity of the outcomes. Non adherence to drugs therapy is well evidenced. Large body of literature presenting strategies to tackle it, including digital approaches, i.e. electronic pill counts and phone apps.

3. Background
Clinicians report that unscheduled removal can cause irreversible deterioration to injured structure.
Without data, difficult to know how important are these devices.
Paucity of information regarding adherence to rehabilitative or ‘removable’ orthotic devices

4. Aims
Systematically review the literature to identify and summarise methods for assessing patient adherence to removable orthotic devices
Provide guidance to improve the interpretation of outcomes
Use information to provide guidance for future studies to improve interpretation of outcomes.

5. Methods Inclusion and exclusion criteria Studies: All types of trial
Observational study
Case study/series
Studies:
Editorial/practice guidelines
Trial protocols/currently recruiting
Systematic reviews/qualitative
Device:
Removable by the patient
Post trauma or surgery
Full or partial immobilisation
Applied to appendicular skeleton
Device:
Surgically fixed
Chronic treatment
Loose support
Applied to axial skeleton
EXCLUDED: no methods for adherence assessment or consider impact of patient adherence on outcomes  Editorial/guidelines/protocols/recruiting, excluded
DEVICE: removable by patient – smthg to adhere to
We also waned to homogenise the analysis as much as possible so we restricted to devices for acute conditions, that fully or partially immobilised and were applied to a limb of the body.

6. Methods Search strategy Develop Pilot Refine Search
Initial pilot carried out in Ovid MEDLINE.
Further articles identified to help refine the strategy.
Seven electronic databases searched in the final version.
Strategy developed with team members and a research librarian.
A pilot in MEDLINE  identify relevant articles, refine strategy for the final version.

7. Methods Search strategy Adherence Acute injury to limbs
Methods of adherence
Removable orthoses
Search strategy included free text and MeSH terms for each of these four concepts.

8. Identification Total records (n=2135) Screening Eligibility Included
Cochrane & Central
(n=180)
MEDLINE (OVID)
(n=718)
EMBASE (OVID)
(n=856)
AMED (OVID)
(n=38)
CINAHL (EBSCO)
(n=307)
SportDiscus
(n=36)
Identification
Total records (n=2135)
Duplicates (n=476)
Screening
Excluded (n= 1551)
Further duplicates n=25
Language n=64
Study design n=126
Patient population n=1090
Orthosis n=201
No adherence =45
Title/abstract screen (n=1659)
Systematic reviews for snowballing (n=14)
Eligibility
Excluded (n=80)
Language n=1
Study design n=10
Patient population n=22
Orthosis n=10
No adherence n=22
Systematic reviews: 14
Full text screening (n =108)
Final analysis (n =28)
Included

9. Title/abstract screen (n=1659)
Cochrane & Central
(n=180)
MEDLINE (OVID)
(n=718)
EMBASE (OVID)
(n=856)
AMED (OVID)
(n=38)
CINAHL (EBSCO)
(n=307)
SportDiscus
(n=36)
Identification
Total records (n=2135)
Duplicates (n=476)
Screening
Excluded (n= 1551)
Further duplicates n=25
Language n=64
Study design n=126
Patient population n=1090
Orthosis n=201
No adherence =45
Title/abstract screen (n=1659)
Systematic reviews for snowballing (n=14)
Eligibility
Excluded (n=80)
Language n=1
Study design n=10
Patient population n=22
Orthosis n=10
No adherence n=22
Systematic reviews: 14
Full text screening (n =108)
Final analysis (n =28)
Included

10. Full text screening (n =108)
Cochrane & Central
(n=180)
MEDLINE (OVID)
(n=718)
EMBASE (OVID)
(n=856)
AMED (OVID)
(n=38)
CINAHL (EBSCO)
(n=307)
SportDiscus
(n=36)
Identification
Total records (n=2135)
Duplicates (n=476)
Screening
Excluded (n= 1551)
Further duplicates n=25
Language n=64
Study design n=126
Patient population n=1090
Orthosis n=201
No adherence =45
Title/abstract screen (n=1659)
Systematic reviews for snowballing (n=14)
Eligibility
Excluded (n=80)
Language n=1
Study design n=10
Patient population n=22
Orthosis n=10
No adherence n=22
Systematic reviews: 14
Full text screening (n =108)
Final analysis (n =28)
Included

11. Identification Screening Eligibility Final analysis (n =28) Included
Cochrane & Central
(n=180)
MEDLINE (OVID)
(n=718)
EMBASE (OVID)
(n=856)
AMED (OVID)
(n=38)
CINAHL (EBSCO)
(n=307)
SportDiscus
(n=36)
Identification
Total records (n=2135)
Duplicates (n=476)
Screening
Excluded (n= 1551)
Further duplicates n=25
Language n=64
Study design n=126
Patient population n=1090
Orthosis n=201
No adherence =45
Title/abstract screen (n=1659)
Systematic reviews for snowballing (n=14)
Eligibility
Excluded (n=80)
Language n=1
Study design n=10
Patient population n=22
Orthosis n=10
No adherence n=22
Systematic reviews: 14
Full text screening (n =108)
Final analysis (n =28)
Included

12. Results Methods of adherence assessment 18 out of 28 studies:
Nine types of adherence method were identified
Patient reported methods were the most popular, with patient questionnaires most frequently used (none of which were validated).
Others include, patient report, attendance at appt., observation, etc. Overall no single method particularly robust.
High tech methods were used in one study only, where pressure sensors were fitted to the orthosis to provide a robust objective measure.

13. Results Subjective measures Objective measures
Multiple methods
Subjective measures
Objective measures
More robust and reliable
PATIENT DIARY
APPOINTMENT ATTENDANCE
However, we did see three studies combining subj. and obj. measures, e.g. patient diary and attendance at follow up appointments. So although individually they were simple measures, combined they somewhat overcame the disadvantages of each one.
VERBAL REPORT
THERAPIST REFLECTIVE NOTES

14. Results Adherence data and study outcomes
Important part of the treatment?
Unnecessary part of the treatment?
Alternative, better treatment options should be explored?
We also recorded whether studies report adherence data and, importantly which studies associated these data with study outcomes. These are important associations to make otherwise the efficacy of the device remains unknown. E.g. if adherence to the orthosis is good but the study outcome is poor, then the orthosis may be an unnecessary part of the treatment and alternative treatment options should be explored.

15. Results Adherence data RCT Comparative Non- comparative Total
RCT
Comparative
Non- comparative
Total
Data reported for adherence: Yes/No
Type of data*
Number (n=)
Proportion
Adherence score
Formal comparison between groups
Adherence trends over time
Narrative presentation
Other
6/1 4 3 1
2/2 2
12/5 8
20/8 14 10 7 5
Firstly in terms of adherence data:
From the 28 studies in the analysis, 20 presented adherence data.
Data were mainly presented as a number as well as a proportion of pxs adhering to the orthosis with a formal comparison performed between groups in a number of studies.
There were other data presented besides this, such as an adherence score provided by simple metric tools developed in-house.
*More than one category may apply per study.

16. Association between adherence data and study outcomes
Results
Adherence data and study outcomes
Association between adherence data and study outcomes
RCTS
(n=2)
Comparative (n=1)
Non comparative (n=10)
Total
(n=13)
Formal statistical analysis 1 3 5
Descriptive quantitative analysis
Narrative analysis/Text 2
20 studies presented adherence data, but only 13 studies used that data to interpret study outcomes. Less than half of these associations were derived from formal statistical analyses and a few of them were simple narrative interpretations by the authors. Therefore we saw that studies aren’t using the data very well to provide information that clinical decisions can be based on.

17. Results Reasons for non-adherence Reasons for non-adherence
Act on reasons to create change/improvement
We also extracted whether studies recorded reasons why patients didn’t adhere to the orthosis. This should be collected as part of the protocol because, for example, it can provide information that can implement change, such as improving the orthosis design or clearing up any badly worded care instructions to patients.
IMPROVE ORTHOSIS DESIGN
IMPROVE PATIENT MONITORING
IMPROVE CARE INSTRUCTIONS

18. Results Reasons for non-adherence 9 2
However, only 9 studies, about a 1/3 of the total, reported reasons for non-adherence to the orthosis.
This was badly recorded across the board, incl. RCTs

19. Results Reasons for non-adherence Reasons provided* RCT Comp Non-Comp
Total
Discomfort 1 - 3 4
Poor fit 2
Impracticality/inconvenience
Skin irritation/maceration
Feeling better
Clinical
Other
And of these 9 studies, reasons were fairly evenly spread, including discomfort, poor fit, inconvenience, and feeling better. Therefore, we saw some patients prioritising getting on with their daily life over adhering to the treatment and others deciding themselves that they had finished treatment before the clinician had.
*More than one category may apply per study.

20. Conclusions
The majority of studies in this analysis employed low burden methods easily implemented into a trial protocol.
A combination of methods would therefore represent a practical approach for future trials whilst providing both objective and subjective data for increased validity.
Expensive and high tech methods such as sensors may not be required but may be useful tools for validation studies
Development of a validated questionnaire is required to increase validity of results and allow for comparison between studies.
The paucity of data regarding reasons for non-adherence calls for a mixed methods approach involving qualitative research to identify and explore patient factors.

21. Conclusions
The majority of studies in this analysis employed low burden methods easily implemented into a trial protocol.
A combination of methods would therefore represent a practical approach for future trials whilst providing both objective and subjective data for increased validity.
Expensive and high tech methods such as sensors may not be required but may be useful tools for validation studies
Development of a validated questionnaire is required to increase validity of results and allow for comparison between studies.
The paucity of data regarding reasons for non-adherence calls for a mixed methods approach involving qualitative research to identify and explore patient factors.

22. Conclusions
The majority of studies in this analysis employed low burden methods easily implemented into a trial protocol.
A combination of methods would therefore represent a practical approach for future trials whilst providing both objective and subjective data for increased validity.
Expensive and high tech methods such as sensors may not be required but may be useful tools for validation studies
Development of a validated questionnaire is required to increase validity of results and allow for comparison between studies.
The paucity of data regarding reasons for non-adherence calls for a mixed methods approach involving qualitative research to identify and explore patient factors.

23. Conclusions
The majority of studies in this analysis employed low burden methods easily implemented into a trial protocol.
A combination of methods would therefore represent a practical approach for future trials whilst providing both objective and subjective data for increased validity.
Expensive and high tech methods such as sensors may not be required but may be useful tools for validation studies
Development of a validated questionnaire is required to increase validity of results and allow for comparison between studies.
The paucity of data regarding reasons for non-adherence calls for a mixed methods approach involving qualitative research to identify and explore patient factors.

24. Conclusions
The majority of studies in this analysis employed low burden methods easily implemented into a trial protocol.
A combination of methods would therefore represent a practical approach for future trials whilst providing both objective and subjective data for increased validity.
Expensive and high tech methods such as sensors may not be required but may be useful tools for validation studies
Development of a validated questionnaire is required to increase validity of results and allow for comparison between studies.
The paucity of data regarding reasons for non-adherence calls for a mixed methods approach involving qualitative research to identify and explore patient factors.

25. Acknowledgements
MRC Network of Hubs for Trials Methodology Research (methodologyhubs.mrc.ac.uk)
ConDuCT-II Hub for Trials Methodology Research (Bristol.ac.uk/social-community-medicine/ centres/conduct2)
British Society for Surgery of the Hand (bssh.ac.uk)
British Association of Plastic, Reconstructive and Aesthetic Surgeons (bapras.org.uk)
Royal College of Surgeons England (rcseng.ac.uk)