1. Using Routinely Recorded Data in a UK RCT: A Comparison to Standard Methods
Graham Powell
MRC HTMR Clinical Research Fellow
University of Liverpool
Professor Tony Marson
Professor Paula Williamson
Professor Dyfrig Hughes
Dr Catrin Tudur-Smith
Dr Laura Bonnett
Introduction, thank you for allowing me the opportunity to present my research…

2. Routinely Recorded Data
Clinical
Non-Clinical
Potential use in prospective research (RCTs):
Recruitment feasibility assessments (Dataline.co.uk)
Measuring study outcomes (Lewsey; 2000, Williams; 2003)
‘Conducting’ a RCT
Recruitment, randomisation, intervention, outcome measurements (Guilliford; 2014)
Data routinely recorded to fulfil a primary function
Clinical:
Healthcare data in the UK National Health Service (NHS), for payment/performance
Non-Clinical:
UK Taxation and Personal Benefits Data
However, limitations in feasibility, access and accuracy have been identified

3. Objectives
Assess the Accessibility of Routinely Recorded Data and Feasibility of Use in a UK RCT
Assess the Attributes of Routinely Recorded Data Compared to Data Collected Using Standard Methods in a UK RCT:
Assessment of the ‘quality’ of routinely recorded data
Assessment of the agreement between routinely recorded data and data collected using standard methods
For this project, regional / national sources of routinely recorded data have been the focus

4. Standard and New Antiepileptic Drugs (SANAD) II
Phase VI RCT (ISRCTN )
Population:
1510
New diagnosis of epilepsy:
Two or more seizures
Intervention:
Arm A: Focal onset seizures:
Lamotrigine OR Levetiracetam OR Zonisamide
Arm B: Generalised onset / unclassified seizures:
Levetiracetam OR Valproate
SANAD II – UK Multicentre phase IV RCT, assessing the clinical and cost effectiveness of antiepileptic drug treatments for epilepsy
Recruitment 2013 – ongoing
FU – 2 – 5.5. yrs

5. Standard and New Antiepileptic Drugs (SANAD) II
Primary Objective:
Time to 12 month remission
Secondary Objectives:
Time to treatment failure
Time to first seizure
Time to 24 month remission
Adverse events
QOL
Cost effectiveness
Baseline Assessment:
Medical History
Clinical Investigations
Follow-Up Assessments
3, 6, 12 months and annually
Seizures
Attendances
Treatment
Adverse events
Self-Report Questionnaires:
QOL
Healthcare resource use

6. Routine Data Sources Clinical Data: Secondary Care:
England: NHS Digital, Hospital Episode Statistics (HES)
Wales: The Secure Anonymised Information Linkage Databank (SAIL)
£10,500
£3,000
Clinical Data: Primary Care:
England:
The Clinical Practice Research Datalink (CPRD)
QResearch
The Health Improvement Network (THIN)
ResearchOne
North West eHealth
£17,000
Economic Data:
HM Revenue and Customs (HMRC)
The Department for Work and Pensions (DWP)
Licensing Data:
The Driver and Vehicle Licensing Authority (DVLA)
‘Linked’ Administrative Data
The Administrative Data Research Network (ADRN)
We underwent a significant period of scoping discussions to identify data sources feasible for inclusion in this study before REC/HRA
All potentially hold relevant data, non-clinical sources were inaccessible for this study, majority of primary care sources are anonymised and non-identifiable therefore not feasible for use in this study
Included data sources and costs, calculated by complexity of data requested, datasets and years accessed rather than number of participants

7. Study Data NHS Digital Hospital Episode Statistics (HES)
Accident & Emergency (66)
Admitted Patient Care (44)
Outpatient (71)
Adult Critical Care (1)
The Secure Anonymised Information Linkage Databank (SAIL)
Emergency Department Dataset (EDDS) (22)
Patient Episode Database for Wales (PEDW) (17)
Outpatient (27)
Primary Care (23)
Data Coverage: 23/05/13 – 31/12/15
() = Number of pts included in each dataset, some have numerous attendances
HES = 71
SAIL = 27
Data was available from both SANAD II and routine sources for a period of 18/12

8. Methods Descriptive Assessment of Data Quality:
‘Comparability’
‘Completeness’
Assessment of Agreement:
Continuous Data
Bland Altman plots
Acceptable clinical limits of agreement specified a priori
Calculation of the mean difference and 95% limits of agreement
Categorical Data
Calculation of Cohens Kappa
Equivalence = comparability, completeness = degree of missing data

9. Routine Data Extraction
Algorithmic approach for the definition of relevant clinical events:
Coding systems, clinical behaviours and organisational pathways
Similar approaches have been used in studies assessing seizures (Grainger 2016) and other disease areas in the UK (Walker 2013, Shawihidi 2014)
Data was extracted for attendances not meeting the algorithmic definitions, but potentially relevant:
Missing diagnostic data
Discrepant diagnostic data
Algorithmic approach for the definition and identification or relevant clinical events FOR EACH COMPARISION was developed using knowledge of…

10. This is an example algorithmic approach for the identification of seizure occurrence
Sx codes were defined a priori from each coding system and identification of relevant codes in the inpatient, emergency or primary care datasets according to the algorithm indicated the occurrence of a seizure.

11. Results
SANAD II Events
Routine Events
Mean Difference
95% Limits of Agreement
‘Acceptable’ Agreement
Baseline Assessment
Date of First Seizure 62 23
-84
30 Days
Diagnosis of Epilepsy 78 37 30
Follow-Up Assessments
Date 12 Month Remission Achieved 46 74 34
Date and Nature of Adverse Events
102 2 NA
90 Days
Date of First Prescription of Antiepileptic Drug 26 25
-20
Dates of Follow-Up Assessments
350
317
0.07
-4 - 4
This slide summarises the results for selected variables, explain the columns
To focus on date of first ever sx – 62 pts in SANAD had date of first sx recorded and were eligible for this comparison. First sx was identified in only 23 pts in routine dataset. For pts with paired data, there was poor agreement with 95% limits of agreement well beyond the acceptable threshold. So what is happening to the pts with no first sx? Approx one third no attendances, one third relevant attendances within 48 hrs of seizure recorded in SANAD II but missing data, one third relevant attendances but discrepant diagnostic codes.
If we consider diagnosis of epilepsy, just 37 of an eligible 78 qualified for a diagnosis of epilepsy at the time of SANAD II randomisation.
With regards to follow-up assessments – For clinical outcomes we see the same pattern of missing data and poor agreement.
Just two variables had relatively complete datasets and were in acceptable agreement: AEDs / HRU

12. The is an example BA Plot for the 23 participants with date of 1st Sx identified…
Mean between datasets, X axis, Difference between datasets, Y axis
A Wilcoxon Signed Rank Test P=0.002, indicates the mean dates of first seizure are significantly different
BA Plot, demonstrates mean of the difference is -84, indicating 1st Sx is identified in SANAD II dataset a mean of 84 days earlier

13. Conclusions Limitations: Benefits: Future Direction?
Access to data is poor and data is expensive
Inadequate for identification of incident cases
Inadequate for assessment of clinical RCT outcomes
Benefits:
Identify additional clinical events (loss to follow-up)
Data regarding compliance
Data regarding healthcare resource use
Future Direction?
National disease registries
Agreed minimum datasets
In the context of a pragmatic RCT assessing treatments for epilepsy, the attributes of routinely recorded data compared to data collected using standard prospective methods are limited:
Routinely recorded data may offer ‘added value’ to standard RCT data:
So where do we go from here?
The codes within existing systems exist, but the problems are missing data and poor agreement compared to RCT methods.
A proposed solution within a publicly funded healthcare system such as the NHS – is the development of regional or national disease registries with agreed minimum datasets, to inform both clinical practice and research. Determination of variables requires input from patient, clinician and researcher and other stakeholder groups such as regulators

14. Questions Acknowledgements: Professor Tony Marson
Professor Paula Williamson
Professor Dyfrig Hughes
Dr Catrin Tudur-Smith
Dr Laura Bonnett
MRC HTMR
Thank you for your time, I would be happy to take any questions