Maximising Return from Cohorts Project: Prevention of Attrition Findings Cara Booker, Ph.D. MRC SPHSU October 12, 2009

Overview Systematic Review Survey of Retention Among MRC Cohort Studies Preliminary Findings Future Analyses

Systematic Review Objectives To determine the effectiveness of retention strategies in improving response rates in prospective population-based cohort studies. To identify possible characteristics (i.e. locality, age, etc.) of studies that may have affected the retention of cohort members.

Definition, Inclusion & Exclusion Criteria Population-based cohort study: Any well-defined population defined by geographic boundaries, membership or occupation. Szklo M. Population-based cohort studies. Epidemiologic Reviews. 1998;20(1): Exclusion Criteria: Clinical or non-clinical trials Non-population-based cohort studies Cohorts with record linkage as the only method of follow-up Studies which only looked at effectiveness of tracking methods Inclusion Criteria: At least one wave of follow-up data collection in which the participant was personally contacted by the study Retention rates were reported Some description of the retention methods was available

Review Methods Electronic Database Search 13 databases searched 5 search terms Manual searches of bibliographies Internet and website searches for technical reports and internal documents Contact with Principal Investigators and other study personnel for access to unpublished manuscripts and grey literature

Flowchart Potentially relevant prospective population- based cohort studies (n=913) Excluded: Trials & non-population- based cohort studies (n=12,596) Papers retrieved for further evaluation (n=290) Excluded from review (n=265) No evaluation (n=245) Reviews (n=13) Tracing methods only (n=7) 28 Studies in 32 Papers Total papers found from search (n=17,210) Excluded: Recruitment (n=3,701) Excluded (n=623) No information on strategies (n=577) No information on follow-up (x=46) Working Papers /Handbooks/Users Guide (n=5) Unpublished papers (n=1) Book chapter (n=1)

Results 28 Studies were identified Countries 15 in the US 5 in the UK 3 in Canada 2 in Sweden 1 in Norway, Spain & Australia 11 were randomized trials Response rates ranges from 34% - 98%

Retention Strategies by Data Collection Method Evaluated Retention Method Data Collection Method Postal Questionnaires Face-to-Face Interviews Telephone Interviews Mixed Methods Incentives4222 Monetary1112 Non-Monetary21 Mixed11 Reminder Methods9 Repeat Questionnaires/Visits/Calls 812 Alternative Methods of Data Collection 532 Other2 Length of Questionnaire1 Postal Method1

Incentives Randomized Trials (n=8) Incentives associated with increased response rates Average increase ranged from 4% to 27% Type of incentive (i.e. gift, money, cash card) did not appear to have an effect on response rate Use of cash cards and percentage of checks cashed varied Previous round responders had higher rates than non-responders Non-randomized studies (n=2) Comparing monetary to gifts, monetary had increased response Non-monetary and informational incentives compared against each other produced no difference in response Greater value of incentive great response, less money spent overall

Reminder Letters/Calls Randomized Trials (n=1) Higher response among participants with second reminder of phone call Non-randomized studies (n=8) Reminder letters increased response However compared to a second questionnaire there was lower response from the reminder letters Overall, use of multiple reminders or methods of data collection increased response Average initial response rate was 61% The average increase of response rate via Reminder letters was 17% Reminder calls was 11%

Multiple Posting/Calls/Visits 8 studies posted multiple questionnaires, 1 had multiple visits and 2 with multiple calls Posting additional questionnaires increased response by an average of 15% Costs increase with subsequent posting Multiple visits to schools increased response by 34% More call attempts appeared to increase response rates

Alternative Data Collection Method 10 studies offered alternative methods Postal studies that offered telephone interviews increased response by an average of 5% Face-to-face interviews which offered alternatives increased response by 25% Two studies started in a central location, one started with clinic visits Telephone studies that sent postal questionnaires increased response by 1% Costs are higher for postal vs. in-school questionnaires (costs of mail & project coordinator) and telephone interviews are more expensive than post (due to personnel costs, and tracing costs)

Other 2 Randomized Trials Length of questionnaire Received higher response with short form of questionnaire (not significant) Postal method Randomized type of envelope and certified mail vs. UPS –Certified mail had better response –Handwritten envelopes had better response than other types of envelopes

Study Characteristics Different designs, sample sizes, reporting etc make it difficult to address our secondary aims. Broad pattern of: Lowest response rates in US-based studies Higher response rates in studies started in the 1980s and 1990s In general, participants who were ethnic minorities, younger and of lower SES had lower response regardless of retention method Higher education associated with earlier response Postal studies with males only had higher response rates than mixed and female only studies In mixed gender studies females had higher response rates in 8 of 11 studies. Proportion of males increased with increasing contacts

Conclusions Incentives were the only strategy that had a clear and positive association with response rates Multiple mailings of questionnaires and reminder letters also appear to increase response Alternative data collection methods had minimal increases for postal questionnaire studies, but large increases for in-person interview studies Use of multiple strategies increases overall response rate

Discussion/Limitations Very few cohort studies with analyses of retention methods Reporting of attrition is not standardized Often have some mention in various manuscripts detailing characteristics of attriters, which may vary by wave of data collection More often dont have technical reports, or detailed manuscript about the strategies used the characteristics of the attriters across the study Calculation of response rates is also difficult due to eligibility criteria, definition of denominator and booster samples

Recommendations Pilot studies or sub-samples to evaluate retention strategies Focus more expensive strategies to non- responders Most initial responders will do so without the need for incentives or other more vigorous reminders Explore effectiveness and ethical issues associated with internet searches and use of social networking sites in relation to participation not just for tracking

Survey of Retention Among MRC Cohort Studies Objectives To examine what retention methods are associated with reductions in attrition To explore what other study design features, if any, influence minimisation of attrition Sample Drawn from studies which are housed in one of the 12 MRC Population Health Research Network (PHSRN) units Thirty-eight studies were identified

Methods of Survey Data collection Conducted between December 2007 and August 2008 Reminder methods 1 Reminder telephone call 3 Reminder s Questionnaire Contents General study design issues Recruitment Consent issues Compensation/Incentives/Reimbursement Tracking of study members Attrition

Retention, Tracking, Tracing... Retention Strategies Used to increase response at time of data collection Tracking/Tracing Strategies Used to find participants between methods of data collection Ambiguous Strategies Often used between data collection but often used to increase loyalty & not specifically to track/trace

General Study Design 24 studies responded (75% response rate) 7 were dropped from analyses One study had two distinct samples, increasing analysed studies to 25 4 settings Data presented here is from 18 studies

Data Collection How studies collected data Face-to-face: Over 90% Post: 70% Telephone: 40% Internet: 10% Studies collected between four and eight different types of data

Retention Strategies 6 studies allocated funding for retention 10 studies offered incentives 1 study evaluated use of incentives Unconditional gift voucher found to be the most successful Retention Methods 50% of the studies used 2-6 different retention methods Most commonly used , newsletters, newspapers/magazines GPs/CCs, schools Key Leaders/Gatekeepers, administrative/supervisory bodies, parent assessments

Tracking Methods Tracking/Tracking Methods Minimum number of methods used was 1 & 14 was maximum Most used: Change of address cards, GP records, routinely registered events, stable address of close relative Ambiguous methods 80% provided findings 40% provided holiday cards & 20% provided birthday cards

Strategies: Numbers, Barriers & Successes Successful Retention Strategies Family/friend contact, change of address card and telephone/directory assistance Barriers to retention Residential mobility, disinterest in issues covered by study, incorrect address and people too busy to join study.

Further Analyses Analyze the impact of retention methods and different data collection methods on retention What retention methods are the most significant in reducing attrition? What is the impact on attrition when more invasive data collection methods are used (i.e. Vene-puncture, tissue collection, etc.) Use a multi-level modelling approach Waves nested within studies Wave variables include: retention methods, data collection methods Study level variables include: setting, average age of sample at baseline, gender of sample, year study started

Future Analytic Methods Multivariate modelling Combine methods into larger categories i.e. Radio + Internet = Media Examine the effects of different sample population characteristics on subsequent retention rates Data collection types Age Gender Study setting

Thank you!