3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Are ‘better’ interviewers more successful at engaging reluctant respondents? Evidence from an experiment conducted on the 1970 British Cohort Study Hannah Carpenter – TNS BMRB Lisa Calderwood – Centre for Longitudinal Studies Matt Brown - Centre for Longitudinal Studies

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS What is the 1970 British Cohort Study? 2 A longitudinal socio- economic study following everyone born in Britain in a particular week in 1970 The Age 42 survey in 2012/13 was entirely face to face. It consisted of an hour long CAPI interview and a 16 page paper self completion questionnaire cohort members were included in the Age 42 survey, of which 9064 were interviewed. Has used a variety of data collection methods in the past but mostly face to face interviewing. Currently fieldwork takes place every years Regular questions about family composition, employment & income, health, education and qualifications. Other content varies at each wave.

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Identified high performing interviewers Allocated some of the ‘likely refusers’ to high performing interviewers and some to other interviewers to see if the high performing interviewers would achieve a better response rate Outline of interviewer experiment at Age 42 survey 3 Identified cohort members that were more likely to refuse (based on previous response) Aim to increase overall response rate

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 We have a scoring system for interviewers. On each assignment interviewers are set a target response rate. Interviewer performance index – an indicator based on scores achieved on all assignments (across all projects) an interviewer has worked on in the last quarter. Interviewers were classed as “high performing” if they had an index score of 0.95 or higher. ¼ of interviewers working on BCS70 were classified as high performers. How were ‘high performing’ interviewers identified? Score % Response rate High performance zone

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Characteristics of interviewers 5 %

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Characteristics of interviewers 6 % Less than 2 years service 2 years to less than 4 years 4 years to less than 6 years 6 years to less than 8 years 8 years to less than 10 years 10+ years service

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 How were ‘likely refusers’ identified? 7 Anyone who refused to take part in the last survey in 2008 Anyone who refused to take part in the 2004 survey and also did not take part (although did not necessarily refuse) in 2008 Anyone who refused to take part in the 2000 survey and also did not take part (although did not necessarily refuse) in 2004 and 2008 This identified 1,009 ‘likely refusers’ ( 8.7% of the sample)

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 How were ‘likely refusers’ allocated to interviewers? 8 Type of refusal case High performers Other interviewers Refused in %50% Refused in 2004 and not taken part since41%44% Refused in 2000 and not taken part since4%6% Base p=0.16 p=0.40 p=0.22

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Other comparisons of ‘likely refusers’ 9 Type of refusal case High performers Other interviewers Last took part in % Last took part in %35% Last took part longer ago31%32% Male52%53% Female48%47% Proportion thought to have moved22% Base281611

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Did high performing interviewers get a better response rate amongst ‘likely refusers’? 10 Outcome High performers Other interviewers Interview achieved30%29% Refusal51%50% No contact2%3% Untraced mover11%13% No longer eligible (died/moved abroad)3%2% Other unproductive3%4% Base p=0.76 p=0.78 p=0.39 p=0.40 p=0.36 p=0.46

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Response rates on remaining sample (not ‘likely refusers’) 11 Outcome High performers Other interviewers Interview achieved87.4%84.0% Refusal6.6%8.7% No contact0.6%1.0% Untraced mover3.5%3.8% No longer eligible (died/moved abroad)1.0%0.8% Other unproductive0.9%1.7% Base p<0.001 p=0.05 p=0.46 p=0.32 p=0.002

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Comparisons of other cases (not ‘likely refusers’) 12 Type of refusal case High performers Other interviewers Last took part in % Last took part in % Last took part in 20002% Last took part longer ago4% Male48%49% Female52%51% Proportion thought to have moved4% Base

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Did high performers have any other benefits? 13 The average number of contact attempts to obtain an interview was similar for high performers and other interviewers (around 6 for both). Higher performers and other interviewers had a similar level of success tracing people who had moved. High performers achieved very slightly higher rates of consent to data linkage with govt. data sources (67% compared with 64% for other interviewers) High performers were better at following survey protocol on one measure – in 89% of cases they has posted/given the paper questionnaire to respondents in advance of the interview. Other interviewers had done this in 83% of cases.

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Conclusions 14 High performing interviewers were no more successful at engaging reluctant respondents on this study This may be due to the way high performers were identified Or perhaps interviewers are less able to influence longitudinal study members

3.14 X AXIS 6.65 BASE MARGIN 5.95 TOP MARGIN 4.52 CHART TOP LEFT MARGIN RIGHT MARGIN ©TNS 2013 Thank you 15