Fieldwork efforts  Monitoring fieldwork efforts  Monitoring fieldwork efforts: Did interviewers /survey organisations implement fieldwork guidelines and procedures?  Evaluation of fieldwork efforts  Evaluation of fieldwork efforts: Did fieldwork efforts result in increased response rates and more representative surveys?  Understanding  Understanding the reasons of success/failure by combining information on monitoring and evaluation

1. Data  Call record data from 12 countries: The Netherlands, Hungary, Poland, Portugal, Finland, Switzerland, Great Britain, Ireland, Hungary, Spain, Greece, Israël each visiteach sample unit  Information on timing, outcome and mode of each visit made to each sample unit

2. Reducing noncontact rates  Reduce noncontact rate by: – making many calls or visits – making visits at varied times of day and week  Contacting procedure – minimum of four calls – minimum of 1 weekend call – minimum of 1 evening call – all visits before first contact face-to-face

Average number of calls/visits made to noncontacts  Very High  Very High: Switzerland (73), Great Britain (9.4), Spain (7.8)  High  High: Finland (5.0), Greece (5.0), Poland (4.8), Slovenia (4.9), The Netherlands (5.1), Portugal (5.1)  Moderate  Moderate: Hungary (4.0)  Low  Low: Ireland (3.3), Israël (2.6)

Number of call attempts  Countries with rather high noncontact rates and low number of call attempts (e.g. Ireland) might increase call attempts  No “clear-cut” relationship between number of call attempts and noncontact rates  Some countries achieve the target noncontact rate with only moderate or low efforts extended interviewer efforts  Other countries rely heavily on extended interviewer efforts

Noncontact rates before and after extended interviewer efforts (>4 calls)

Number of call attempts  Necessity to make many calls/visits is especially high in Great Britain, Spain, Portugal and Switzerland: Why? at-home patterns  Less favourable “at-home patterns” ? timing of visits  Less optimal timing of visits?

Percentage of visits made on a weekday morning or afternoon for the first four visits

Percentage of sample units successfully contacted at first visit by timing of visit

Timing of visits  Countries with less favourable at-home patterns can adapt strategies and make more evening/weekend calls  Portugal : high benefits of weekend visits, high % weekend visits  Great Britain & Ireland: high benefits of evening visits but low % of evening visits  However, making many evening and weekend calls is not necessarily more efficient

3. Increasing survey participation  Advance letter  Respondent Incentives  Interviewer training  Refusal conversion

Refusal conversion procedures  Re-approaching reluctant sample units and asking them to reconsider participation  ESS recommendations: reissue all soft and as many hard refusals as possible to another interviewer  Implementation will vary because of differences in ressources, necessity and definition of “soft” refusal

Percentage of refusals re-approached  Very High  Very High: The Netherlands (88%), Switzerland (84%), Great Britain (77%)  High  High : Finland (50%), Greece (54%)  Moderate  Moderate: Spain (34%), Slovenia (33%), Poland (24%), Israël (17%)  Low  Low: Ireland (1.9%), Hungary (5.3%)

Conversion success rate  40%: The Netherlands  30-40% : Slovenia  20-30%: Israël, Poland, Finland  10-20%: Greece, Great Britain  <10%: Switzerland

Response rates before and after refusal conversion

How to explain success in the Netherlands?  Second letter to reluctant sample units  Incentives increased with financial donations up to 5 Euro  Incentives supplemented with quiz  Highly motivated survey organisation and interviewers?

Does refusal conversion reduce nonresponse bias?  Evaluation of refusal conversion procedures should also take into account relationship between increasing response rates and reduction of bias (Stoop,2003)  This can be done by comparing converted refusals with cooperative respondents  The Netherlands  The Netherlands represents an interesting case: what happens with survey estimates if response rates increase from 54% to 68%

Effects on survey estimates % Cooperative respondents Converted refusals Total EDUCATION (HIGH) INCOME (HIGH) SEX (MALE) AGE (+65) SINGLE MEMBER ORG. (NO) FRIENDS (NO) SOC. TRUST (LOW) POL. INTEREST (HIGH) IMMIGRANTS N

Effects on survey estimates  Most differences are in the expected direction  Large & significant differences for political interest (cfr. Voogt & Saris, 2003)  Increasing response rate from 54% to 68% seems to reduce bias and improve at least some survey estimates

Evaluation of refusal conversion  Efforts in Great Britain & Switzerland were not cost-effective: hardly any effect on response rates  Refusal conversion did work in the Netherlands, probably due to a range of conversion strategies  In the Netherlands, the profile of converted refusals indicates that they are different on a number of key survey variables  Results suggest that countries with low response rates might reach higher response rates and more representative samples by adopting the successfull approach of the Netherlands

4. Conclusions  Call record data are a very usefull and essential tool for monitoring and evaluating fieldwork strategies  The analysis raises some interesting questions: e.g. about refusal conversion and timing of visits  Results should feed back into survey process and lead to actions to improve fieldwork procedures