An ambiguous relationship : Interview standardisation and Survey accuracy

1.Presentation of Insee interviewers 2.Main features of the job

I Presentation of Insee interviewers

I. Presentation of Insee interviewers 1.Work organisation Insee is a decentralised organisation: The headquarters in Paris 21 regional offices covering the whole territory (metropolitan) Interviewers are allocated to the regional offices Around 900 interviewers in 21 regional offices UMS (Unité des Méthodes Statistiques) coordinates this widespread network : For management issues (CAEMC)… … As well as methodology issues (ETSD)

I. Presentation of Insee interviewers 2.Interviewers employment conditions A very unusual type of contract: pay per result Current negotiations for more stable contracts (impulsed by Insee) Interviewers do both field and telephone interviews For interviewers: a flat organisation that doesnt open any carrier perspective (the management staff is based on Insee civil servants)

I. Presentation of Insee interviewers 3.Key features about interviewers Gender: 82% are female Age: 48 Experience: 8 years Turnover Unionisation : rather high, with active unions An ambiguous relationship to the job : involvement and confrontation

II Insee interviewers: Main features of the job

II. Main features of the job 1.Principles from the methodological othodoxy With its wording, each question has a meaning for the respondents This meaning is the same for all respondents Data standardisation mainly depends on the standardisation of the interviewing process

II. Main features of the job 2.Instructions for interviewers In coherence, with the former principles: Word by word reading of the questionnaires No re-wording Very detailed survey protocols Regular interviewers training Involvement of the interviewers as experts for the conception and wording of questionnaires

II. Main features of the job 3.A large autonomy Once on field (or telephone), interviewers are autonomous Ex post controls are possible (via keyboarding historic) but… … rarely used.

II. Main features of the job 3.A large autonomy A quality approach based on : Trust on interviewers professionalism and their involvement in quality issues for public statistics Incentives carried on by objectives (pay by results) It seems to work: rather low non-response rate in France, satisfying quality

III From the strict principles… to the actual work

III. The actual work 1.The need for destandardisation Main concern for interviewers : To get the right answers for the questions in the questionnaire at stake Global quality of a statistical corpus is an abstract concept for interviewers An expert claim for destandardisation as a condition for successful interviews

III. The actual work 1.The need for destandardisation A rational response to quality objectives : Translating the questionnaire: adaptation to various linguistic registers Helping the respondent: sometimes necessary for collecting the correct and accurate information Taking field specificities into account to ensure against misunderstandings

III. The actual work 1. The need for destandardisation A rational response to social constraints in an interaction: Refusing to limit the interaction to its functional dimension Avoiding to bore the respondents and individualise the interaction (very important for panel surveys !)

III. The actual work 1. The need for destandardisation For the interviewers themselves, the need to give sense to their work and skills : Avoiding repetitiveness Appropriating the job by intervening on the working process and using ones creative skills to enhance the quality

III. The actual work 2.At the end, a partial standardisation that tends to follow Insees methodological principles, but preserves some wiggle room for interviewers and interviewees that enhance in some cases the quality of the responses (depending on the interviewers skills) but potentially creates variance (heterogeneity of interviewers skills) Problem: this variance is impossible to control ex post Anyhow, we have to deal with an empirical hard core

III. The actual work 3.Main leads for the future Enhancement of the interviewers employment conditions Toward a greater involvement in Insee work organisation ? Set up of an Insee telephone unit (Metz), with the following agenda: Tasks divided between field and telephone interviewers The telephone unit is the opportunity to better standardise and supervise all the telephone interviews A great change : the whole interviewers organisation has to be audited

Conclusions The set up of a telephone unit will allow: to enhance control procedures to better standardise the interviews To give carrier perspectives to interviewers But what will be the consequences on data quality (interviewee weariness, non-response growth) ? on interviewers quality of job (loss of interest, withdrawal behaviour, turnover) ? Round table debate: How to ensure against the pervert effects of telephone units activity ?