2. TQM in Statistics Finland’s Interview Data Collection Process Matti Simpanen, Kai Vikki and Eero Tanskanen

3. 11th of July 20082M.Simpanen, K.Vikki, E.Tanskanen Tutkimusongelma  Data collection!   Weighting, corrections Analyse, reporting   Quality output Good reputation and glory!   Survey plan, sampling

4. 11th of July 20083M.Simpanen, K.Vikki, E.Tanskanen Topics The employment system The recruitment process Interviewer training The standardised interviewing method Setting the objectives Quality monitoring

5. 11th of July 20084M.Simpanen, K.Vikki, E.Tanskanen Survey process is a coherent process A survey is a process in which every individual stage is influenced by the other stage Every part of the survey process has to have adequate understanding of the survey process, its backgrounds, goals and operating principles Even interviewers Needs long-term, continuous quality monitoring (various perspectives) Interviewers employment system What kind of employment system helps to keep quality high? Remuneration From what interviewers are paid? Efficienty or quality? System has to be encouraging and fair

6. 11th of July 20085M.Simpanen, K.Vikki, E.Tanskanen Quality starts from the recruitment process Systematic recruitment process to measure applicant’s ability to understand and follow the standardised interview method to measure interaction and style of interview (positiveness) to measure ability to adopt backgrounds and goals of surveys quickly (general understanding of surveys and interview work) to measure general commitment abilities and a tolerance of routines Interviewers represent survey organisation, are impartials In Statistics Finland recruitment process divides into three phases: Phone contact Structured job application form Job interviews

7. 11th of July 20086M.Simpanen, K.Vikki, E.Tanskanen The quality demands training and orientation To orietate interviewers to survey process basic training, project training, further training Basic training The goal is to orientate new interviewers to interviewer’s work to familiarise interviewers to standardised interviewing method to orientate new interviewers to the survey process as a whole to orientate new interviewers to the organisation (commitment) In SF one week basic training + repetition and follow-up after 1-2 months

8. 11th of July 20087M.Simpanen, K.Vikki, E.Tanskanen Interviewer skills need continuing training Project training to give adequate understanding of the individual survey project and its backgrounds Further training to develop and to maintain interviewers skills to motivate interviewers to commit interviewers to the survey process and to the quality work to repeat items of standardised interviewing method Goal of training is also to get interviewers understand their role in the survey process and the value of their work

9. 11th of July 20088M.Simpanen, K.Vikki, E.Tanskanen Questionnaire standards Interviewing techniques Interviewing style Interview interaction Standarised interviewing method

10. 11th of July 20089M.Simpanen, K.Vikki, E.Tanskanen Questionnaire standards Structured way to make questions, answer alternatives, introductions, descriptions to the questionnaires Helps the interviewer realise how to ask different kind of questions gives time to interviewer to use his/her skills of interaction to get the interviews interviewer´s work is to do interviews, not to think how different questions should be asked to reduce interviewer effect

11. 11th of July 200810M.Simpanen, K.Vikki, E.Tanskanen Interviewing techniques the manner in which interviewer presents questions and answer alternatives, introductions, descriptions the manner interviewer presents the question modules and introductions and clarifies questions if the respondent does not understand the question (probing) Interview techniques allow more time to use to the interaction Using interviewing techniques the interviewer ”teaches” the respondent to understand our way to do interviewing work (the ”right” answer atmosphere) To keep quality high (to ask questions similarily leads to less interviewer effect and more comparable and reliable data)

12. 11th of July 200811M.Simpanen, K.Vikki, E.Tanskanen Interviewer: Always read the following text as well: Did you return in April, May or June from a domestic trip with overnight stay, with a destination of at least 30 km from your home. Included are all trips with overnight stay in Finland; also trips to own free-time residence, visits to friends or relatives, and business and professional trips?

13. 11th of July 200812M.Simpanen, K.Vikki, E.Tanskanen Interviewing style Articulation Pace Use of pauses and emphasis Speech manners Neutrality of language Voice in general

14. 11th of July 200813M.Simpanen, K.Vikki, E.Tanskanen Interview interaction We put effort to interaction: The basic precondition is to get the respondent to agree to the interview Interviewers behave to convince the respondent (persuation) To create right kind of atmosphere by applying the interviewing technique and to use the correct interviewing style In SF we measure by recordings the interviewers´ interviewing techniques, interviewing style and interaction

15. 11th of July 200814M.Simpanen, K.Vikki, E.Tanskanen Objectives, follow-up To maintain quality high needs the objectives, the goal settings efficiency and quality to the projects and to the interviewers schedule, response rate, quality Quality control by monitoring (recordings, supervisor monitoring) Objectives to commit interviewer to the project to the survey to the (survey)organisation Cost control system

16. 11th of July 200815M.Simpanen, K.Vikki, E.Tanskanen Feedback system Appraisals Interviewer feedback surveys Respondent feedback surveys

17. 11th of July 200816M.Simpanen, K.Vikki, E.Tanskanen Appraisal discussions Feedback of the individual survey project is not enough to keep quality of the interviews high Needs broader discussions with interviewers how the employee has succeeded how to improve co-operation in the work society motivation professional development and again to commit him-/herself to the survey and to interview work In SF each employee has annual development and performance discussions (appraisals) with their immediate or own supervisor personal performance -> flexible personal pay component

18. 11th of July 200817M.Simpanen, K.Vikki, E.Tanskanen Interviewer feedback surveys Systematic, comparable feedback surveys from each projects to develope questionnaires and survey processes systematic feedback Also systematic feedback on the interviewers’ working conditions, work methods and job orientation to motivate to commit the interviewers part of the survey processes Feedback surveys show also that researchers/organisation respects interviewers’ attitudes and their work

19. 11th of July 200818M.Simpanen, K.Vikki, E.Tanskanen Respondent feedback surveys Systematic, comparable feedback surveys from each projects to develope questionnaires and survey processes systematic feedback response burden validation

20. 11th of July 200819M.Simpanen, K.Vikki, E.Tanskanen At the end... The main point is NOT to say every organisation should have its own interviewing organisation but... survey is the coherent process - including several subprojects every party of the whole process has to have similar values of the process every party has to have similar understanding of the process every party needs to commit themselves to the survey process - not only an individual project Processes should be systematic and systematically evaluated To maintain quality of data collection demands continuing follow-up All these mentioned before have a great influence to the quality Poor data collection cannot be corrected with mathematical tricks!

21. 11th of July 200820M.Simpanen, K.Vikki, E.Tanskanen Grazie! Thank you! matti.simpanen@stat.fi (back in the office 4.8.2008) matti.simpanen@stat.fi