2. TQM in Interview Data Collection Process Matti Simpanen

3. 22.9.20102M.Simpanen  Data collection=Interviews!  Weighting, corrections Analyse, reporting  Quality outcome Good reputation and glory!  Survey plan, sampling Research problem

4. 22.9.20103M.Simpanen Interview is not a constant variable! My mission is try researchers to bear in mind the data collection should not be forgotten in quality reports Researchers should demand quality in data collection process, and not to keep it as given, constant quality Also financiers should consider the quality of data, not only how get data with lowest expensies

5. 22.9.20104M.Simpanen Topics The employment system The recruitment process Interviewer training The standardised interviewing method Setting the objectives Quality monitoring

6. 22.9.20105M.Simpanen Survey process is a coherent process A survey is a process in which every individual stage is influenced by the other stage Every part of the survey process has to have adequate understanding of the survey process, its backgrounds, goals and operating principles Even interviewers or other field workers Needs continuous quality monitoring (various perspectives) Interviewers employment system What kind of employment system helps to keep quality high? Remuneration From what interviewers are paid? Efficienty or quality? System has to be encouraging and fair

7. 22.9.20106M.Simpanen Quality starts from the recruitment process Systematic recruitment process to measure applicant’s ability to understand and follow the standardised interview method (see later) to measure interaction and style of interview (positiveness) to measure ability to adopt backgrounds and goals of surveys quickly (general understanding of surveys and interview work) to measure general commitment abilities and a tolerance of routines Interviewers represent survey organisation, are impartials In Statistics Finland recruitment process divides into three phases: Phone contact (systematic evaluation of applicant's voice, interaction, etc). Structured job application form (internet-based) Job interviews (f-to-f)

8. 22.9.20107M.Simpanen The quality demands training and orientation To orietate interviewers / other field workers to survey process basic training, project training, further training Basic training The goal is to orientate new interviewers to do interviewer’s work to familiarise interviewers to standardised interviewing method to orientate new interviewers to the survey process as a whole to orientate new interviewers to the organisation (commitment) In SF one week basic training + repetition and follow-up after 1-2 months

9. 22.9.20108M.Simpanen Interviewer skills need continuing training Further training to develop and to maintain interviewers skills to motivate interviewers to commit interviewers to the survey process and to the quality work to repeat items of standardised interviewing method Goal of training is to get interviewers understand their role in the survey process and the value of their work

10. 22.9.20109M.Simpanen Questionnaire standards Interviewing techniques Interviewing style Interview interaction Standarised interviewing method

11. 22.9.201010M.Simpanen Questionnaire standards Structured way to make questions, answer alternatives, introductions, descriptions to the questionnaires factual and other (opinion) questions standards are guiding the interviewer helps researcher to formulate questions for example does the question really measure the researcher wants it to measure Helps the interviewer realise how to ask different kind of questions gives time to interviewer to use his/her skills of interaction to get the interviews interviewer´s work is to do interviews, not to think how different questions should be asked (to reduce interviewer effect)

12. 22.9.201011M.Simpanen Interviewing techniques the manner in which interviewer presents questions and answer alternatives, introductions, descriptions the manner interviewer presents the question modules and introductions and clarifies questions if the respondent does not understand the question (neutral probing) Interview technique is NOT monotonous repeating the questions Interview techniques allow more time to use to the interaction Using interviewing techniques the interviewer ”teaches” the respondent to understand our way to do interviewing work (the right answer atmosphere) To keep quality high (to ask questions similarily leads to less interviewer effect and more comparable and reliable data)

13. 22.9.201012M.Simpanen Examples: Question order How satisfied are you to your life as general? How satisfied are you to your spouse? Correlation: +0,37 How satisfied are you to your spouse? How satisfied are you to your life as general? Correlation: +0,62

14. 22.9.201013M.Simpanen Interviewer: Always read the following text as well: Did you return in August or July from a domestic trip with overnight stay, with a destination of at least 30 km from your home. Included are all trips with overnight stay in Finland; also trips to own free-time residence, visits to friends or relatives, and business and professional trips?

15. 22.9.201014M.Simpanen Interviewing style Articulation Pace Use of pauses and emphasis Speech manners Neutrality of language Voice in general

16. 22.9.201015M.Simpanen Interview interaction We put effort to interaction: The basic precondition is to get the respondent to agree to the interview Interviewers behave to convince the respondent (persuasion) To create right kind of atmosphere by applying the interviewing technique and to use the correct interviewing style In SF we measure by recordings the interviewers´ interviewing techniques, interviewing style and interaction

17. 22.9.201016M.Simpanen Objectives, follow-up To maintain quality high needs the objectives, the goal settings efficiency and quality to the projects and to the interviewers short-term and long-term goals, even mid-term goals on project schedule, response rate, quality Quality control by monitoring (recordings, supervisor monitoring) Objectives to commit interviewer to the project to the survey to the organisation Cost control system

18. 22.9.201017M.Simpanen Feedback system Appraisals or personal discussions Interviewer feedback surveys Respondent feedback surveys

19. 22.9.201018M.Simpanen Appraisals, discussions Basic/project training is not enough to keep quality of interviews high Needs also discussions with interviewers time to time how the employee has succeeded how to improve his/her performance co-operation in the work society motivation professional development and again to commit him-/herself to the survey and to interview work In SF each employee has annual development and performance discussions (appraisals) with their immediate or own supervisor personal performance -> flexible personal pay component

20. 22.9.201019M.Simpanen Interviewer feedback surveys Systematic, comparable feedback surveys from each projects to develope questionnaires and survey processes systematic feedback Feedback surveys show also that researchers/organisation respects interviewers’ attitudes and their work Also systematic feedback on the interviewers’ working conditions, work methods and job orientation to motivate to commit the interviewers part of the survey processes

21. 22.9.201020M.Simpanen Respondent feedback surveys Systematic, comparable feedback surveys from each projects to develope questionnaires and survey processes systematic feedback response burden validation

22. 22.9.201021M.Simpanen At the end... The main point is NOT to say every organisation should have its own interviewing organisation but... survey is the coherent process - including several subprojects every party of the whole process has to have similar values of the process every party has to have similar understanding of the process every party needs to commit themselves to the survey process - not only an individual project Processes should be systematic and systematically evaluated To maintain quality of data collection demands continuing follow-up All these mentioned before have a great influence to the quality Research, analyse and report also paradata of interviews Poor data collection cannot be corrected with mathematical tricks!

23. 22.9.201022M.Simpanen Thank you! matti.simpanen@stat.fi