1 Chapter 13 Collecting the Data: Field Procedures and Nonsampling Error © 2005 Thomson/South-Western

2 Figure 1: Overview of Nonsampling Errors Nonsampling Biases  Nonobservation  Noncoverage  Nonresponse Not-at-homes Refusals  Observation  Data Collection Errors  Office Processing Errors

3 Figure 3: Biases Brought to Marketing Research Interview Background (age, education, socioeconomic status, ethnicity, religion, gender, etc.) Background (age, education, socioeconomic status, ethnicity, religion, gender, etc.) Psychological factors (perceptions, attitudes, expectations, motives, etc..) Psychological factors (perceptions, attitudes, expectations, motives, etc..) Behavioral factors (mistakes made: in asking questions, probing, recording, etc.) Behavioral factors (responses inadequate, responses inaccurate, etc.) InterviewerInterviewee

4 Methods for Handling Noncoverage Errors Definition: Failure to include some units or entire sections of the defined survey population in the sampling frame. Methods for Handling: 1. Improve basic sampling frame using other sources. 2. Select sample in such a way as to reduce incidence such as by ignoring ineligibles on a list. 3. Adjust the results by appropriately weighting the subsample results.

5 Methods for Handling Nonresponse Errors Definition: Failure to obtain information from some elements of the population that were selected for the sample. Not-At-Homes: Designated respondent is not home when the interviewer calls. 1.Have interviewers make advance appointments. 2.Call back at another time preferably at a different time of day. 3. Attempt to contact the designated respondent using another approach, i.e., use a modified callback.

6 Methods for Handling Nonresponse Errors Refusals: Respondent refuses to cooperate in the survey. 1. Attempt to convince respondents of the value of the research and the importance of their participation. 2.Provide advance notice that the survey is coming. 3.Guarantee anonymity. 4.Provide an incentive for participating. 5.Hide the identification of the sponsor by using an independent research organization. 6.Try and get a foot-in-the-door by getting the respondent to comply with some small task before getting the survey. 7.Use personalized cover letters. 8.Use a follow-up contact at a more convenient time. 9.Avoid interesting, but not vital, questions. 10.Adjust the results to account for the nonresponse.

7 Methods for Handling Data Collection Errors Definition: When the individual participates in the study, he or she refuses to answer specific questions or provides incorrect answers to them. 1. Match the background characteristics of the interviewer and respondent as closely as possible. 2. Make sure interviewer instructions are clear and written down. 3. Conduct practice training sessions with interviewers. 4. Examine the interviewer’s understanding of the study’s purposes and procedures. 5. Have interviewers complete the questionnaire and examine the replies they secure to see if there is any relationship. 6. Verify a sample of each interviewer’s interviews.

8 Methods for Handling Office Processing Errors Definition: Errors that arise when coding, tabulating, or analyzing the data. 1. Use field edit to detect the most glaring omissions and inaccuracies in the data. 2. Use a second edit in the office to decide how data collection instruments containing incomplete answers, obviously wrong answers, and answers that reflect a lack of interest are to be handled. 3. Use closed questions to simplify the coding, but when open-ended questions need to be used, specify the appropriate codes that will be allowed before collecting the data. 4. When open-ended questions are being coded, and multiple coders are being used, divide the task by questions and not by data collection forms. 5. Have each coder code a sample of the other’s work to ensure a consistent set of coding criteria is being employed. 6. Follow established conventions when coding the data for computer analysis (e.g., use numeric codes and not letters of the alphabet). 7. Prepare a codebook that lists the codes for each variable and the categories included in each code. 8. Use appropriate methods to analyze the data.

9 FIGURE 2: OUTCOMES IN TELEPHONE SURVEYS Pre-Screening  Refusal by Respondent  Refusal by Interviewer  Cooperating (Post-Screening):  Eligible  Completed  Terminated  Refusal  Reject  Not usable  Eligible but over Quota  Ineligible  Nonhousehold  Did not pass screening Respondent Contacted?  No Number Available  No Answer  Not at Home  Household Refusal  Disconnected  Busy  Nonworking Number  Answering Machine/Service  Other Yes No