Survey Design and Data Collection Most Of the Error In Surveys Comes From Poorly Designed Questionnaires And Sloppy Data Collection Procedures

Questionnaire Design Preliminary Decisions – What information is required? –Who are the target respondents? –How will we communicate with the respondents? Decisions About Question Content –Is the question needed? –Is the question sufficient? –Can the question be answered? –Will the question be answered? –Are there external events that could influence response?

Questionnaire Design Decisions Concerning Question Phrasing –Do the words have the same meaning to all respondents? –Are the words misleading? –Will the respondents adopt the same frame of reference as the researcher? Decisions About Format Response –How should the respondents be allowed to answer?

Questionnaire Design Decisions About Question Sequence –Does the question order make sense? Decisions About Layout –Is the questionnaire easy to read and answer? Pretest and Revise

Nonsampling Error In Terms of Total Survey Error Nonsampling Error : –Is the Most Serious –Is the Hardest to Detect –Is the Toughest to Fix

Total Error Sampling Error Increasing Sample Size (n) Will Decrease Improved Sampling Procedure Will Decrease Probabilistic Bounds Can Be Placed on Our Level of Precision Nonsampling Error Increasing Sample Size (n) May Increase Improved Research Methods Will Decrease Difficult to Determine the Magnitude

Nonsampling Error Nonobservation –Noncoverage Sampling Frame Doesn’t Match Population Nonresponse Respondents Do not Provide the Information You Need Observation –Field Information Is Not Valid (Poor Questionnaire, Bad Interviewing, Misrepresentat ion) –Office Survey Processing Sloppy

Noncoverage Error: How To Correct Think Hard About Who Our Population Is- This Isn’t As Easy As It Seems Search For A Complete Sampling Frame Fix the Sampling Frame We Have

Nonresponse Error: How To Correct Making Contact: Continue to Make Contact With Sampled Respondents (e.g. 5 call backs). Obtaining Cooperation: Respondents Must be Approached in a Professional Manner. Obtaining Desired Information: Questionnaire and/or Interview Must Appear Reasonable, Understandable, and Confidential.

Field Error: How to Correct The Entire Data Collection Process Must Be Thought Through Carefully. Devote Time to Developing Survey Instrument. Train All People Connected With the Research Project. Pretest Procedures

Office Errors: How To Correct Conduct Checks in the Field to Head Off Potential Problems Early. Check for Deletions, Inconsistencies, and Wrong Answers. Use Closed Ended Questions if Possible. Use Same Coder Per Question Use Statistical Procedures to Catch Mistakes.