1. 6/3/2015Bill Reimer1 Sampling and Questionnaires Bill Reimer October 31, 2005 reimer@vax2.concordia.ca http://reimer.concordia.ca/teaching

2. 6/3/2015Bill Reimer2 Outline Sampling Questionnaire construction

3. 6/3/2015Bill Reimer3 Sampling Objective Get a lot of information with little cost Population Sample

4. 6/3/2015Bill Reimer4 Sample sizes for a population of 10,000

5. 6/3/2015Bill Reimer5 3 Rules of Scientific Sampling each unit in the population has an equal chance of being chosen we must know the chance of each member being chosen each selection must be independent from the others

6. 6/3/2015Bill Reimer6 To what do you want to generalize? People Households Trees Leaves These are the Units of Analysis

7. 6/3/2015Bill Reimer7 To which units do you want to generalize? People in Canada? Women in Québec? Citizens of Montréal? Trees in boreal forests? This is the Population

8. 6/3/2015Bill Reimer8 The Population Is the collection of units to which you want to generalize It may be abstract It may be concrete But it must be clear

9. 6/3/2015Bill Reimer9 From what will you choose your sample? A list of units? A region or place? A drawer or computer file? A particular time? This is your sample frame

10. 6/3/2015Bill Reimer10 The Sample Frame may not match the Population Population Sample Frame Units in the SF but not the P Units in the P but not the SF …but try to get them close

11. 6/3/2015Bill Reimer11 Sampling Requirements: Identify the population Is the sample frame reasonable? Does it meet the 3 conditions of scientific sampling? We can seldom meet all 3 conditions!

12. 6/3/2015Bill Reimer12 Types of Samples Probability Generalizability Efficient Non-probability Exploratory Strategic

13. 6/3/2015Bill Reimer13 Simple Random Sampling Equal chance of being chosen We know the chance Each selection independent           From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

14. 6/3/2015Bill Reimer14 Systematic Sampling Determine sample rate (8/40) Random selection of 1 st case Every nth case (n=5) Watch for regularities in the sample frame         From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. Random selection

15. 6/3/2015Bill Reimer15 Stratified Random Sampling Separate sample frame into strata Take random (or systematic) sample from each strata May be proportionate or disproportionate         From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

16. 6/3/2015Bill Reimer16 Cluster Sampling Divide population into clusters Randomly select clusters Collect data on all cases in cluster Watch cluster composition         From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

17. 6/3/2015Bill Reimer17 Cluster Sampling - Montréal

18. 6/3/2015Bill Reimer18 Non-Probability Samples Accidental Purposive

19. 6/3/2015Bill Reimer19 Convenience Sample Choose most convenient people Subject to multiple biases         From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

20. 6/3/2015Bill Reimer20 Volunteer Sampling Ask for volunteers Directly Ads Selected groups Unclear biases       From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

21. 6/3/2015Bill Reimer21 Strategic Sampling Handpick respondents for strategic purposes Useful for marginalized or small groups Control biases by specifying criteria From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

22. 6/3/2015Bill Reimer22 Snowball Sampling Identify strategic respondent Ask for referrals Continue the process Use multiple starts to avoid network biases From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.

23. 6/3/2015Bill Reimer23 Probability Samples – Multiple Mixture of various techniques

24. 6/3/2015Bill Reimer24 Sample Size Depends On Size of the population Accuracy desired Confidence desired Variation in the phenomenon investigated

25. 6/3/2015Bill Reimer25 Assignment Specify a research question you would like to answer. Describe a sampling procedure along with the data you would collect to answer the question. Identify the following: the population the sample unit the sample frame the type of sample

26. 6/3/2015Bill Reimer26 Samples Non-probability Accidental Purposive Probability Simple random Systematic Stratified: Proportionate Disproportionate Cluster Multiple

27. 6/3/2015Bill Reimer27 Constructing Questionnaires

28. 6/3/2015Bill Reimer28 Types of Research Interviews Questionnaires Closed and some open-ended questions Semi-structured interviews Open and some closed-ended questions Unstructured interviews Open-ended questions Research Interviews are NOT what you see on TV

29. 6/3/2015Bill Reimer29 Advantages and Disadvantages Questionnaires Economic Speedy Minimal interviewer bias Anonymity Low response rates Limited exploration Comparison easy Semi-structured High response rates Can ask complicated questions Can follow-up Qs Can observe Expensive Time consuming Comparison difficult

30. 6/3/2015Bill Reimer30 Preparing Questions Resist the temptation to start with specific questions Clarify the RESEARCH question Identify the types of information required to answer the research question Within each of the types of information brainstorm for specific questions to ask Rearrange questions

31. 6/3/2015Bill Reimer31 Question Order Leave sensitive issues for later - once rapport has been established Follow from general to specific Use tables and grids where appropriate Use cards for sensitive issues if appropriate Prepare your probes if interview Follow a natural flow

32. 6/3/2015Bill Reimer32 Anticipate the Analysis Produce dummy tables Work backwards from your analysis

33. 6/3/2015Bill Reimer33 Format examples Contingency question Full filter question Quasi-filter question Closed question Partially open question Open question Probes

34. 6/3/2015Bill Reimer34 Simple, Mutually Exclusive What is your gender? (Circle the appropriate number) Female...............1 Male.................2 5

35. 6/3/2015Bill Reimer35 Multiple Choice, not Mutally Exclusive What modes of transportation did you used to get to school last week? (Circle all that apply) Metro.................1 Bus...................1 Car...................1 Bicycle.................1 Walk..................1 Other (please specify) _____________________1 6 7 8 9 10 |__|__| 11-12

36. 6/3/2015Bill Reimer36 Rank-order Rank order the 3 most important aspects of your job (Place 1 beside the most important, 2 beside the next most important, and 3 beside the next most important) Money..................|___| Security.................|___| New experiences.......... |___| Power.................. |___| Excitement.............. |___| Other (please specify) ________________________|___| 13 14 15 16 17 |__|__| 18-19

37. 6/3/2015Bill Reimer37 Pretest, pretest, pretest Use your friends and family Discuss Where it was clear or not clear What they thought about when responding to each question What they thought you were getting at Move to colleagues Check it out on people who are similar to those you will eventually survey

38. Problem Questions Jargon, slang, abbreviations What is your Internet browser? Ambiguity, confusion, vagueness What is your income? Emotional language and prestige bias Should we put murderous terrorists in jail? Do you support Prime Minister Chretien’s policy on world trade? Double-barrelled questions Should marijuana be legalized for medical or other purposes?

39. 6/3/2015Bill Reimer39 Problem Questions – con’t Leading questions Do you feel that governments should have less power to interfere in business? Beyond respondent’s competence Does your mother support extramarital sex? False premises How can we halt the rising crime rate?

40. 6/3/2015Bill Reimer40 Problem Questions – con’t Future intentions How will you vote in the next federal election? Overlapping or unbalanced response categories What is your opinion regarding abortion? Very favourable Favourable Neutral Opposed

41. 6/3/2015Bill Reimer41 Materials http://artsandscience.concordia.ca/soci613br Check the Readings link: SPSS Keywords support for questions Jackson article – questionnaire design Selltiz – checklist This Powerpoint presentation Texts

42. 6/3/2015Bill Reimer42 Questionnaire Construction The amount of time spent in the preparation of the questionnaire is directly related to the value of the results. Surveys usually cost a lot and you can't redo them. Know what you are wanting to discover

43. 6/3/2015Bill Reimer43 Research Strategies Questionnaires for generalizability Exploration: well selected, in-depth interviews Use interviews to design survey Use comparisons

44. 6/3/2015Bill Reimer44 Sampling and Questionnaires Bill Reimer October 31, 2005 reimer@vax2.concordia.ca http://reimer.concordia.ca/teaching

45. 6/3/2015Bill Reimer45 Assignment