1. Determining How to Select a Sample

2. Ch 122 Basic Concepts in Sampling Population: the entire group under study as defined by research objectives –Researchers define populations in specific terms such as “heads of households located in areas served by the company who are responsible for making the pest control decision.”

3. Ch 123 Basic Concepts in Sampling Sample: a subset of the population that should represent the entire group Sample unit: the basic level of investigation Census: an accounting of the complete population

4. Ch 124 Basic Concepts in Sampling Sampling error: any error in a survey that occurs because a sample is used A sample frame: a master list of the entire population Sample frame error: the degree to which the sample frame fails to account for all of the population…a telephone book listing does not contain unlisted numbers

5. Ch 125 Reasons for Taking a Sample Practical considerations such as cost and population size Inability of researcher to analyze huge amounts of data generated by census Samples can produce precise results

6. Ch 126 Two Basic Sampling Methods Probability samples: ones in which members of the population have a known chance (probability) of being selected into the sample Non-probability samples: instances in which the chances (probability) of selecting members from the population into the sample are unknown

7. Ch 127 Probability Sampling Methods Simple random sampling Systematic sampling Cluster sampling Stratified sampling

8. Ch 128 Probability Sampling Methods

9. Ch 129 Probability Sampling: Simple Random Sampling Simple random sampling: the probability of being selected into the sample is “known” and equal for all members of the population –E.g., Blind Draw Method –Random Numbers Method (see MRI 12.1, p. 335)

10. Ch 1210 Probability Sampling: Simple Random Sampling –Advantage: Known and equal chance of selection –Disadvantages: Complete accounting of population needed Cumbersome to provide unique designations to every population member

11. Ch 1211 Probability Sampling Systematic Sampling Systematic sampling: way to select a random sample from a directory or list that is much more efficient than simple random sampling –Skip interval=population list size/sample size

12. Ch 1212 Probability Sampling Systematic Sampling –Advantages: Approximate known and equal chance of selection…it is a probability sample plan Efficiency…do not need to designate every population member Less expensive…faster than SRS –Disadvantage: Small loss in sampling precision

13. Ch 1213 Probability Sampling Cluster Sampling Cluster sampling: method in which the population is divided into groups, any of which can be considered a representative sample –Area sampling

14. Ch 1214 Probability Sampling Cluster Sampling –Advantage: Economic efficiency…faster and less expensive than SRS –Disadvantage: Cluster specification error…the more homogeneous the clusters, the more precise the sample results

15. Ch 1215 Cluster Sampling In cluster sampling the population is divided into subgroups, called “clusters.” Each cluster should represent the population. Area sampling is a form of cluster sampling – the geographic area is divided into clusters.

16. Ch 1216 Cluster Sampling One cluster may be selected to represent the entire area with the one-step area sample. Several clusters may be selected using the two-step area sample.

17. Ch 1217 A Two-Step Cluster Sample A two-step cluster sample (sampling several clusters) is preferable to a one- step (selecting only one cluster) sample unless the clusters are homogeneous.

18. Ch 1218 Stratified Sampling When the researcher knows the answers to the research question are likely to vary by subgroups…

19. Ch 1219 Stratified Sampling –Research Question: “To what extent do you value your college degree?” Answers are on a five point scale: 1= “Not valued at all” and 5= “Very highly valued” We would expect the answers to vary depending on classification. Freshmen are likely to value less than seniors. We would expect the mean scores to be higher as classification goes up.

20. Ch 1220 Stratified Sampling –Research Question: “To what extent do you value your college degree?” We would also expect there to be more agreement (less variance) as classification goes up. That is, seniors should pretty much agree that there is value. Freshmen will have less agreement.

21. Ch 1221 Stratified Sampling

22. Ch 1222 Probability Sampling Stratified Sampling Stratified sampling: method in which the population is separated into different strata and a sample is taken from each stratum –Proportionate stratified sample –Disproportionate stratified sample

23. Ch 1223 Probability Sampling Stratified Sampling –Advantage: More accurate overall sample of skewed population…see next slide for WHY –Disadvantage: More complex sampling plan requiring different sample size for each stratum

24. Ch 1224 Stratified Sampling Why is stratified sampling more accurate when there are skewed populations? –The less variance in a group, the less sample size it takes to produce a precise answer. –Why? If 99% of the population (low variance) agreed on the choice of Brand A, it would be easy to make a precise estimate that the population preferred Brand A even with a small sample size.

25. Ch 1225 Stratified Sampling –But, if 33% chose Brand A, and 23% chose B, and so on (high variance) it would be difficult to make a precise estimate of the population’s preferred brand…it would take a larger sample size…

26. Ch 1226 Stratified Sampling –Stratified sampling allows the researcher to allocate more sample size to strata with less variance and less sample size to strata with less variance. Thus, for the same sample size, more precision is achieved. –This is normally accomplished by disproportionate sampling. Seniors would be sampled LESS than their proportionate share of the population and freshmen would be sampled more.

27. Ch 1227 Stratified Sampling Note that we would expect this question to be answered differently depending on student classification. Not only are the means different, variance is less as classification goes up…Seniors tend to agree more than Freshmen!

28. Ch 1228 Nonprobability Sampling With nonprobability sampling methods selection is not based on fairness, equity, or equal chance. –Convenience sampling –Judgment sampling –Referral sampling –Quota sampling

29. Ch 1229 Nonprobability Sampling

30. Ch 1230 Nonprobability Sampling May not be representative but they are still used very often. Why? –Decision makers want fast, relatively inexpensive answers… nonprobability samples are faster and less costly than probability samples.

31. Ch 1231 Nonprobability Sampling May not be representative but they are still used very often. Why? –Decision makers can make a decision based upon what 100 or 200 or 300 people say…they don’t feel they need a probability sample.

32. Ch 1232 Nonprobability Sampling Convenience samples: samples drawn at the convenience of the interviewer –Error occurs in the form of members of the population who are infrequent or nonusers of that location

33. Ch 1233 Nonprobability Sampling Judgment samples: samples that require a judgment or an “educated guess” as to who should represent the population –Subjectivity enters in here, and certain members will have a smaller chance of selection than others

34. Ch 1234 Nonprobability Sampling Referral samples (snowball samples): samples which require respondents to provide the names of additional respondents –Members of the population who are less known, disliked, or whose opinions conflict with the respondent have a low probability of being selected

35. Ch 1235 Nonprobability Sampling Quota samples: samples that use a specific quota of certain types of individuals to be interviewed –Often used to ensure that convenience samples will have desired proportion of different respondent classes

36. Ch 1236 Online Sampling Techniques Random online intercept sampling: relies on a random selection of Web site visitors Invitation online sampling: is when potential respondents are alerted that they may fill out a questionnaire that is hosted at a specific Web site

37. Ch 1237 Online Sampling Techniques Online panel sampling: refers to consumer or other respondent panels that are set up by marketing research companies for the explicit purpose of conducting surveys with representative samples Other online sampling approaches

38. Ch 1238 Developing a Sample Plan Sample plan: definite sequence of steps that the researcher goes through in order to draw and ultimately arrive at the final sample

39. Ch 1239 Developing a Sample Plan 1.Define the relevant population. 2.Obtain a listing of the population. –The incidence rate is the percentage of people on a list who qualify as members of the population 3.Design the sample plan (size and method).

40. Ch 1240 Developing a Sample Plan 4.Draw the sample. –Substitution methods: Drop-down substitution Oversampling Resampling

41. Ch 1241 Developing a Sample Plan 5.Validate the sample. –Sample validation is a process in which the researcher inspects some characteristic(s) of the sample to judge how well it represents the population. 6.Resample, if necessary.

42. Ch 12 Determining the Size of a Sample

43. Ch 1243 Sample Accuracy Sample accuracy: refers to how close a random sample’s statistic is to the true population’s value it represents Important points: –Sample size is not related to representativeness –Sample size is related to accuracy

44. Ch 1244 Sample Size and Accuracy Intuition: Which is more accurate: a large probability sample or a small probability sample? The larger a probability sample is, the more accurate it is (less sample error).

45. Ch 1245 A Picture Says 1,000 Words ± Probability sample accuracy (error) can be calculated with a simple formula, and expressed as a ± % number. n 550 - 2000 = 1,450 4% - 2% = ±2%

46. Ch 1246 How to Interpret Sample Accuracy From a report… –The sample is accurate ± 7% at the 95% level of confidence… From a news article –The accuracy of this survey is ± 7%…

47. Ch 1247 How to Interpret Sample Accuracy Interpretation –Finding: 60% are aware of our brand –So… between 53% (60%-7%) and 67% (60%+7%) of the entire population is aware of our brand

48. Ch 1248 Sample Size Axioms To properly understand how to determine sample size, it helps to understand the following axioms…

49. Ch 1249 Sample Size Axioms The only perfectly accurate sample is a census. A probability sample will always have some inaccuracy (sample error). The larger a probability sample is, the more accurate it is (less sample error). Probability sample accuracy (error) can be calculated with a simple formula, and expressed as a +- % number.

50. Ch 1250 Sample Size Axioms You can take any finding in the survey, replicate the survey with the same probability sample size, and you will be “very likely” to find the same finding within the +- range of the original finding. In almost all cases, the accuracy (sample error) of a probability sample is independent of the size of the population.

51. Ch 1251 Sample Size Axioms A probability sample can be a very tiny percentage of the population size and still be very accurate (have little sample error).

52. Ch 1252 Population Size e=±3% Sample Sizee=±4% Sample Size 10,000____ 100,000____ 1,000,000____ 100,000,000____ Sample Size and Population Size Where is N (size of the population) in the sample size determination formula? 1,067600 1,067600 1,067600 1,067600 In almost all cases, the accuracy (sample error) of a probability sample is independent of the size of the population.

53. Ch 1253 Sample Size and Population Size Does the size of the population, N, affect sample size or sample error? A probability sample size can be a very tiny percentage of the population size and still be very accurate (have little sample error). Population Sizee=±3% Sample SizeSample as % of the Population 10,000_______% 100,000_______% 1,000,000_______% 100,000,000_______% 1,0671% 1,067.1% 1,067.01% 1,067.0001%

54. Ch 1254 Sample Size Axiom The size of the probability sample depends on the client’s desired accuracy (acceptable sample error) balanced against the cost of data collection for that sample size.

55. Ch 1255 Putting It All Together MR – What level accuracy do you want? MM – I don’t have a clue. MR – National opinion polls use 3.5%. MM – Sounds good to me. MR – Okay, that means we need a sample of 1,200. MM – Gee Whiz. That small? MR – Yup, and at a cost of $20 per completion, it will be $24,000. MM – Holy Cow! That much? MR – I could do 500 for $10,000, and that would be 4.4% accurate, or 300 for $6,000 at 5.7%. MM – 500 sounds good to me. The size of a probability sample depends on the client’s desired accuracy (acceptable sample error) balanced against the cost of data collection for that sample size.

56. Ch 1256 There is only one method of determining sample size that allows the researcher to PREDETERMINE the accuracy of the sample results… The Confidence Interval Method of Determining Sample Size

57. Ch 1257 The Confidence Interval Method of Determining Sample Size This method is based upon the Confidence Interval and the Central Limit Theorem… Confidence interval: range whose endpoints define a certain percentage of the response to a question

58. Ch 1258 The Confidence Interval Method of Determining Sample Size Confidence interval approach: applies the concepts of accuracy, variability, and confidence interval to create a “correct” sample size Two types of error: –Nonsampling error: pertains to all sources of error other than sample selection method and sample size –Sampling error: involves sample selection and sample size

59. Ch 1259 The Confidence Interval Method of Determining Sample Size Sample error formula:

60. Ch 1260 The Confidence Interval Method of Determining Sample Size The relationship between sample size and sample error:

61. Ch 1261 Computations Help Page Let’s try 3 n’s 1000 500 100 1.96 50 times 50 Answers this way…

62. Ch 1262 And the answers are… Let’s try 3 n’s 1000±3.1% 500±4.4% 100±9.8% 1.96 50 times 50

63. Ch 1263 Review: What does sample accuracy mean? 95% Accuracy –Calculate your sample’s finding, p% –Calculate your sample’s accuracy, ± e% –You will be 95% confident that the population percentage (π) lies between p% ± e%

64. Ch 1264 Review: What does sample accuracy mean? Example –Sample size of 1,000 –Finding: 40% of respondents like our brand –Sample accuracy is ± 3% (via our formula) –So 37% - 43% like our brand

65. Ch 1265 The Confidence Interval Method of Determining Sample Size Variability: refers to how similar or dissimilar responses are to a given question P: percent Q: 100%-P Important point: the more variability in the population being studied, the higher the sample size needed to achieve a stated level of accuracy.

66. Ch 1266 With nominal data (i.e. yes, no), we can conceptualize variability with bar charts…the highest variability is 50/50

67. Ch 1267 Confidence Interval Approach The confidence interval approach is based upon the normal curve distribution. We can use the normal distribution because of the CENTRAL LIMITS THEOREM…regardless of the shape of the population’s distribution, the distribution of samples (of n at least =30) drawn from that population will form a normal distribution.

68. Ch 1268 Central Limits Theorem The central limits theorem allows us to use the logic of the normal curve distribution. Since 95% of samples drawn from a population will fall + or – 1.96 x sample error (this logic is based upon our understanding of the normal curve) we can make the following statement…

69. Ch 1269 If we conducted our study over and over, 1,000 times, we would expect our result to fall within a known range. Based upon this, we say that we are 95% confident that the true population range value falls within this range.

70. Ch 1270 The Confidence Interval Method of Determining Sample Size 1.96 x s.d. defines the endpoints of the distribution.

71. Ch 1271 We also know that, given the amount of variability in the population, the sample size will affect the size of the confidence interval.

72. Ch 1272 So, what have we learned thus far? There is a relationship between: –The level of confidence we wish to have that our results would be repeated within some known range if we were to conduct the study again, and… –Variability in the population and… –The amount of acceptable sample error (desired accuracy) we wish to have and… –The size of the sample!

73. Ch 1273 Sample Size Formula Fortunately, statisticians have given us a formula which is based upon these relationships. –The formula requires that we Specify the amount of confidence we wish Estimate the variance in the population Specify the amount of desired accuracy we want. –When we specify the above, the formula tells us what sample we need to use…n

74. Ch 1274 Sample Size Formula Standard sample size formula for estimating a percentage:

75. Ch 1275 Practical Considerations in Sample Size Determination How to estimate variability (p times q) in the population –Expect the worst cast (p=50; q=50) –Estimate variability: Previous studies? Conduct a pilot study?

76. Ch 1276 Practical Considerations in Sample Size Determination How to determine the amount of desired sample error –Researchers should work with managers to make this decision. How much error is the manager willing to tolerate? –Convention is + or – 5% –The more important the decision, the more (smaller number) the sample error.

77. Ch 1277 Practical Considerations in Sample Size Determination How to decide on the level of confidence desired –Researchers should work with managers to make this decision. The more confidence, the larger the sample size. –Convention is 95% (z=1.96) –The more important the decision, the more likely the manager will want more confidence. 99% confidence, z=2.58.

78. Ch 1278 Example Estimating a Percentage in the Population What is the required sample size? –Five years ago a survey showed that 42% of consumers were aware of the company’s brand (Consumers were either “aware” or “not aware”) –After an intense ad campaign, management wants to conduct another survey and they want to be 95% confident that the survey estimate will be within ±5% of the true percentage of “aware” consumers in the population. –What is n?

79. Ch 1279 Estimating a Percentage: What is n? Z=1.96 (95% confidence) p=42 q=100-p=58 e=5 What is n?

80. Ch 1280 Estimating a Percentage: What is n? What does this mean? –It means that if we use a sample size of 374, after the survey, we can say the following of the results: (assume results show that 55% are aware) –“Our most likely estimate of the percentage of consumers that are ‘aware’ of our brand name is 55%. In addition, we are 95% confident that the true percentage of ‘aware’ customers in the population falls between 50% and 60%.” N=374

81. Ch 1281 Estimating a Mean Estimating a mean requires a different formula (See MRI 13.2, p. 378) Z is determined the same way (1.96 or 2.58) E is expressed in terms of the units we are estimating (i.e., if we are measuring attitudes on a 1-7 scale, we may want error to be no more than ±.5 scale units S is a little more difficult to estimate…

82. Ch 1282 Estimating s Since we are estimating a mean, we can assume that our data are either interval or ratio. When we have interval or ratio data, the standard deviation, s, may be used as a measure of variance.

83. Ch 1283 Estimating s How to estimate s? –Use standard deviation from a previous study on the target population. –Conduct a pilot study of a few members of the target population and calculate s. –Estimate the range the value you are estimating can take on (minimum and maximum value) and divide the range by 6.

84. Ch 1284 Estimating s –Why divide the range by 6? The range covers the entire distribution and ± 3 (or 6) standard deviations cover 99.9% of the area under the normal curve. Since we are estimating one standard deviation, we divide the range by 6.

85. Ch 1285 Example Estimating the Mean of a Population What is the required sample size? –Management wants to know customers’ level of satisfaction with their service. They propose conducting a survey and asking for satisfaction on a scale from 1 to 10. (since there are 10 possible answers, the range=10). –Management wants to be 99% confident in the results and they do not want the allowed error to be more than ±.5 scale points. –What is n?

86. Ch 1286 Estimating a Mean: What is n? S=10/6 or 1.7 Z=2.58 (99% confidence) e=.5 scale points What is n?

87. Ch 1287 Estimating a Percentage: What is n? What does this mean? –After the survey, management may make the following statement: (assume satisfaction mean is 7.3) –“Our most likely estimate of the level of consumer satisfaction is 7.3 on a 10-point scale. In addition, we are 99% confident that the true level of satisfaction in our consumer population falls between 6.8 and 7.8 on a 10-point scale” N=77

88. Ch 1288 Other Methods of Sample Size Determination Arbitrary “percentage of thumb” sample size: –Arbitrary sample size approaches rely on erroneous rules of thumb. –Arbitrary sample sizes are simple and easy to apply, but they are neither efficient nor economical.

89. Ch 1289 Other Methods of Sample Size Determination Conventional sample size specification: –Conventional approach follows some convention: or number believed somehow to be the right sample size. –Using conventional sample size can result in a sample that may be too large or too small. –Conventional sample sizes ignore the special circumstances of the survey at hand.

90. Ch 1290 Other Methods of Sample Size Determination Statistical analysis requirements of sample size specification: –Sometimes the researcher’s desire to use particular statistical technique influences sample size

91. Ch 1291 Other Methods of Sample Size Determination Cost basis of sample size specification: –“All you can afford” method –Instead of the value of the information to be gained from the survey being primary consideration in the sample size, the sample size is determined by budget factors that usually ignore the value of the survey’s results to management.

92. Ch 1292 Special Sample Size Determination Situations Sampling from small populations: –Small population: sample exceeds 5% of total population size –Finite multiplier: adjustment factor for sample size formula –Appropriate use of the finite multiplier formula will reduce a calculated sample size and save money when performing research on small populations.

93. Ch 1293 Special Sample Size Determination Situations Sample size using nonprobability sampling: –When using nonprobability sampling, sample size is unrelated to accuracy, so cost-benefit considerations must be used.

94. Ch 1294 Practice Examples We will do some examples from the questions and exercises at the end of the chapter on sample size…question 5 on page 386.

95. Ch 1295 Practice Examples 5a. Using the formula provided in your text, determine the approximate sample sizes for each of the following cases, all with precision (allowable error) of ±5%: –Variability of 30%, confidence level of 95%

96. Ch 1296 Practice Examples 5b. Using the formula provided in your text, determine the approximate sample sizes for each of the following cases, all with precision (allowable error) of ±5%: –Variability of 60%, confidence level of 99%

97. Ch 1297 Practice Examples 5c. Using the formula provided in your text, determine the approximate sample sizes for each of the following cases, all with precision (allowable error) of ±5%: –Unknown variability, confidence level of 95%

98. Ch 12 Data Collection in the Field, Nonresponse Error, and Questionnaire Screening

99. Ch 1299 Nonsampling Error in Marketing Research Nonsampling error includes –All types of nonresponse error –Data gathering errors –Data handling errors –Data analysis errors –Interpretation errors

100. Ch 12100 Possible Errors in Field Data Collection Field worker error: errors committed by the persons who administer the questionnaires Respondent error: errors committed on the part of the respondent Errors may be either intentional or unintentional.

101. Ch 12101 Data Collection Errors

102. Ch 12102 Possible Errors in Field Data Collection: Field-Worker Errors Intentional field worker error: errors committed when a data collection person willfully violates the data collection requirements set forth by the researcher –Interviewer cheating occurs when the interviewer intentionally misrepresents respondents –Leading respondents occurs when the interviewer influences respondent’s answers through wording, voice inflection, or body language

103. Ch 12103 Possible Errors in Field Data Collection: Field-Worker Errors Unintentional field worker error: errors committed when an interviewer believes he or she is performing correctly –Interviewer personal characteristics occurs because of the interviewer’s personal characteristics such as accent, sex, and demeanor

104. Ch 12104 Possible Errors in Field Data Collection: Field-Worker Errors –Interviewer misunderstanding occurs when the interviewer believes he or she knows how to administer a survey but instead does it incorrectly –Fatigue-related mistakes occur when interviewer becomes tired

105. Ch 12105 Possible Errors in Field Data Collection: Respondent Errors Intentional respondent error: errors committed when there are respondents that willfully misrepresent themselves in surveys –Falsehoods occur when respondents fail to tell the truth in surveys –Nonresponse occurs when the prospective respondent fails to take part in a survey or to answer specific questions on the survey

106. Ch 12106 Possible Errors in Field Data Collection: Respondent Errors Unintentional respondent error: errors committed when a respondent gives a response that is not valid but that he or she believes is the truth –Respondent misunderstanding occurs when a respondent gives an answer without comprehending the question and/or the accompanying instructions –Guessing occurs when a respondent gives an answer when he or she is uncertain of its accuracy

107. Ch 12107 Possible Errors in Field Data Collection: Respondent Errors –Attention loss occurs when a respondent’s interest in the survey wanes –Distractions (such as interruptions) may occur while questionnaire administration takes place –Fatigue occurs when a respondent becomes tired of participating in a survey

108. Ch 12108 How to Control Data Collection Errors: Fieldworkers

109. Ch 12109 How to Control Data Collection Errors: Respondents

110. Ch 12110 How to Control Data Collection Errors: Respondents

111. Ch 12111 Data Collection Errors with Online Surveys Multiple submissions by the same respondent Bogus respondents and/or responses Misrepresentation of the population

112. Ch 12112 Nonresponse Error Nonresponse: failure on the part of a prospective respondent to take part in a survey or to answer specific questions on the survey –Refusals to participate in survey –Break-offs during the interview –Refusals to answer certain questions (item omissions) Completed interview must be defined

113. Ch 12113 Nonresponse Error Response rate enumerates the percentage of the total sample with which the interviews were completed

114. Ch 12114 Nonresponse Error CASRO response rate formula:

115. Ch 12115 Reducing Nonresponse Error Mail surveys: –Advance notification –Monetary incentives –Follow-up mailings Telephone surveys: –Callback attempts

116. Ch 12116 Preliminary Questionnaire Screening Unsystematic and systematic checks of completed questionnaires What to look for in questionnaire inspection Incomplete questionnaires Nonresponses to specific questions (item omissions) Yea- or nay-saying patterns Middle-of-the-road patterns

117. Ch 12117 Unreliable Responses Unreliable responses are found when conducting questionnaire screening, and an inconsistent or unreliable respondent may need to be eliminated from the sample.

118. Ch 12118 Minimizing Non-Sampling Error Cannot eliminate and cannot measure (except for non-response error) Implement CONTROLS to minimize error: –Close supervision of data collectors –Training

119. Ch 12119 Minimizing Non-Sampling Error –Care in constructing questionnaire and instructions…pretest! –Provide incentives to respondents –VALIDATION…industry standard is 10%