EMR 6500: Survey Research Dr. Chris L. S. Coryn Kristin A. Hobson Spring 2013

Agenda Stratified random sampling for means and totals Review

Stratified Random Sampling

A stratified random sample is one in which some form of random sampling is applied in each of a set of separate groups formed from all entries on a sampling frame from which a sample is to be drawn

Strata In stratified random sampling, strata are nonoverlapping groups separating population elements By strategically forming these groups, stratification becomes a feature of the sample design that can improve the statistical quality of survey estimates

Notation for Stratified Random Sampling

Allocation to Strata Deciding how a stratified sample will be distributed among all strata is called stratum allocation The most appropriate allocation method depends on how the stratification will be used

Equal Allocation If the main purpose of stratification is to control subgroup sample sizes for important population subgroups, stratum sample sizes should be sufficient to meet precision requirements for subgroup analysis An important part of the analysis is to produce comparisons among all subgroup strata In this instance, equal allocation (i.e., equal sample sizes) would be appropriate

Proportionate Allocation Proportionate allocation is a prudent choice when the main focus of the analysis is characteristics of several subgroups or the population as a whole and where the appropriate allocations for these analyses are discrepant Proportionate allocation involves applying the same sampling rate to all strata, thus implying that the percent distribution of the selected sample among strata is identical to the corresponding distribution for the population

Optimum Allocation Optimum allocation, in which the most cost-efficient stratum sample sizes are sought, can lead to estimates of overall population characteristics that are statistically superior to those from proportionate allocations When all stratum unit costs are the same, the stratum sampling rates that yield the most precise sample estimates are proportional to the stratum-specific standard deviations (Neyman allocation)

Estimation of a Population Mean and Total

Estimate of Population Mean

Example for a Population Mean NnMSD Town A Town B Rural

Example for a Population Mean

Estimate of Population Total

Example for Population Total

Selecting the Sample Size for Estimating Population Means and Totals

Sample Size for Estimating Population Means and Totals

Example for a Population Mean

Neyman Allocation

Proportionate Allocation

Comparison of Allocation Methods Proportionate Neyman General framework

Review

The Tailored Design Method

Uses multiple motivational features in compatible and mutually supportive ways to encourage high quantity and quality of responses

The Tailored Design Method Premised on social exchange perspective on human behavior Assumes that the likelihood of responding is greater when the expected rewards outweigh the anticipated costs

The Tailored Design Method Gives attention to all aspects of contacting and communicating with respondents Encourages response by considering survey sponsorship, the nature of the population and variations within it, and content of questions

The Tailored Design Method Emphasizes reducing errors of coverage, sampling, nonresponse, and measurement

Coverage Error Occurs when all members of a population do not have a known, non-zero probability of selection Occurs when those who are excluded are different from those who are included

Sampling Error Results from surveying only some rather than all members of a population Represented by B, the bound on the error of estimation

Nonresponse Error Occurs when people selected do not respond and are different than those who do Nonresponse can occur at the level of items within a survey or at the level of the survey – MAR – MCAR

Measurement Error Occurs when responses are inaccurate or imprecise Primarily related to poor layout and poor design and wording of questions

Social Exchange and Surveys Addresses three central questions about design and implementation 1.How can the perceived rewards for responding be increased? 2.How can the perceived costs of responding be reduced? 3.How can trust be established so that people believe the rewards will outweigh the costs of responding?

Increasing Benefits Provide information about the survey Ask for help or advise Show positive regard Say thank you Support group values Give tangible rewards Make the questionnaire interesting Provide social validation Inform people that opportunities to respond are limited

Decreasing Costs Make it convenient to respond Avoid subordinating language Make the questionnaire short and easy to complete Minimize requests for personal or sensitive information Emphasize similarity to other requests or tasks to which a person has already responded

Establishing Trust Obtain sponsorship by legitimate authority Provide a token of appreciation in advance Make the task appear important Ensure confidentiality and security of information

Features that can be Tailored Survey mode – Singular or multiple Sample design – Type of sample – Number of units sampled Incentives – Type of incentive – Amount or cost of incentive – Before or after

Features that can be Tailored Contacts – Number of contacts – Timing of initial and subsequent contacts – Mode of each contact – Whether contacts will be personalized – Sponsorship information – Visual design of each contact – Text or words in each contact

Features that can be Tailored Additional materials – Whether to provide them at all – Type of materials (e.g., research report) – Visual design of materials – Text or wording of materials

Features that can be Tailored Questionnaire – Topics included – Length (duration, number of pages/screens, number of questions) – First page or screen – Visual design – Organization and order of questions – Navigation through questionnaire

Features that can be Tailored Individual questions – Topic (sensitive, of interest to the respondent) – Type (open-ended versus closed-ended) – Organization of information – Text or wording – Visual design

Coverage and Sampling

Central Terminology An element is an object on which a measurement is taken A population is a collection of elements to which an inference is made from a sample A sample is a collection of sampling units drawn from a frame or frames Sampling units are nonoverlapping collections of elements from the population that cover the entire population A frame is a list of sampling units

Central Terminology A completed sample is the units that respond Sampling error is the result of collecting data from only a subset, rather than all, units from a frame – Again, represented by B, the bound on the error of estimation

Coverage The degree to which the units in a sampling frame correspond to the population of interest Coverage is likely one of the most serious problems in most surveys

Coverage and Frame Problems

Reducing Coverage Error Central questions: – Does the list contain everyone in the survey population? – Does the list include people who are not in the study population? – How is the list maintained and updated? – Are the same sample units included on the list more than once? – Does the list contain other information that can be used to improve the survey?

Estimate of Population Mean

Estimate of Population Total

Selecting the Sample Size for Estimating Population Means and Totals

Sample Size for Estimating Population Means where

Often, the population variance,, is unknown An approximate value of can be obtained by Sample Size for Estimating Population Means

Sample Size for Estimating Population Totals where

Estimation of a Population Proportion

Estimate of Population Proportion where

Selecting the Sample Size for Estimating a Population Proportion

Sample Size for Estimating Population Proportions where and

An Overview of Crafting Good Questions

Issues to Consider 1.What survey mode(s) will be used to ask the questions? 2.Is the question being repeated from another survey, and/or will answers be compared to previously collected data? 3.Will respondents be willing and motivated to answer accurately? 4.What type of information is the question asking for?

Choosing Words and Forming Question 1.Make sure the question applies to the respondent 2.Make sure the question is technically accurate 3.Ask one question at a time 4.Use simple and familiar words 5.Use specific and concrete words to specify the concepts clearly 6.Use as few words as possible to pose the question 7.Use complete sentences with simple sentence structures 8.Make sure “yes” means yes and “no” means no 9.Be sure the question specifies the response task

Visual Presentation of Survey Questions 1.Use darker and/or larger print for the question and lighter and/or smaller print for answer choices and answer spaces 2.Use spacing to create subgrouping within a question 3.Visually standardize all answer spaces or response options 4.Use visual design properties to emphasize elements that are important to the respondent and to deemphasize those that are not 5.Make sure words and visual elements that make up the question send consistent messages 6.Integrate special instructions into the question where they will be used rather than including them as freestanding entities 7.Separate optional or occasionally needed instructions from the question stem by font or symbol variation 8.Organize each question in a way that minimizes the need to reread portions in order to comprehend the response task 9.Choose line spacing, font, and text size to ensure the legibility of the text