1. Complex Surveys
Sunday, April 16, 2017

2. Assembling Design Components
Building blocks
Probability sampling
Simple random sampling (SRS)
Unequal probability sampling
Stratification
Purpose: to increase the precision of estimates by grouping similar items together
Cluster sampling
Purpose: convenience.
Ratio estimation

3. Simple Random Sampling (without replacement)
There are (N choose n) possible samples
Each with probability 1/(N choose n)
Point estimate and C.I.
Sample size (n) calculation

4. Stratification (Ch 3) The estimate of the population total
It’s variance
Sample size allocation - proportional, optimal
In general, stratified sampling with proportional allocation is more efficient than SRS
The more unequal the stratum means, the more benefits

5. Ratio estimation (ch4) Biased May results in smaller MSE
Useful when variables are linearly correlated
Regression estimation

6. Cluster Sampling (ch 5&6)
Usually less efficient than other methods
The relative efficiency of it and SRS depends on intra-class correlation coefficient
The larger the correlation coefficient, the less efficient
Can reduce cost and lead to administrative convenience
One-stage, two-stage, with equal or unequal probs, point estimate, variance, c.i.
Allocation of m and n for two-stage cluster sampling

7. Cluster Sampling without Replacement
Select a sample of n clusters with replacement based on
Estimate cluster total and variance
Estimate population total
Variance can be estimated by formulas in ch5,6 or resampling methods

8. Cluster Sampling with Replacement
Select a sample of n clusters with replacement based on
Estimate cluster total
Calculate
Estimate population total and variance

9. In practice Most of large surveys involves several ideas of techniques
Different types of estimators

10. An example: background
Malaria is a common public health problem in tropical and subtropical regions
It is infectious. People get it by being bitten by a kind of female mosquito
Without timely and proper treatment, the death rate can be very high
Can be prevented by using mosquito nets
The prevention is only affective if the nets are in widespread use

11. Summary Goal: To estimate the prevalence of bed net use in rural areas
Sampling frame: all rural villages of <3,000 people in The Gambia

12. The survey in Gambia (1991) 3000 rural villages Stage Sampling unit
Sampling method
eastern
central
western
Stratified by region
Prob district size
5 districts per region 1
district
PHC
Non-PHC
Stratified by PHC
Prob village size
4 villages per district 2
village
SRS
6 compounds / village 3
compound
Top-down

13. The survey in Gambia (1991) 3000 rural villages eastern central
western
Stratified sampling
Sampling with unequal probs,
two-stage cluster, Ch 6
PHC
Non-PHC
district
Stratified sampling
Sampling with unequal probs,
two-stage cluster, Ch 6
village
compound
SRS
(average number of nets per compound)
Top-down
Bottom-up

14. The survey in Gambia (1991)
The way to calculate the estimated total and its variance seems to be complicated
It can be worse if we include ratio estimators
In practice, we can
Use sampling weights to obtain point estimates
Use computer intensive methods to obtain standard error (ch9)
Such as jackknife, bootstrap

15. Sampling weights
The sampling weight is the reciprocal of Pr(being selected)
Each sampled unit “represents” certain number of units in the population
The whole sample “represents” the whole population

16. Sampling weights
Weights are used to deal with the effects of stratification and clustering on point estimate
Stratified sampling

17. Sampling weights
Cluster sampling with equal probabilities

18. Sampling weights For three-stage sampling
Very large weights are often truncated
Biases results
Reduces the mean squared error
p: primary
s: secondary
t: tertiary

19. Sampling weights
Weights contain the information needed to construct point estimates
Weights do not contain enough information for computing variance
Weights can be used to find point estimates because calculating variance requires prob(pairs of units are selected)
Computer-intensive methods can be used to find variances

20. Sampling weights: the malaria example
Pr(a compound in central region PHC villages is selected)=

21. Self-weighting and Non-self-weighting
Self-weighting: sampling weights for all observation units are equal
A self-weighing sampling is representative of the population if nonsampling errors are ignored
Most large self-weighting samples are not SRS
Standard software with the usual assumption of iid leads to correct estimate of mean, proportion, percentiles; but erroneous estimation for variance

22. Ratio Estimation in Complex Surveys
Ratio estimation is part of the analysis, not the design
Can be used at any level. Usually used near the top

23. Ratio Estimation in the Malaria example
Region level:
Above the region level

24. Ratio Estimation in Complex Surveys
The bias of ratio estimation can be large when sample sizes are small
Separate ratio estimator for a population total
Improves efficiency when ratios vary from stratum to stratum; works poorly for small strata sample sizes
Combined ratio estimator for a population total
Has less bias when strata sizes are small; works poorly when ratios vary from stratum to stratum

25. Estimating a Distribution Function
Historically, sampling theory was developed to find population means, totals, and ratios.
Other quantities, such ass,
Pr(Statistics > means or totals)
Median? 95th percentile? Probability mass function?
Sampling weights can be used in constructing an empirical distribution of the population

26. Population quantities and functions
Probability mass function (pmf)
Distribution function

27. Empirical Functions Empirical probability mass function
Empirical distribution function
Empirical functions can be used to estimate population quantities such as mean, median, percentiles, variance, ect.

28. Plotting data from a complex survey
SRS
Histograms/smoothed density estimates
Scatterplots and scatterplot matrices
In a complex sampling design, simple plots can be missleading

29. Incorporating weights

30. Incorporating weights

31. Plotting data from a complex survey
The 1987 Survey of Youth in Custody
Family background, previous criminal history, drug and alcohol use
206 PSU’s (facilities) were divided by 16 strata
SSUs were the 1985 Children in Custody (CIC)

32. The 1987 Survey of Youth in Custody
The two figures are very similar because the survey was aimed to be self-weighting
Youths aged 15 were undersampled due to unequal selection prob and nonresponse
Youths aged 17 were oversampled

33. The 1987 Survey of Youth in Custody

34. The 1987 Survey of Youth in Custody

35. The 1987 Survey of Youth in Custody

36. The 1987 Survey of Youth in Custody

37. Design effects Cornfield’s ratio (1951)
Measure the efficiency of a sampling plan by the ratio of the variance that would be obtained from an SRS of k observation units to the variance obtained from the complex sampling plan with k observation units
The design effect (deff, Kish 1965)
The reciprocal of Cornfield’s ratio

38. The design effects
The design effect provides a measure of the precision gained/lost by use of the more complex design instead of SRS
For estimating a mean

39. The design effects
Stratified
Cluster