Presentation is loading. Please wait.

Presentation is loading. Please wait.

Multiple Indicator Cluster Surveys Survey Design Workshop

Similar presentations

Presentation on theme: "Multiple Indicator Cluster Surveys Survey Design Workshop"— Presentation transcript:

1 Multiple Indicator Cluster Surveys Survey Design Workshop
Advanced Sampling MICS4 Survey Design Workshop

2 Major steps in designing MICS sample
Define objectives Key indicators Desired level of precision Subnational domains of estimation Identify most appropriate sampling frame Sample for another survey conducted recently Most recent census of population and housing Determine sample size and allocation Determine availability of previous MICS or DHS results to provide measures of sampling parameters MICS4 Survey Design Workshop

3 MICS4 Survey Design Workshop
Sampling frame Sampling frame should be nationally-representative and have complete coverage, with measures of size (households or population) Most countries conduct Census of Population and Housing every 10 years Generally provides most effective sampling frame for household surveys Sample for another survey conducted recently In case of older frame, geographic areas with substantial changes, such as peri-urban in larges cities, may need to be updated When no census is available Identify most complete geographic frame available Example – Southern Sudan – list of villages from WHO immunization program with estimated population MICS4 Survey Design Workshop

4 MICS recommendations on sample size determinants
FACTOR RECOMMENDATION 1.Expected size estimate of indicators (next slide) 2.Expected size estimate of target population mos [3%] 3.Average household size 6 persons 4.Relative margin of error wanted 12% of coverage rate 5.Level of confidence wanted 95 percent 6.Design effect in cluster surveys 1.5 7.Expected non-response rate 10 percent 8.Number of clusters or PSUs - minimum [ ] 9.Cluster size [15-35 households] 10.Number of estimation “domains” wanted [5 or fewer] 11.Survey budget (country specific) For items 2, 3, 6, 7 use available country data (recent survey or census); if not available, use value above. MICS4 Survey Design Workshop

5 Indicators for sample size determination
Sample size is different for each MICS indicator. Must choose a key indicator, since only one sample size can be used in MICS Recommendations for choosing key indicator: Choose from among main indicators of interest in your country Choose the one which will yield largest sample size Usually for a single-year age group Usually DPT, measles, polio or tuberculosis immunization - or birth weight below 2.5 kg Exceptions: Do not choose infant or maternal mortality rates as the key indicators. Do not choose a low prevalence indicator that is desirably low (such as malnutrition prevalence). MICS4 Survey Design Workshop

6 MICS4 Survey Design Workshop
Sample size formula where n is the required sample size, expressed as number of households, for the KEY indicator, 4 is factor to achieve 95 percent level of confidence, r is anticipated prevalence rate for key indicator, 1.1 is factor to raise sample size by 10 percent for potential nonresponse, deff is shortened symbol for design effect, 0.12r is margin of error to be tolerated, defined as 12 percent of r (12 percent thus represents the relative margin of error of r), p is proportion of total population that smallest group comprises, and is average household size. MICS4 Survey Design Workshop

7 MICS4 Survey Design Workshop
Example Target group: Children 12 to 23 months old Percent of population: 3.5 percent Key indicator: DPT immunization coverage Prevalence (Coverage): 25 percent Deff: Non-response adjustment: 1.05 (response rate 95%) Average household size: 6 MICS4 Survey Design Workshop

8 MICS4 Survey Design Workshop
Sample size (Households) to estimate coverage rates for smallest target population MICS4 Survey Design Workshop

9 Note on precision requirements
In case of MICS2, precision requirements expressed in terms of acceptable margin of error (ME), which varied according to the size of the estimate (5% absolute error for high rate indicators or 3% for low rate indicators) For MICS3 and MICS4, this was simplified to a relative margin of error (RME) of 0.12 Follow guidelines in sampling chapter carefully; avoid indicators with a high rate Final criterion for acceptable precision: is the confidence interval useful? If confidence interval is too wide, estimate may not be useful MICS4 Survey Design Workshop

10 Stratification and sample allocation
Stratification is the process of dividing the sampling frame into sub-groups (strata) of homogeneous (similar) PSUs Advantages: better precision, flexible design, sub-national estimates for smaller domains (differential sampling rates) Reduced variance within stratum given similarity of units Example of stratification: region, urban/rural Existing sampling frame, such as master sample, may have socioeconomic stratification for large cities Should improve statistical efficiency of sample design Geographic domains defined as strata Possible to use variable sampling rates by domain to ensure sufficient sample size for each MICS4 Survey Design Workshop

11 Implicit stratification
Sort the sampling frame according to certain characters such as regions, urban-rural residence, sub-regions, districts, etc., then select a systematic pps sample. Ensures a representative sample for each subgroup Automatically provides proportional allocation by size of subgroup MICS4 Survey Design Workshop

12 Allocation of sample to strata
Proportional allocation Effective for precision of estimates at the national level Equal allocation to each domain Used when each domain requires same level of precision Optimum allocation – takes into account differential variance and costs by stratum For example, variability may be higher in urban areas and enumeration costs may be higher in rural areas – use higher sampling rate for urban areas MICS4 Survey Design Workshop

13 Number of PSUs and cluster size
Survey costs depend not only on number of households but their distribution among primary sampling units (PSUs) Important to determine effective balance between number of sample PSUs and cluster size In general, the more PSUs the better for reliability but the greater the cost (mostly costs of travel and listing) At national level, minimum of 300 to 400 PSUs should be selected Subnational domains require larger samples Cluster size should be as small as practical for reliability Example: 8000 households selected in 400 PSUs of 20 sample households each is a much more reliable sample than 200 PSUs of 40 households each, but more expensive MICS4 Survey Design Workshop

14 MICS4 Survey Design Workshop
Design effect Deff - ratio of variance of estimate based on stratified multi-stage sample design and corresponding variance from simple random sample of same size Measure of the relative efficiency of the sample design Effective stratification reduces the deff Cluster sampling increases the deff Deft = square root of Deff, expressed as ratio of standard errors Generally presented in tables of standard errors for the DHS MICS4 Survey Design Workshop

15 Design effect (continued)
In case of cluster sampling, deff generally measures effect of clustering δ = intraclass correlation coefficient, or measure of homogeneity within cluster = average cluster size (households per cluster) Design effect increases with intraclass correlation and cluster size MICS4 Survey Design Workshop

16 MICS Sampling Option 1 – use an existing sample
Design MICS as a rider to another survey if timely and feasible Use sample from a previous survey and re-interview households for MICS Or, use old survey sample EAs and construct new listing of households to select for MICS Old sample must be probability-based, national in scope Possibilities – DHS, other national health survey, recent labour force survey Important: design parameters must be known (such as selection probability, stratification, etc.) MICS4 Survey Design Workshop

17 Sampling option 1 - continued
Advantages of using previous sample - cost savings - maps available for interviewers - appropriate sampling plan available - simplicity Limitations of using old sample - burden on respondents - sample design may need modification * sample size * sub-national coverage * number of PSUs or clusters Balance between loss and gain MICS4 Survey Design Workshop

18 MICS Sampling Option 2 – new sample with household listing
Design new MICS sample based on prototype Two stages with census as frame Use of implicit stratification, systematic selection of census EAs at first stage with pps Create standard segments (DHS approach) List households in selected segments Select households systematically from listing Interview selected households, no replacement will be allowed MICS4 Survey Design Workshop

19 Sampling Option 2 - continued
Advantages of option 2 - simple design - probability-based - if possible self-weighting (national level) Limitations of option 2 - expense of listing households - time necessary to list households [Example, sample size of 5000 households may require to households to be listed] MICS4 Survey Design Workshop

20 MICS4 Survey Design Workshop
DHS Method - Option 2 Create “standard” segments Divide census population in each EA by 500 to determine number of standard segments Map sketch segments in each EA Choose 1 segment at random List households in selected segment only (instead of entire EA) Purpose is to reduce listing workload to a manageable size MICS4 Survey Design Workshop

21 MICS Sampling Option 3 – use “compact clusters” with no listing
Modified segment, or cluster, design) Design new MICS sample based on prototype Two stages with census as frame Use of implicit stratification, systematic selection of census EAs at first stage with pps Pre-determine number of segments (measure of size) based on desired cluster size Map sketch segments in each EA Choose 1 segment at random Interview all households in selected segment MICS4 Survey Design Workshop

22 MICS Sampling Option 3 - continued
Illustration: Suppose desired cluster size is 20 households. Suppose first sample EA contains 112 census households (according to frame) Divide 112 by 20 = 5.6 (round to 6) Map sketch exactly 6 segments based on canvass of EA Select one segment at random Interview all households (no matter how many are currently in the selected segment) MICS4 Survey Design Workshop

23 MICS Sampling Option 3 - continued
Advantages of option 3 avoids listing completely probability-based self-weighting (national level) Limitations of option 3 less reliable than option 2 (households are “clustered” together in compact segments) segmentation itself can be time-consuming and complicated difficult to control overall sample size MICS4 Survey Design Workshop

24 Common sampling option used by some countries
Select EAs systematically with PPS, where measure of size is based on number of households (or population) In case of large EAs in sample, subdivide into standard segments, similar to Option 2 Advantage: measures of size more exact, easier to implement a self-weighting design and control sample size MICS4 Survey Design Workshop

25 PPS systematic selection of PSUs
Selection of PSUs with PPS provides a self-weighting sample when a fairly constant number of sample households selected in each PSU at second stage Systematic sampling of PSUs from a geographically ordered list ensures that the sample is geographically representative, with a proportional allocation to the different levels of geography Examine template for PPS systematic sampling MICS4 Survey Design Workshop

26 Listing of households in sample segments
Importance of new listing to represent current population Problems with using previous listing (older than 1 year) Does not represent newer households Distribution of sample population by age group distorted, generally with higher median age Difficulty of finding households in old list MICS4 Survey Design Workshop

27 Listing of households (continued)
Common problems found in listing operations Problem with quality of sketch maps – difficult to determine segment boundaries Sometimes large differences found between number of households in frame (census) and number listed MICS4 Survey Design Workshop

28 Selection of sample households from listing
Selection of households in the office following listing operation Advantages – conducted by specialized staff, possible to avoid selection bias in the field, possible to control overall sample size Disadvantage – increased costs from having two field visits Selection of households in field Advantage – cost savings of having one integrated field operation Disadvantage - correct sampling may be difficult for field staff, selection may be biased Self-weighting samples – cluster sizes somewhat variable Selection of fixed number of sample households per cluster Controls sample size, allowing weights to vary somewhat by EA Use of household selection table in field Easy to use, minimizes selection bias MICS4 Survey Design Workshop

29 Considerations for designing self-weighting samples
Main advantage of self-weighting sample is to simplify the estimation procedures Also effective for national-level estimates Disadvantages of self-weighting samples May not be possible to obtain reliable estimates for smaller subnational groups, given proportional allocation of sample Difficult to control overall sample size Use of SPSS and other software packages that automatically weight survey tables reduce advantages of self-weighting samples Most countries are not using self-weighting samples for MICS Prefer selection of fixed number of households per EA MICS4 Survey Design Workshop

30 Subnational estimates
Number of separate areas (domains) for which separate, equally reliable estimates are wanted affects sample size For example, if 10 regional estimates are wanted, theoretically the sample should be increased by factor of 10 As a compromise, larger sampling errors accepted for subnational estimates One proposal (by Dr. Vijay Verma) – increase national sample size by factor of D0.65, where D is the number of domains Results in an average increase in the sampling errors for domain estimates by a factor of about 1.5 Minimum number of PSUs required for each domain – for example, 30 clusters Allocation of sample to domains Equal allocation Modified proportional allocation, with a minimum and maximum number of sample PSUs per domain MICS4 Survey Design Workshop

31 Survey weighting procedures
All analysis based on survey data must apply survey weights in order to prevent biased results Survey weighting is design-specific Overall probability of selection has component from each sampling stage. Design weight is inverse of final probability of selection Non-response must be taken into account Separate non-response adjustment for households, women age years and children under 5 years MICS4 Survey Design Workshop

32 Survey weighting procedures
Formulas for calculating weights depend on the exact sample design used in each country Design weights important for validating calculation of weights and coverage of frame Weighted total number of households by region, urban and rural strata should be compared to corresponding distribution from census data or projections Normalized weights – each weight is divided by the overall average weight Using normalized weights, the weighted and unweighted total number of sample cases (households, women and children) are equal Review of templates for calculating weights MICS4 Survey Design Workshop

33 Sampling error estimation
Calculation of sampling errors necessary to evaluate reliability of survey estimates Should be done for important indicators Methodology is complex and design-specific There are several software options for sampling error calculations: SPSS – Complex Samples add-on – calculation of standard errors, confidence intervals and design effects Other existing software can be used (Stata, Clusters, WesVar, CENVAR, PCCarp, etc.) Soon variance component will be added to CSPro Review of SPSS sampling error application MICS4 Survey Design Workshop

34 MICS4 Survey Design Workshop
Reducing bias Accuracy of survey results depends on both variance and bias (mostly from nonsampling errors) Bias should be minimized with quality control for all survey operations Basic data quality determined during enumeration Important to have good training and supervision in the field Data capture should include 100% or sample verification Important to have quality control for editing and coding procedures Computer consistency and range checks MICS4 Survey Design Workshop

35 MICS4 Survey Design Workshop
Country example 2008 Mozambique MICS3 Use of existing survey Subsample of EAs from the other survey Shared listing with another survey Different households selected for each survey MICS4 Survey Design Workshop

Download ppt "Multiple Indicator Cluster Surveys Survey Design Workshop"

Similar presentations

Ads by Google