2 Learning Objectives By the end of this session, you will be able to describe what is meant by sample, target population, sampled (study) population, sampling frame, sampling unitsexplain what is meant by a representative samplediscuss the importance of sampling to ensure survey results are generalisable – use of probability based samplesdescribe key issues central to the sampling process
3 PopulationThe population is the entire group of all entities about which we want information, e.g.All children below 16 years of age ~ to assess their nutritional levelsAll farmers growing maize ~ to learn about total maize production in the countryAll schools in a country ~ to learn about educational achievementsSuppose you want to estimate the proportion of adults in rural areas with no primary education. What do you think is your population?
4 More on PopulationsPopulation sizes may be finite or infinite. We will allow the possibility of finite populations in this module.Describing the population clearly is necessary before sampling is considered. Why do you think this is so?Need also to distinguish between the target population, as defined above, and the study (or sampled) population. Latter excludes the inaccessible members of the target group e.g. temporary international migrant workers.
5 SamplesA sample is a subset of units drawn from the population. This module is largely on sampling procedures.Sampling units refer to the entities on which measurements are made during a surveyDuring planning (and implementation) of a survey, check the extent to which the target population can be (or has been) covered. Recognise that survey conclusions apply only to the population (study popn) which has a chance of inclusion in the sample.
6 Issues related to sampling units Part of the planning of the sampling processincludes:Identifying the sampling unit. Is ita village, household, farm, school, etc?Recognising occurrence of units at different levels.In most surveys, sampling is done at different levels, e.g. district, then village, then household, then household member. Need to be specific about the selection of sampling units and sampling effort at each stage and specify this in the sampling protocol.
7 More on sampling units Observing units over time Unequal sized units In longitudinal surveys, or monitoring and evaluation surveys, should the same units be chosen for measurement each year?Unequal sized unitsAre all units, whose measurements are being summarised, of the same size? If not, e.g. with business surveys, should measurements be weighted according to size?Number of unitsIs the sample large enough to enable an analysis that addresses the study objectives?
8 Sampling framesThe sampling frame is a list of all sampling units, for examplelist of villages in a regionlist of households within a villagelist of schools in the districtWhy do we need a sampling frame?Tool essential for objectively selecting a sample of units from the population of all unitsDifferent frames are needed if sampling at different hierarchical levelsMay develop relevant sub-frames at each stage as sampling progresses down the hierarchy
9 RepresentativenessThe primary aim in a survey is to make inferences about the target populationFor this, need to ensure population is well represented in the sampleBring in qualitative aspects of the population to ensure there is adequate representation of divisions of the population, e.g. rural/urban, different wealth categories, geographical coverage, etcMay be achieved by sampling from these different sub-groups so that all necessary factors likely to influence survey results are representedIn doing so, ensure selection is such that variability of sampling units within each sub-division is captured, i.e. not just 1 unit per group.
10 Non-representativeness? Example: in a survey of Kenya, urban Nairobi is represented by a random sample of size 10, which, by chance, includes eight women, five being office workers and four of those employed in Ministry offices. Both the two men, by chance, are unemployed.Example: a random sample of two areas in Malawi happened to come up with urban Mzuzu and urban Zomba [2 of the 6 largest towns]In both cases, these (honest) random samples were of very small size, and by chance produced quite untypical results.
11 GeneralisabilityAlso key to survey success is to ensure the sampling is such that survey results can be generalised to the study population.Generalisability requires taking probability-based samples during the sampling process.Probability sampling is the general term for methods where sample selection is objectively-based on known chances of inclusion in the sample.If the probabilities are known and non-zero, they don’t have to be equal: corrections can be made at the data analysis stage.
12 Non-generalisability Example – a researcher selected her sample from a district, including one farm each with all possible combinations of (i) child-/female-/male-headed; (ii) Christian/Muslim; (iii) dambo/hillside land; (iv) purely vegetable-/mixed/purely maize-growingSome cases very hard to find e.g. there were few Muslim, child-headed, purely maize dambo farms.Sample of 36 does not relate to population proportions: not enough cases to be able to represent the commoner combinationsResearcher’s selections not objective/random and cases only came from one smallish area, maybe untypical. Cannot generalise to country!
13 Summary of issues to consider when sampling Have the objectives been clearly specified?Has the target population been clearly defined and the possibility of survey results being applicable to a different ‘study’ population been recognised?What is the geographical coverage?What factors are likely to influence survey results – have they been considered in the sampling?What should be the sampling unit(s) for fieldwork?
14 Summary of issues to consider when sampling - continued Will the sample results lead to generalisable conclusions?Will the proposed sampling plan be possible within time and budget limitations?Is the sampling procedure practically feasible?Will the adopted sampling scheme provide results that address survey objectives with appropriate measures of precision?Developing a good sampling plan requirestime and effort! Full plan should be welldocumented with full justification!
15 ReferencesPettersson H. (2004) “Design of master sampling frames and master samples for household surveys in developing countries”. Chapter V of the UN Publication An Analysis of Operating Characteristics of Household Surveys in Developing and Transition Countries: Survey Costs, Design Effects and Non-Sampling Errors. Pp Available atDe Vaus, A.D. (2001) “Research Design in Social Research”. Sage Publications, London - for discussion on generalisabilityBechhofer, F. & Paterson, L. (2000) “Principles of research design in the social sciences”. Routledge, London – for a discussion on representativeness