Presentation on theme: "Sampling and Sample Size Calculation"— Presentation transcript:
1 Sampling and Sample Size Calculation Lazereto de Mahón, Menorca, SpainSeptember 2006Sources:-EPIET Introductory course,Thomas Grein, Denis Coulombier, Philippe Sudre, Mike Catchpole, Denise Antona-IDEABrigitte Helynck, Philippe Malfait, Institut de veille sanitaireModified: Viviane Bremer, EPIET 2004, Suzanne Cotter 2005,Richard Pebody 2006
2 Objectives: sampling To understand: Why we use sampling Definitions in samplingSampling errorsMain methods of samplingSample size calculation
3 Why do we use sampling? Get information from large populations with: Reduced costsReduced field timeIncreased accuracyEnhanced methods
4 Definition of sampling Procedure by which some membersof a given population are selected as representatives of the entire populationSampling is the use of a subset of the population to represent the whole population
5 Definition of sampling terms Sampling unit (element)Subject under observation on which information is collectedExample: children <5 years, hospital discharges, health events…Sampling fractionRatio between sample size and population sizeExample: 100 out of 2000 (5%)
6 Definition of sampling terms Sampling frameList of all the sampling units from which sample is drawnLists: e.g. children < 5 years of age, households, health care units…Sampling schemeMethod of selecting sampling units from sampling frameRandomly, convenience sample…
8 Representativeness (validity) A sample should accurately reflect distribution ofrelevant variable in populationPerson e.g. age, sexPlace e.g. urban vs. ruralTime e.g. seasonalityRepresentativeness essential to generaliseEnsure representativeness before starting,Confirm once completed
9 Sampling and representativeness PopulationSampleTarget PopulationTarget Population Sampling Population Sample
10 Sampling errorRandom difference between sample and population from which sample drawnSize of error can be measured in probability samplesExpressed as “standard error”of mean, proportion…Standard error (or precision) depends upon:Size of the sampleDistribution of character of interest in populationan error is the amount by which an observation differs from its expected value.
11 Sampling errorWhen simple random sample of size ‘n’ is selected from population of size N, standard error (s) for population mean or proportion is:σ p(1-p) n nUsed to calculate, 95% confidence intervalsan error is the amount by which an observation differs from its expected value.Estimated 95% confidence interval
12 Quality of a sampling estimate No precisionRandom errorPrecision butno validitySystematic error (bias)Precision & validityprécision = Precision (also called reproducibility or repeatability) is the degree to which further measurements or calculations will show the same or similar results.validité = capacité à estimer la vraie valeur du paramètre dans la populationan error is the amount by which an observation differs from its expected value.the latter being based on the whole population from which the statistical unit was chosen randomly. The expected value, being the average of the entire population, is typically unobservable. If the average height of 21-year-old men is 5 feet 9 inches, and one randomly chosen man is 5 feet 11 inches tall, then the "error" is 2 inches; if the randomly chosen man is 5 feet 7 inches tall, then the "error" is −2 inches. The nomenclature arose from random measurement errors in astronomy. It is as if the measurement of the man's height were an attempt to measure the population average, so that any difference between the man's height and the average would be a measurement error.A residual, on the other hand, is an observable estimate of the unobservable error. The simplest case involves a random sample of n men whose heights are measured. The sample average is used as an estimate of the population average. Then we have:The difference between the height of each man in the sample and the unobservable population average is an error, andThe difference between the height of each man in the sample and the observable sample average is a residual.Residuals are observable; errors are not.Note that the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. The sum of the errors need not be zero; the errors are independent random variables if the individuals are chosen from the population independently.In statistics, when analyzing collected data, the samples observed may not be a representative sample from all possible samples. This is sampling error and is controlled by ensuring that, as much as possible, the samples taken have no systematic characteristics and are a true random sample from all possible samples. If the observations are a true random sample, statistics can make probability estimates of the sampling error and allow the researcher to estimate what further experiments are necessary to minimize it.A systematic error is any biasing effect, in the environment, methods of observation or instruments used, which introduces error into an experimentConstant systematic errors are very difficult to deal with, because their effects are only observable if they can be removed. To remove systematic error is simply to make a better experiment7
13 Survey errors: example Measuring height:Measuring tape held differently by different investigators→ loss of precisionLarge standard errorTape shrunk/wrong→ systematic errorBias (cannot be corrected afterwards)179178177176175174173
14 Types of samplingNon-probability samplesProbability samples
15 Non probability samples Convenience samples (ease of access)Snowball sampling (friend of friend….etc.)Purposive sampling (judgemental)You chose who you think should be in the studyNormally sample should reflect the population structure. A convenience sample does not! In a convenience sample some persons have a higher chance of being sampled than others. For example, if we choose people on the main shopping street as a sample for a city ward, those who are frequently doing their shopping there are at higher risk of being chosen than those working during the day, bedriddenNonprobability sampling techniques cannot be used to infer from the sample to the general population. Any generalizations obtained from a nonprobability sample must be filtered through one's knowledge of the topic being studied. Performing nonprobability sampling is considerably less expensive than doing probability sampling, but the results are of limited value.Examples of nonprobability sampling include:Convenience sampling - members of the population are chosen based on their relative ease of access. To sample friends, co-workers, or shoppers at a single mall, are all examples of convenience sampling.Snowball sampling - The first respondent refers a friend. The friend also referes a friend, etc.Judgmental sampling or Purposive sampling - The researcher chooses the sample based on who they think would be appropriate for the study. This is used primarily when there is a limited number of people that have expertise in the area being researched.Case study - The research is limited to one group, often with a similar characteristic or of small size.ad hoc quotas - A quota is established (say 65% women) and researchers are free to choose any respondent they wish as long as the quota is metEven studies intended to be probability studies sometimes end up being non-probability studies due to unintentional or unavoidable characteristics of the sampling method. In public opinion polling by private companies (or organizations unable to require response), the sample can be self-selected rather than random. This often introduces an important type of error: self-selection error. This error sometimes makes it unlikely that the sample will accurately represent the broader population. Volunteering for the sample may be determined by characteristics such as submissiveness or availability. The samples in such surveys should be treated as non-probability samples of the population, and the validity of the estimates of parameters based on them unknownProbability of being chosen is unknownCheaper- but unable to generalise, potential for bias
16 Probability samples Random sampling Each subject has a known probability of being selectedAllows application of statistical sampling theory to results to:GeneraliseTest hypotheses
17 Methods used in probability samples Simple random samplingSystematic samplingStratified samplingMulti-stage samplingCluster sampling
18 Simple random sampling PrincipleEqual chance/probability of drawing each unitProcedureTake sampling populationNeed listing of all sampling units (“sampling frame”)Number all unitsRandomly draw units
19 Simple random sampling AdvantagesSimpleSampling error easily measuredDisadvantagesNeed complete list of unitsDoes not always achieve best representativenessUnits may be scattered and poorly accessiblee.g simple random sampling of telephone numbers- is everyone in the telephone book? Are there people without phones? Or non-functioning phonesBecause units may be scattered may be more time consuming difficult to involve units
20 Simple random sampling Example: evaluate the prevalence of tooth decay among 1200 children attending a schoolList of children attending the schoolChildren numerated from 1 to 1200Sample size = 100 childrenRandom sampling of 100 numbers between 1 and 1200How to randomly select?
24 Systematic sampling Principle Advantages Disadvantages Select sample at regular intervals based on sampling fractionAdvantagesSimpleSampling error easily measuredDisadvantagesNeed complete list of unitsPeriodicitye
25 Systematic sampling 1st person selected = the 8th on the list N = 1200, and n = 60 sampling fraction = 1200/60 = 20List persons from 1 to 1200Randomly select a number between 1 and 20 (ex : 8) 1st person selected = the 8th on the list 2nd person = = the 28th etc .....8
27 Stratified sampling Principle : Divide sampling frame into homogeneous subgroups (strata) e.g. age-group, occupation;Draw random sample in each strata.
28 Stratified sampling Advantages Disadvantages Can acquire information about whole population and individual strataPrecision increased if variability within strata is less (homogenous) than between strataDisadvantagesCan be difficult to identify strataLoss of precision if small numbers in individual strataresolve by sampling proportionate to stratum population
30 Cluster sampling Principle Sample units not identified independently but in a group (or “cluster”)Provides logistical advantage.
31 Cluster sampling Principle Whole population divided into groups e.g. neighbourhoodsRandom sample taken of these groups (“clusters”)Within selected clusters, all units e.g. households included (or random sample of these units)
33 Cluster sampling Advantages Simple as complete list of sampling units within population not requiredLess travel/resources requiredDisadvantagesPotential problem is that cluster members are more likely to be alike, than those in another cluster (homogenous)….This “dependence” needs to be taken into account in the sample size….and the analysis (“design effect”)
34 Selecting a sampling method Population to be studiedSize/geographical distributionHeterogeneity with respect to variableAvailability of list of sampling unitsLevel of precision requiredResources available
35 Sample size estimation Estimate number needed toreliably measure factor of interestdetect significant associationTrade-off between study size and resources….Sample size determined by various factors:significance level (“alpha”)power (“1-beta”)expected prevalence of factor of interest
36 Type 1 errorThe probability of finding a difference with our sample compared to population, and there really isn’t one….Known as the α (or “type 1 error”)Usually set at 5% (or 0.05)
37 Type 2 errorThe probability of not finding a difference that actually exists between our sample compared to the population…Known as the β (or “type 2 error”)Power is (1- β) and is usually 80%
38 A question? Are the English more intelligent than the Dutch? H0 Null hypothesis: The English and Dutch have the same mean IQHa Alternative hypothesis: The mean IQ of the English is greater than the Dutch
39 Type 1 and 2 errors Truth Decision H0 true H0 false Reject H0 Type I error Correct decisionAccept H Correct Type II errordecision
40 Power The easiest ways to increase power are to: increase sample size increase desired difference (or effect size)decrease significance level desired e.g. 10%
41 Steps in estimating sample size for descriptive survey Identify major study variableDetermine type of estimate (%, mean, ratio,...)Indicate expected frequency of factor of interestDecide on desired precision of the estimateDecide on acceptable risk that estimate will fall outside its real population valueAdjust for estimated design effectAdjust for expected response rate
42 Sample size for descriptive survey Simple random / systematic samplingz² * p * q1.96²*0.15*0.85n == 544d²0.03²Cluster samplingz² * p * q2*1.96²*0.15*0.85n = g*= 1088d²0.03²z: alpha risk expressed in z-scorep: expected prevalenceq: 1 - pd: absolute precisiong: design effect
43 Case-control sample size: issues to consider Number of casesNumber of controls per caseOdds ratio worth detectingProportion of exposed persons in source populationDesired level of significance (α)Power of the study (1-β)to detect at a statistically significant level a particular odds ratio
44 Case-control: STATCALC Sample size If we have too few cases we may improve the power of our study by selecting more controls, as we said almost no extra power is gained selecting more than 4 controls but
45 Case-control: STATCALC Sample size Risk of alpha error %Power %Proportion of controls exposed 20%OR to detect > 2The power of a statistical test is the probability that the test will reject a false null hypothesis, or in other words that it will not make a Type II error. The higher the power, the greater the chance of obtaining a statistically significant result when the null hypothesis is false.Statistical power depends on the significance criterion, the size of the difference or the strength of the similarity (that is, the effect size) in the population, and the sensitivity of the data.
46 Case-control: STATCALC Sample size If we have too few cases we may improve the power of our study by selecting more controls, as we said almost no extra power is gained selecting more than 4 controls but
47 Statistical Power of a Case-Control Study for different control-to-case ratios and odds ratios (with 50 cases)a probability is a number between 0 and 1;the probability of an event or proposition and its complement must add up to 1; andthe joint probability of two events or propositions is the product of the probability of one of them and the probability of the second, conditional on the first.Representation and interpretation of probability valuesThe probability of an event is generally represented as a real number between 0 and 1, inclusive. An impossible event has a probability of exactly 0, and a certain event has a probability of 1, but the converses are not always true: probability 0 events are not always impossible, nor probability 1 events certain. The rather subtle distinction between "certain" and "probability 1" is treated at greater length in the article on "almost surely".Most probabilities that occur in practice are numbers between 0 and 1, indicating the event's position on the continuum between impossibility and certainty. The closer an event's probability is to 1, the more likely it is to occur.For example, if two mutually exclusive events are assumed equally probable, such as a flipped coin landing heads-up or tails-up, we can express the probability of each event as "1 in 2", or, equivalently, "50%" or "1/2".Probabilities are equivalently expressed as odds, which is the ratio of the probability of one event to the probability of all other events. The odds of heads-up, for the tossed coin, are (1/2)/(1 - 1/2), which is equal to 1/1. This is expressed as "1 to 1 odds" and often written "1:1".Odds a:b for some event are equivalent to probability a/(a+b). For example, 1:1 odds are equivalent to probability 1/2, and 3:2 odds are equivalent to probability 3/5.There remains the question of exactly what can be assigned probability, and how the numbers so assigned can be used; this is the question of probability interpretations.
48 Conclusions Probability samples are the best Ensure RepresentativenessPrecision…..within available constraints
49 ConclusionsIf in doubt…Call a statistician !!!!