# Sampling and Sample Size Calculation

## Presentation on theme: "Sampling and Sample Size Calculation"— Presentation transcript:

Sampling and Sample Size Calculation
Lazereto de Mahón, Menorca, Spain September 2006 Sources: -EPIET Introductory course, Thomas Grein, Denis Coulombier, Philippe Sudre, Mike Catchpole, Denise Antona -IDEA Brigitte Helynck, Philippe Malfait, Institut de veille sanitaire Modified: Viviane Bremer, EPIET 2004, Suzanne Cotter 2005, Richard Pebody 2006

Objectives: sampling To understand: Why we use sampling
Definitions in sampling Sampling errors Main methods of sampling Sample size calculation

Why do we use sampling? Get information from large populations with:
Reduced costs Reduced field time Increased accuracy Enhanced methods

Definition of sampling
Procedure by which some members of a given population are selected as representatives of the entire population Sampling is the use of a subset of the population to represent the whole population

Definition of sampling terms
Sampling unit (element) Subject under observation on which information is collected Example: children <5 years, hospital discharges, health events… Sampling fraction Ratio between sample size and population size Example: 100 out of 2000 (5%)

Definition of sampling terms
Sampling frame List of all the sampling units from which sample is drawn Lists: e.g. children < 5 years of age, households, health care units… Sampling scheme Method of selecting sampling units from sampling frame Randomly, convenience sample…

Survey errors Systematic error (or bias) Sampling error (random error)
Sample not typical of population Inaccurate response (information bias) Selection bias Sampling error (random error)

Representativeness (validity)
A sample should accurately reflect distribution of relevant variable in population Person e.g. age, sex Place e.g. urban vs. rural Time e.g. seasonality Representativeness essential to generalise Ensure representativeness before starting, Confirm once completed

Sampling and representativeness
Population Sample Target Population Target Population  Sampling Population  Sample

Sampling error Random difference between sample and population from which sample drawn Size of error can be measured in probability samples Expressed as “standard error” of mean, proportion… Standard error (or precision) depends upon: Size of the sample Distribution of character of interest in population an error is the amount by which an observation differs from its expected value.

Sampling error When simple random sample of size ‘n’ is selected from population of size N, standard error (s) for population mean or proportion is: σ p(1-p)  n n Used to calculate, 95% confidence intervals an error is the amount by which an observation differs from its expected value. Estimated 95% confidence interval

Quality of a sampling estimate
No precision Random error Precision but no validity Systematic error (bias) Precision & validity précision = Precision (also called reproducibility or repeatability) is the degree to which further measurements or calculations will show the same or similar results. validité = capacité à estimer la vraie valeur du paramètre dans la population an error is the amount by which an observation differs from its expected value. the latter being based on the whole population from which the statistical unit was chosen randomly. The expected value, being the average of the entire population, is typically unobservable. If the average height of 21-year-old men is 5 feet 9 inches, and one randomly chosen man is 5 feet 11 inches tall, then the "error" is 2 inches; if the randomly chosen man is 5 feet 7 inches tall, then the "error" is −2 inches. The nomenclature arose from random measurement errors in astronomy. It is as if the measurement of the man's height were an attempt to measure the population average, so that any difference between the man's height and the average would be a measurement error. A residual, on the other hand, is an observable estimate of the unobservable error. The simplest case involves a random sample of n men whose heights are measured. The sample average is used as an estimate of the population average. Then we have: The difference between the height of each man in the sample and the unobservable population average is an error, and The difference between the height of each man in the sample and the observable sample average is a residual. Residuals are observable; errors are not. Note that the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. The sum of the errors need not be zero; the errors are independent random variables if the individuals are chosen from the population independently. In statistics, when analyzing collected data, the samples observed may not be a representative sample from all possible samples. This is sampling error and is controlled by ensuring that, as much as possible, the samples taken have no systematic characteristics and are a true random sample from all possible samples. If the observations are a true random sample, statistics can make probability estimates of the sampling error and allow the researcher to estimate what further experiments are necessary to minimize it. A systematic error is any biasing effect, in the environment, methods of observation or instruments used, which introduces error into an experiment Constant systematic errors are very difficult to deal with, because their effects are only observable if they can be removed. To remove systematic error is simply to make a better experiment 7

Survey errors: example
Measuring height: Measuring tape held differently by different investigators → loss of precision Large standard error Tape shrunk/wrong → systematic error Bias (cannot be corrected afterwards) 179 178 177 176 175 174 173

Types of sampling Non-probability samples Probability samples

Non probability samples
Convenience samples (ease of access) Snowball sampling (friend of friend….etc.) Purposive sampling (judgemental) You chose who you think should be in the study Normally sample should reflect the population structure. A convenience sample does not! In a convenience sample some persons have a higher chance of being sampled than others. For example, if we choose people on the main shopping street as a sample for a city ward, those who are frequently doing their shopping there are at higher risk of being chosen than those working during the day, bedridden Nonprobability sampling techniques cannot be used to infer from the sample to the general population. Any generalizations obtained from a nonprobability sample must be filtered through one's knowledge of the topic being studied. Performing nonprobability sampling is considerably less expensive than doing probability sampling, but the results are of limited value. Examples of nonprobability sampling include: Convenience sampling - members of the population are chosen based on their relative ease of access. To sample friends, co-workers, or shoppers at a single mall, are all examples of convenience sampling. Snowball sampling - The first respondent refers a friend. The friend also referes a friend, etc. Judgmental sampling or Purposive sampling - The researcher chooses the sample based on who they think would be appropriate for the study. This is used primarily when there is a limited number of people that have expertise in the area being researched. Case study - The research is limited to one group, often with a similar characteristic or of small size. ad hoc quotas - A quota is established (say 65% women) and researchers are free to choose any respondent they wish as long as the quota is met Even studies intended to be probability studies sometimes end up being non-probability studies due to unintentional or unavoidable characteristics of the sampling method. In public opinion polling by private companies (or organizations unable to require response), the sample can be self-selected rather than random. This often introduces an important type of error: self-selection error. This error sometimes makes it unlikely that the sample will accurately represent the broader population. Volunteering for the sample may be determined by characteristics such as submissiveness or availability. The samples in such surveys should be treated as non-probability samples of the population, and the validity of the estimates of parameters based on them unknown Probability of being chosen is unknown Cheaper- but unable to generalise, potential for bias

Probability samples Random sampling
Each subject has a known probability of being selected Allows application of statistical sampling theory to results to: Generalise Test hypotheses

Methods used in probability samples
Simple random sampling Systematic sampling Stratified sampling Multi-stage sampling Cluster sampling

Simple random sampling
Principle Equal chance/probability of drawing each unit Procedure Take sampling population Need listing of all sampling units (“sampling frame”) Number all units Randomly draw units

Simple random sampling
Advantages Simple Sampling error easily measured Disadvantages Need complete list of units Does not always achieve best representativeness Units may be scattered and poorly accessible e.g simple random sampling of telephone numbers- is everyone in the telephone book? Are there people without phones? Or non-functioning phones Because units may be scattered may be more time consuming difficult to involve units

Simple random sampling
Example: evaluate the prevalence of tooth decay among 1200 children attending a school List of children attending the school Children numerated from 1 to 1200 Sample size = 100 children Random sampling of 100 numbers between 1 and 1200 How to randomly select?

EPITABLE: random number listing

EPITABLE: random number listing
Also possible in Excel

Simple random sampling

Select sample at regular intervals based on sampling fraction Advantages Simple Sampling error easily measured Disadvantages Need complete list of units Periodicity e

Systematic sampling  1st person selected = the 8th on the list
N = 1200, and n = 60  sampling fraction = 1200/60 = 20 List persons from 1 to 1200 Randomly select a number between 1 and 20 (ex : 8)  1st person selected = the 8th on the list  2nd person = = the 28th etc ..... 8

Systematic sampling

Stratified sampling Principle :
Divide sampling frame into homogeneous subgroups (strata) e.g. age-group, occupation; Draw random sample in each strata.

Can acquire information about whole population and individual strata Precision increased if variability within strata is less (homogenous) than between strata Disadvantages Can be difficult to identify strata Loss of precision if small numbers in individual strata resolve by sampling proportionate to stratum population

Multiple stage sampling
Principle: consecutive sampling example : sampling unit = household 1st stage: draw neighborhoods 2nd stage: draw buildings 3rd stage: draw households 8

Cluster sampling Principle
Sample units not identified independently but in a group (or “cluster”) Provides logistical advantage.

Cluster sampling Principle
Whole population divided into groups e.g. neighbourhoods Random sample taken of these groups (“clusters”) Within selected clusters, all units e.g. households included (or random sample of these units)

Example: Cluster sampling
Section 1 Section 2 Section 3 Section 5 Section 4

Simple as complete list of sampling units within population not required Less travel/resources required Disadvantages Potential problem is that cluster members are more likely to be alike, than those in another cluster (homogenous)…. This “dependence” needs to be taken into account in the sample size….and the analysis (“design effect”)

Selecting a sampling method
Population to be studied Size/geographical distribution Heterogeneity with respect to variable Availability of list of sampling units Level of precision required Resources available

Sample size estimation
Estimate number needed to reliably measure factor of interest detect significant association Trade-off between study size and resources…. Sample size determined by various factors: significance level (“alpha”) power (“1-beta”) expected prevalence of factor of interest

Type 1 error The probability of finding a difference with our sample compared to population, and there really isn’t one…. Known as the α (or “type 1 error”) Usually set at 5% (or 0.05)

Type 2 error The probability of not finding a difference that actually exists between our sample compared to the population… Known as the β (or “type 2 error”) Power is (1- β) and is usually 80%

A question? Are the English more intelligent than the Dutch?
H0 Null hypothesis: The English and Dutch have the same mean IQ Ha Alternative hypothesis: The mean IQ of the English is greater than the Dutch

Type 1 and 2 errors Truth Decision H0 true H0 false
Reject H0 Type I error Correct decision Accept H Correct Type II error decision

Power The easiest ways to increase power are to: increase sample size
increase desired difference (or effect size) decrease significance level desired e.g. 10%

Steps in estimating sample size for descriptive survey
Identify major study variable Determine type of estimate (%, mean, ratio,...) Indicate expected frequency of factor of interest Decide on desired precision of the estimate Decide on acceptable risk that estimate will fall outside its real population value Adjust for estimated design effect Adjust for expected response rate

Sample size for descriptive survey
Simple random / systematic sampling z² * p * q 1.96²*0.15*0.85 n = = 544 0.03² Cluster sampling z² * p * q 2*1.96²*0.15*0.85 n = g* = 1088 0.03² z: alpha risk expressed in z-score p: expected prevalence q: 1 - p d: absolute precision g: design effect

Case-control sample size: issues to consider
Number of cases Number of controls per case Odds ratio worth detecting Proportion of exposed persons in source population Desired level of significance (α) Power of the study (1-β) to detect at a statistically significant level a particular odds ratio

Case-control: STATCALC Sample size
If we have too few cases we may improve the power of our study by selecting more controls, as we said almost no extra power is gained selecting more than 4 controls but

Case-control: STATCALC Sample size
Risk of alpha error % Power % Proportion of controls exposed 20% OR to detect > 2 The power of a statistical test is the probability that the test will reject a false null hypothesis, or in other words that it will not make a Type II error. The higher the power, the greater the chance of obtaining a statistically significant result when the null hypothesis is false. Statistical power depends on the significance criterion, the size of the difference or the strength of the similarity (that is, the effect size) in the population, and the sensitivity of the data.

Case-control: STATCALC Sample size
If we have too few cases we may improve the power of our study by selecting more controls, as we said almost no extra power is gained selecting more than 4 controls but

Statistical Power of a Case-Control Study for different control-to-case ratios and odds ratios (with 50 cases) a probability is a number between 0 and 1; the probability of an event or proposition and its complement must add up to 1; and the joint probability of two events or propositions is the product of the probability of one of them and the probability of the second, conditional on the first. Representation and interpretation of probability values The probability of an event is generally represented as a real number between 0 and 1, inclusive. An impossible event has a probability of exactly 0, and a certain event has a probability of 1, but the converses are not always true: probability 0 events are not always impossible, nor probability 1 events certain. The rather subtle distinction between "certain" and "probability 1" is treated at greater length in the article on "almost surely". Most probabilities that occur in practice are numbers between 0 and 1, indicating the event's position on the continuum between impossibility and certainty. The closer an event's probability is to 1, the more likely it is to occur. For example, if two mutually exclusive events are assumed equally probable, such as a flipped coin landing heads-up or tails-up, we can express the probability of each event as "1 in 2", or, equivalently, "50%" or "1/2". Probabilities are equivalently expressed as odds, which is the ratio of the probability of one event to the probability of all other events. The odds of heads-up, for the tossed coin, are (1/2)/(1 - 1/2), which is equal to 1/1. This is expressed as "1 to 1 odds" and often written "1:1". Odds a:b for some event are equivalent to probability a/(a+b). For example, 1:1 odds are equivalent to probability 1/2, and 3:2 odds are equivalent to probability 3/5. There remains the question of exactly what can be assigned probability, and how the numbers so assigned can be used; this is the question of probability interpretations.

Conclusions Probability samples are the best Ensure
Representativeness Precision …..within available constraints

Conclusions If in doubt… Call a statistician !!!!

Similar presentations