2 Ways to Collect Data Survey Observational Study Experiment Select a sample, ask questions, record answers.Observational StudyObserve individuals and measure variables of interest, but do not attempt to influence the response, (lack of control, often no random assignment to experimental groups).ExperimentStudy in which we deliberately manipulate and control the sample and measure variables.
3 Survey TermsPopulation: The entire group of individuals that we want information about.Sample: A part of the population that we actually examine in order to gather information.Sampling: The process of choosing and studying a part, in order to get information about the whole.Census: Attempts to contact every individual in the population.
4 Bad Sampling Methods Voluntary Response Sample Convenience Sampling Consists of people who choose themselves by responding to a general appeal.Convenience SamplingChoosing individuals who are the easiest to contact.Bias….the result of poor samplingSystematic favoring of certain outcomes
5 Proper Sampling Methods Probability Sample: Sample chosen by chance.Simple Random Sample (SRS)n individuals from the population chosen in such a way that every set of n individuals has an equal chance to be the sample actually selected.Stratified Random SamplePopulation is first divided into groups called strata, that are similar in a way that is important to the response. SRS is then taken from each stratum to form the sample.# of individuals taken from the strata should be proportional to the number of individuals in the strata.
6 Other Sampling Methods Cluster SamplingDivide population into clustersRandomly select some clusters.All individuals in chosen clusters are included in the sampleMultistage samplingSampling done in steps using a combination of methods, (used for very large populations).
7 Problems with surveys (even when sampling methods are good) UndercoverageSome groups in the population are left out of the process of choosing a sample.NonresponseIndividual chosen for the sample can’t be contacted or does not cooperate*These problems may or may not cause bias.*Bias will result if the people left out are different, as a group, than the people included.
8 Error/Bias Sampling Error Response Bias Poor Question Wording Occurs because the sample rarely reflects the population perfectly.Can’t be avoided…we just have to account for it in our calculations (example: margin of error).Response BiasOccurs when a respondent does not give an accurate response.Causes: characteristics of the interviewer, lying, etc.Poor Question WordingOne-sided, leading
9 Parts of an ExperimentExperimental Units: Individuals on which the experiment is being performed, (called subjects or participants when human)Treatment: An experimental condition applied to the units.Factors: The explanatory variables in an experiment.Level: A specific value of a factorExamples: Dosage, temperatureCombination of levels and factors form the treatment.Example: 200mg given orally, 400mg administered intravenously
10 Principles of Experimental Design 1. ControlMinimize the effects of lurking variables by comparing several treatments in the same environment. (utilize placebos and control groups)Placebo Effect: response to a dummy treatment2. ReplicationUse many experimental units to reduce chance variation in the results3. RandomizationUse impersonal chance to assign experimental units to treatments.Goal: Find statistical significance…the observed effect is so large that it is unlikely to have occurred by chance.
11 Types of Experimental Designs Completely Randomized Designaka a basic comparative experimentAll experimental units are allocated at random among all the treatments
13 Types of Experimental Designs Block DesignAn experiment is conducted separately for different groups (blocks) of experimental units.Use blocks if you expect certain groups of units/subjects to systematically affect the response to the treatments.It is similar to stratified random sampling.
15 Types of Experimental Designs Matched Pairs Design (type of block design)Compares two treatments by comparing the response of two matched experimental units.Units are matched one of two ways….(a) Two different units/subjects matched based on similar characteristics (e.g. identical twins)(b) One subject/unit receives both treatments (i.e. A person is paired with him/herself. Each subject serves as his/her own control.)Randomization is still used to determine who gets which treatment, or which treatment is given first.
17 Other Considerations with Experiments It is sometimes better if the experiment is conducted in a double-blind manner.Neither the subjects nor the people administering the experiment know which treatment the subjects received.Sometimes a lack of realism is a problem for experiments.A laboratory setting is not always the same as real life, which makes it difficult to generalize your findings.
18 Other Considerations Cont… Don’t forget to describe your randomization process in detail when writing an open-ended response.Random sampleAllows you to generalize your results to the populationRandom allocation to treatment groupsAllows you to state that the difference between the responses in the treatment groups was due to the effects of the explanatory variable, not the personal characteristics of the subjects.