Chapter 11 EDPR 7521 Dr. Kakali Bhattacharya Experimental Designs Chapter 11 EDPR 7521 Dr. Kakali Bhattacharya

Why Experiments? Establish cause and effect Two or more groups to study Get grants 

Key Characteristics Random assignment Control over extraneous variables Manipulation of treatment conditions Outcome measures Group comparisons Threats to validity

Control Procedures Pretests and posttests Covariates Expectations can affect outcomes Covariates Relate only to dependent variable Used to control for effects of pretests, or other variables that need to be controlled for E.g. scores on a pretest correlated with dependent variable Analysis of covariance – adjusts scores on dependent variable to account for covariance Matching of participants Gender, pretest scores, individual abilities Homogenous samples Blocking variables Break into subgroups, effects of subgroups on outcome

Manipulating Treatment Conditions Treatment variables independent variables that are manipulated by the researcher Conditions Levels of treatment variables Manipulate the treatment conditions Who gets to receive what treatment in control and experimental groups

Outcome Measures Dependent variable presumed effect of the treatment variable Effect predicted in the hypothesis Good outcome measures are sensitive to smallest amount of intervention

Group Comparisons Compare means and variance both within and between groups You would want to do this to make your study stronger Show that you have considered multiple forms of manipulation of independent variables

Threats to Internal Validity History Maturation Regression Selection Mortality Interactions with selection Diffusion of treatments Experimental and control group can communicate with each other Compensatory equalization One group gets benefits and other doesn’t. Use comparison groups instead of control groups and offer some mitigated benefits Switch groups (education) Compensatory rivalry Publicly announce benefit of being in the experimental group Resentful demoralization Less desirable treatment Testing (remember) Instrumentation (change) Bad design, bad conclusions Bad inferences

Threats to External Validity Interaction of selection and treatment Inability to generalize beyond the groups in the experiment Interaction of setting and treatment Inability to generalize to another setting Analyze effect of setting and treatment BAD GENERALIZABILITY

Types of Experimental Designs Between Group Designs (2 or more groups) true, quasi, factorial designs (multiple independent variables, p. 299) Within Group or Individual Designs Limited participants, can’t compare groups Time series designs (Table 11.5, p. 302) Repeated Measures Design All participants in a single group participate in all experimental treatments with each group becoming its own control (compare outcomes, need to make treatments distinct) Single-subject Designs Baseline, intervention, outcome, reversal (p. 305)

Steps in Conducting Experimental Research Decide if experiment addresses research purpose and questions Form hypotheses to test cause and effect Select experimental unit Identify participants Select experimental treatment Introduce experimental treatment Choose a type of experimental design Conduct the experiment Organize data Analyze data Write up findings