1. l1 Chapter 11 Experimental and Quasi- experimental Designs Dr. Bill Bauer

2. l2 Key Ideas History of experimental resign Key characteristics of experimental design Types of experimental design Steps in conducting experimental research Criteria for evaluating experimental research

3. l3 A Brief History of Experimental Designs Schuyler 1903: control groups McCall 1916: randomly selected groups McCall 1925: book on experiments Fisher 1936: statistical methods book

4. l4 A Brief History of Experimental Designs Campbell and Stanley 1963: 15 types of experimental designs evaluated in terms of threats to validity Cook and Campbell 1979: four types of validity 1980 and beyond: computer enhancements to experimental design2

5. l5 Key Characteristics of Experimental Designs Participants selected and assigned to groups control experimental An intervention is applied to one or more groups Outcomes are measured at the end of the experiment

6. l6 Key Characteristics of Experimental Designs Procedures are designed that address potential threats to validity Internal External Construct Statistical Conclusion Statistical comparisons of different groups are conducted

7. l7 Selecting Participants and Assigning Them to Treatments Decide on the experimental unit of analysis to be treated individual group or groups organization Randomly assign individuals to groups control for extraneous characteristics that might influence the outcome

8. l8 Selecting Participants and Assigning Them to Treatments Control for extraneous factors random assignment (equating groups) pretest/posttest covariates matching participants selecting homogenous samples using blocking variables

9. l9 Independent Variable Independent Variable: Type of Instruction Dependent Variable: Rates of Smoking Controlling for Covariates No Covariates Dependent Variable Covariate Introduced Covariate: Parents Who Smoke Variance Removed Variance

10. l10 Matching Process Based on Gender Experimental Group Control Group John Jim James Josh Jackson Jane Johanna Julie Jean Jeb

11. l11 Applying an Intervention or Treatment Identify a treatment variable identify the conditions or levels of the variable Manipulate the treatment conditions

12. l12 The Experimental Manipulation of a Treatment Group Independent Variables 1. Age (can’t manipulate) 2. Gender (can’t manipulate) 3. Types of Instruction (can manipulate) a. Lecture (control) b. Lecture + Hazard Instruction (Comparison) c. Lecture + Hazard Instruction + slides of damaged lungs (experiment) Dependent Variable Frequency of Smoking

13. l13 Threats to Statistical Conclusion Validity Low statistical power due to low sample size Violation of assumptions of statistical tests Use of unreliable measures

14. l14 Threats to Internal Validity History Maturation Regression Selection Mortality Interactions with selection

15. l15 Threats to Internal Validity Diffusion of treatments Compensatory equalization Compensation rivalry Resentful demoralization Testing Instrumentation

16. l16 Threats to Construct Validity Lack of good operational definitions Apprehensiveness by participants Participants “guessing” what the researcher hopes to find

17. l17 Threats to External Validity Interaction of selection and treatment Interaction of setting and treatment interaction of history and treatment

18. l18 Treatment Comparisons in an Experiment Phase 1: Relationship Picture Error Correction Treatment + Spelling Accuracy Phase 2: Timeline Picture Test 1 Test 2 Test 3 6 Weeks 6 Weeks 6 Weeks Class A: Regular Spelling Practice (Control) Class B: Reduced word list (Comparison) Phase 3: Statistical Comparisons Class A Class B Class C F-value Test 1 Test 2 Test 3 10.3 (3.6) 10.7 (3.3) 11.1 (3.3) 10.8 (4.3) 10.6 (3.8) 10.3 (3.6) 9.9 (3.9) 13.9 (4.2) 13.1 (3.8) 0.27 4.90* 3.31* * p<.05 Error Correction Treatment 6 Weeks 6 Weeks 6 Weeks Class C: Error Correction (Experimental)

19. l19 Types of Experimental Designs: Between Groups True Experiments Pre- and posttest Posttest Only Quasi Experiments Pre- and posttest Posttest Only Factorial Designs

20. l20 Types of Experimental Designs: Within-Group or Individual Time series experiments interrupted uninterrupted Repeated measures experiments Single-subject experiments A/B design Multiple baseline design Alternating treatments

21. l21 Means and Main Effects of Eight groups in Factorial Design Depression Mean rate of smoking Low Mean rate of smoking Mean rate of smoking Mean rate of smoking Mean rate of smoking Mean rate of smoking Health lecture Type of Instruction Standard lecture Main Effects of Type of Instruction Main Effects of Depression MediumHigh

22. l22 Graphs Showing Main and Interaction Effects High Low MediumHigh Low MediumHigh Low MediumHigh (a) No interaction Effects (Parallel) Extent of Smoking (b) Interaction Effects (Crossed) (c) Interaction Effects (Not Parallel) (a) Interaction Effects (Crossed) Standard lecture Health lecture Standard lecture Health lecture Standard lecture Health lecture

23. l23 Steps in Conducting Experimental Research Decide if an experiment addresses the research problem Form hypotheses to test cause- effect relationships Select an experimental treatment and introduce it Identify study participants

24. l24 Steps in Conducting Experimental Research Choose a type of experimental design Conduct the experiment Organize and analyze the data Develop an experimental research report

25. l25 Criteria for Evaluating Experimental Research Does the experiment have a powerful intervention? Does it employ few treatment groups (e.g. only two)? Will participant profit from the intervention? Is there a systematic way the researcher derived the number of participants per group?

26. l26 Criteria for Evaluating Experimental Research Were there an adequate number of participants used in the study? Were valid, reliable, and sensitive measures or observations used? Did the study control for extraneous factors? Did the researcher control for threats to internal validity?

27. l27 Applying What you Have Learned: An Experimental Study Review the article and look for the following: The research problem and use of quantitative research Use of the literature The purpose statement and research hypothesis Types and procedures of data collection Types and procedures of data analysis and interpretation The overall report structure