Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 1 Chapter 11 Experimental and Quasi-experimental Designs

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 2 Key Ideas 4 History of experimental resign 4 Key characteristics of experimental design 4 Types of experimental design 4 Steps in conducting experimental research 4 Criteria for evaluating experimental research

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 3 A Brief History of Experimental Designs 4 Schuyler 1903: control groups 4 McCall 1916: randomly selected groups 4 McCall 1925: book on experiments 4 Fisher 1936: statistical methods book

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 4 A Brief History of Experimental Designs 4 Campbell and Stanley 1963: 15 types of experimental designs evaluated in terms of threats to validity 4 Cook and Campbell 1979: four types of validity and beyond: computer enhancements to experimental design2

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 5 Key Characteristics of Experimental Designs 4 Participants selected and assigned to groups –control –experimental 4 An intervention is applied to one or more groups 4 Outcomes are measured at the end of the experiment

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 6 Key Characteristics of Experimental Designs 4 Procedures are designed that address potential threats to validity –Internal –External –Construct –Statistical Conclusion 4 Statistical comparisons of different groups are conducted

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 7 Selecting Participants and Assigning Them to Treatments 4 Decide on the experimental unit of analysis to be treated –individual –group or groups –organization 4 Randomly assign individuals to groups control for extraneous characteristics that might influence the outcome

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 8 Selecting Participants and Assigning Them to Treatments 4 Control for extraneous factors –random assignment (equating groups) –pretest/posttest –covariates –matching participants –selecting homogenous samples –using blocking variables

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 9 Independent Variable Independent Variable: Type of Instruction Dependent Variable: Rates of Smoking Controlling for Covariates No Covariates Dependent Variable Covariate Introduced Covariate: Parents Who Smoke Variance Removed Variance

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 10 Matching Process Based on Gender Experimental Group Control Group John Jim James Josh Jackson Jane Johanna Julie Jean Jeb

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 11 Applying an Intervention or Treatment 4 Identify a treatment variable 4 identify the conditions or levels of the variable 4 Manipulate the treatment conditions

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 12 The Experimental Manipulation of a Treatment Group Independent Variables 1. Age (can’t manipulate) 2. Gender (can’t manipulate) 3. Types of Instruction (can manipulate) a. Lecture (control) b. Lecture + Hazard Instruction (Comparison) c. Lecture + Hazard Instruction + slides of damaged lungs (experiment) Dependent Variable Frequency of Smoking

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 13 Threats to Statistical Conclusion Validity 4 Low statistical power due to low sample size 4 Violation of assumptions of statistical tests 4 Use of unreliable measures

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 14 Threats to Internal Validity 4 History 4 Maturation 4 Regression 4 Selection 4 Mortality 4 Interactions with selection

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 15 Threats to Internal Validity 4 Diffusion of treatments 4 Compensatory equalization 4 Compensation rivalry 4 Resentful demoralization 4 Testing 4 Instrumentation

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 16 Threats to Construct Validity 4 Lack of good operational definitions 4 Apprehensiveness by participants 4 Participants “guessing” what the researcher hopes to find

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 17 Threats to External Validity 4 Interaction of selection and treatment 4 Interaction of setting and treatment 4 interaction of history and treatment

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 18 Treatment Comparisons in an Experiment Phase 1: Relationship Picture Error Correction Treatment + Spelling Accuracy Phase 2: Timeline Picture Test 1 Test 2 Test 3 6 Weeks 6 Weeks 6 Weeks Class A: Regular Spelling Practice (Control) Class B: Reduced word list (Comparison) Phase 3: Statistical Comparisons Class A Class B Class C F-value Test 1 Test 2 Test (3.6) 10.7 (3.3) 11.1 (3.3) 10.8 (4.3) 10.6 (3.8) 10.3 (3.6) 9.9 (3.9) 13.9 (4.2) 13.1 (3.8) * 3.31* * p<.05 Error Correction Treatment 6 Weeks 6 Weeks 6 Weeks Class C: Error Correction (Experimental)

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 19 Types of Experimental Designs: Between Groups 4 True Experiments –Pre- and posttest –Posttest Only 4 Quasi Experiments –Pre- and posttest –Posttest Only 4 Factorial Designs

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 20 Types of Experimental Designs: Within-Group or Individual 4 Time series experiments –interrupted –uninterrupted 4 Repeated measures experiments 4 Single-subject experiments –A/B design –Multiple baseline design –Alternating treatments

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 21 Means and Main Effects of Eight groups in Factorial Design Depression Mean rate of smoking Low Mean rate of smoking Mean rate of smoking Mean rate of smoking Mean rate of smoking Mean rate of smoking Health lecture Type of Instruction Standard lecture Main Effects of Type of Instruction Main Effects of Depression MediumHigh

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 22 Graphs Showing Main and Interaction Effects High Low MediumHigh Low MediumHigh Low MediumHigh (a) No interaction Effects (Parallel) Extent of Smoking (b) Interaction Effects (Crossed) (c) Interaction Effects (Not Parallel) (a) Interaction Effects (Crossed) Standard lecture Health lecture Standard lecture Health lecture Standard lecture Health lecture

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 23 Steps in Conducting Experimental Research 4 Decide if an experiment addresses the research problem 4 Form hypotheses to test cause-effect relationships 4 Select an experimental treatment and introduce it 4 Identify study participants

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 24 Steps in Conducting Experimental Research 4 Choose a type of experimental design 4 Conduct the experiment 4 Organize and analyze the data 4 Develop an experimental research report

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 25 Criteria for Evaluating Experimental Research 4 Does the experiment have a powerful intervention? 4 Does it employ few treatment groups (e.g. only two)? 4 Will participant profit from the intervention? 4 Is there a systematic way the researcher derived the number of participants per group?

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 26 Criteria for Evaluating Experimental Research 4 Were there an adequate number of participants used in the study? 4 Were valid, reliable, and sensitive measures or observations used? 4 Did the study control for extraneous factors? 4 Did the researcher control for threats to internal validity?

Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 27 Applying What you Have Learned: An Experimental Study Review the article and look for the following: 4 The research problem and use of quantitative research 4 Use of the literature 4 The purpose statement and research hypothesis 4 Types and procedures of data collection 4 Types and procedures of data analysis and interpretation 4 The overall report structure