Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright © Allyn & Bacon (2010) Controls to Reduce Threats to Validity Graziano and Raulin Research Methods: Chapter 9 This multimedia product and its.

Similar presentations


Presentation on theme: "Copyright © Allyn & Bacon (2010) Controls to Reduce Threats to Validity Graziano and Raulin Research Methods: Chapter 9 This multimedia product and its."— Presentation transcript:

1 Copyright © Allyn & Bacon (2010) Controls to Reduce Threats to Validity Graziano and Raulin Research Methods: Chapter 9 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.

2 Copyright © Allyn & Bacon (2010) Threats to Validity Covered in Chapter 8 Covered in Chapter 8 Validity can be threatened in many ways Validity can be threatened in many ways –Presence of confounding variables –Unrepresentative samples –Inappropriate statistical tests or violations of statistics assumptions –Subject and experimenter effects All these threats can be controlled All these threats can be controlled

3 Copyright © Allyn & Bacon (2010) Control Procedures General control procedures (applicable to virtually all research) General control procedures (applicable to virtually all research) Control over subject and experimenter effects Control over subject and experimenter effects Control through the selection and assignment of participants Control through the selection and assignment of participants Control through experimental design Control through experimental design

4 Copyright © Allyn & Bacon (2010) General Control Procedures Preparation of the setting Preparation of the setting –Free of distractions that might interfere –A natural setting increases external validity Response Measurement Response Measurement –Use reliable and valid measures Replication Replication –Demonstrates that findings are consistent and robust

5 Copyright © Allyn & Bacon (2010) Types of Replication Exact Replication Exact Replication –Repeating a study using identical procedures to the original Systematic Replication Systematic Replication –Using a theoretical or procedural change Conceptual Replication Conceptual Replication –Varying the operational definitions of the variables to get new research hypotheses

6 Copyright © Allyn & Bacon (2010) Subject and Experimenter Effects Blind procedures Blind procedures –Best control for expectancy effects –Single-blind: The experimenter does not know what condition the participant is in –Double-blind: Neither the experimenter nor the participant knows what condition the participant is in

7 Copyright © Allyn & Bacon (2010) Subject and Experimenter Effects Automation Automation –Reduces contact between participants and the experimenter –Gives the experimenter less opportunity to affect participants Using objective measures Using objective measures –Objective measure require less judgment –Provides less opportunity for subtle experimenter biases to affect the data

8 Copyright © Allyn & Bacon (2010) Subject and Experimenter Effects Multiple observers Multiple observers –Reduces bias because it challenges observers to be as precise and objective as possible –Can measure amount of observer agreement (percent agreement or Kappa) Using deception Using deception –Hides purpose of the study from participants –Balanced placebo design is a good example

9 Copyright © Allyn & Bacon (2010) Balanced Placebo Design Separates the pharmacological effects from the expectancy effects of alcohol Separates the pharmacological effects from the expectancy effects of alcohol A two-factor design A two-factor design –Factor 1 is whether the person drinks alcohol –Factor 2 is whether the person thinks he or she is drinking alcohol

10 Copyright © Allyn & Bacon (2010) Balanced Placebo Design This design crosses the consumption of alcohol with the belief that alcohol is being consumed People Led to Believe Drinking Alcohol Not Drinking Alcohol Actual Situation Drinking Alcohol Not Drinking Alcohol

11 Copyright © Allyn & Bacon (2010) Participant Selection Can generalize only if your sample is representative Can generalize only if your sample is representative Populations and samples Populations and samples –General population: all potential participants –Target population: those participants you are interested in –Accessible population: portion of target population that is available to the researcher –Sample: drawn from the accessible population

12 Copyright © Allyn & Bacon (2010) Populations and Sampling This figure shows the relationship between the various populations This figure shows the relationship between the various populations –General Population –Target Population –Accessible Population –Sample

13 Copyright © Allyn & Bacon (2010) Sampling Procedures Random sampling Random sampling –Every participant has an equal chance of being sampled Stratified random sampling Stratified random sampling –Random sampling within strata (subgroups) Ad hoc samples Ad hoc samples –Random sample from accessible population –Must generalize cautiously –Should describe sample to help define limits of generalization

14 Copyright © Allyn & Bacon (2010) Participant Assignment Assignment Procedures Assignment Procedures –Free random assignment Random assignment of participants to groups Random assignment of participants to groups –Randomize within blocks Randomly assign in blocks of one participant per condition Randomly assign in blocks of one participant per condition –Matched random assignment Random assignment of participants in matched sets to groups Random assignment of participants in matched sets to groups –Other matching procedures e.g., match groups on key characteristics e.g., match groups on key characteristics

15 Copyright © Allyn & Bacon (2010) Matched Random Assignment Match on relevant variables Match on relevant variables –Variables likely to affect the dependent measure –Variables that show the largest variability in the population Procedures Procedures –Match in sets on the relevant variable Set size is the number of groups in the study Set size is the number of groups in the study –Randomly assign participants from the set, one to each group –Keep track of matching data for the statistical analysis

16 Copyright © Allyn & Bacon (2010) Experimental Design Main focus of Chapters 10 through 13 Main focus of Chapters 10 through 13 Experimental design maximizes validity Experimental design maximizes validity –Need to also include the other control procedures covered in this chapter Key elements of experiments Key elements of experiments –One or more control groups –Random assignment of participants to groups

17 Ethical Principles Balanced Placebo Design raises several ethical issues Balanced Placebo Design raises several ethical issues –Alcohol is a controlled substance –Intoxication poses risks –Some individuals are at especially high risk (e.g., people with certain medical conditions) Must address these issues Must address these issues Copyright © Allyn & Bacon (2010)

18 Screening Participants Must assure that participants are legally old enough to consume alcohol Must assure that participants are legally old enough to consume alcohol Must screen out people who abuse alcohol and those with no experience with alcohol Must screen out people who abuse alcohol and those with no experience with alcohol Must exclude those with medical problems that might be exacerbated by alcohol Must exclude those with medical problems that might be exacerbated by alcohol Must inform potential participants that alcohol may be consumed, so those with moral objects can decline Must inform potential participants that alcohol may be consumed, so those with moral objects can decline Copyright © Allyn & Bacon (2010)

19 Deception Participants know that, depending on the condition, they may or may not be consuming alcohol Participants know that, depending on the condition, they may or may not be consuming alcohol –They agree to this in the informed consent –What they don’t know is that they may be deceived about the condition to which they are assigned Debriefing is required to clear up misconceptions Debriefing is required to clear up misconceptions Copyright © Allyn & Bacon (2010)

20 Protecting Participants Must have medical safeguards available in the event of an adverse reaction Must have medical safeguards available in the event of an adverse reaction Must assure that the participant does not drive intoxicated Must assure that the participant does not drive intoxicated –Usually by keeping them in the lab until the blood alcohol level has dropped Copyright © Allyn & Bacon (2010)

21 Summary Most threats to validity can be minimized with proper use of control procedures Most threats to validity can be minimized with proper use of control procedures Broad classes of control procedures Broad classes of control procedures –General control procedures –Control over subject and experimenter effects –Control through participant selection and assignment –Control through specific experimental design


Download ppt "Copyright © Allyn & Bacon (2010) Controls to Reduce Threats to Validity Graziano and Raulin Research Methods: Chapter 9 This multimedia product and its."

Similar presentations


Ads by Google