1. Experimental Research Design Causality & Validity Threats to Validity –Construct (particular to experiments) –Internal –External – already discussed

2. Threats to Construct Validity Mono Method bias –Remove common method variance –Establish convergent & discriminant validity Hypothesis guessing Inappropriate level of construct –Usually lower than what occurs in natural setting

3. Threats to Internal Validity Time –History, Maturation Measurement –Testing, Instrumentation Selection –Randomization, Statistical regression, Mortality Contamination –Diffusion, Compensation Interaction among threats

4. Internal Validity Establish that cause led to the effect –Identify non-experimental (uncontrolled) factors that affect the dependent variable or the relation b/w the independent and dependent variables

5. 10 Threats to Internal Validity Time History –Factors occurring along with the independent variable An event occurring at the time of the intervention that overshadows (or reduces the effect of iv) and causes changes in dv –E.g., change in manufacturing material along with training influences performance quality

6. FIGURE 7.1 Copyright © 2003 John Wiley & Sons, Inc. Sekaran/RESEARCH 4E

7. 10 Threats to Internal Validity Time Maturation –Biological or psychological processes or factors that occur w/the passage of time & affect the dependent variable E.g., job experience, socialization etc. affect performance

8. FIGURE 7.2 Copyright © 2003 John Wiley & Sons, Inc. Sekaran/RESEARCH 4E

10. 10 Threats to Internal Validity Measurement Testing –Studies w/pre-post measures or multiple measures Increased or decreased sensitization to post- test measures Changes in participants due to measurement itself Increased fatigue or resentment Self-consistency effects

11. 10 Threats to Internal Validity Instrumentation –Change in the measures from pre to post test Change in observers’ frames of reference Changes in participants’ frames of reference (e.g., beta & gamma changes p. 43 Saks) Change in instrumentation quality (e.g., performance quality measurement)

12. 10 Threats to Internal Validity Instrumentation –Use of different measurement techniques Establish expected convergent & discriminant validity Importance of using different measures to overcome percept-percept problem

13. Threats to Internal Validity Selection Randomization –Randomly assign to experimental conditions Only in experimental designs Matching on key variable if random assignment not possible

14. Threats to Internal Validity Selection Randomization –Randomly select from population Calculate response rate Establish comparability of respondents to population on key features Stratified (v. cluster?) sampling

15. 10 Threats to Internal Validity Selection Statistical Regression –Upon repeat measurement, scores usually “regress” or “move toward” the mean Usually when extreme scorers are selected into the study –Can be classified as ‘error’ variation

16. 10 Threats to Internal Validity Selection Mortality –Features of participants who do not complete the study affect the results –Reduced power in testing effects

17. 10 Threats to Internal Validity Contamination Diffusion/imitation of treatments –When people in control groups inadvertently know about or are affected by intervention and that changes their behavior

18. 10 Threats to Internal Validity Contamination Compensatory Effects –Knowledge about intervention and changes in control group members’ behavior is more purposeful

19. 10 Threats to Internal Validity Interaction among threats –When multiple threats operate at the same time obscuring the effect of the IV on the DV

20. Threats to Internal Validity Time-related –History, maturation Measurement –Testing, Instrumentation Selection –Randomization, Statistical regression, mortality Contamination –Diffusion, Compensation Interaction among threats