Presentation on theme: "Internal Validity EAF 410 July 25, 2001 Is the observed relationship due to something other than what I studied?"— Presentation transcript:
Internal Validity EAF 410 July 25, 2001
Is the observed relationship due to something other than what I studied?
Many alternative hypotheses. They are “threats to internal validity.”
Validity -- 3 types b Instrument (measurement) b External (generalization) b Internal (ambiguity)
Internal Validity b Observed relationship among two or more variables unambiguous. b Relationship not due to “something else.”
Threats to Internal Validity
Subject Characteristics b Individuals/groups differ in unintended ways agegenderagegender ethnicityintelligenceethnicityintelligence attitudevocabularyattitudevocabulary
Mortality b Loss of subjects drop outdrop out don’t complete testsdon’t complete tests don’t complete questionnairesdon’t complete questionnaires
b Limits generalizability b Introduces bias different responses from those who dropped outdifferent responses from those who dropped out b What can be done about this threat?
Location b Influence of where research is conducted resourcesresources environmentenvironment powerpower b Hold location constant
Instrumentation b Instrument Decay nature of instrument changednature of instrument changed –scoring changes –item changes
Instrumentation b Data Collector Characteristics characteristics of researcher affects datacharacteristics of researcher affects data –gender –age –ethnicity –language
Instrumentation b Data Collector Bias unconsciously distort dataunconsciously distort data –treat groups differently –leading questions –judgement/interpretation Standardize proceduresStandardize procedures Planned ignorancePlanned ignorance
Testing b Improvement due to pretest subjects “alerted” to topic by pretestsubjects “alerted” to topic by pretest practice on pretest helpspractice on pretest helps
History b Unanticipated, unplanned for events occur during study b Individual/group experiential differences
Maturation b Change due to passing of time b Serious threat to studies using pre-post data for intervention group b Control with well-selected comparison group
Attitude of Subjects b How subjects view the study b Hawthorne Effect effects because of increased attention to subjectseffects because of increased attention to subjects
Regression b Group selected because of low or high scores will, on average, score closer to the average on subsequent tests b Control with equivalent comparison group
Implementation b Experimental group treated in unintended ways different individuals implement different treatmentsdifferent individuals implement different treatments personal bias for one treatmentpersonal bias for one treatment
Finding a Relationship b Threats to internal validity can reduce likelihood of finding a relationship
Minimize the Threats b Standardize study conditions b Obtain subject information b Obtain study details where, when, extraneous eventswhere, when, extraneous events b Choose appropriate design
b Threats reduced by planning. b Planning requires collection of information before the study begins. b Often too late to control them once the data have been collected.