1. Internal Validity and Confounding Variables
Jenny and Nasim

2. Review - Terminology Independent Variable Dependent Variable
Researchers study as having a possible effect on one or more variables [1]
“mathematical variable that is independent of the other variables in an expression or function and whose value determines one or more of the values of the other variables” [2]
Dependent Variable
Variable that is maybe affected by independent variable [1]
“mathematical variable whose value is determined by that of one or more other variables in a function” [3]

3. Validity - Review
Source: Research Methods Knowledge Base
rchmethods.net/kb/intr oval.php

4. Was there an alternate explanation?
Internal Validity
Can the results of your experiment be attributed to the manipulation of the independent variable?
… or …
Was there an alternate explanation?

5. Internal Validity - Demonstrate Causality
Covariance - The Dependent Variable changes with the Independent Variable
Temporal Precedence - The change in the Independent Variable occurred before the change in the Dependent Variable
Did the cause occur before the effect?
No Plausible Alternatives
Confounding Variables - variation in the experiment other than the Independent Variable

6. Threats to Internal Validity
Single Group Experiment:
A: Run Experiment -> Measure Results
B: Measure Baseline -> Run Experiment -> Measure Results

7. Single Group - Threats to Internal Validity
History - Something outside of the study is causing the change
Maturation - Change in the measure would have occurred naturally over time
Testing - Pre-test might prepare participants for post-test
Instrumentation - Change in the measurement between pretest and posttest
Attrition - Participants dropping out of the study
Statistical Regression - Group that doesn’t measure close to the mean in the pre-test moves closer to it in the post-test

8. Statistical Regression
Also called Regression to the mean
Nonrandom sample of the population
Metric distribution is asymmetrical compared the population in one test
Next test shows movement towards the mean even when nothing changed
Occurs in both directions
Statistical Phenomenon
The result of random error
Not related to Instrumentation Bias or other experiment design issues
Group Phenomenon
Relative Phenomenon
Not related to the Maturation Threat

9. Statistical Regression - Example
A Health Science Graduate Student is studying the impact a specific set of anaerobic exercises on HDL levels in adults. After months of developing his exercise program, it’s ready for evaluation. The experiment requires participants to follow the exercise program for 10 weeks. Participants are asked not change their diets or add an additional exercise not specified by the program. The Student measures HDL levels both at the beginning and conclusion of the study. To his dismay, the results show no statistically significant change in the mean HDL Levels. After half a year of work, the student has nothing to write about!
The student start looking at different subsets in the test results and notices that the participants in the lowest quantile of pretest HDL value showed noticeable improvement in their posttest values. The research effort is saved and the student prepares to write his thesis.

10. Multiple Group Threats
Same threats as single group threat with the added factor of differing impact on each group
Focus on the differential changes between each group for each threat
History
Maturation
Testing
Instrumentation
Attrition
Statistical Regression

11. Social Threats Imitation of Treatment Compensatory Rivalry
Resentful Demoralization
Equalization of Treatment

12. Internal Validity - Example 1
A psychology research assistant is tasked with conducting a simple experiment that will take approximately 20 minutes. She decides recruit participants from the two classes goes to her advisors teaches; one is Introduction to the Practice of Psychology, PSY 110, offered at 5:45pm on Monday the other is a Graduate Seminar in Adolescent Psychotherapy, PSY 528, offer at 2:00pm on Tuesday. For convenience, she asks for 15 volunteers who can stay immediately after each class. She gets enough participants from each class and uses students from one class as a control and the others as the experimental group. She conducts the experiment with the same equipment, in similar classroom environments. Unfortunately, she the experiment runs longer than anticipated in both cases and 5 participants from the evening course have to leave before it’s finished to catch the last train home. The student still has data from 25 participants and decides to move forward with it.

13. Internal Validity - Example 2
A researcher is conducting a visual learning and retention experiment using Geography subject matter. He begins his research conducting a survey to identify regions of the world that would be least familiar to likely participants in his experiment. He then recruits participants who did not take his survey to attend seminars studying 1 of 2 geographical regions and randomly assigns them control and experimental groups for each region (a total of 4 groups in the study). He begins with a pretest for each group and concludes that the mean test scores are have similar distributions between control and experimental groups for both regions. This experiment requires groups to take a 4-week seminar, taking up approximately 30 minutes per week of the participant’s time. The participants are compensated with Starbucks gift cards to motivate them to continue taking the seminar. After the first week, a humanitarian crisis puts the one of the regions in the news regularly. Over the course of 4 weeks, the researcher found that participants in a group that meets at 6pm had attendance rates at 2/3 of the other groups and 25% more dropouts, this group was the control group for the region in the news. The pretest and posttest were similar to each other for both regions and the control and experimental group for region were administered the same test. In the end, the research found experimental group the region that wasn’t in the news showed noticeable improvement with the experimental learning program compared to the controlled, while control and experimental groups for the region in the news both showed similar improvements in test scores.

14. Mitigating Threats to Internal Validity
Use a control group
Hawthorne Effect
Use of a Placebo
Participants might be denied existing beneficial treatments or services
Requirements for
Informed consent must discuss use of the placebo
Participants in placebo group should be offered treatment afterwards, if it’s beneficial
Research should weigh value of study vs. risk to participants
Keep things constant
Constant factors can’t be confounding variables

15. Mitigating Threats to Internal Validity
Randomly assign groups
Any differences are due to chance
Use pretest(s)
Determine the equivalence of different groups
Useful when groups cannot be random
Expose participants to all treatments
Within-subjects variable - Independent variable is systematically manipulated for all participants
Statistically Control of Confounding Variables

16. References
Leedy, P. D., & Ormrod, J. E. (2016). Practical research: planning and design. Boston: Pearson.
Independent Variable. (n.d.). Retrieved April 30, 2017, from webster.com/dictionary/independent%20variable
Dependent Variable. (n.d.). Retrieved April 30, 2017, from webster.com/dictionary/dependent%20variable
Introduction to Validity. (n.d.). Retrieved May 01, 2017, from