1. Research Methods in Psychology Second Edition
Lecture Slides by
Dana B. Narter, Ph.D.

2. Three Claims, Four Validities: Interrogation Tools for Consumers of Research
Chapter 3
I spend three 50-minute class periods on this chapter.

3. Chapter Overview Variables Three claims
Interrogating the three claims using the four big validities
Prioritizing validities
This chapter sets up the framework for the textbook, so it is a very important chapter to understand. You will start by learning about what variables are. Then I will talk about the three claims (frequency, association, and causal) and the four validities (construct, external, statistical, internal) and how the validities are prioritized differently depending on what kind of claim you’re making.

4. Variables Variables vs. constants Measured and manipulated variables
From conceptual variable to operational definition
Variable: It’s something that changes or varies, so it needs to have at least two levels or values (but they can have more).
Examples from made-up headlines:
“80% of college undergraduates have sexted.” Sexting is the variable, and there are two levels: Those who have sent sexually explicit photos electronically and those who have not.
“Those who attend church regularly lie more often.” Two variables: church attendance (attend church at least once a month or do not attend church at least once a month) and lying (some numerical score on a lying inventory).
Constant: A constant does not vary. In other words, it stays the same. Made-up headline: “30% of women have been sexually harassed in the workplace.” Gender is a constant.
Next we will look at different types of variables: measured variables, manipulated variables, conceptual variables, and operationalized variables.

5. Measured and Manipulated Variables
Measured variables are observed and recorded.
Manipulated variables are controlled.
Some variables can only be measured—not manipulated.
Some variables can be either manipulated or measured.
Researchers either measure or manipulate variables in a study.
Measured variable: Levels are observed and recorded by the researcher. Examples: weight, eye color, age, cholesterol level. Psychologists also measure more abstract variables such as well-being, aggression.
Manipulated variable: A variable that the researcher controls, usually by assigning participants to different levels of that variable. Example: One group drinks no coffee before taking an exam, one group has one cup of coffee, and one group has two cups of coffee.
Some variables can only be measured—not manipulated. Example: You can’t assign participants to be a particular age, as age is a naturally occurring variable. Sometimes it is unethical to manipulate variables. Example: Providing one group of children with nutritious school lunches and another group of children with high calorie, high fat, and high sugar lunches.
Some variables can be either manipulated or measured. Example: You could measure mood by using an instrument such as the Beck Depression Inventory, and you can also manipulate mood through mood induction, asking some participants to think about a happy event and some to think about a sad event.

6. From Conceptual Variable to Operational Definition
Conceptual variables/constructs/conceptual definitions: abstract, theoretical concepts such as “infant temperament” or “anxiety.” I think of them as being “up in the clouds,” but we need to bring them down to the ground from the lofty theoretical level so that we may measure them.
Operational variables/operational definitions/operationalize: In order to test their hypotheses with empirical data, researchers need to develop operational definitions/operational variables. Operationalizing means turning a conceptual definition into a measured or manipulated variable. Example: conceptual variable = physical aggression and operational variable = the number of times a child pushes, hits, kicks and/or punches another child.

7. Operationalizing “Texting While Driving”
A conceptual variable can be operationalized in different ways.
Conceptual level = “texting while driving”
Operational definitions include direct observation, friends’ observations and a self-report questionnaire.

8. Three Claims Frequency claims Association claims Causal claims
Not all claims are based on research.
Claim: argument someone is trying to make. Psychological scientists use data to test and refine theories and claims.
Frequency claim: one variable
Association claim: two variables that are related
Causal claim: two variables in which one causes the other

9. Frequency Claims
Frequency claims describe a particular rate or degree of a single variable.
Frequency claims involve only ONE MEASURED VARIABLE.
Examples: Four out of five doctors recommend taking vitamin D; half of all college students have cheated on an exam.

10. Association Claims
Association claims argue that one level of a variable is likely to be associated with a particular level of another variable.
Association claims involve at least TWO MEASURED VARIABLES.
Variables that are associated are correlated.
Association claim: One level of a variable is likely to be associated with a particular level of another variable. Examples: Students who sit in the front row tend to score better on exams. Children who eat more green vegetables have more energy. Association claims involve at least two measured variables.
Correlate: covary or are related; as one variable changes, the other tends to change, too.
There are three basic types of associations: positive, negative, and zero. (Go to next slide.)

11. Positive Association
Positive association (a.k.a. positive correlation): high scores on one variable associated with high scores on other variable OR low scores on one variable associated with low scores on other variable. Example in Figure 3.2a: Shy people are better at reading facial expressions.
Scatterplot: One way to represent an association is to plot one variable on the x-axis and the other variable on the y-axis; mention that the slope of a positive association goes up as you move from left to right.

12. Negative Association
Negative association (a.k.a. inverse association or negative correlation): high scores on one variable associated with low scores on other variable OR low scores on one variable associated with high scores on other variable. Example: “People who multitask the most are the worst at it.” Describe the negative slope that decreases as you move from left to right across the scatterplot.

13. Zero Association
Zero association (a.k.a. zero correlation): no association between the two variables. Both high and low scores on one variable are associated with all levels of the other variable. There is no slope (i.e., slope is close to horizontal, slope = 0).
Example: screen time not linked to physical activity

14. Making Predictions Based On Associations
Some association claims are useful because they help us make predictions.
The stronger the association between the two variables, the more accurate the prediction.
Both positive and negative associations can help us make predictions, but zero associations cannot.
Some association claims are useful because they help us make predictions: Prediction = using the association to make our estimates more accurate; doesn’t necessarily mean predicting into the future
The stronger the association between the two variables, the more accurate the prediction: the weaker the relationship, the less accurate the prediction
Both positive and negative associations can help us make predictions, but zero associations cannot.

15. Causal Claims
Causal claim: One of the variables is responsible for changing the other; one measured variable and one manipulated variable. Like an association claim, the two variables covary.
Example: Not drinking enough milk causes a calcium deficiency.

16. Verbs for Association and Causal Claims
Verbs for casual claims: cause, enhance, curb
Verbs for association claims: link, associate, correlate, predict, tie to, being at risk for.
A causal claim that contains tentative language (could, may, seem, suggest) is still a causal claim. Example: Not getting enough sleep could result in lower school performance.

17. Not All Claims Are Based On Research
Not all claims we read about in the popular press are based on research.
Some claims are based on experience, intuition, or authority.
If you will recall from Chapter 2, there are other ways of knowing besides research (e.g., experience, intuition, authority). Not all claims that you see, hear, or read about in the popular media are based on research.
Examples:
“Here’s some help if you don’t want the bed bugs to bite”
“Saving man’s best friend: Tucson’s thriving animal rescue scene”
“Children on the run”

18. Interrogating the Three Claims Using the Four Big Validities
Interrogating frequency claims
Interrogating association claims
Interrogating causal claims
Valid: reasonable, accurate, and justifiable; validity is the appropriateness of a conclusion or decision.

19. The Four Big Validities
Introduce the four big validities in general before describing how they are interrogated for the three different types of claims.

20. Interrogating Frequency Claims
Construct validity
External validity (generalizability)
Statistical validity
When interrogating frequency claims, you will want to focus on construct validity and external validity. Statistical validity may be relevant as well.
Construct validity: how well a conceptual variable is operationalized. Example: 80% of college students have been depressed during the last year. How was depression operationally defined? Were college students simply asked if they’d ever been depressed during the last year, or was the frequency claim based on a score on a depression inventory, or based on records from psychological services on campus.
External validity: how well the results of a study represent the people or contexts besides those in the study itself; a.k.a. generalizability. Example: 40% of Americans have cheated on their significant others. How were these participants chosen?
Statistical validity (a.k.a. statistical conclusion validity): the extent to which the study’s conclusion are reasonable and accurate. The questions asked will vary depending on the claim. When interrogating frequency claims, percentages are usually accompanied by a margin of error (a statistic based on sample size which indicates where the true value in the population probably lies. For example, if the margin of error is +/- 5 percentage points, then it means that the true percentage of “depressed college students” probably lies between 75–85%.

21. Interrogating Association Claims
Construct validity
External validity
Statistical validity
Reminder: Association claims involve the relationship between two measured variables. Interrogate construct, external, and statistical validities.
Construct validity: similar to construct validity for a frequency claim except that you now have TWO conceptual variables that need to be operationalized appropriately. Example: People who multitask are the worst at it. How were the frequency and ability to multitask measured? Ability to multitask could be measured by self-report (How good are you at multitasking?) or by an observational measure in which participants are given a task that requires them to multitask. Frequency of multitasking could also be self-report (keep track of how many times you multitask) or by observing participants.
External validity: Does the association claim generalize to other populations, contexts, times, or places? Example: In the multitasking example, who were the participants? Were they college undergraduates? If so, do you think the findings might generalize to later in adulthood? In terms of generalizing to other contexts, let’s say the two tasks you used to assess multitasking ability were a mathematical task and a verbal task. Would the results generalize to two physical tasks, for example?
Statistical validity: When referring to an association claim, statistical validity is the extent to which the statistical conclusions are accurate and reasonable. There are two aspects of statistical validity to consider: strength and statistical significance AND two kinds of mistakes. (See next slide for details.)

22. Statistical Validity of Association Claims
Strength and significance
Avoiding two mistaken conclusions
Type I error
Type II error
Strength: How strong is the association? Some associations are strong and others are weak. Remember that the stronger the association, the more accurate our predictions will be.
Significance: If an association is statistically significant, then the result is probably not due to chance based on that sample. If an association isn’t statistically significant, then the result probably is due to chance.
Avoiding two mistaken conclusions: When interrogating the statistical validity of an association claim, there are two kinds of mistakes that can be made.
Type I error: false positive; a study might mistakenly conclude that there is an association between two variables in their sample when there actually is no association in the population. You want to increase the chances that you will find an association only when there really is an association.
Type II error: a “miss”; a study might mistakenly conclude from a sample that there is no association between two variables when there actually is an association in the population. You want to minimize/reduce the chances missing associations that are really there.

23. Table 3.5: Interrogating the Three Types of Claims Using the Four Big Validities
This table is a summary of how we interrogate the three claims (frequency, association, and causal) using the four big validities (construct, statistical, internal, and external).

24. Interrogating Causal Claims
Three criteria for causation
Covariance
Temporal precedence
Internal validity
Because causal claims state that one variable causes another variable, they use directional verbs like leads to, affects, and influences.
There are three criteria for causation:
1. Covariance is simply that the two variables are related. Association claims fulfill this criterion.
2. Temporal precedence: one variable comes before the other variable in time. Because the research is manipulating one variable and then measuring the other variable, she knows the manipulated variable comes before the outcome variable, which is measured after the manipulation.
3. Internal validity (a.k.a. third variable criterion): a study should be able to eliminate alternative explanation. In other words, Variable A is the only thing that changed.

25. Experiments Can Support Causal Claims
Independent variable
Dependent variable
Random assignment
In order to support a causal claim, a researcher needs to conduct an experiment in which one variable is manipulated and the other is measured.
Independent variable: manipulated or variable (cause) in an experiment (e.g., type of lessons (four levels: keyboard, voice, drama, no lessons).
Dependent variable: measured variable (effect) in an experiment (e.g., IQ)
Random assignment: A method of assigning participants to levels of the independent variable such that each group is as similar as possible; flipping a coin or rolling a die; increases internal validity by controlling for potential alternative explanations. Example: The lessons caused the differences in IQ and not something else because IQ was similar in the four groups due to random assignment.

26. When Causal Claims Are A Mistake
Do family meals really curb eating disorders?
Do early language skills reduce preschool tantrums?
Sometimes we read an article in the popular press that sounds like it’s a causal claim when it’s not. Here are two examples:
Do family meals really curb eating disorders? Covariance: Yes, there is an association between family meals and eating disorders. Temporal precedence: Did the family meals increase BEFORE the eating disorders decreased? Not clear. Internal validity: No. We can’t rule out third variable explanations without an experiment.
Do early language skills reduce preschool tantrums? It sounds causal, but is it? Covariance: Yes. Good language skills go with reduced anger during the waiting task. Temporal precedence: Yes. Language skills measured at 18 months; waiting task conducted at 4 years. Internal validity: No. This is not an experiment, so we cannot rule out possible alternative explanations.

27. Other Validities To Interrogate In Causal Claims
Construct validity
External validity
Statistical validity
We have been emphasizing internal validity because it is only important when interrogating causal claims. However, we still need to consider the three other types of validity.
Construct validity: always important regardless of what kind of claim you’re making; need to consider the construct validity of both the manipulated variable and the measured variable
External validity: Do the results generalize to other people or contexts? This probably isn’t paramount since internal validity tends to be emphasized in experiments, and internal and external validity are kind of like two sides of a coin. As internal validity increases, external validity decreases and vice versa.
Statistical validity: How strong is the relationship between the IV and the DV? Is the difference between groups statistically significant?

28. Prioritizing Validities
Which of the four validities is the most important?
It depends on what kind of claim the researcher is making and what her priorities are.
It’s impossible to find a study that satisfies all four validities at once. But depending on the goals of the study, researchers decide what are their priorities. Example: Internal validity is typically a top priority when making causal claims but not for making frequency or association claims.

29. This concludes the Lecture Slides for Chapter 3
Research Methods in Psychology Second Edition by
Beth Morling
For more resources to accompany this text, see wwnorton.com/instructors and everydayresearchmethods.com.