1. Psych 231: Research Methods in Psychology
Finishing up
Psych 231: Research Methods in Psychology

2. Announcements This is the final new content lecture Labs this week
Thursday (optional) – Course overview, Review, Q&A, etc.
Turn in extra-credit journal summaries
Make sure that your SONA credits are assigned where you want them
Labs this week
Poster presentations
Turn in group ratings sheets
Turn in the GP: results and discussion sections
Announcements

3. Non-Experimental designs
Sometimes you just can’t perform a fully controlled experiment
Because of the issue of interest
Limited resources (not enough subjects, observations are too costly, etc).
Surveys
Correlational
Quasi-Experiments
Developmental designs
Small-N designs
This does NOT imply that they are bad designs
Just remember the advantages and disadvantages of each
Going to finish up after the stats lectures
Non-Experimental designs

4. Quasi-experiments What are quasi-experiments?
Almost “true” experiments, but with an inherent confounding variable
Typically, they lack random assignment for one of the independent variables
General types
An event occurs that the experimenter doesn’t manipulate or have control over
Interested in subject variables
Time is used as a variable
Commonly used with
Developmental Designs
Non-equivalent control group designs
In Program Evaluations
Quasi-experiments

5. Quasi-experiments Nonequivalent control group designs
with pretest and posttest (most common)
(think back to the second control lecture)
participants
Experimental
group
Control
Measure
Non-Random
Assignment
Independent Variable
Dependent Variable
But remember that the results may be compromised because of the nonequivalent control group (review threats to internal validity)
Quasi-experiments

6. Quasi-experiments Program evaluation
Systematic research on programs that is conducted to evaluate their effectiveness and efficiency.
e.g., does abstinence from sex program work in schools
Steps in program evaluation
Needs assessment - is there a problem?
Program theory assessment - does program address the needs?
Process evaluation - does it reach the target population? Is it being run correctly?
Outcome evaluation - are the intended outcomes being realized?
Efficiency assessment- was it “worth” it? The the benefits worth the costs?
Quasi-experiments

7. Quasi-experiments Advantages Disadvantages
Allows applied research when experiments not possible
Threats to internal validity can be assessed (sometimes)
Disadvantages
Threats to internal validity may exist
Designs are more complex than traditional experiments
Statistical analysis can be difficult
Most statistical analyses assume randomness
Quasi-experiments

8. Non-Experimental designs
Sometimes you just can’t perform a fully controlled experiment
Because of the issue of interest
Limited resources (not enough subjects, observations are too costly, etc).
Surveys
Correlational
Quasi-Experiments
Developmental designs
Small-N designs
This does NOT imply that they are bad designs
Just remember the advantages and disadvantages of each
Non-Experimental designs

9. Small N designs What are they?
In contrast to Large N-designs (comparing aggregated performance of large groups of participants)
One or a few participants
Data are typically not analyzed statistically; rather rely on visual interpretation of the data
Historically, these were the typical kind of design used until 1920’s when there was a shift to using larger sample sizes
Even today, in some sub-areas, using small N designs is common place
(e.g., psychophysics, clinical settings, animal studies, expertise, etc.)
Small N designs

10. Small N designs Baseline experiments – the basic idea is to show:
= observation
Treatment introduced
Steady state (baseline)
Baseline experiments – the basic idea is to show:
Observations begin in the absence of treatment (BASELINE)
Essentially our control/comparison level
Then treatment is implemented and changes in frequency, magnitude, or intensity of behavior are recorded
Small N designs

11. Small N designs Baseline experiments – the basic idea is to show:
= observation
Reversibility
Treatment removed
Transition steady state
Steady state (baseline)
Treatment introduced
Baseline experiments – the basic idea is to show:
When the IV occurs, you get the effect
When the IV doesn’t occur, you don’t get the effect (reversibility)
This allows other comparisons, to the original baseline as well as to the transition steady state
Small N designs

12. Unstable
Stable
Before introducing treatment (IV), baseline needs to be stable
Measure level and trend
Level – how frequent (how intense) is behavior?
Are all the data points high or low?
Trend – does behavior seem to increase (or decrease)
Are data points “flat” or on a slope?
Small N designs

13. ABA design ABA design (baseline, treatment, baseline)
Steady state (baseline)
Transition steady state
Reversibility
ABA design (baseline, treatment, baseline)
The reversibility is necessary, otherwise
something else may have caused the effect
other than the IV (e.g., history, maturation, etc.)
There are other designs as well (e.g., ABAB see figure13.6 in your textbook)
ABA design

14. Small N designs Advantages
Focus on individual performance, not fooled by group averaging effects
Focus is on big effects (small effects typically can’t be seen without using large groups)
Avoid some ethical problems – e.g., with non-treatments
Allows to look at unusual (and rare) types of subjects (e.g., case studies of amnesics, experts vs. novices)
Often used to supplement large N studies, with more observations on fewer subjects
Small N designs

15. Small N designs Disadvantages
Difficult to determine how generalizable the effects are
Effects may be small relative to variability of situation so NEED more observation
Some effects are by definition between subjects
Treatment leads to a lasting change, so you don’t get reversals
Small N designs

16. Small N designs Small vs. Large N debate
Some researchers have argued that Small N designs are the best way to go.
The goal of psychology is to describe behavior of an individual
Looking at data collapsed over groups “looks” in the wrong place
Need to look at the data at the level of the individual
Small N designs

17. Course Review: The Research Process

18. Course Review: The Research Process
Slide from Day 1
Course Review: The Research Process

19. Course Review: The Research Process
Presenting your work
Get an idea
A set of skills leading to knowledge & understanding
A way of thinking (beware small samples, correlation is not causation, etc.)
A way of life?
Stats.org: Stats in the news
Would you recommend that people should know basic research methodology (and statistics)? Why?, what benefits do you think that they get from it?
Why do you think that it is important in psychology; why do we require so many methods and statistics courses for our majors?
Is knowing this “set of skills” important only for doing research? Or is it equally important for consuming (reading or hearing about) research?
Is it important to recognize the difference between “science” and “pseudoscience”? Does knowing the “set of skills” introduced here help with that?
Do you find the scientific method lead to strong, convincing argumentation? How does it compare to other sources of knowledge (e.g., authority, faith, intuition, etc)?
Course Review: The Research Process

20. The Research Process Get an idea Often the hardest part
No firm rules for how to do this
Observations
Past research
Review the literature
The Research Process

21. The Research Process Review the literature What has already been done?
What variables have people looked at
What hasn’t been looked at
How are other experiments in the area done?
What methods are used?
To measure the dependent variable
To manipulate the independent variable
To control extraneous variables
The Research Process

22. The Research Process Formulate a testable hypothesis
What is a hypothesis?
A predicted relationship between variables
What does it mean to be testable?
Must be falsifiable
Can it be replicated
Must be able to observe/measure (and manipulate for experiments) the variables
Directly
Indirectly
Operational definitions
The Research Process

23. The Research Process Design the research What method?
Experiment, Survey, Developmental designs, …
What kind of comparisons are used
Control groups
Baseline conditions
What are your variables?
How many levels of your Independent variable(s)
How do you measure your dependent variable(s)
What can be done to control for biases and confounds?
The Research Process

24. The Research Process Collect Data Importance of pilot research
Who do you test?
What is your population?
Your sample?
Your sampling method?
The Research Process

25. The Research Process Analyze the data Design drives the statistics
Understanding Variables and variability
Descriptive statistics (summarizing)
Means, standard deviations
Graphs, tables
Correlation
Inferential statistics (drawing conclusions)
What kind of analysis is appropriate for your design
T-tests
ANOVA
Between or within versions
The Research Process

26. The Research Process Interpret the results
Correlation versus causation
Reject or fail to reject null hypotheses
Statistical vs. theoretical significance
Support or refute the theory (or revise)
Generalizability of the results
The Research Process

27. The Research Process Present the results
Getting the research “out there”
Conference presentations
Posters
Talks
Written reports
APA style
Supports clarity
The Research Process

28. The Research Process Repeat
Each set of results leads to more research questions
Refine the theory
Test a refined theory
Test alternative explanations
The Research Process

29. Reviewing for the final exam
1:00P
It is cumulative, covers the entire course. The majority is on new material (roughly 65%), the rest is material covered on Exams 1 & 2.
All multiple choice/scantron for the final
Reviewing for the final exam

30. Reviewing for the final exam
Final 1/3 of the course
Non experimental methods
Survey, correlational, & developmental
Statistics
Descriptive
Inferential
Presentations
Papers, Posters, & Talks
Reviewing for the final exam

31. Reviewing for the final exam
First 2/3 of the course
Scientific method
Getting ideas
Developing (good) theories
Reviewing the literature
Psychological Science
Ethics
Basic methodologies
APA style
Underlying reasons for the organization
Parts of a manuscript
Variables
Sampling
Control
Experimental Designs
Vocabulary
Single factor designs
Between & Within
Factorial designs
Reviewing for the final exam