1. End of Lecture? Active Learning Works Across STEM Disciplines
Sarah L. Eddy

2. My Context: What does this mean?
It means having education researchers embedded in a discipline. At FIU we have education researchers who focus on how to teach physics, computer sicence, earth and environmental sciences, math, chemistry and, of course, biology.

3. Outline What is the evidence for active learning?
Can active learning also reduce achievement gaps between student groups?
How does active learning produce these outcomes?

4. What is the Evidence for Active Learning?
Calls from national agencies to incorporate active learning into all college STEM classrooms.
But what is the evidence for these calls?
Needed a synthesis of experiments on STEM college classrooms

5. What is the Evidence for Active Learning?
Turn to your neighbor:
Define Active Learning
1 minute

6. What is the Evidence for Active Learning?
Active Learning (a working definition):
Engages students in the process of learning through activities and/or discussions in class
Emphasizes higher order thinking
Often involves groupwork
Traditional Lecture (control condition):
Students passively listen to expert
Methods: Meta-analysis
Hand searched 55 STEM education journals from 6/1/1998 to 1/1/2010
642 initial studies identified
225 studies met criteria for rigor
Freeman, Eddy et al. 2014
PNAS

7. What is the Evidence For active Learning?
Highly
Collaborative
Effort
Dozie Okoroafor
Michelle Smith
Scott Freeman
Mary Pat
Wenderoth
Miles
McDonough
Hannah Jordt

8. What is the Evidence For Active Learning?
95%
confidence
interval
Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture?
Estimate of
Effect Size
Effect Size:
Hedges’s g = Mean Performance in Trt – Mean in Control
Pooled SD
Number of
Studies
Two groups
not statistically
different
from each other
Freeman, Eddy et al. 2014
PNAS

9. What is the Evidence For Active learning?
Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture?
Yes! Student in Active Learning classes perform almost ½ a standard deviation better on identical assessments than students in Traditional Lecture classes.
Freeman, Eddy et al. 2014
PNAS

10. What is the Evidence for Active Learning?
Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture?
Putting results in context:
Collected SDs for exam performance from multiple years of three introductory STEM courses at UW.
In intro STEM, 6% increase in exam scores; 0.3 increase in average course grade (or B to B+, C+ to B, etc).
In addition in a study looking at number of students who fail or withdraw from course, students in lecture classes were 1.5 time more likely to fail than students in active learning courses.
And what’s most exciting is that when this study came out it actually started conversations about teaching around the world both in the STEM community and with the general public. In terms of academic press, it was written up in Schience Insider, was featured in the Chronicle of Higher Ed and Inside Higher Ed. The paper itself has been downloaded almost 100,000 times and even though its only 2 years old its been cited over 400 times.
It also had an impact in with the general public. It has been featured in new york times articles, it has had write ups in iternational papers, and it is probably the one time in my life that I will make it to the front page of reddit.
Effect Size in Standard Deviations

11. What is the evidence for active learning?
Question: Does active learning decrease Ds, Fs earned in and/or withdrawals from courses across STEM disciplines relative to traditional lecture?
Yes: Risk difference = 12.0%
Avg. Traditional: 33.8% DFW
Avg. Active Learning: 21.8% DFW
Students 1.5 times more likely to fail in traditional course than an active learning course.
Risk Difference:
% DFW in Control – % DFW in Trt

12. What is the Evidence for active learning?
Question: Does active learning decrease Ds, Fs earned in and/or withdrawals from courses across STEM disciplines relative to traditional lecture?
Putting results in context:
Meta-analysis of biomedical trials showed trials stopped early for benefit when:
P-value < 0.001
Our p-value: p<< 0.001
relative risk: 0.53 ( )
Our relative risk: 0.64
In our sample: 3,516 fewer students would fail
Pocock 2006

13. Outline What is the evidence for active learning?
Increases student performance by ½ SD (~6%)
12% fewer students would fail
Can active learning also reduce achievement gaps between student groups?
How does active learning produce these outcomes?
Active
Learning
Increased
Student
Performance
Decreased
DFW rates

14. What is the Evidence for Active Learning?
Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture?
Putting results in context:
Collected SDs for exam performance from multiple years of three introductory STEM courses at UW.
In intro STEM, 6% increase in exam scores; 0.3 increase in average course grade (or B to B+, C+ to B, etc).
In addition in a study looking at number of students who fail or withdraw from course, students in lecture classes were 1.5 time more likely to fail than students in active learning courses.
And what’s most exciting is that when this study came out it actually started conversations about teaching around the world both in the STEM community and with the general public. In terms of academic press, it was written up in Schience Insider, was featured in the Chronicle of Higher Ed and Inside Higher Ed. The paper itself has been downloaded almost 100,000 times and even though its only 2 years old its been cited over 400 times.
It also had an impact in with the general public. It has been featured in new york times articles, it has had write ups in iternational papers, and it is probably the one time in my life that I will make it to the front page of reddit.
Effect Size in Standard Deviations

15. What is the Evidence for Active Learning?
Given this new information, what more do you want to know before beginning to conclude that Active Learning improves learning for all students?
What questions remain for you?
1 minute
Studies in Meta-analysis:
Most done at one type of institution: selective R1s.
Majority of students at these institutions: White Americans.
Looked at instruction as a binary – really it’s a range.
Pooled across all types of active learning.
Freeman, Eddy et al. 2014
PNAS

16. Outline What is the evidence for active learning?
Increases student performance by ½ SD (~6%)
Decreases failure rates by 12%
Can active learning also reduce achievement gaps between student groups?
How does active learning produce these outcomes?
Active
Learning
Increased
Student
Performance
Decreased
DFW rates

17. Can Active learning Close achievement gaps?
Why might active learning work differently for different students?
Students differ in their preparedness for college
High schools differ in how well they prepare students for college (ETS 2015)
Families differ in how much they can help students prepare and adjust to college
Students come from different cultural contexts
Interdependence vs. Independence: Latina/o, Asian, and Working Class cultures > White and Middle Class cultures (Stephens et al. 2014; Kim and Markus 2002; Crisp 2014)
Talking While Learning: Black, Working Class cultures > White or Middle Class > Asian (Kim and Markus; Hitchcock 2011)
Prediction: Active learning will help students from historically underrepresented groups in science more than those from majority groups.

18. Can active learning Reduce achievement gaps?
Context:
Large Southeastern US university
Course: all of biology in one semester
Student population:
Non-majors and underprepared majors
Race/Ethnicity:
Native American: 1.1%
Asian: 7.4%
10.3%
Black: 13.9%
White: 59%
Undeclared, Mixed or International: 8%
Generation Status: 24% First-Generation
Gender: 66.3% Female
From chemical reactions in cells to global warming all in one semester
Kelly Hogan, Senior Lecturer
Assistant Dean for the Office of Instructional Innovation, College of Arts & Sciences

19. Can Active learning Reduce achievement gaps?
Structure:
Graded Preparatory Assignments
Student
in-class engagement
Increased Structure
(Active Learning)
Online homework
Students working on problems 34.5% of course time
Low structure (Lecture)
None or
< 1 per week
Talk < 15%
of course time
Kelly Hogan facilitating her class

20. Can active learning Reduce achievement gaps?
Study Design:
Low Structure: 3 terms
Increased Structure: 3 terms
Statistical methods:
Outcome: Total exam points earned
Checked for exam equivalency between terms: Cognitive task exam questions ask them to do
Model: Linear Regression
Controlled for: Student SAT score
Kelly Hogan facilitating her class

21. Can active learning Reduce achievement gaps?
Predicted Performance based on regression model. Students matched by SAT score, gender, Term.

22. Can Active learning Reduce Achievement gaps?
Yes! Everyone sees an increase in exam performance with ‘increased structure’, but Black students and First-Generation experience an additional increase.
3.2%
6.3%
3.7%
6.1%
Halves the achievement gap
with White students
Closes the achievement gap with
Continuing-Generation students
Performance Predicted based on regression model. Students matched by SAT score, gender, Term.

23. Outline What is Discipline Based Education Research?
What is the evidence for active learning?
Increases student performance by ½ SD (~6%)
Decreases failure rates by 12%
Can active learning also reduce achievement gaps between student groups?
Decreases gap between Black and White students
Closes gap between first- and continuing generation students
How does active learning produce these outcomes?
Structured Learning Environment that meets best practices for how people learn
Classroom climate where students feel safe to focus on learning
Active
Learning
Increased
Student
Performance
Decreased
DFW rates

24. Reduces Achievement Gaps
Outline
Active
Learning
What is the evidence for active learning?
Increases student performance by ½ SD (~6%)
Decreases failure rates by 12%
Can active learning also reduce achievement gaps between student groups?
Decreases gap between Black and White students
Closes gap between first- and continuing generation students
How does active learning produce these outcomes?
Structured Learning Environment that meets best practices for how people learn
Classroom climate where students feel safe to focus on learning
Reduces Achievement Gaps
Increased
Student
Performance
Decreased
DFW rates

25. How does active learning produce outcomes?
Why should we care about the ‘how’?
We want people to be able to adapt intervention to meet their needs, but these adaptions need to be wise
Can flexibility around surface features
Need to have fidelity of implementation of critical features
Understanding how something works helps us figure out what is a critical and what is a surface feature

26. How does active learning produce outcomes?
Structure:
Graded Preparatory Assignments
Student
in-class engagement
Graded
Review
Increased Structure
(Active Learning)
Online homework
(one per week)
Students working on problems 34.5% of course time
Optional:
1 per week
Low structure (Traditional Lecture)
None or
< 1 per week
Talk < 15%
of course time
High Structure
(Haak et al. 2011)
>1 per week
> 40% of course time
≥ 1 per week
Not the only instance of high structure reducing achievement gaps

27. How does active learning produce outcomes?
Why do you think the changes in these courses led to greater student achievement?
Come up with a hypothesis for at least 2 that is 10 words or less.
2 minutes
Structure:
Graded Preparatory Assignments
Student
in-class engagement
Graded
Review
Increased Structure
(Active Learning)
Online homework
(one per week)
Students working on problems 34.5% of course time
Optional:
1 per week
Low structure (Traditional Lecture)
None or
< 1 per week
Talk < 15%
of course time
High Structure
(Haak et al. 2011)
>1 per week
> 40% of course time
> 1 per week
What’s your 10 word hypothesis? And talk to 5 folks? Or ask who else said that?

28. How does Active learning Produce outcomes?
Why did ‘increased structure’ increase student performance?
We see hints from a classroom survey:
Administered survey in one semester of low structure and three semesters of increased structure.
Survey focused on course related behaviors and attitudes:
How students allocate time outside of class
Perception of classroom climate
Value students saw in class
Kelly Hogan facilitating her class

29. How does Active Learning Produce outcomes?
How did ‘increased structure’ increase student performance?
All students:
2.6 times more likely to spend more hours studying a week for course.
2 times more likely to read textbook before coming to class.
2 times more likely to see class as a community.
Specifically Black students:
More likely to participate in class (White students did not change).
See more value in out of class work than White Students.
From 1-3 hrs to 4-7
From rarely to sometimes
Kelly Hogan facilitating her class

30. How does active learning produce outcomes?
Hypothesis 1: Structures course so students employ best practices for learning.
Required Preparatory and Review assignments:
Distributed learning vs. cramming
Testing effect
In class practice:
Elaborative interrogation and Self-explaining
Students explain to themselves/their group in their own words why something is true when answering questions
“Testing… improves learning” even when you get the answer wrong
“practice testing facilitates the encoding of more effective mediators (i.e., elaborative information connecting cues and targets) during subsequent restudy opportunities”
“Supports the integration of new information with existing prior knowledge”

31. How does active learning produce outcomes?
Other increased structure replications fail to reduce performance gaps:
Comprehensive University in Midwest Replication:
High Structure (modified)
4% gains for all students
No impact on Black/White exam performance gap
Caveat: Did reduce failure rate gap
Community College in PNW Replication:
Moderate structure
3.7% gains for all students
No impact on achievement gap
Casper, Eddy et al., in prep
Pape-Lindstrom, Eddy et al., in prep

32. How does active learning produce outcomes?
Hypothesis 2: Creates a climate more conducive to risk taking which is important for learning.
Norms: implementations of active learning can establish a norm of ‘sense making’ rather than ‘answer-making’ (Turpen and Finkelstein 2009)
Relationships: Student perception of their relationship to the instructor, and their classmates can impact outcomes:
Feeling the instructor cares about students increases how much student enjoys subject and how much they learn (Ellis, 2000; 2004; Goodboy & Myers, 2008)
When students believe the instructor believes they can improve they are more motivated and their performance increases ( Good et al. 2012; Cohen 1999)
Climate may be particularly important for students from historically underserved populations.
Identity Threat

33. Structured Learning Environment
Outline
Active
Learning
What is Discipline Based Education Research?
What is the evidence for active learning?
Increases student performance by ½ SD (~6%)
Decreases failure rates by 12%
Can active learning also reduce achievement gaps between student groups?
Decreases gap between Black and White students
Closes gap between first- and continuing generation students
How does active learning produce these outcomes?
Structured Learning Environment that matches best practices for how people learn
Classroom climate where students feel safe to focus on learning
Structured Learning Environment
Perceived
Classroom
Climate
Increased
Student
Performance
Increased
Student
Performance
Decreased
DFW rates

34. Thank You and Questions
Thank you to:
The students and instructors who agreed to participate in these studies.
Collaborators on Gender Projects:
Sara Brownell, Bio Educ, Arizona State Univ
Mary Pat Wenderoth, Bio Educ, Univ of Washington (UW)
Dan Grunspan, Anthropology, UW
Steve Goodreau, Anthropology, UW
Collaborators on Values Affirmation project:
Hannah Jordt, Biology, UW
Scott Freeman, Bio Educ, UW
Collaborator on Persistence Patterns Across STEM disciplines:
Erin Dolan, Bio Educ, Univ Georgia
Collaborators on student motivation projects:
Lisa Corwin, Bio Educ, UC Boulder
Mark Graham, Educ Psych, Yale
Melissa Aikens, Bio Educ, Univ New Hampshire
Chris Runyon, Bio Educ, UT Austin
Collaborators on Active Learning in introductory Biology projects:
Pam Pape-Lindstrom, Biology, Everrett Community College
Anne Casper, Biology, Eastern Michigan University
Bill Hoese, Biology, Cal State Fullerton
Carol Pollock, Bio Educ, University of British Columbia
Jerry Timbrook, Quantitative Educ Psych, University of Nebraska-Lincoln
Alison Crowe, Bio Educ, UW
Ben Wiggins, Educ, UW

36. Methods: A quantitative review of the literature
7 criteria for inclusion of a study:
Contrast any active learning intervention with traditional lecturing (same class and institution).
Occurred in a regularly scheduled course for undergrads.
Involve changes in the conduct of class sessions (or recitation/discussion).
Involved a course in Astronomy, Bio, Chem, CompSci, Engineering, Geo, Math, Physics, Psych, Stats.
Included data on some aspect of academic performance—exam/concept inventory scores or failure rates (DFW).
If measure achievement: assessment equivalent in both treatments.
Presents data necessary to compute test statistic for meta-analysis.
Meta-analysis: a quantitative statistical analysis of separate but similar studies in order to test the pooled data for statistical significance.
6: equivalence = identical assessment, questions draw at random from a shared test bank, assessed as equivalent by a third party blind to study hypothesis