1. Assessing GE Goals and Capacities with Concept Inventories: One Path Forward . . . Part I
Overview: General Education
Overview: Concept Inventories
Confluence: SLCI Results from S13 GE Natural Sciences Courses
Confluence: Math Concept Inventories for GE Quantitative Reasoning
Moving Forward: Next steps and segue into Part II
Assessment takes a village . . .
Catherin Atkins, College of Science
Janet Bowers, Mathematics
Geoff Chase, Undergraduate Studies
Douglas Deutschman, Biology
Reynaldo Monzon, STAR
Chris Rasmussen, Mathematics
Stephen Schellenberg, Geological Science
Kathy Williams, CTL
. . . with thanks to S13 GE Nat. Sci. Faculty and Ed Nuhfer of Humboldt State
Fun facts:
~1/3 of BS/BA is GE
WASC is interested in GE
GE is integral part of mission

2. Goals and Capacities (G&Cs) of SDSU General Education Program
Goals for Natural Sciences (Three Foundation, One Exploration)
Explain basic concepts and theories of the natural sciences
Use logic and scientific methods to analyze the natural world and solve problems
Argue from multiple perspectives about issues in natural sciences that have personal and global relevance
Use technology in laboratory and field situations to connect concepts and theories with real-world phenomena
Are these G&Cs being assessed at the course level? If so, how?
How can these G&Cs be introduced, developed, and demonstrated within and across courses?
How can we assess these G&Cs at a
program level?
Goals for Quantitative Reasoning (One Foundation)
Apply appropriate computational skills and use basic mathematical concept to analyzed problems in the natural and social sciences
Use methods of quantitative reasoning to solve and communicate answers to real-world problems
Hyperdistributional – often dozens of courses to fullfill a specific GE program requirement
Essential Capacities Developed through General Education
Construct, analyze, and communicate arguments
Apply theoretical models to the real world
Contextualized phenomena
Negotiate differences
Integrate global and local perspectives
Illustrate relevance of concepts across boundaries
Evaluate consequences of actions

3. One Path Forward for Programmatic Assessment and Curricular Revision is the Concept Inventory
Definition: Collection of questions designed to assess student understanding of the foundational knowledge, concepts, and procedures for a given topic, discipline, etc.
Goals:
Assess scientific habits of mind and literacy
Identify gaps in approach understanding
Inform curricular reform
Inform on multiple levels in multiple ways when linked with demographics:
From synoptic overview of student population and needs to insights from distribution of distractor responses
Concept inventories have traditionally been discipline-focused, but recent efforts include metadisciplinary assessment of science literacy
We hypothesize that such concept inventories provide a means for programmatic assessment and curricular revision that will serve our students, faculty, and institution

4. Two Example Questions from 25-Item Science Literacy Concept Inventory (SLCI of Nuhfer et al., pers. comm.)
6. To help us to understand the lunar phases, we have set up a basketball, a baseball, and a golf ball to represent respectively the Sun, Earth and the moon. What method of science are we employing? A. Experiment. Moving the balls can allow us to measure the size of the shadow that one ball casts on another ball. B. Modeling. Moving the balls helps us to perceive the positions of the celestial bodies that might explain the observed phases. C. Multiple working hypotheses. Moving the balls can allow us to determine whether the lunar phases were different during the ice ages. D. None. Moving the balls differs from reality to such an extent that it is an ineffective way to understand lunar phases. 24. Which of the following assumptions is important to all sciences? A. Humans can understand the physical world through laws they can discover. B. The experimental method is the only valid way to test hypotheses. C. Life is not governed by the same physical laws as non-living systems. D. Random events have no role in the actual physical world.
GE Capacities:
GE Goals (Nat. Sci.):
GE Capacities:
GE Goals (Nat. Sci.):

5. Summary of Results from SCLI Deployment across Suite of GE Foundation and Exploration Courses during early Spring 2013
2,794 Initial Records
Non-Response Bias:
Female 25.2%
Male 39.1%
GEOL %
BIOL %
BIOL % ASTR %
870 (31%)
SLCI? No
Yes
SAT/ACT Not Reported:
Native 3.8%
Transfer 60.8%
Taking Foundations:
Native 71.6%
Transfer 31.4%
Native?
548 (20%) No
Yes
First Generation:
CHEM %
All Others 19.0%
Minority:
CHEM %
All Others 54.3%
216 (8%)
CHEM100?
For later in discussion, need table showing completion percentages by course to promote discussion on incentivizing future implementations.
Effect of incentivization
Yes No
Missing Data
67 (2%)
Yes No
1,093 Retained Records (39%)

6. Overall SLCI Performance and Potential Explanatory Variables
Demographics
Student Preparedness and Performance
SDSU “Treatment”
Age
SAT (ACT) Score
Student Level
Gender
GPA
Foundations, Exploration
Ethnicity, Race
Total Units Completed
First Generation
Course Modality
English as 1st Language
Major (Sci., Eng., Other)
Service Area
Residence Hall

7. Correlations of Explanatory Variables with SLCI Performance
Univariate (Unadjusted)
Multivariate (R2=20.5%)
Explanatory Variable R2 F p
Age
0.5%
1.93
0.12
0.0%
0.19
0.91
Gender
<.1%
0.42
0.51
0.2%
2.25
0.13
Ethnicity / Race
4.1%
4.57
<.001
1.2%
1.63
0.09
First Generation
1.3%
7.14
0.33
0.72
English as 1st Lang.
0.4%
4.78
0.03
0.1%
1.27
0.26
Service Area
0.15
0.70
2.66
0.10
SAT (ACT) Score
12.6%
157
7.1%
95.3
GPA
6.3%
73.4
1.9%
25.2
Student Level
1.8%
6.70
0.52
0.67
Found/Exploration
1.1%
11.6
0.11
0.74
Units Completed
2.4%
27.1
2.93
Course Modality
0.3%
1.76
0.17
0.6%
3.74
0.02
Major (Sci, Eng, Oth)
1.98
0.14
0.48
Residence Hall
0.66
1.21
0.27
Couldn’t format last two columns to align with others horizontally – no big deal.

8. Correlations of SAT (ACT) and GPA with SLCI Performance

9. Aside: Snapshot of when Foundation and Exploration courses are being taken by our students Implications for scaffolding, advising, and degree progress?
Freshman
Sophomore
Junior
Senior
Total
Exploration
2 (<1%)
66 (21%)
154 (49%)
91 (29%)
313
Foundation
340 (44%)
351 (45%)
62 (8%)
27 (3%)
780

10. Patterns, trends, and explanatory hypotheses?
Explorations
R2 = 5.2%
R2 = 6.0%
Foundations
R2 = 7.0%
R2 = 15.0%
Patterns, trends, and explanatory hypotheses?
As student develop within college, SAT become a weaker predictor of SLCI performance while GPA continues to play a strong role – essentially issue of resilience and effort versus high-stakes summative assessment of SAT/ACT

11. SLCI Performance Across Courses

12. End-Member Comparison
Performance difference cannot be attributed to incoming SAT scores
Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables
Course 18
19-21
22+
BIOL32x 0%
76%
24%
ENVS100
43%
53% 4%
Course 18
19-21
22+
Science Major
BIOL32x 0%
76%
24%
30%
ENVS100
43%
53% 4%
14%
Course 18
19-21
22+
Sci. Major
Units Earned
BIOL32x 0%
76%
24%
30%
95 (+/- 29)
ENVS100
43%
53% 4%
14%
42 (+/- 23)

13. End-Member Comparison End-Member Comparison
Performance difference cannot be attributed to incoming SAT scores
Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables
Univariate (Unadjusted)
Multivariate (R2=35.9%)
Variable R2 F p
Course
19.9%
27.4
<.001
Gender
SAT (ACT) Score
GPA
Student Level
Units Completed
Total

14. 8.22 1.00 5.75 3.93 0.14 0.10 End-Member Comparison
Performance difference cannot be attributed to incoming SAT scores
Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables
Univariate (Unadjusted)
Multivariate (R2=35.9%)
Variable R2 F p
Course
19.9%
27.4
<.001
4.7%
8.22
0.01
Gender
0.1%
1.00
0.32
SAT (ACT) Score
3.7%
5.75
0.02
GPA
2.6%
3.93
0.03
Student Level
0.14
0.94
Units Completed
0.10
0.77
Total
11.3%

15. 8.22 1.00 5.75 3.93 0.14 0.10 End-Member Comparison
Performance difference cannot be attributed to incoming SAT scores
Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables
Majority of the R2 is not uniquely assignable since it is shared among related variables (i.e., confounded)
Univariate (Unadjusted)
Multivariate (R2=35.9%)
Variable R2 F p
Course
19.9%
27.4
<.001
4.7%
8.22
0.01
Gender
0.1%
1.00
0.32
SAT (ACT) Score
3.7%
5.75
0.02
GPA
2.6%
3.93
0.03
Student Level
0.14
0.94
Units Completed
0.10
0.77
Total
11.3%

16. A Working Model . . . Limited by the Nature of the Current Data
Demographics
Student Preparedness and Performance
SDSU “Treatment”
Age
SAT (ACT) Score
Student Level
Gender
GPA
Foundations/Exploration
Ethnicity / Race
Total Units Completed
First Generation
Course Modality
English as a 1st Language
Major (Sci, Eng, Other)
Service Area
Residence Hall
SLCI
Issue of attribution.
Next Steps: Deploy more broadly over multiple semesters to allow focus on science literacy gains through time within individuals

17. Alternate or Parallel Concept Inventory?
Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012)

18. Alternate or Parallel Path?
Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012)

19. Alternate or Parallel Path for GE Natural Sciences?
Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012)
Ongoing Efforts:
Programmatic: Mapping Concept Inventory skills onto GE G&Cs
Administrative: Implementing deployment across GE Nat. Sci. courses
Grass-Roots: Using course content to introduce, practice, and master GE G&C

20. From GE Natural Science to GE Quantitative Reasoning:
Fall 2012 Calculus Concept Inventory
Goals:
Assess scientific habits of mind and literacy
Identify gaps in approach understanding
Inform curricular reform
Operate and inform on multiple levels in multiple ways:
Synoptic overview of student population to insights from distribution of distractor responses
TECH
TRADITIONAL
EXPERIENCED
FLIPPED
TECH
TRADITIONAL
EXPERIENCED
FLIPPED
September 2012
Pre-Test
December 2012
Post Test

21. Fall 2012 Calculus Concept Inventory
Pre-Post Test Gains:
Fall 2012 Calculus Concept Inventory

22. Example Question
16. The drawing represents a loaf of bread with a slice shown x inches from the left-hand end of the bread. Which of the following graphs could represent the volume V of the bread to the left of the slice as a function of the distance x from the left-hand end of the slice?

23. Operate and inform on multiple levels in multiple ways:
Synoptic overview of student population

24. Assessing GE Goals and Capacities with Concept Inventories: One Path Forward . . . Part I
Overview: General Education
Overview: Concept Inventories
Confluence: SLCI Results from S13 GE Natural Sciences Courses
Confluence: Math Concept Inventories for GE Quantitative Reasoning
Moving Forward: Next steps and segue into Part II
Assessment takes a village . . .
Catherin Atkins, College of Science
Janet Bowers, Mathematics
Geoff Chase, Undergraduate Studies
Douglas Deutschman, Biology
Reynaldo Monzon, STAR
Chris Rasmussen, Mathematics
Stephen Schellenberg, Geological Science
Kathy Williams, CTL
. . . with thanks to S13 GE Nat. Sci. Faculty and Ed Nuhfer of Humboldt State
Fun facts:
~1/3 of BS/BA is GE
WASC is interested in GE
GE is integral part of mission