Presentation is loading. Please wait.

Presentation is loading. Please wait.

A SSESSING GE G OALS AND C APACITIES WITH C ONCEPT I NVENTORIES : O NE P ATH F ORWARD... P ART I 1.Overview: General Education 2.Overview: Concept Inventories.

Similar presentations


Presentation on theme: "A SSESSING GE G OALS AND C APACITIES WITH C ONCEPT I NVENTORIES : O NE P ATH F ORWARD... P ART I 1.Overview: General Education 2.Overview: Concept Inventories."— Presentation transcript:

1 A SSESSING GE G OALS AND C APACITIES WITH C ONCEPT I NVENTORIES : O NE P ATH F ORWARD... P ART I 1.Overview: General Education 2.Overview: Concept Inventories 3.Confluence: SLCI Results from S13 GE Natural Sciences Courses 4.Confluence: Math Concept Inventories for GE Quantitative Reasoning 5.Moving Forward: Next steps and segue into Part II Assessment takes a village... Catherin Atkins, College of Science Janet Bowers, Mathematics Geoff Chase, Undergraduate Studies Douglas Deutschman, Biology Reynaldo Monzon, STAR Chris Rasmussen, Mathematics Stephen Schellenberg, Geological Science Kathy Williams, CTL... with thanks to S13 GE Nat. Sci. Faculty and Ed Nuhfer of Humboldt State Fun facts: ~1/3 of BS/BA is GE WASC is interested in GE GE is integral part of mission

2 Goals and Capacities (G&Cs) of SDSU General Education Program Essential Capacities Developed through General Education 1.Construct, analyze, and communicate arguments 2.Apply theoretical models to the real world 3.Contextualized phenomena 4.Negotiate differences 5.Integrate global and local perspectives 6.Illustrate relevance of concepts across boundaries 7.Evaluate consequences of actions Goals for Natural Sciences (Three Foundation, One Exploration) 1.Explain basic concepts and theories of the natural sciences 2.Use logic and scientific methods to analyze the natural world and solve problems 3.Argue from multiple perspectives about issues in natural sciences that have personal and global relevance 4.Use technology in laboratory and field situations to connect concepts and theories with real-world phenomena Goals for Quantitative Reasoning (One Foundation) 1.Apply appropriate computational skills and use basic mathematical concept to analyzed problems in the natural and social sciences 2.Use methods of quantitative reasoning to solve and communicate answers to real-world problems Are these G&Cs being assessed at the course level? If so, how? How can these G&Cs be introduced, developed, and demonstrated within and across courses? How can we assess these G&Cs at a program level? Are these G&Cs being assessed at the course level? If so, how? How can these G&Cs be introduced, developed, and demonstrated within and across courses? How can we assess these G&Cs at a program level?

3 One Path Forward for Programmatic Assessment and Curricular Revision is the Concept Inventory Definition: Collection of questions designed to assess student understanding of the foundational knowledge, concepts, and procedures for a given topic, discipline, etc. We hypothesize that such concept inventories provide a means for programmatic assessment and curricular revision that will serve our students, faculty, and institution Goals: Assess scientific habits of mind and literacy Identify gaps in approach understanding Inform curricular reform Inform on multiple levels in multiple ways when linked with demographics: From synoptic overview of student population and needs to insights from distribution of distractor responses Concept inventories have traditionally been discipline-focused, but recent efforts include metadisciplinary assessment of science literacy

4 Two Example Questions from 25-Item Science Literacy Concept Inventory (SLCI of Nuhfer et al., pers. comm.) 6. To help us to understand the lunar phases, we have set up a basketball, a baseball, and a golf ball to represent respectively the Sun, Earth and the moon. What method of science are we employing? A. Experiment. Moving the balls can allow us to measure the size of the shadow that one ball casts on another ball. B. Modeling. Moving the balls helps us to perceive the positions of the celestial bodies that might explain the observed phases. C. Multiple working hypotheses. Moving the balls can allow us to determine whether the lunar phases were different during the ice ages. D. None. Moving the balls differs from reality to such an extent that it is an ineffective way to understand lunar phases. 24. Which of the following assumptions is important to all sciences? A. Humans can understand the physical world through laws they can discover. B. The experimental method is the only valid way to test hypotheses. C. Life is not governed by the same physical laws as non-living systems. D. Random events have no role in the actual physical world. GE Capacities: GE Goals (Nat. Sci.): GE Capacities: GE Goals (Nat. Sci.): GE Capacities: GE Goals (Nat. Sci.): GE Capacities: GE Goals (Nat. Sci.):

5 2,794 Initial Records SLCI? 870 (31%) No Yes Non-Response Bias: Female 25.2% Male 39.1% GEOL3039.2% BIOL % BIOL % ASTR % Native? 548 (20%) No Yes CHEM100? 216 (8%) Yes No Missing Data 67 (2%) Yes No 1,093 Retained Records (39%) SAT/ACT Not Reported: Native3.8% Transfer60.8% Taking Foundations: Native71.6% Transfer31.4% First Generation: CHEM % All Others19.0% Minority: CHEM % All Others54.3% Summary of Results from SCLI Deployment across Suite of GE Foundation and Exploration Courses during early Spring 2013

6 DemographicsStudent Preparedness and Performance SDSU “Treatment” AgeSAT (ACT) ScoreStudent Level GenderGPAFoundations, Exploration Ethnicity, RaceTotal Units Completed First GenerationCourse Modality English as 1 st LanguageMajor (Sci., Eng., Other) Service AreaResidence Hall Overall SLCI Performance and Potential Explanatory Variables

7 Correlations of Explanatory Variables with SLCI Performance Univariate (Unadjusted)Multivariate (R 2 =20.5%) Explanatory Variable R2R2 FpR2R2 Fp Age 0.5% % Gender <.1% % Ethnicity / Race 4.1%4.57< % First Generation 1.3%7.14< % English as 1 st Lang. 0.4% % Service Area <.1% % SAT (ACT) Score 12.6%157< % 95.3 <.001 GPA 6.3%73.4< % 25.2 <.001 Student Level 1.8%6.70< % Found/Exploration 1.1%11.6< % Units Completed 2.4%27.1< % Course Modality 0.3% % Major (Sci, Eng, Oth) 0.4% % Residence Hall <.1% % R 2 = 20.5%

8 Correlations of SAT (ACT) and GPA with SLCI Performance R 2 = 12.6% R 2 = 6.3%

9 FreshmanSophomoreJuniorSeniorTotal Exploration 2 (<1%) 66 (21%) 154 (49%) 91 (29%) 313 Foundation 340 (44%) 351 (45%) 62 (8%) 27 (3%) 780 Aside: Snapshot of when Foundation and Exploration courses are being taken by our students... Implications for scaffolding, advising, and degree progress?

10 Explorations R 2 = 5.2% R 2 = 6.0% Foundations R 2 = 7.0% R 2 = 15.0% Patterns, trends, and explanatory hypotheses?

11 SLCI Performance Across Courses

12 Course Sci. MajorUnits Earned BIOL32x0%76%24%30%95 (+/- 29) ENVS10043%53%4%14%42 (+/- 23) Course Science Major BIOL32x0%76%24%30% ENVS10043%53%4%14% Course BIOL32x0%76%24% ENVS10043%53%4% Performance difference cannot be attributed to incoming SAT scores Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables End-Member Comparison

13 Performance difference cannot be attributed to incoming SAT scores Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables End-Member Comparison Univariate (Unadjusted)Multivariate (R 2 =35.9%) VariableR2R2 FpR2R2 Fp Course19.9%27.4<.001 Gender SAT (ACT) Score GPA Student Level Units Completed Total 19.9% End-Member Comparison

14 Performance difference cannot be attributed to incoming SAT scores Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables End-Member Comparison Univariate (Unadjusted)Multivariate (R 2 =35.9%) VariableR2R2 FpR2R2 Fp Course19.9%27.4< % Gender 0.1% SAT (ACT) Score 3.7% GPA 2.6% Student Level 0.1% Units Completed 0.1% Total 19.9% 11.3% End-Member Comparison

15 Performance difference cannot be attributed to incoming SAT scores Performance difference may be partially explained by differences in age, major, units earned, and other confounded variables End-Member Comparison Majority of the R 2 is not uniquely assignable since it is shared among related variables (i.e., confounded) End-Member Comparison Univariate (Unadjusted)Multivariate (R 2 =35.9%) VariableR2R2 FpR2R2 Fp Course19.9%27.4< % Gender 0.1% SAT (ACT) Score 3.7% GPA 2.6% Student Level 0.1% Units Completed 0.1% Total 19.9% 11.3%

16 DemographicsStudent Preparedness and Performance SDSU “Treatment” AgeSAT (ACT) ScoreStudent Level GenderGPAFoundations/Exploration Ethnicity / RaceTotal Units Completed First GenerationCourse Modality English as a 1 st LanguageMajor (Sci, Eng, Other) Service AreaResidence Hall A Working Model... Limited by the Nature of the Current Data Next Steps: Deploy more broadly over multiple semesters to allow focus on science literacy gains through time within individuals SLCI

17 Alternate or Parallel Concept Inventory? Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012)

18 Alternate or Parallel Path? Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012)

19 Alternate or Parallel Path for GE Natural Sciences? Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012) Alternate or Parallel Path for GE Natural Sciences? Test of Scientific Literacy Skills (TOSLS) – Gormally et al. (2012) Ongoing Efforts: Programmatic: Mapping Concept Inventory skills onto GE G&Cs Administrative: Implementing deployment across GE Nat. Sci. courses Grass-Roots: Using course content to introduce, practice, and master GE G&C Ongoing Efforts: Programmatic: Mapping Concept Inventory skills onto GE G&Cs Administrative: Implementing deployment across GE Nat. Sci. courses Grass-Roots: Using course content to introduce, practice, and master GE G&C

20 TECH TRADITIONAL EXPERIENCED FLIPPED September 2012 Pre-Test December 2012 Post Test TECH TRADITIONAL EXPERIENCED FLIPPED From GE Natural Science to GE Quantitative Reasoning: Fall 2012 Calculus Concept Inventory From GE Natural Science to GE Quantitative Reasoning: Fall 2012 Calculus Concept Inventory Goals: Assess scientific habits of mind and literacy Identify gaps in approach understanding Inform curricular reform Operate and inform on multiple levels in multiple ways: Synoptic overview of student population to insights from distribution of distractor responses

21 Pre-Post Test Gains: Fall 2012 Calculus Concept Inventory Pre-Post Test Gains: Fall 2012 Calculus Concept Inventory

22 16. The drawing represents a loaf of bread with a slice shown x inches from the left-hand end of the bread. Which of the following graphs could represent the volume V of the bread to the left of the slice as a function of the distance x from the left-hand end of the slice? Example Question

23 Operate and inform on multiple levels in multiple ways: Synoptic overview of student population Operate and inform on multiple levels in multiple ways: Synoptic overview of student population

24 A SSESSING GE G OALS AND C APACITIES WITH C ONCEPT I NVENTORIES : O NE P ATH F ORWARD... P ART I 1.Overview: General Education 2.Overview: Concept Inventories 3.Confluence: SLCI Results from S13 GE Natural Sciences Courses 4.Confluence: Math Concept Inventories for GE Quantitative Reasoning 5.Moving Forward: Next steps and segue into Part II Assessment takes a village... Catherin Atkins, College of Science Janet Bowers, Mathematics Geoff Chase, Undergraduate Studies Douglas Deutschman, Biology Reynaldo Monzon, STAR Chris Rasmussen, Mathematics Stephen Schellenberg, Geological Science Kathy Williams, CTL... with thanks to S13 GE Nat. Sci. Faculty and Ed Nuhfer of Humboldt State Fun facts: ~1/3 of BS/BA is GE WASC is interested in GE GE is integral part of mission


Download ppt "A SSESSING GE G OALS AND C APACITIES WITH C ONCEPT I NVENTORIES : O NE P ATH F ORWARD... P ART I 1.Overview: General Education 2.Overview: Concept Inventories."

Similar presentations


Ads by Google