1. Truman State University Kirksville, Missouri Glenn Wehner – Agriculture Ian Lindevald – Physics Philip Ryan – Mathematics Karen Smith - Psychology

2. Truman State: Who are we? Public liberal arts and science institution Public liberal arts and science institution Highly selective Highly selective Undergraduate with selected graduate programs Undergraduate with selected graduate programs Approx. 6,000 student body Approx. 6,000 student body Long history of assessment (>25 years) Long history of assessment (>25 years) Rural setting Rural setting

3. Assessment Overview Collegiate Learning Assessment (CLA) * Value added model with Freshmen * Value added model with Freshmen and Seniors and Seniors Junior Testing Junior Testing * CAAP * CAAP * AP * AP * 2007-2008 JMU * 2007-2008 JMU Senior tests in the major (nationally normed)

4. Student Interview Project Writing Across the University Project Senior Portfolios Senior Capstone Experiences

5. Attitudinal Assessment NSSE CSEQ and CSXQ Graduating Student Questionnaire (GSQ) Alumni Survey Staff Survey HERI Survey (faculty)

6. Scientific Reasoning TSU has used senior portfolios to assess scientific reasoning in the past. a. On a 0-3 scale, SR scores have been a. On a 0-3 scale, SR scores have been steady over the past 4 years with a steady over the past 4 years with a mean score around 1.2 with: mean score around 1.2 with: 0 = no competence 0 = no competence 3 = strong competence 3 = strong competence

7. b. Appears to be confusion with students b. Appears to be confusion with students as to scientific reasoning and scientific as to scientific reasoning and scientific knowledge or scientific method. knowledge or scientific method. c. Quantitative reasoning has not been c. Quantitative reasoning has not been measured with portfolios for several measured with portfolios for several years years Junior CAAP scores have had stronger findings than the portfolios in that 80%+ of juniors taking the SR and QR modules have scores above the 50%tile. For QR, above the 80%tile, 86% of juniors and for SR 49%. 86% of juniors and for SR 49%.

8. Both assessments allow snap-shots of student progress but these results are very difficult to take back to the classroom with specific recommendations or suggestions for improvement. TSU became interested in the JMU project since the results appeared to allow us to correlate disciplines and possibly class types that would promote our stated outcomes for the scientific (both life and physical sciences) mode of inquiry and for the quantitative mode.

9. Testing Procedure Juniors of standing (75+ hours) were randomly assigned to one of two groups: Juniors of standing (75+ hours) were randomly assigned to one of two groups: 1) JMU SR/QR test 1) JMU SR/QR test 2) CAAP – SR plus Math modules 2) CAAP – SR plus Math modules Both vehicles were administered in the Fall 2007 and Spring 2008 - JMU N=591 (64 declined) - JMU N=591 (64 declined) - CAAP N=634 - CAAP N=634

10. Test Results: JMU test at Truman: Cronbach’s Alpha =.80 Mean Score = 81% (std dev. – 6.99) Score ranged 29% - 99% CAAP reports Kuder-Richardson Formula 20 Science =.86 Math =.84

11. Research Questions Research Questions How do JMU and CAAP SR/QR scores correlate with: A) Number of SR or QR classes taken: Science Social Sci. Math Agriculture Psychology Algebra Biology Sociology Trig. Chemistry Poli. Sci. Calculus* Physics Economics Statistics CS CS * Particularly interested in LAS calculus

12. B) Number of 100 level classes in each category C) Which individual classes or cluster of classes contribute to good performance on either instrument D) Is there a correlation between ACT scores and classes and performance on either instrument