1. KSC Mathematics Creating Rubrics for Assessment of General Education Mathematics Dick Jardine SUNY General Education Assessment Conference Syracuse, NY April 27, 2005 rjardine@keene.edu

2. KSC Mathematics Overview The language of rubrics Some examples Make your own rubric! What to do with the results

3. KSC Mathematics What is a rubric? A rubric is a scoring device designed to assist in the process of communicating expectations and assessing performance Based on a range of criteria rather than a single numerical score A working guide for both students and faculty Non-traditional, “authentic” assessment instrument

4. KSC Mathematics Analytic and Holistic Rubrics Analytic: Identify and assess components of the task A score for each component Provide more detail about standards Holistic: Assesses the whole work One score for the entire product

5. KSC Mathematics Holistic Rubric Template CriterionScore

6. KSC Mathematics Example holistic rubric Student will conduct a hypothesis test 4 Uses appropriate test with correct with correct interpretation. Identifies correct null and alternative hypothesis. Uses correct test statistic. Obtains correct p-value and correctly interprets that result in terms of the null hypothesis and the context with clear exposition. 3 Provides most of level 4 but with unclear/incorrect interpretation 2 Uses incorrect test but remaining work follows from error 1 Uses incorrect test and remaining work does not follow 0 Little or no work of value

7. KSC Mathematics Holistic rubric for hypothesis test CriterionScore Uses appropriate test with correct interpretation. Identifies correct null and alternative hypothesis… 4 Provides most of level 4 but with unclear/incorrect interpretation 3 Uses incorrect test but remaining work follows from error 2 Uses incorrect test and remaining work does not follow 1 Little or no work of value 0

8. KSC Mathematics Analytic and Holistic Rubrics Analytic: Identify and assess components A score for each component Provide more detail about standards Holistic: Assesses the whole work One score for the entire product

9. KSC Mathematics Analytic Rubric Template Score Criterion 012

10. KSC Mathematics Problem solving evaluation matrix 0123 Identify appropriate variables No attemptInappropriate variable definition Appropriate variables defined imprecisely Appropriate variables defined precisely w/units Organize available information No attemptInappropriate attempt Incomplete attemptAppropriate attempt with clear explanation Model the problem mathematically No attemptInappropriate attempt Appropriate attempt with minor errors Appropriate attempt with no errors Execute the model No attemptInappropriate attempt Appropriate attempt with arithmetic errors Appropriate method with no errors Evaluatethe results No attemptAn attempt but fails to explain solution, provide units Incomplete explanationComplete explanation with correct labeling Problem solving matrix (Rhodes State College)

11. KSC Mathematics Problem solving evaluation scoresheet ScoreComments Identify appropriate variables Organize available information Model the problem mathematically Execute the model Evaluate the results Total

12. KSC Mathematics Possible terms for scoring levels Needs Improvement...Satisfactory...Good…Exemplary Beginning...Developing...Accomplished…Exemplary Needs work...Good…Excellent Novice...Apprentice...Proficient…Distinguished Numeric scale ranging from 0 to n

13. KSC Mathematics Course-embedded assessment (University of Northern Colorado) Align course objectives with Gen Ed outcomes Identify course objectives and student outcomes Explain tasks to measure each objective/outcome Explain performance criteria to evaluate each objective (what constitutes meeting expectations)

14. KSC Mathematics Course objectiveAdvancedProficientIn Progress Not Assessable 1. Student will demonstrate proficiency in the use of math/stats to structure their understanding of and investigate questions in the world around them 2. The student will demonstrate proficiency in treating math/stats content at an appropriate level 3. The student will demonstrate competence in the use of numerical, graphical, and algebraic representations 4. The student will demonstrate proficiency in the use of math/stats in written and oral from. 5. The student will demonstrate proficiency in the use of math/stats to formulate and solve problems 6. The student will demonstrate proficiency in the use of technology to support their use of math/stats

15. KSC Mathematics Area 1 : The student will demonstrate proficiency in the use of mathematics and/or statistics to structure their understanding of and investigate questions in the world around them Task: (Test question) How long will it take for $8600 to grow to $31,300 at an interest rate of 5.5% if the interest is compounded continuously? Student uses logarithms and rounds for a reasonable answer 4 Student uses logarithms and gives answer copied from calculator 3 Student uses logarithms but incorrectly 2 Student uses guess and check 1 No effort 0 Exceeds expectations Meets expectations Does not meet expectations

16. KSC Mathematics Performance Assessment Scoring Rubric 4 – Thorough Understanding Demonstrates a thorough understanding of the concepts/content Demonstrates mastery of the strategies and skills Solves new problems by creative and appropriate use of knowledge/skills 3 – Good understanding Demonstrates a satisfactory understanding Carries out skills and strategies with few minor errors Applies prior knowledge when prompted 2 – Satisfactory understanding Demonstrates partial understanding of most important concepts Carries out skills and strategies with minor errors Makes progress toward the solving of new problems 1 – Needs improvement Demonstrates weak understanding of concepts/content Carries out skills and strategies with significant critical errors

17. KSC Mathematics Developing your own rubrics With a colleague: oDetermine the learning objective to be assessed. oList the criteria to be evaluated. oDescribe degrees of quality (in a matrix), starting with the best and worst quality and then filling in the middle. Share the rubric with students before they complete the assignment so students understand the criteria for evaluation. With a colleague: oUse the rubric to evaluate the students’ work to determine if they have met the learning objective. oAdjust rubric as needed for next assessment

18. KSC Mathematics Some limitations of rubrics Time it takes to develop quality rubrics Reduction of “knowledge rich” activities to selected easily observable behaviors Achieving consensus to “norm” the rubric standards

19. KSC Mathematics Some advantages of rubrics Faculty provide focus, emphasis, and attention to details for students Students have explicit details regarding teacher expectations Tool for student development Grading more efficient and effective

20. KSC Mathematics What to do with the results? Improve student learning Improve curriculum and instruction Improve assessment process How to use the numbers?

21. KSC Mathematics Summary and conclusion Creating rubrics should be a collaborative process Communicate the standards to students Calibrating the scoring requires significant discussion Improve learning with the results http://www.maa.org/saum email: rjardine@keene.edurjardine@keene.edu