1. Paths to Program Assessment through Rubrics Dannelle D. Stevens, Ph.D. Cal Poly, Pomona May 23 & 24, 2010

2. Objectives To learn how to use rubrics for program assessment Case #1, A sociology dept. listens to its graduates Case #2, Bottom-up or Top-down Creation To learn several ways to create and use rubrics for program assessment To begin to create a rubric for a SLO

3. Agenda 1. Identifying assumptions about assessment 2. Using rubrics in assessment: Case #1 – How to use rubrics 3. Creating rubrics Case #2- Bottom-up, top-down 4. Let’s create a rubric for a program outcome… What do you need to know?

4. Basic assumptions about program assessment Assessment is not new. Yet, grades not enough. Data, data, seems to be everywhere. No, not about faculty evaluation. Yes, about “conversation”, student learning. Ultimate goals => better programs, student learning

5. KEY Program Assessment Components Student Learning OUTCOMES STUDENT “WORK ” TIME! A way to assess quality- RUBRIC + + + +

6. 3 basic steps 2. GATHER 1. REFLECT/PLAN 3. USE

7. Case Study #1 Case: An example of SLOs and a rubric to improve student learning. Context: Sociology Department, Seniors HANDOUT, Case #1 Product: Oral presentation of Senior Project

8. Your task 1. Review materials for the case Context, SLO, Rubric, Scores on a rubric 2. Decide how to use these rubric scores to improve the program.

9. Questions for program assessment 1. What patterns do you see in the data? 2. What statements could you make based on these data? 3. What SPECIFIC suggestions do you have for the program on how to improve student learning of this outcome? List these and, then, rank order them on a flip chart. Be ready to say why you chose one suggestion over another. 4. What will you tell the graduates?

10. Case #1- Follow-up What did you think about the case? Typical? Data helpful? Or not? Further questions

11. 3 basic steps in program assessment: Case #1 2. GATHER 1. REFLECT/ PLAN 3. USE Concern over oral presentations. Identified related Student Learning Outcome. Gather student work samples. Selected 10 out of 100 Disaggregated the scores. Take notes. Share with others. Look at the data. Look for patterns. Ask questions. Make suggestions to department. Follow-up. See what happened. Take notes. Share with others.

12. K- W- L What do you KNOW now about creating rubrics? What do you WANT to know? What have you LEARNED?

13. Case #2: Creating rubrics Bottom-up ? Top- down?

14. Case #2: Bottom-up Path Start with student work Read, stack them as exemplary, competent, weak LIST short descriptions on stickies of each stack GROUP & LABEL the descriptions APPLY to rubric grid. Check descriptions with SLOs; refine SLOs…

15. Case #2: Top-down Path Start with SLOs. REFLECT. LIST many descriptions of how students would demonstrate SLOs at the highest level first. GROUP and LABEL descriptions. APPLY to rubric grid. Add other levels. SCORE sample of student work with rubric. (5- 10) Refine SLOs or rubric.

16. Case #2: Discussion What is the difference between Bottom-up and Top-Down paths? Which makes more sense to you? Why? What does this tell you about rubric creation?

17. Now, let’s begin to create a rubric from a Student Learning Outcome Read SLO carefully. REFLECT: Imagine student work or performance that would demonstrate this outcome. Think of highest level of performance for now. LIST: Many descriptors of student work- evidence that students had achieved this outcome. GROUP & LABEL: Group descriptors & label. APPLY: Apply.

18. LISTING

19. GROUPING & LABELING

20. APPLYING

21. The ever-so-generative rubric What does it generate? Data, useful data Case #1 Look down the student scores in columns. Choices.. Disaggregate dimensions (e.g, content, style, org.) Disaggregate scores by student demographics Can this lead to program improvement. How?

22. 3 basic steps in program assessment 2. GATHER 1. REFLECT/ PLAN 3. USE Identify potential area of interest/concern. Identify related Student Learning Outcome. Gather student work, examples of where students should be demonstrating this outcome. Many decisions here! What kind of evidence? What class? How many products? Is there a rubric? Start somewhere. Record the process! Look at the data. Ask questions. Use the data to make decisions to improve the program. Do it! Then, gather data on that! Take notes. Share with others.

23. Thank you… http://www.csupomona.edu/~academic/programs/ programreview/ http://www.csupomona.edu/~academic/programs/ programreview/ www.csupomona.edu/facultycenter www.introductiontorubrics.com Look for rubrics made at University of Alabama