1. Learning Outcomes, Authentic Assessments and Rubrics Erin Hagar ehagar@jhu.edu

2. Workshop Objectives  Describe the importance of aligning program goals, course goals, and learning outcomes with assessment strategies  Write measurable learning outcomes for their courses  Design an authentic assessment technique that measures a desired learning outcome  Create a rubric that articulates the evaluation criteria and standards of performance for that assessment

3. Facilitator Commitments  Minimum jargon  Open to questions at any point  Grounded in theory, but  Focused on practical applications

4. Program Goals Keeping the Big Picture in Mind Course Goals Unit Objectives/ Outcomes Activities Assessments

5. http://www.jhu.edu/advanced/biotechnology/curr iculum.html AAP Biotechnology Program Goals “The curriculum is designed so that graduates can  participate fully in research;  lead lab teams,  help to make development and planning decisions,  apply research modalities to larger schemes set in large research projects.  bridge the worlds between [non-scientists’] roles and the scientists in their organization, allowing for effective communications and decision making. ”

6. Course Goals  Large or general intentions of a course of instruction Ex: “To introduce a group of academic staff new or comparatively new to teaching to the basic principles of teaching and learning, and to give them some opportunity to practice the application of these principles.” Source: Good Practices in Teaching and Learning, University College Dublin, http://www.ucd.ie/teaching/good/cou3.htm

7. Unit Outcomes/Objectives  Statements that describe, in behavioral terms, the skill or knowledge a student should demonstrate as a result of the instruction Outcome: “Participants will be able to create a rubric for a subjective assessment.” Objectives:  Articulate criteria for the assignment  Define standards for each criteria  Describe levels of performance for each criteria

8. Activity: Articulate Outcomes/Objectives for an Upcoming Unit

9. The Assessment Cycle Learning Outcomes Instructional Activities Assessment Mechanisms

10. Principles of Effective Assessment  Valid: The assessment measures what you want measured  Reliable: The same assessment given after similar instruction produces similar results over time  Aligned with learning outcomes

11. “Traditional” Vs. “Authentic” Assessment  Select Response  Contrived  Knowledge Recall  Teacher-structured  Indirect evidence of student thinking  Perform a task  Real-life  Knowledge Application  Student-structured  Direct evidence of student thinking Source: http://jonathan.mueller.faculty.noctrl.edu/toolbox/index.htm

12. What Might Authentic Assessments in Biotech Courses Look Like?

13. Authentic Assessment Activity: I Think about the learning outcomes you identified earlier in the workshop, and begin to formulate an authentic assessment that reflects desired outcome(s). What are you assessing? What skills/behaviors should be evidenced in the final product? What kind of guiding instructions will you provide?

14. Authentic Assessment Activity: II In pairs, share your ideas for this assessment, and provide feedback to your partner:  How well does activity reflect desired learning outcomes?  Does it meet some of the criteria of “authentic” assessment? (real-world, student-structured, etc.)  If you were a student given this assignment, what questions would you have about it?

15. “Traditional” Vs. “Authentic” Assessment  Select Response  Contrived  Knowledge Recall  Teacher-structured  Indirect evidence of student thinking  Perform a task  Real-life  Knowledge Application  Student-structured  Direct evidence of student thinking Source: http://jonathan.mueller.faculty.noctrl.edu/toolbox/index.htm

16. BREAK TIME

17. Part III: Evaluating Subjective Assignments with Rubrics

18. What is a Rubric?  An assessment tool used to evaluate student products that… are complex and subjective. have various components. have no easily discernable right or wrong answers, but rather, can be described as “good” vs. “poor” products.

19. A Silly Example: The Chocolate Chip Cookie  If you were the judge in a bake-off, what criteria would you look for in a chocolate chip cookie? (ex: texture)  Within those general criteria, what would constitute a “delicious” cookie? (ex: texture— slightly chewy)

20. Specifically, a Rubric is...  a printed set of scoring guidelines that communicate the criteria being evaluated and the various standards of performance  Let’s look at some now...

21. A Rubric Answers the Questions…  By what criteria will the work be judged?  What is the difference between good and weaker work?  How can we make sure our judgments or scores are valid and reliable?  How can students prepare for excellence?

22. The Advantages of Rubrics  Force the teacher to clarify his/her standards and criteria in specific terms.  Allow assessment to be more objective and consistent.  Clearly show the student how their work will be evaluated and what is expected (must be given to student in advance.)  Promote student awareness of the criteria to use in assessing peer performance  Provide benchmarks to measure and document progress (i.e. an assignment you give periodically throughout the semester)

23. Things to Consider When Using Rubrics  What are the major elements of this assignment? What exactly are you looking for?  Within these elements, what constitutes “excellence” and “unacceptable,” and everything in between?  Can you describe “excellent”, “average” or “poor” etc. using the most concrete terms possible?

24. Steps to Create a Rubric  Step 1: Decide what the product is (in our case, a the authentic assessment you’ve begun to design).  Step 2: Determine the elements or features of the product you will assess (4-5 elements).

25. Steps to Create a Rubric  Step 3: Decide what you will call the levels of performance: 4-Exemplary, Excellent, Superior, Master, (Delicious.) 3-Accomplished, Skilled, Competent, Good 2- Fair, Needs Improvement, Developing; 1-Poor, Novice, Beginning, Unsatisfactory, Unacceptable.

26. Steps to Create a Rubric  Step 4. For every criteria you identify, describe what would be exemplary. Use former student products as a guide (more than one). Be as concrete as possible.  Step 5. Repeat this process for the lower levels in descending order, also using student samples to help you.

27. Steps to Create a Rubric  Step 6: Give the rubric to the students in advance.  Step 7: Apply the rubric to the assignment.  Step 8: Modify the rubric based on feedback from students and reflection on its effectiveness.

28. Activity: Begin to design a rubric for the authentic assessment you’re planning

29. Closing thoughts  Our courses are ultimately a reflection of the program’s goals  Those program goals usually reflect advanced levels of thinking, focusing on synthesizing and applying knowledge appropriately  Authentic assessments provide an opportunity for students to engage in those higher levels of thinking  Rubrics can help us evaluate the products of those assessments that are complex and subjective