Applying Assessment to Learning Unit Time: 2 minutes Introductions: Introduce yourself Share your credentials Share your experiences as an educator Share why you are qualified to facilitate the session Provide an opportunity for your participants to introduce themselves and share a little about themselves Assessment Tier II Professional Development

Assessment Tier Professional Development Tier I – (required) Purposes and functions of assessment and evaluation tools. Tier II- (required) Development and use of assessments and evaluation tools, including using Rigor and Relevance (you are here). Tier III – (optional) Professional development for future Assessment Tier Champions. Unit Time: 2 minutes Cumulative Time: 4 minutes Read this slide to introduce the Assessment Tier Training structure. The Assessment Tier trainings are one piece of the AIM (Academic Improvement Model) initiative. This is derived from Highly Effective Programs and Delivery Systems, one of the 11 Guiding Principles.

Assessment Tier II Objectives Develop reliable and purposeful measures related to SLOs through use of evaluation tools e.g., rubrics, and checklists. Apply appropriate assessment tools. Review SLOs, EOs, and Rigor and Relevance to create and select assessment items. Explore the use of assessment as part of the broader teaching and learning experience. Unit Time: 1 minute Cumulative Time: 5 minutes Read over the objectives.

Tier I Review What can you tell me about: Authentic v. Traditional Assessments Formative and Summative Assessment When and why do I use it? Program v. Classroom Assessment Rigor and Relevance Framework Rubrics Why rubrics? Types of rubrics Unit Time: 15 minutes Cumulative Time: 20 minutes Use small groups or table groups to discuss each bullet (5 minutes) and report out (10 minutes). Assist as needed or This can also be done as an “arrival” assignment. If you would like to do this, refer participants to their handouts. Upon arrival, ask them to pair-up with another participant, discuss, and be ready to report out. If needed remind participants about each item: Traditional v. Authentic Assessments: On traditional assessments, students are typically given several choices (e.g., a,b,c or d; true or false; which of these match with those) and asked to select the right answer. In contrast, authentic assessments ask students to demonstrate understanding by performing a more complex task usually representative of more meaningful application. Formative: a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve students’ achievement of intended instructional outcomes. When: daily, weekly, or periodically Why: Clarify and share learning intentions and criteria for success, engineer effective classroom discussions, questions, and learning tasks that elicit evidence of learning, provide feedback that moves learning forward, activate students as the owners of their own learning, activate students as instructional resources for one another. Summative: the process of gathering quantitative performance data to evaluate progress on student learning outcomes, benchmarks, and class or programmatic efficacy. When: periodically (mid-term, final, annual) Why: To collect quantitative performance data for students, to assess the efficacy of the session’s delivery and instruction, to quantitatively assess student learning outcomes for a unit(s) and/or class. How assessment is used: Of learning when using evidence of student learning to make judgements on student achievement against goals and standards. For learning when using inferences about student progress to inform their teaching. As learning when students reflect on and monitor their progress to inform their future learning goals. Program v. Classroom: Classroom: Focuses on outcomes from a single course of instruction, can be assessed daily, weekly, midterm, final etc., focuses on and aligned to SLOs and EOs defined by a section of content and the syllabus, focus on providing student feedback for improvement of the learner. Program: Focuses on learner outcomes for degree completion (capstone outcomes), typically assessed annually or every few years, focuses on exit competencies and ISLO’s and PO’s, focuses on gathering data for institutional and programmatic improvements The Rigor and Relevance Frame work has two continuums working together. One side of the framework focuses on levels of thought using Bloom’s Taxonomy. Each level demonstrates a more complex and involved treatment of a given piece of information or content. At the bottom learners understand terms and meanings and at the top they are able to apply them in complex ways. There are Six levels: Level 1 – Remembering (Knowledge) Level 2 – Understanding (Comprehension) Level 3 – Applying (Application) Level 4 – Analyzing (Analysis) Level 5 – Evaluating (Synthesis) Level 6 – Creating (Evaluation) The other side focuses on the application of information. Similar to Bloom, this continuum involves increasingly complex application of knowledge in increasingly diverse scenarios. Whereas Bloom deals primarily with understanding and information processing, the application framework deals with how that information is put into practice and used. When combined, the Framework provides 4 quadrants of performance. The end goal of an educational process is to get learners into the D-quadrant however daily/weekly SLOs may need only end in one of the others. All SLOs and EOs are plotted on the Rigor and Relevance framework to determine what quadrant they fall. Once we determine the quadrant each is in we are able to determine the appropriate assessment types. Rubrics: Why: a way to grade student work, to clearly communicates how work will be judged, to ensure consistent standards are used in judging all students and their work Types: Holistic Rubrics provide a single score based on an overall impression of a student’s performance on a task. Analytic Rubrics provide specific feedback along several dimensions. Checklists contain a list of behaviors or specific steps.

Professor Dancealot Unit Time: 5 minutes Cumulative Time: 25 minutes Click on the picture to play the video: If the link does not work it can be found at the following URL: http://corvus.baker.edu/departments/etl/videos/professor-dancealot/?iframe=true&width=550&height=350 The video runs 3 minutes 8 seconds, so you have only a small amount of time to debrief. You can use the video to link to previous content and to upcoming content. Use the video to link to the review just finished. Ask participants their thoughts about the Professor’s methods in terms of: Authentic v. Traditional Assessments [Facilitator note: —While Prof. Dancealot taught in a very traditional manner (tell and show with no learner-centered teaching strategies to involve the students), he tried to use an authentic assessment for an exam.] Formative and Summative Assessment [Facilitator note: —Prof. Dancealot used no formative assessment. In fact, he left the room once he gave them the instructions for the video. The video was to be used for Summative Assessment. “I’ll have your results to you by Friday.” Practice with feedback (and even video feedback) could have been very powerful formative assessment tools.] Transition to next slide: Think about Professor Dancealot as we once again discuss academic Rigor and Relevance.

Rigor and Relevance Creating Evaluating Analyzing Applying Comprehending Remembering Unit Time: 8 minutes Cumulative Time: 33 minutes Ask for volunteers to answer the following questions referring to the Rigor and Relevance Framework: Using the Knowledge Taxonomy. Ask “At what levels were Prof. Dancealot’s SLOs written? “Perform the basic steps of…” and “Demonstrate confidence on the dance floor.” [Applying] At what levels did Prof. Dancealot teach his class?” [Remembering and Comprehending] “At what level did he assess?” [Applying] While comical in this video, this is quite common. We want to make sure this doesn’t happen to you. Using the Application Model. Then ask “At what levels did Prof. Dancealot teach his class?” [1-Knowledge in one discipline. He taught them the steps.] “At what level did he assess?” [4-Apply to real-world predictable situation. He turned on the music and just said “Dance” with no indication of what kind of dance they should do to the song and no previous practice whatsoever.] Okay. Enough of Prof. Dancealot. He’s not teaching here anymore. Check for understanding: How do you determine the quadrant your SLOs are in? In which quadrant are authentic assessments more appropriate? Transition to next slide: Now, let’s use the Rigor and Relevance Framework with your own SLOs and EOs. Below is a reminder of what was discussed in Tier I: The Rigor and Relevance Frame work has two continuums working together. One side of the framework focuses on levels of thought using Bloom’s Taxonomy. Each level demonstrates a more complex and involved treatment of a given piece of information or content. At the bottom learners understand terms and meanings and at the top they are able to apply them in complex ways. There are Six levels: Level 1 – Remembering (Knowledge) Level 2 – Understanding (Comprehension) Level 3 – Applying (Application) Level 4 – Analyzing (Analysis) Level 5 – Evaluating (Synthesis) Level 6 – Creating (Evaluation) The other side focuses on the application of information. Similar to Bloom, this continuum involves increasingly complex application of knowledge in increasingly diverse scenarios. Whereas Bloom deals primarily with understanding and information processing, the application framework deals with how that information is put into practice and used. When combined, the Framework provides 4 quadrants of performance. The end goal of an educational process is to get learners into the D-quadrant however daily/weekly SLOs may need only end in one of the others. All SLOs and EOs are plotted on the Rigor and Relevance framework to determine what quadrant they fall. Once we determine the quadrant each is in we are able to determine the appropriate assessment types. *adapted from the explanation in Daggett (2000) Daggett, W. R. (2000). Moving from standards to instructional practice. National Association of Secondary School Principals.NASSP Bulletin, 84(620), 66-72. http://search.proquest.com/docview/216047841?accountid=34899

Activity Using the SLOs and EOs that you brought: Identify the level on the Knowledge Taxonomy axis and on the Application Model axis using the Rigor and Relevance Framework provided in your folder. Identify the quadrant in which they fall on the Rigor and Relevance Framework. Exchange with a partner to review your work. Raise your hand if you need assistance. Unit Time: 15 minutes Cumulative Time: 48 minutes Refer participants to their handouts for the Rigor and Relevance Framework where they can plot the SLOs and EOs (if they have EOs) and Bloom’s Taxonomy chart. Participants use the SLOs and EOs they brought and will work independently to complete the task. If participants did not bring SLOs to use they may use the CRJ121 SLOs located in their handouts. There is no report out on this activity. Be available to answer questions.

Building An Assessment Steps in building an assessment: Determine which SLOs and EOs it will assess. Plot the SLOs and EOs on Rigor and Relevance Framework. Use the Recommendations for Assessments, Instructional Strategies, and Assignments handout to determine the best assessment methods. Determine criteria for success for the assessment. Unit Time: 2 minutes Cumulative Time: 50 minutes 1. Review the steps necessary when building an assessment. In order to begin, you need to know which SLOs and EOs you are assessing and which criteria a student would have to meet in order to complete the assessment successfully.

Creating an Authentic Assessment Activity Based on the previous activity in which you identified the levels of Rigor and Relevance and the corresponding quadrants: Develop an appropriate authentic assessment for your course. Choose the SLOs and EOs that you would like to assess in an authentic and real world way. What is the best method of assessment? Determine criteria for success. Report out. Unit Time: 20 minutes Cumulative Time: 70 minutes 8 minutes to work 12 minutes to report out Work with the same person that you paired with in the last activity. This way pairs will be familiar with the content and Rigor and Relevance plotting. If a participant’s SLOs and EOs fall into quadrants A or C, assist them in “challenging” their students to accomplish a higher level of application by moving the SLOs into quadrant B or D.

Rubrics 3 types of rubrics Examples of rubrics Creating a rubric Assessing a rubric Unit Time: 1 minutes Cumulative Time: 71 minutes Rubric creation overview

3 Types of Rubrics Analytic Rubrics Holistic Rubrics Checklists Unit Time: 1 minutes Cumulative Time: 72 minutes Facilitator Notes: 1. Review the three types of rubrics that will be central to this PD.

Analytic Rubric Provide specific feedback along several dimensions. Advantages: more detailed feedback, scoring more consistent across students and graders Disadvantage: time consuming to score Unit Time: 1 minutes Cumulative Time: 73 minutes Facilitator Notes: 1. Rubric creation overview

Example of Analytic Rubric Unit Time: 1 minutes Cumulative Time: 74 minutes Facilitator Notes: 1. Rubric creation overview

Holistic Rubric Provide a single score based on an overall impression of a student’s performance on a task. ƒAdvantages: quick scoring, provides overview of student achievement ƒDisadvantages: does not provide detailed information, may be difficult to provide one overall score Unit Time: 2 minutes Cumulative Time: 76 minutes Facilitator Notes: 1. Review the contents on the slide.

Example of a Holistic Rubric Unit Time: 1 minutes Cumulative Time: 77 minutes Facilitator Notes: 1. Review the contents on the slide.

Checklist Contains a list of behaviors or specific steps Checklists are a simple list of assessment criteria or components that must be present in student work. All that is needed is a place to mark whether or not the student has accomplished the task or not, there is no judgment on the quality of the work. Unit Time: 2 minutes Cumulative Time: 79 minutes Facilitator Notes: 1. Review the contents on the slide.

Example of a Checklist Unit Time: 1minutes Cumulative Time: 80 minutes Facilitator Notes: 1. Review the contents on the slide.

Guidelines Objective descriptors Holistic and Analytic Rubrics Use a 3+ zero scale Use a 4+ zero scale Use a point range starting at zero Waypoint ready Checklists Yes and No Unit Time: 2 minutes Cumulative Time: 82 minutes Facilitator Notes: Review the contents on the slide. Emphasize and discuss the value of having a zero scale. Tell the participants that as we move forward the guidelines listed above will be practiced by all individuals who are creating rubrics including our instructional design team. 4. Waypoint is the software system that we are using to collect student learning data. 5. Capstone and direct measure courses that have been through Assessment Process are the priorities to go into Waypoint.

Weighting Points on Rubric When reviewing or developing your rubric consider the weight that is distributed to measuring the SLOs. Consider ISLOs like… Effective communication in various academic and career setting using technology as appropriate Unit Time: 2 minutes Cumulative Time: 84 minutes Facilitator Notes: Review the contents on the slide. Note that weighting is optional it is dependent on the course.

Group Activity Using the assessment you created, construct an appropriate rubric. Identify the following briefly for your rubric: Components Scale Criteria for at least one component Do you need component weights? Point Be ready to share with the group. Unit Time: 25 minutes Cumulative Time: 109 minutes Facilitator Notes: *Refer the participants to the “Types of Rubrics Basics” documents for support and for detailed steps as to how to create each of the type of rubrics. Participants may not get the entire rubric done within this time, but will have enough time to get a decent start.

Assess the Rubric Conduct a peer review. Ask one or two other instructors to review your rubric. Provide time for student review. Allow students to ask questions and make comments. Unit Time: 2 minutes Cumulative Time: 111 minutes Facilitator Notes: Validity – comes from Rigor and Relevance – are these valid assessments? Inter-rater reliability – if at all possible, collect a group of faculty who will teach the course and use the rubric. Have them use the rubric to score a sample paper then discuss the scoring.

Implement and Refine Refine your rubric based on feedback from other instructors and students. Make notes each time you use the rubric for continuous improvement purposes. Share with others. Unit Time: 2 minutes Cumulative Time: 113 minutes

Review… Built an authentic assessment. Determined: Which SLOs and EOs it will assess. Where the SLOs and EOs fall on Rigor and Relevance. What quadrants your SLO and EO fall into by using the Recommendations for Assessments, Instructional Strategies, and Assignments handout. The best assessment method to use to assess your SLOs and EOs. Criteria for success for the assessment. Created an analytic rubric. Unit Time: 2 Minutes Cumulative Time: 115 Minutes Facilitator Notes: Review topics that have been discussed in this professional development.

What Now? What is one thing that you are going to take away and implement from this professional development session? Unit Time: 1 minutes Cumulative Time: 116 minutes Facilitator Notes: Individually have participants complete the My Take Away from Tier Two Assessment handout. This is something they can take with them.

Potential Column Headings Outstanding | Accomplished | Proficient | Developing | Beginning Accomplished | Average | Developing | Beginning Excellent | Good | Needs Improvement | Unsatisfactory Exceptional | Acceptable | Marginal | Unacceptable Expert | Practitioner | Apprentice | Novice Professional | Adequate | Needs Work | You’re Fired Exceeds Expectation | On Target | Beginning Exemplary | Competent | Developing High | Medium | Low Outstanding | Proficient | Shows Potential This slide is to serve as an additional resource.