1. IMPROVING THE QUALITY OF LOCALLY-DEVELOPED ASSESSMENTS MSTC FEBRUARY 18, 2016 DR. ELLEN VORENKAMP WAYNE RESA MAC BOARD MEMBER DR. ELLEN VORENKAMP WAYNE RESA MAC BOARD MEMBER

2. WHO’S IN THE ROOM? Who am I? Who are you?

3. WORKING AGREEMENTS ATTN: First Signal means complete conversations and bring attention to the speaker. Take responsibility for your own learning. Enrich and support the learning for everyone. Take care of your personal needs.

4. LEARNING OUTCOMES Participants will… recognize the need for ensuring quality classroom assessments identify and distinguish the elements of quality assessment understand a process for ensuring quality assessments recognize appropriate resources to aid in the development/modification of locally-developed assessments

5. Table activity Protocol – Chalk Talk Center of chart paper write Quality Assessments Without comment… What comes to mind when you think of quality assessments? Debrief/Categorize SETTING THE STAGE…

6. SETTING THE STAGE We need to do something different!

7. OUR CHARGE… We have reached a tipping point: We either change our assessment beliefs and act accordingly, or we doom struggling learners to inevitable failure. ---Rick Stiggins

8. THE HOW 5 Keys to Quality Assessment Carousel

9. What is the purpose of our classroom assessments? Do we share the results with the correct users? How can we improve in this area?

10. Do students and teachers have clarity around what we need students to know and be able to do and what proficiency on it looks like? How can we make learning more transparent?

11. How do we ensure our assessments are of high quality? What more can we do to ensure sound design?

12. How is information from our assessments communicated to the correct users? How can we ensure better communication with our learning partners?

13. In what ways are students involved in the assessment process? What more can be done to make them partners in the assessment process?

14. KEYS TO QUALITY ASSESSMENTS Clear Purposes Why Assess? What’s the purpose? Who will use results? Clear Targets Assess What? What are the learning targets? Are they clear? Are they good? Good Design Assess How? What method? Sampled how? Avoid bias how? Sound Communication Communicate How? How manage information? How report? Accurate Assessment Effectively Used Student Involvement Students are users, too. Students need to understand learning targets, too. Students can participate in the assessment process, too. Students can track progress and communicate, too.

15. KEY 1: CLEAR PURPOSE Evidence gathered from assessments informs instructional decisions Some decisions occur frequently: As learning progresses teachers and students need to know what comes next in student learning or determine what is blocking student learning Some decisions are made periodically: When we certify learning for purposes of report card grades or identify students for special services Some decisions are made less frequently: Such as when districts need to make adjustments in instructional programs or resources

16. CLEAR PURPOSE As a result it becomes apparent that different assessments serve a variety of users and uses, centering on student achievement at varying levels, and require a variety of different assessment information gathered at different times

17. CLEAR PURPOSE Student Bill of Rights By Rick Stiggins excerpted from Improve assessment literacy outside of school to, PDK October, 2014 Students are entitled to know the purpose of each assessment in which they participate; that is, they have a right to know specifically how the results will be used and by whom. Students are entitled to know and understand the learning target(s) to be reflected in the exercises and scoring guides that make up any and al assessments. Students are entitled to understand the differences between good and poor performance on pending assessment and to learn to self-assess their progress toward mastery. Students are entitled to dependable assessment of their achievement gathered using quality assessments.

18. CLEAR PURPOSE As you develop quality assessments, keep the following questions in mind… What is the purpose of the assessment? Who will use the information? How will it be used? Is the use formative or summative?

19. A FORMATIVE VIEW As you develop quality assessments, keep the following questions in mind… What is the purpose of the assessment? To provide teachers immediate information on student learning Who will use the information? Teachers and students in the classroom How will it be used? To inform next steps in the learning process Is the use formative or summative? Formative

20. A SUMMATIVE VIEW As you develop quality assessments, keep the following questions in mind… What is the purpose of the assessment? Educator Evaluation / Accountability Who will use the information? Teachers and Administrators How will it be used? To certify the learning process Is the use formative or summative? Summative

21. ASSESSMENT: COMPARING PURPOSES Assessment of learning…(summative) strives to document student achievement. diagnoses a program’s strengths and weaknesses by providing comparable results. provides summative results at the end of a unit or course of study. informs others (teachers, parents, administrators, community members) about students and their achievement. assumes the teacher’s role is to gauge student success. reflects the standards themselves. Assessment for learning…(formative) strives to increase student achievement diagnoses a student’s strengths and weaknesses by providing results that are unique to individual students. provides data throughout a unit or course of study that allows tailoring instruction and motivation for improvement. informs students about themselves and helps them learn how to take charge of their own progress. assumes the teacher’s role is to promote student success. reflects the knowledge, skills, and understandings that underpin standards.

22. ASSESSMENT: COMPARING PURPOSES

23. META MOMENT How confident are you that assessments given in classrooms in your school/district have a clear purpose? How is that purpose communicated to the students and other interested parties? Is the data from the assessments used for the intended purpose?

24. KEY 2: CLEAR LEARNING TARGETS We must have a clear sense of the achievement expectations we need our students to master Expectations must be worded in student-friendly language and be shared with students We need to ensure the targets are tightly aligned with the standards and are taught/learned at the appropriate cognitive demand

25. CLEAR LEARNING TARGETS Research…John Hattie High Leverage Strategies Activity What did we find? Impact of our findings?

26. HIGH LEVERAGE STRATEGIES 1.44 Student Expectations.90 Formative Evaluation for Educators.90 Teacher Credibility in the Eyes of the Students.82 Classroom Discussion.75 Feedback.75 Teacher Clarity.74 Reciprocal Teaching.72 Teacher-Student Relationships.69 Metacognitive Strategies.67 Vocabulary Programs.64 Self-Verbalization/Self-Questioning.61 Problem solving Teaching

27. CLEAR LEARNING TARGETS “Making the learning intentions and success criteria transparent, having high, but appropriate, expectations, and providing feedback at the appropriate levels is critical to building confidence in taking on challenging tasks.” John Hattie, Visible Learning for Teachers: Maximizing Impact on Learning, 2012

28. CLEAR LEARNING TARGETS Learning Targets, Connie Moss and Susan Brookhart Quality Formative and Summative Assessments

29. LEARNING TARGETS… A target becomes a learning target only when students use it to aim for understanding throughout a particular lesson, and students aim for a target only when they know what it is. Learning targets must be shared and actively used by both halves of the classroom learning team—the teacher and the students.

30. LEARNING TARGETS Clear Learning Targets shift us away from what we, as teachers, are covering towards what our students are learning.

31. SUCCESS CRITERIA? Success criteria are statements that specifically outline what the students need to do in order to achieve their learning target. “…success criteria summarize the key steps or ingredients the student needs in order to fulfill the learning intention (target) – the main things to do, include or focus on.” -Shirley Clarke (Ireland School Board)

32. SUCCESS CRITERIA Setting Criteria with students is a powerful tool that helps students talk through and think about what mastery of the learning target looks like. Success criteria describes what it means to do quality work and be proficient on “today’s” learning target. Criteria is often displayed in well developed rubrics. These can be developed for or with students.

33. EFFECTIVE SUCCESS CRITERIA… Are linked to the learning target Are discussed and agreed with students prior to undertaking the activity Are linked to what students learned yesterday and what the will learn tomorrow Are used to determine learning progression Are used to activate student prior knowledge Provide a scaffold and focus for students while engaged in the activity Are used as the basis for feedback and peer/self assessment Are used to determine mastery of the learning target through assessment, both formative and summative

34. META MOMENT Does the assessment author have a clear idea of what she or he is trying to measure? Do all teachers agree on the evidence to be collected? Are the learning targets aligned and derived from the curriculum standards? Is there a clear/shared criteria for success? Are the learning targets shared in student friendly language and used throughout the lesson? Do the learning targets create a clear, reasonable learning progression? Do the learning targets represent the emphasis of what was taught at the appropriate cognitive demand?

35. KEY 3: SOUND DESIGN Items must be well written and developed using sound item development guidelines. Items should be field tested to determine their reliability Items should be arranged appropriately within the test construct and hold true the intent of the blue print Test should be field tested Data reviewed Revised and Modified as necessary

36. SOUND DESIGN Assessments must be accurate Accuracy requires selecting the appropriate assessment method for the context of what is being assessed Once the method have been selected, we must ensure the assessment items, tasks, questions are of high- quality Finally, we must anticipate what could go wrong with the assessment and prevent those issues if possible

37. SOUND DESIGN Quality Assessments…are Reliable and yield Valid data. In order for these two requirements to be met assessment developers must pay special attention to the following: Standard/Item Alignment Balance of Representation Target-Method Match Quality Items The best way to ensure your assessment is reliable and valid is to create a test blueprint and follow the blueprint while developing the assessment.

38. WHAT’S YOUR DOK?

40. METHODS OF ASSESSMENTS Multiple Choice True-False Matching Selected Response Diagram Fill-in-the-blank (words, phrases) Essay Short answer (sentences, paragraphs) Web Concept Map Flowchart Graph Table Matrix Illustration Presentation Movement Science lab Athletic skill Dramatization Enactment Project Debate Model Exhibition Recital Performance Task Oral questioning Observation Interview Conference Process description Checklist Rating scale Journal sharing Thinking aloud a process Student self- assessment Peer review Constructed Response Performance Assessment Observations/ Conversations Adapted from the work of Dr. Robert Marzano 40

41. Target-Method Match Activity How well does your method of assessment match your target? Target to be Assessed Assessment Method Selected Response/ Short-Response Response/ Short-Response Extended - Response Performance Assessment Observation/ Conversations Knowledge/ Recall Reasoning/ Strategic Thinking Performance Skills Applications/ Products

42. Target-Method Match How well does your method of assessment match your target? Target to be Assessed Assessment Method Selected Response/ Short-Response Response/ Short-Response Extended - Response Performance Assessment Observation/Conversations Knowledge/Recall Good match Not a good match Partial match Reasoning/ Strategic Thinking Partial match Good match Performance Skills Not a good match Good match Partial match Application/Products Not a good match Partial match Good match Not a good match

43. SOUND DESIGN Methods of Assessment Written Response Selected Response Short Written Response Extended Response/Essay Performance Tasks Observation/Conversation Collection of existing work in portfolios Each has advantages and disadvantages

44. SOUND DESIGN

45. Gives knowledge of methods, procedures, and analysis of skills related to student learning. Performance assessments, in concert with more traditional forms of assessment can give us a more complete picture of student achievement. Provides better insight into a student’s level of conceptual and procedural knowledge. PERFORMANCE TASKS

46. Demonstration of skills through process or product Designed to judge students abilities to USE knowledge and skills Performance matches the depth of knowledge of skills and abilities being measured Usually requires students to manipulate equipment, solve a problem, or make an analysis Provide Scoring is based on observation and judgment

47. META MOMENT Is the assessment method most appropriate for the learning target being assessed? Are the assessment items and/or tasks assessing at an appropriate depth of knowledge? Are the assessment questions and/or tasks written well? In the case of a constructed response item and/or a performance task, does the scoring guide or rubric clearly indicate the criteria on which the student performance is judged, a scale that identifies levels of performance and a detailed description of student performance at varying levels of quality? 2 Are the items free from bias? Have the items been field tested?

48. KEY 4: EFFECTIVE COMMUNICATION Assessment results need to be communicated effectively to the appropriate intended user Everyone must understand the meaning of the achievement target Information in the communication must be accurate Everyone must understand the symbols being used to report information Communication must be tailored to the intended audience (level of detail, timing, format)

49. ISRAEL STUDY Three kinds of feedback… Scores Comments Both Scores and Comments Results?

50. RESULTS FEEDBACK TYPE Scores Comments Both Scores and Comments PERCENT GAIN 0% Gain 30% Gain 0% Gain

51. ELEMENTS OF FEEDBACK Timing Amount Mode Audience Individual Group

52. FEEDBACK

53. TYPES OF FEEDBACK Motivational EvaluativeDescriptiveActionable Feedback is primarily motivational Feedback is primarily evaluative Descriptive feedback primarily tells the student how to correct their reasoning. Actionable feedback asks the student to move their learning to the next level and gives them ideas on how. Purpose: to encourage and support the learner Purpose: to measure student achievement with a score or a grade Purpose: to improve learning by indicating to the student what needs to be improved Purpose: to improve learning, by moving student reasoning to the next level MoreSummativeMoreFormative

54. DESCRIPTIVE, ACTIONABLE FEEDBACK Goal is for students to internalize and use the actionable feedback. Feedback that is intended to be used by the learner to independently move their reasoning to the next level. “I agree with the pattern that you have identified in the table. I am not convinced that the rule you wrote works for all the values in the table. How could you prove this?”

55. KEY IMPLICATION Students must be given the opportunity to apply the feedback by trying again (Black & Wiliam, 1998)

56. META MOMENT How will the information from the assessment be gathered and reported? Has communication been planned as part of the assessment process? Do teachers record assessment information accurately over time and combine it appropriately for reporting? Will the users of the results understand them and find them useful?

57. KEY 5: STUDENT INVOLVEMENT Critical shift in our current perspective regarding assessment Most important decisions based on the data or not made by the adults, but the students themselves. They decide whether the learning is worth the effort required to retain it. They decide if they are capable of reaching the target. It is only after students make these decisions in the affirmative that teachers can impact their learning

58. WHAT IS STUDENT SELF- ASSESSMENT? Student self-assessment occurs when students review their own work and identify strengths and weaknesses for the purpose of improving their performance and attaining mastery of the learning targets. This happens when students compare their work against the criteria for success. This “criteria” can be set by the teacher, the student or both.

59. WHAT IS STUDENT SELF- ASSESSMENT? Student self assessment and student goal-setting are directly related to feedback. By offering descriptive, actionable feedback we model for students the kind of thinking we want them to engage in as they revisit and further their own learning. The next step of implementation is to empower students to take on this skill and responsibility.

60. WHAT IS STUDENT SELF- ASSESSMENT They also must reflect on the learning tactics used when the evidence was collected and how successful (or unsuccessful) that strategy was for them.

61. WHY STUDENT SELF- ASSESSMENT? According to Jan Chappius…”When students self- assess and set goals they develop an internal sense of control over the conditions of their success and greater ownership of the responsibility of improving.” She continues…”Engaging in self-assessment and goal setting can also cause students to value feedback more highly than the grade.”

62. HOW DOES SELF-ASSESSMENT SUPPORT STUDENT LEARNING? Develop insight into their own learning Rather than rely solely on teacher feedback, they begin to monitor their own learning Consider what part of an assignment meets set criteria and which part needs more attention Empowers students to think about their thinking (meta- cognition) Students become more engaged in the curriculum Students become more aware of the learning tactics that benefit them

63. HOW DOES SELF-ASSESSMENT SUPPORT TEACHERS? Teachers are able to see gaps in student learning As teachers collect student self-assessments they get additional assessment data to inform their instruction Teachers go beyond the product and get to assess data on student thinking

64. HOW DOES SELF-ASSESSMENT SUPPORT TEACHERS? Teachers provide students with time to process new learning Providing time for students to pause and think, to look for proof and to connect to criteria allows teachers to slow the pace of their teaching to match the speed of student learning Teachers provide students time to consolidate their learning before moving on

65. WHAT IS STUDENT PEER- ASSESSMENT? Peer assessment is the assessment of student work with input from other students and is an integral part of the formative assessment process.

66. WHAT STUDENT PEER ASSESSMENT IS NOT… Students grading papers, so the teacher does not have to… It goes much deeper than that…

67. WHY STUDENT PEER ASSESSMENT? It has been recognized that engaging students in peer assessment can help them in learning to evaluate their own learning, the success of their learning strategies, and in interpreting assessment criteria. Further benefits might also include: increasing feedback to others; giving students a sense of ownership of the assessment process; encouraging students to be more involved and take more responsibility for their learning; Increasing awareness of varied learning tactics, and encouraging critical analysis of others work, so students see beyond a mark/grade.

68. HOW DOES PEER-ASSESSMENT SUPPORT STUDENT LEARNING? Peer assessment involves students taking responsibility for assessing the work of their peers against set assessment criteria. They can therefore be engaged in providing feedback to their peers. It's a powerful way for your students to act as the 'assessor' and to gain an opportunity to better understand assessment criteria.

69. HOW DOES PEER-ASSESSMENT SUPPORT STUDENT LEARNING? It can also transfer some ownership of the assessment process to them, thereby potentially increasing their motivation and engagement. In doing so, your students might be encouraged to learn more deeply, building up their understanding, rather than just their knowledge of the facts, as well as gaining an insight into their own approach to an assessment task in comparison to their peers.

70. HOW DOES PEER-ASSESSMENT SUPPORT STUDENT LEARNING? You may find that peer assessment is particularly useful in aiding your students to develop judgment skills, critiquing abilities, adjust learning tactics and build their own self- awareness.

71. WHY SHOULD I USE PEER- ASSESSMENT? Students are doing it already in different ways. Students will get the chance to find out more about assessment culture. Learning is enhanced when students have contributed to their marking criteria. Assessing is a good way to achieve deep learning. Students can learn from the successes of others. Successful learning strategies can be shared. Students can learn from other's mistakes. It is a good way to generate timely feedback.

72. HOW DOES PEER-ASSESSMENT SUPPORT TEACHERS? The process of explicitly defining and explaining assessment criteria can be of benefit to your own assessment practices. Deeper learning will translate into better summative scores. May reduce teacher load.

73. META MOMENT Are students involved in the assessment process? Are students involved with tracking progress and setting goals? Is descriptive feedback given to the students in a timely manner? Is reflective use of assessment data an integral part of learning?

74. CONTACT INFORMATION Ellen Vorenkamp 810-923-0327 vorenke83@gmail.com