Presentation on theme: "Developing Quality Assessments"— Presentation transcript:
1Developing Quality Assessments Presented by Ellen Vorenkamp for the MI Assessment Consortium at Lewis Cass ISD Best of Formative Assessment Series Session 5
2Overview The need for Quality Assessments The Development Process Accurate DataUse of DataStudent GrowthEducator EvaluationThe Development ProcessMAC ModulesPlanning for your Assessment (CASL)Key Components—Common Core ConnectionsTest Blue PrintsQuality ItemsConclusion
3Good Assessment applies to all Learners Regardless of student ability or communication level, our assessments must have a clear purpose and stretch our students to higher levels of doing and thinkingGood assessment design is fundamental to allEach student is uniqueAs ever – take the best and adapt to our kids!
4Essential Questions to Dialogue Around… As you develop quality assessments, keep the following questions in mind…What is the purpose of the assessment?Who will use the information?How will it be used?Is the use formative or summative?
5A Formative ViewAs you develop quality assessments, keep the following questions in mind…What is the purpose of the assessment?To provide teachers immediate information on student learningWho will use the information?Teachers and students in the classroomHow will it be used?To inform next steps in the learning processIs the use formative or summative?Formative
6A Summative ViewAs you develop quality assessments, keep the following questions in mind…What is the purpose of the assessment?Educator Evaluation / AccountabilityWho will use the information?Teachers and AdministratorsHow will it be used?To certify the learning processIs the use formative or summative?Summative
7Think…Pair…Share…What elements are necessary to ensure quality assessments?List these qualitiesDiscuss why these are important
8MAC’s Thinking… 24 Modules that cover the entire development process… PlanningModules 1-7DevelopingModules 8-20ImplementingModule 21ReflectingModules 21-24
9Quality Assessments… are Reliable and yield Valid data. In order for these two requirements to be met assessment developers must pay special attention to the following:Standard/Item AlignmentBalance of RepresentationTarget-Method MatchQuality ItemsThe best way to ensure your assessment is reliable and valid is to create a test blueprint and follow the blueprint while developing the assessment.7-11 on chart
10Test BlueprintsRequires a bit of planning and forethought as you begin this process…Several steps should be completed that will help make developing your test blueprint easier
11Standard AlignmentAre the assessment items tightly aligned with the standards?Do the assessment items address the targeted standards sufficiently and at all levels of cognitive demand?Are there an equal number of assessment items per standard? If not, is there “rationale”?Are there enough assessment items per standard to determine mastery?
12Balance of Representation Items are balanced and should reflect what was taught and how it was taught within the classroom environmentThe assessment as a whole should reflect the emphasis of the lesson, unit or other educational experience on which you are assessing.For example, a social studies assessment might cover 4 related standards or learning targets…
15Target-Method MatchIdentify the Depth of Knowledge (DOK) for the standardMatch the cognitive demand of the standard with an appropriate item typeDetermine which item types will be included in the assessment and in what proportion.
18SYNTHESIS AND EVALUATION BLOOM’S TAXONOMYWEBB’S DOKKNOWLEDGE“The recall of specifics and universals, involving little more than bringing to mind the appropriate material”RECALLRecall of a fact, information, or procedure (e.g., What are 3 critical skill cues for the overhand throw?)COMPREHENSION“Ability to process knowledge on a low level such that the knowledge can be reproduced or communicated without a verbatim repetition.”APPLICATION“The use of abstractions in concrete situations.”SKILL/CONCEPTUse of information, conceptual knowledge, procedures, two or more steps, etc.STRATEGIC THINKINGRequires reasoning, developing a plan or sequence of steps; has some complexity; more than one possible answerANALYSIS“The breakdown of a situation into its component parts.”SYNTHESIS AND EVALUATION“Putting together elements & parts to form a whole, then making value judgments about the method.” EXTENDED THINKINGRequires an investigation; time to think and process multiple conditions of the problem or task.Wyoming School Health and Physical Education Network (2001). Standards, Assessment, and Beyond. Retrieved May 25, 2006, from
19Prepared and published by the Educational Policy Improvement Center
20Prepared and published by the Educational Policy Improvement Center
21Reflection… What are the implications for classroom instruction? What are the implications for classroom assessment?
22Types/Levels of Standards Recall and Reproduction– (Knowledge) Facts and concepts we want students to know (Usually DOK Level 1 or 2)Skills & Concepts – (Skills) Students use their knowledge and reasoning to act skillfully (Usually DOK Levels 2; Can go higher based on context)Strategic Thinking– (Reasoning) Using what they know to reason and solve problems (Usually DOK Levels 2-4)Extended Thinking– (Application/Products) use knowledge, reasoning, and skills to create a concrete product (Usually DOK 3-4)
23Methods of Assessments Selected ResponseConstructedResponsePerformance AssessmentObservations/ConversationsMultiple ChoiceTrue-FalseMatchingDiagramFill-in-the-blank (words, phrases)EssayShort answer (sentences, paragraphs)WebConcept MapFlowchartGraphTableMatrixIllustrationPresentationMovementScience labAthletic skillDramatizationEnactmentProjectDebateModelExhibitionRecitalPerformance TaskOral questioningObservationInterviewConferenceProcess descriptionChecklistRating scaleJournal sharingThinking aloud a processStudent self-assessmentPeer review23Adapted from the work of Dr. Robert Marzano23
24Response/ Short-Response Performance Assessment Target-Method MatchHow well does your method of assessment match your target?Target to be AssessedAssessment MethodSelectedResponse/ Short-ResponseExtended -ResponsePerformance AssessmentObservation/ConversationsKnowledge/RecallGood matchNot a good matchPartial matchReasoning/Strategic ThinkingPerformance SkillsApplication/Products
25Test Blueprint Putting it all together: Standard Specific Learning Target derived from StandardTarget DOK LevelAssessment MethodNumber of Items/Points
26Performance TasksPerformance tasks challenge students to apply their knowledge and skills to respond to complex real-world problems. They can best be described as collections of questions and activities that are coherently connected to a single theme or scenario. These activities are meant to measure capacities such as depth of understanding, writing and research skills, and complex analysis, which cannot be adequately assessed with traditional assessment questions. The performance tasks may take one to two class periods to complete.
27Components of a Performance Assessment Performance Assessments:call upon the examinee to demonstrate skills and competencies (application)are comprised of 4 basic components:A reason for the assessmentIdentify the content and/or skill standard(s) to be assessedA specific performance to be evaluatedConstruct the prompt or item and determine the criteria for successExercises to elicit that performance, andDescribe the way you will collect evidence, when you will collect the evidence and how much evidence you will collect.Systemic rating proceduresDetermine the score for proficiency or mastery; determine who will rate the performance, scoring method (rubric, checklist, anecdotal record, portfolio)
28SBAC Performance Tasks (PT) Performance tasks, the most complex of all items, include the following elements: Integrate knowledge and skills across multiple claims. Measure capacities such as depth of understanding, research skills, and/or complex analysis with relevant evidence. Require student-initiated planning, management of information/data and ideas, and/or interaction with other materials. Reflect a real-world task and/or scenario-based problem Allow for multiple approaches. Represent content that is relevant and meaningful to students. Allow for demonstration of important knowledge and skills. Require scoring that focuses on the essence of the Claim(s) for which the task was written. Seem feasible for the school/classroom environment.
38Additional Sample Items ents/asmt-sbac-math-hs-sample-items.pdfScroll down to page 3
39Implications for the Classroom Assessments will evolve to be more rigorous and real world relevant, must match our teaching to this same standardAssessment data must be used in the moment to inform “next steps” in the learning processSchools must have a balanced assessment system in place within their classrooms
40Not the end of the journey… MAC ModulesMuch left to do to ensure quality assessments
41A Final Thought…“Remember, formative assessment works,” says Popham. “When it is used, students learn better. By using this assessment-rooted instructional process, teachers can increase the test-based achievement of their students… ‘Student growth’ will be demonstrated on the tests because, in fact, student growth will have occurred.”“Formative Assessment’s ‘Advocatable Moment’” by James Popham in Education Week, Jan. 9, 2013 (Vol. 32, #15, p. 29)