Presentation on theme: "District Determined Measures"— Presentation transcript:
1District Determined Measures Professional Development DayJanuary 27, 2014White Brook Middle School StaffAnd Elementary SpecialistsShirley Gilfether
2District Determined Measures Measures of student learning, growth, and achievement related to the Massachusetts Curriculum Frameworks, Massachusetts Vocational Technical Education Frameworks, or other relevant frameworks, that are comparable across grade or subject level district-wide.
3The Educator Evaluation Framework Beginning educators earn two ratingsMassachusetts Department of Elementary and Secondary Education
4Student Impact Rating Regulations For each educator there must be at least two measures.Options – 603 CMR 35.07(1)(a)(3-5)Statewide growth measure(s)[MCAS SGP must use if applicable]District-determined Measure(s) of student learning comparable across grade or subject district-wide.For educators whose primary role is not as a classroom teacher, the appropriate measures of the educator's contribution to student learning, growth, and achievement set by the district.
5Rating of Impact on Student Learning Two RatingsSummative RatingExemplary1-yr Self-Directed Growth Plan2-yr Self-Directed Growth PlanProficientNeeds ImprovementDirected Growth PlanUnsatisfactoryImprovement PlanLowModerateHighRating of Impact on Student LearningMassachusetts Department of Elementary and Secondary Education
6Multiple sources of evidence inform the summative rating Administrator Observations; Teacher supplied evidenceClassroom assessments; Benchmarks from SLG; recurring assessments- BAS, writing prompts….Explain:Here is a visual representation of the entire process. You can see the three categories of evidence along the left, and how they inform practice associated with each of the four Standards (rubric is the frame), as well as an assessment of progress on educator goals. In its totality, an evaluator uses this information to determine an overall summative performance rating of exemplary, proficient, needs improvement or unsatisfactory.More information on determining a Summative Performance Rating is available in ESE’s related guidance document, located on its website.Other evidence related to EES standards; student feedback6Massachusetts Department of Elementary and Secondary Education
7District-Determined Measures DDMs may inform both an educator’s summative performance rating and impact ratingSummative Performance RatingStudent Impact RatingEvidenceProducts of practice (e.g., observations)Other evidence relevant to one or more of the four Standards of practice (e.g., student surveys)Multiple measures of student learning, growth and achievement, including:Measures of student progress on classroom assessmentsMeasures of student progress on learning goals set between the educator and evaluatorEvidenceTrends and patterns in student learning, growth & achievementAt least two years of dataAt least two measuresStatewide growth measures, where available (including MCAS SGP)Additional DDMs comparable across schools, grades, and subject matter district-wideExplain:Let’s look at precisely how and where DDMs will inform an educator’s evaluation. Here we have the required sources of evidence for both evaluation ratings.With respect to the Summative Performance Rating, multiple measures of student learning, growth and achievement comprise one of the three categories of evidence that must be taken into account, and must include measures of student progress on classroom assessments, as well as measures of student progress related to an individual educator’s learning goals. District-determined measures can serve this role for the Summative Performance Rating, depending on the nature of the educator’s goals or educator plan.With respect to the Student Impact Rating, data from at least two state or district-wide measures of student learning gains over at least two years is used to determine each educator’s impact on student growth. MCAS SGP must be one of these measures for educators with students taking the MCAS, and can be both where available. Bottom line: the Impact Rating is always based on a trend over time of at least two years, and it should reflect a pattern in the results on at least two different assessments, which is where district-determined measures come into play.Massachusetts Department of Elementary and Secondary Education
8The two ratings work together Explain:To wrap up… Here you can see how these processes ultimately work together. Across the top, you’ll see the evidence categories that inform an evaluator’s judgment of practice associated with the four Standards, as well as the goals, and ultimately, the development of a Summative Performance Rating. On the bottom, you see how trends and patterns of student performance, evident within at least two measures of student learning, inform the Rating of Impact.These two ratings come together and determine type and duration of educator plan, as well as when and how to recognize excellent teachers and leaders.8Massachusetts Department of Elementary and Secondary Education
9Student Impact Rating Regulations Evaluators must assign a rating based on trends (at least 2 years) and patterns (at least 2 measures)Options – 603 CMR 35.09(3)(a-c)high indicates significantly higher than one year's growth relative to academic peers in the grade or subject.moderate indicates one year's growth relative to academic peers in the grade or subject.low indicates significantly lower than one year's student learning growth relative to academic peers in the grade or subject.
10SGP from MCAS as Growth Measure Scores not available until fall of following yearStudent impact rating only determines a one, or two-year plan, and is a separate rating from the Summative Rating, which determines the type of Plan you will be onThe State has determined the SGP range for ratingLOW34MODERATEHIGH66
11District-Determined Measures DDMs should measure growth, not achievement. Student growth measures answer the fundamental question of, “Where did my students start and where did they end?”All DDMs have to have baseline data…some point of origin for the growth….some measure of the same core objectivesAssessments should be administered across all schools in the district where the same grade or subject is taught. (e.g. 2nd gr ELA will be BAS for everyone)DDMs should assess learning as directly as possible.Massachusetts Department of Elementary and Secondary Education
12Measures of Growth – 4 Options Every measure MUST have a BASELINE Pre- Test / Post- Test Pre- and post-tests can be identical measures administered twice or comparable versionsRepeated Measures Design Some teachers use short measures throughout the year to monitor student growth on a set of skills.Holistic Evaluation A holistic evaluation of student growth combines aspects of a pre- and post-test model with the regularity of a repeated measures approach. These use a rubric that describes growth over time.Post- Test only Not really feasible for a locally made assessment. Only applies to MCAS and some commercial product because there is not baseline
13Two fundamental questions should be the guideposts for selecting DDMs as a measure of student learning:Is the measure aligned to content?Does it assess what is most important for students to learn and be able to do?Does it assess what the educators intend to teach?2. Is the measure informative?Do the results inform educators about curriculum, instruction, and practice?Does it provide valuable information to educators about their students, helping them identify whether students are making the desired progress, falling short, or excelling?Does it provide valuable information to schools and districts about their educators?
14Measures of Growth with Specific Assessment Types Portfolios If a portfolio is to be used as a DDM that measures growth, it must be designed to capture progress rather than to showcase accomplishments.Unit Assessments While a common form of assessment, it is necessary to have baseline data to compare. Also, one unit alone is not enough of a measure.End-of-Course Exams While many courses have these already, it is again necessary to have baseline data. For this reason, these are rarely used as DDMs, unless pre-assessment data is collected.Capstone Projects Capstone Projects are large-scale student projects that represent a culmination of the work completed in a course. Perhaps the biggest challenge in using capstone projects as DDMs is the difficulty with measuring growth. The goal of DDMs is that they measure student growth over the year.
15Developing a DDM Assessment Step 1 – Identify the key content (CCO, key standards, concepts or skills…)may be taught repeatedly across the year, or once in the yearbe sure that it is informative (informs teachers, students, and administration)Step 2 - Ensure that change in performance represents student growthstarts with a baseline, measures similar content, demonstrates what students know and don’t knowStep 3 - Select an approach for measuring growthpre/post, repeated measures, holistic which might include portfolios, performance tasks, unit assessment, capstone projects, etc. Think about the type of assessments coming with PARCC
16Developing a DDM Assessment (cont.) Step 4 – Begin to build assessmentsalign to content, seek ideas from other resources as available, think about weighting of certain questionsStep 5 - Decide on a scoring protocolraw score, percent score, rubric scorehow will growth be determined [raw to raw, % to %....]Step 6 – Draft a scale for low, moderate and high impactmoderate is based on what would be the expectation for most students or a year’s worth of growth; low is below the expectation; and high is significantly higher than the expectation
17Assessment ProtocolsIt is important to know that there need to be a set of protocols for the assessments used as DDMs. For example, assessments should be done on the same day, have the same set of directions, use the same scoring methods, etc. The protocols are similar to the steps taken to administer MCAS and assure the reliability of the assessment. More information will become available once DDMs have been chosen for courses.
18For More InformationMTA You Tube piece on Student Growth PercentileDESE website – Educator Evaluation – District Determined MeasuresTechnical Guide BExample DDMs – based on Core Course ObjectivesThe examples include Core Course Objectives for many levelsGo to Assessment Literacy Webinar SeriesGo to Presentations on left menu instead of DDM …and look for Getting Started ppt for Educator EvaluationVisit PARCC or Smarter Balance websites for samplesMaterials and information stored on the NEW district website including this ppt and the DDM packet of infoContact Shirley Gilfether who will get answers to your questions
19Questions and AnswersIMPORTANT NOTES:DDM Pilot going on now- grade 10 ELA, grade 7 music and grade 5 and 3 mathematicsFebruary 3rd –DDMs should be decided for each course (only the basics of course/level and type of DDM)April 4th – Drafts of DDMs to administrators