Presentation on theme: "NYCDOE’s Advance: Measures of Student Learning"— Presentation transcript:
1NYCDOE’s Advance: Measures of Student Learning CFN 109 Leadership Meeting, August 26, 2013
2CFN109 Leadership MoSL Training Welcome, Review Agenda, Norms & Goals10:45 – 11:00 a.m.Overview of MoSL Long & Short Term Priorities11:00 – 11:10 a.m.Assessment Types11:10 – 11:30 amBaseline Measures11:30 – 11:35 amGoal Setting vs. Growth Model11:35 – 11:45 amThe Default Option11:45 – 11:55 amReview of Next Steps11:55 am – 12:00 pmQ & A12:00 – 12:10 pmFeedback12:10 – 12:15 pm
3 Goals for the Session School Leaders will: Be able to work with Local Measures Committees to make decisions about Assessments, Target Populations & Growth Measures for MoSL that can be feasibly implemented within their schools.Understand the short and term critical dates and deadlines for implementing MoSLWelcome/introductionsWhy are we here together for the next few days and what are we hoping to accomplish
4Session Norms High Engagement, Low Tech – Please turn off cell phones. Come Together – Please raise your hand when facilitator calls for attention.Contribute to the Learning – Throughout the day, write any outstanding questions on index cards and place them in the center of your table. Q&A at the end or follow up s and conversations will address these and any other lingering questions.Maximize Our Time Together – Use this day to collaborate, troubleshoot and brainstorm with your colleagues to prepare your school for implementing MoSL.Time:1- 2 minFacilitator: JessicaMaterials: NoneOverview: Introducing the norms will begin to assimilate them to our culture and set a standard of expectations for all them as a cohort. This is extremely important to state up front. This can impact the level of quality all participants experience.Facilitation:~ <1 min Facilitator reads the text in the blue boxes and emphasizes the importance of norms and expectations of active participation which will impact what they take away as learners and what their colleagues take away. Next, ask 1 TC each to read the four bullet points. Before transitioning, quickly check that all TCs agree to participate within these norms.~1 min Facilitator transitions: For the next activity we want to get to know each other a little better and know what we are bringing to the party. What experiences make up what is known as the Office of Teacher Effectiveness? What journeys have we all taken to get to this destination?
5New York State Education Law 3012c Passed in 2010, New York State Education Law 3012c introduced significant changes to the Annual Professional Performance Review for teachers and principals.The law is intended to foster clearer expectations for practice aligned to improvements in student achievement.Summary of State Policy on Teacher EvaluationTeachers will be evaluated annually on a 4-point rating scale (Highly Effective, Effective, Developing, Ineffective)40% of a teacher’s evaluation will be based on an assessment of student growthAll classroom teachers will be evaluated against uniform qualitative rating criteria through use of a state-approved research-based rubric of teacher practiceTeachers will receive timely and constructive feedback, including use of improvement plans for tenured teachers who receive a Developing or Ineffective End-of-Year ratingTime:2 minFacilitation: LisaMaterials:Overview of the activity:Overview of the DOE, framing of the division of talent, labor and innovation, learnings from the first 3 years of this work, policy context, etc. (include successes/impact from the history of the work – data)Facilitation:Say: While the details of our new teacher evaluation and development system may seem like new information to some, the DOE has engaged in intensive work and piloting efforts in the past 3 years.The Teacher Effectiveness Program helped New York City prepare to implement NYSED Law 3012 c for the past 3 years.Passed in 2010, the law changed the requirements for teachers’ performance reviewThe law is intended to foster clear understandings of excellent teaching practice, aligned to high standards for student achievementFour parts of the lawEvaluation on a 4 point HEDI scalePortion of evaluation based on student growth (EMPHASIZE THIS MOSL IS LAW)Use of a rubric to measure teaching practiceProvision of feedback to teachers, catered to need5
6All teachers will receive: Teachers in Grade 3-12 will receive: Sixty percent of a teacher’s overall rating will be based on Measures of Teacher Practice (MOTP).All teachers will receive:Initial planning conference and summative end of year conference, including artifact reviewChoice between two observation approachesWritten and/or verbal feedback and observation reportsStudent Feedback via Tripod Student Survey (no stakes pilot in ; worth 5 of 60 points beginning in )Let’s look more closely at the Measures of Teacher Practice, which will make up 60% of a teacher’s overall rating.Facilitator describes the Measures of Teacher Practice, which account for 60% of a teacher’s evaluation rating.One on one initial and end of year conferences with their principal or assistant principal to support the evaluation and development process.Conferences are a valuable opportunity for teacher and school leader together to reflect on a teacher’s practice and how to make improvements to it to further support student learning.In addition, teachers will be able to choose how they want to receive their observations.For teachers in grades 3-12, we will also pilot the use of a student survey in and this survey will count for 5 of a teacher’s 60 measures of teacher practice points beginning in the following school year.Teachers in Grade 3-12 will receive:
7All teachers will receive: Forty percent of a teacher’s overall rating will be based on Measures of Student Learning (MOSL).All teachers will receive:Two different measures of student learning (40%)State MeasureLocal MeasureMultiple measures provide a more valid, robust picture of teacher performance, providing teachers with multiple sources of feedbackFacilitator briefly previews the 40% Measures of Student Learning rating, and indicates that this will be covered more extensively in the next session.
8Other Measures of Teacher Practice (60%) Summative Evaluation Rating Every teacher will receive an overall summative rating based on multiple measures of teacher effectiveness.Other Measures of Teacher Practice (60%)State Measure (20%)Local Measure (20%)Summative Evaluation RatingExampleHighly Effective60%20%Highly Effective OverallTime:2 minFacilitation: LisaMaterials: noneOverview of the activity: Facilitator summarizes the overall summative rating using the talking points listed belowThis next slide gives you a sense of how these scores are going to be combined.The [Measures of] Teacher Practice, the State Growth Score, and Locally Selected Measure will each have a point-value associated with them.They will sum up together to get an overall rating.You see in this example that the teacher was “Highly Effective” on each of those, and ended up at “Highly Effective” overall.Now, there is one element in the law I want to note, that’s important here and that’s gotten a lot of attention: the law specifically says that if a teacher is “Ineffective” on both their State Measure and their Locally Selected Measure, then they’re “Ineffective” overall.We’ve done a lot of analysis of the data, and it is a very rare occurrence, based on the projections that we’re seeing, that someone will be “Ineffective” in both of these and their principal has also rated them highly.It happens in less than half of 1% of the cases, so we don’t think this will affect many people, but it is an important rule for folks to be aware of.
9Focus on the Forty Percent: Transition slide: Now we will focus specifically on the 40% of a Teacher’s Advance rating that is based on student outcomes or Measures of Student Learning.
10Advance MoSL – Implementation Steps September - NovemberMake and submit SelectionsProgram students in STARSTrain Teachers and Relevant Support StaffInform teachers of State and Local MeasuresAdminister, Norm & Score (when necessary) baseline assessmentsComplete Goal Setting Process (schoolwide and teacher level)December – MarchComplete data entry of teacher programs, assessment selections, and student-level goals(High Schools) Prepare for 2nd semesterApril – June3-8 NYS ELA and Math Testing & ScoringNYSESLAT Testing & ScoringGrades 4 & 8 Science Performance & Written Testing and ScoringNYC Performance Assessment & 3rd Party Assessment Administration & Scoring (Teachers cannot score the exams of their own students)Summer / Fall 2013State test scores, NYS Growth Scores, NYC Growth Scores releasedFinalization of teacher scores for Local and State MeasuresTeachers informed of overall performance ratings2 minReview key dates.Draw attention to the “optional” nature of the 10/4 date and what this means for inclusion in goal-setting.
11Upcoming DatesBy 9/9Principal and School Local Measure Committee makes selections. Principal enters selections into Advance Web ApplicationStudents must be programmed in STARS for first teacher-student-course data captureSeptember (ongoing)Schools train teachers on any new protocols and/or assessments in place.Schools administer all baseline assessments, keeping in mind any new security and administration protocols.Schools norm and score NYC Performance Assessments.September 16Principals receive Teacher-Level Selection Tool populated with 9/9 data. Principals correct data, where required.Schools begin printing answer sheets from ATS where teachers record NYC Performance Assessments scores for upload.September 24Principals must inform teachers of State and Local Measures of Student Learning (will be available for print from Advance Web App)For GOAL SETTING, Principals and Teachers begin to set classroom, grade and school-level goals.October10/3 – Data Capture for Version #2 of Teacher-:Level Selection Tool10/4 – For teachers doing GOAL SETTING: enter results of baseline assessment (if school wants this data included in DOE-predicted targets).10/15 – For GOAL SETTING ,School and grade goals due to Superintendent10/31 – Deadline for all baseline assessment administration and submission.November11/1 For GOAL SETTING: Goal setting worksheets with DOE-produced targets available for download from STARS.11/15 For GOAL SETTING: Deadline for Principals to approve teachers’ classroom level goals
12School Local Measures Committee - Role and Responsibilities Before September 9:Principal and UFT chapter leader has each selected 4 members.*School Local Measures Committee uses six-step process (one-hour per step) to make recommendations for Local Measures:Discuss important introductory information.Review assessment options.Finalize assessment selection.Review growth measurementsFinalize growth measurements.Present recommended approach to principalFacilitator Note: Principals are encouraged to be part of their committee.*Principals and UFT Chapter Chairs are encouraged to be part of their School Local Measures Committee.OverviewTimelineRoles & Resp.Application
13After September 9th - Roles and Responsibilities Principal:If implementing goal-setting, work with teachers to adjust targets and approve goals.Discuss progress toward goals throughout the year.Meet with teachers for summative discussion of results.Teacher:Understand the measures that will be used and for which courses taught.Understand the assessment that must be administered and the timeline for doing so.Administer and, where necessary, score baselines.Norm on NYC Performance Assessment scoring rubrics.If goal-setting, adjust DOE targets and submit goals to principal.OverviewTimelineRoles & Resp.Application
14CFN109 Leadership MoSL Training Welcome, Review Agenda, Norms & Goals10:45 – 11:00 a.m.Overview of MoSL Long & Short Term Priorities11:00 – 11:10 a.m.Assessment Types11:10 – 11:30 amBaseline Measures11:30 – 11:35 amGoal Setting vs. Growth Model11:35 – 11:45 amThe Default Option11:45 – 11:55 amReview of Next Steps11:55 am – 12:00 pmQ & A12:00 – 12:10 pmFeedback12:10 – 12:15 pm
15Key Considerations for School Local Measures Committees The following are key considerations schools will need to keep in mind for the first year of implementation:The school community will select its Measures of Student Learning annually. Schools will be able to make different selections for subsequent school years.Schools should use the DOE-supplied tools to focus on critical questions:What assessments and target populations do we use?E.g., Do we want to use an existing formative assessment, such as running records for evaluative purposes, or would we rather use a school-wide measure?What will we do for growth measurements?While goal-setting gives teachers more control, it requires more time and is often less predictive of student growth than growth models.For both decisions, consider:How much time will this take? Do we want to invest this time in light of the other critical work in our school?How does this decision build upon or change current practice in our schools?OverviewTimelineRoles & Resp.Application
16 Assessment Options NYC Assessment Type Example (K-5) Target Population Options*State Assessments4-5 Math and ELA State AssessmentsIndividualGradeSchool3rd Party AssessmentsPerformance SeriesNYC Performance AssessmentsNew!(for release this year)NYC* There are some exceptions to these options – see the Measures of Student Learning Guide for target population options by assessment.Make sure facilitator references additional bulleted rules on in guides (page 23 of elementary guide).For every assessment selected, there must also be a target population selected.Target Population refers to the students who are included in the measure for a given assessment.
17Assessment List – Changes from 8/1 Added“ELA Performance Assessments” in K-8 now include an option between Running Records and writing-based Performance Assessments.8th Grade Science NYC Performance AssessmentSANDI/FASTAlways included in D75 supplementIntegrated Algebra SVMINew Visions schools onlyAdditional clarification for allowable State and Local optionsAPLocal Measure use onlyDiscovery MathLimited use for K-2 only schoolsChanged3 min- Review changes to the assessment list.
18Assessment Options: State Assessments State Assessments: Measure the performance of students based on state-created assessments.Considerations:Can be used as a school-wide measure by most teachers in a school where a State Assessment is administered.If you select the same assessment for Local Measures and State Measures, the target population must be a different set of students.No new assessments or workflow needed (in some cases, a baseline test must be administered)M2: AssessmentsState3rd PartyPerformance
19Assessment Options: State Assessments State Assessments include:Grades 3-8 ELA and Math State AssessmentGrade 4 and Grade 8 State Science AssessmentNYSESLAT (ESL)NYSAA in ELA and MathState Regents (English, Math, Science, Social Studies)Where a state assessment is administered by a teacher, it is required as the State Measures assessment (for common branch K-8, if both ELA and Math are taught, they both must be used for the State Measure*).* For schools that departmentalize, teachers are only responsible for the content they teach.M2: AssessmentsState3rd PartyPerformance
20Assessment Options: State Assessments State assessments may be selected as the Local Measures for:Teachers who administer state assessmentsIn this situation, it must target a different population of students (options: grade, school, or lowest third of performers in the class)Teachers of ELA in grades 4-8 must use a NYC Performance Assessment as one of the Local Measures, but this may be supplemented with other state assessment measuresTeachers who do not administer state assessmentsCan use the state assessment as either a grade-level or school-wide measure.This will frequently be a good option for teachers:Of subjects without individual assessmentsWho teach courses which largely affect performance in a state-tested grade/subjectM2: AssessmentsState3rd PartyPerformance
21Assessment Options: 3rd Party Assessments 3rd Party Assessments: Created by assessment experts; assessment format varies (multiple choice, performance based, etc.)Considerations:Only state-approved 3rd Party Assessments (in Guide) can be used.Not available for all grades and subjects.Include both baseline and end-of-year assessments.Additional administration procedures needed.Teachers (per law) cannot score their own end-of-year assessment results.M2: AssessmentsState3rd PartyPerformance
22Assessment Options: 3rd Party Assessments Considerations for Use:Schools may choose to focus on 3rd Party Assessments that are already in use, rather than implementing new assessments.Some schools use 3rd Party Assessments for formative purposes throughout the year and may not want to alter their intent by using them for evaluation.Teachers may not score their own students’ end-of-year assessments, so additional procedures must be put in place to account for this.For teachers using GOAL SETTING, baseline assessments must be administered and scores uploaded by October 4th. All baselines must be administered by October 31st.M2: AssessmentsState3rd PartyPerformance
233rd Party Assessments – Scantron Performance Series Performance Series (Scantron) MOSL options:ELA: Grades 3, 9-11Math: Grades 3 through GeometryConsiderations:Administration/scoring will not change for MOSL, with the exception of a pre-determined window for pre-/post-test administration.Can also use for Periodic Assessments; beginning- and end-of- year administration will count toward MOSL.No additional baseline selection needed.Online, web-based administration; schools must have adequate bandwidth and hardware to administer.2 min- (Still need confirmation) - If administering twice within testing window, first administration will count as baseline and last administration will count as final assessment.
243rd Party Assessments – Discovery K-2 Considerations:An option for Math in K-2 only schools.K-5 and K-8 schools should use school wide performance on the state math assessment for K-2 teachers’ State Measure.For K-2 only schools:May choose to use beginning-of-year Discovery administration for baseline along with growth model or goal-setting.For Kindergarten ONLY: May choose school-created baseline rather than administering Discovery pre-test in beginning-of- year (must use goal-setting in this instance).2 minThis is new information, so review each point carefully.Explain why this decision was made (K-2 only).Explain baseline implications and why it is different for Kindergarteners (don’t know how to take test) – can select portfolio of work even, does not have to be a traditional assessment, but in this case, must use goal-setting.
253rd Party Assessments: Turn & Talk At your tables, there are examples of some of these 3rd Party Assessments. Take time to review the assessments most applicable to your school.While you do this, discuss:What third party assessments do we currently use and are we considering using for MoSL?What will be the implications be for the school if we a third party assessment? What work would we have to make sure is in place? Who will do this and when?10 minActivity led by table facilitatorsMake sure participants know where at their tables to find the samples that are provided.
26Assessment Options: NYC Performance Assessments NYC Performance Assessments: Authentic tasks (e.g., evidence-based essays), scored against a common rubric. Created by the DOE, NYC teachers, and curriculum and assessment experts to be used as Measures of Student Learning in teacher evaluation.Considerations:Not available for all grades and subjects.Include both baseline and end of year assessments.Require additional time for training, scoring, and recording results.Teachers (per law) cannot score their own end-of-year assessment results.M2: AssessmentsState3rd PartyPerformance
27Assessment Options: NYC Performance Assessments BackgroundCreated and piloted selected sample assessments in SY ’12-13.Goal: Design assessments that achieved guiding principles80 schools participatedNYC Performance Assessments:Required for the State Measures where they exist and no state assessment exists – Grades 6-8 Social Studies & 6-7 ScienceThe ELA Performance Assessment is required as at least one of the Local Measures for teachers of ELA in grades 4-8.May be selected as baseline assessments for specific state assessment (e.g. 4th Gr. Science).For complete listing, see Guide.M2: AssessmentsState3rd PartyPerformance
28Performance Assessments – Overview NYC Performance Assessments are available in multiple K-12 subjects, some as optional, others as required. These include baseline and end-of-year assessments (no additional selections for baseline needed).If the principal and/or committee chooses the default option, the mandated NYC Performance Assessments in grades 4-8 ELA for the Local Measure do not need to be administered as all teachers in the school will use the default measure.There are Performance Assessments required in some grades/subjects for the State Measure. Those assessments are not affected by the selection of the default option for the Local Measure.In most grades/subjects with Performance Assessments, there is one type of assessment. However, in K-8 ELA, committees have two options within the Performance Assessment category:Running Records NYC Performance Assessment (using one of three reading programs)Writing-based NYC Performance Assessment3 minReview all points on slideFocus on the optional vs. mandatory nature of performance assessments and how choosing the default option gets rid of the 4-8 ELA Local Measure requirement (but not in other places where performance assessment mandated for State Requirement)Focus on the fact that there are two options for the K-8 ELA performance assessments – will review both today
29Running Records (K-8) – Overview Three programs may be used for MOSL Running Records.Running Record program options for MOSL include:Fountas and Pinnell (F&P)Developmental Reading Assessment (DRA2), orTeacher’s College (TCRWP)Text funding is included in Periodic Assessment funding for elementary schools.Schools must establish structures so that teachers do not score final assessments of students whose scores will factor into their evaluation rating.Text Security – Texts used for Running Record assessment purposes should be kept separate from other texts (i.e., not used for instructional purposes).2 min- Review running record program options
30Running Records: Implementation Steps By 9/9Principals and School Local Measure Committees work together to select between Running Records and writing-based Performance Assessments for ELA K-8.SeptemberSchools administer first assessment as baseline. Must consider options for scoring: by the classroom teacher (keeping in mind that the spring administration CANNOT be scored by the classroom teacher) or by someone other than the classroom teacher.OctoberBy 10/4 – Submit results of baseline assessment in order to receive DOE-predicted targets with this data included (optional).By 10/31 – All baseline assessments administered, scored, and submitted.2 min- Review timeline. Note that this is the same timeline for all assessments, not just RRs.
31Running Records: Turn and Talk Thinking about Running Records:What are the implications if our school chooses Running Records?What new systems must be put in place in order to ensure testing security & compliance with mandate that teachers cannot score their own students’ end of year assessments?10 min- Led by table facilitators
32Performance Assessments: NYC Performance Assessments Considerations:Review of where they exist on assessment listELA (writing-based): K-12Math: Grade 3, Integrated AlgebraSocial Studies: 6-8, Global History, U.S. HistoryScience: 4, 6-8, Living EnvironmentIn K-8 ELA, also have choice of Running Records.All NYC Performance Assessments (except ELA) translated into 9 languages.Require norming on rubrics at the school level.Samples available for download from Advance Intranet Site (final versions downloaded from Advance Web Application by 9/9).3 min- These were all on the old assessment list except 8th grade science, which was added in the new version- Need to remind people on this slide that we are now talking about the “other” type of performance assessment – this type exists across several grades and subjects, whereas the running records are just specific to ELA K-8
33NYC Performance Assessments: Turn and Talk Thinking about NYC Performance Assessments:How will baseline performance assessments be administered in September?How will scoring be normed across the school (to prepare for requirement that teachers cannot score their own end of year assessments)?What preparation, training, and time is required to administer and score these assessments reliably?10 min- Led by table facilitators
34Performance Assessments: Implementation Steps By 9/9Final performance assessments released online (samples by 8/15).School Local Measure Committees recommend whether or not to administer Performance Assessments where optional. Principal approves or chooses default.For ELA K-8, principals and committees decide between Running Records and a writing-based performance assessment.Principals plan necessary teacher support around administration and scoring.SeptemberSchools develop scoring protocols.Schools norm teachers on Performance Assessment rubrics to ensure a shared understanding of demands of the rubric and scoring across student work.Teachers administer Performance Assessments.OctoberSchools score and scan Performance Assessments. Must submit scores by 10/4 for inclusion in DOE targets (otherwise by 10/31).2 min- Review points on slide. Timeline isn’t different for performance assessments, but does include some additional work in terms of norming in September.
35CFN109 Leadership MoSL Training Welcome, Review Agenda, Norms & Goals10:45 – 11:00 a.m.Overview of MoSL Long & Short Term Priorities11:00 – 11:10 a.m.Assessment Types11:10 – 11:30 amBaseline Measures11:30 – 11:35 amGoal Setting vs. Growth Model11:35 – 11:45 amThe Default Option11:45 – 11:55 amReview of Next Steps11:55 am – 12:00 pmQ & A12:00 – 12:10 pmFeedback12:10 – 12:15 pm
36Baseline AssessmentsWhere choices exist, schools may opt to use a DOE-recommended baseline assessment or a school-selected baseline.Remember:Baselines only need to be selected for a few state assessments (4th grade ELA, Math & Science) and AP exams.* All other 3rd party and Performance Assessments come with baseline assessments.Where baselines must be selected, the PRINCIPAL chooses.If the selected baseline is not on the DOE list, schools must use goal-setting. DOE targets provided to those schools will not include results from the baseline.For schools using goal-setting:By October 4: Administer and submit baseline scores if schools want data to be used in the DOE calculation of target suggestions.By November 15: Goal-setting approval deadline. Schools must administer baseline assessments in time to write and approve goals for this date.All schools must administer, score, and submit baseline assessments by October 31.5 minReview information about baselinesDraw attention to the fact that only principals can select baselines (there are very few instances where a committee would have to select a baseline, but where this comes up, principal needs to select – ex. If using 4th grade Science Assessment for Local but not State, or AP exams in HS)Draw attention to the fact that goal-setting MUST be the growth measurement if baseline is school selected.* Principals also have the option of selecting an alternative baseline assessment for Discovery Math in K only.
37CFN109 Leadership MoSL Training Welcome, Review Agenda, Norms & Goals10:45 – 11:00 a.m.Overview of MoSL Long & Short Term Priorities11:00 – 11:10 a.m.Assessment Types11:10 – 11:30 amBaseline Measures11:30 – 11:35 amGoal Setting vs. Growth Model11:35 – 11:45 amThe Default Option11:45 – 11:55 amReview of Next Steps11:55 am – 12:00 pmQ & A12:00 – 12:10 pmFeedback12:10 – 12:15 pm
38Growth Measurement Options After choosing assessments and target populations, the next step is choosing one of two growth measurements for each assessment.Growth Models: DOE calculates student targets, results, and teachers’ scores. Results are shared after assessments have been administered so student growth can be compared to similar students’ performance on assessments.Goal-Setting: DOE provides targets for how students will perform on assessments that principals and teachers can adjust based on their knowledge of students. Principals approve targets.5 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
39Growth Models: Overview 39Growth models allow us to compare the progress that students make in a year to similar students.In the State Growth Model, similar students are defined by four student-level characteristics at the student and classroom levels:Academic historyEconomic disadvantageDisability statusLanguage learner statusCitywide models account for similar characteristics to the State model.Growth models control for the degree to which students are expected to make gains given their achievement history and demographic characteristics.3 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
40Defining Characteristic of “Similar Students” State Growth ModelCategoryDefining Characteristic of “Similar Students”Academic History:Prior year test score, same subjectPrior year test score, different subjectRetained in gradeNew to school in year other than entry yearAverage prior achievement and range around average prior score in student’s class/course (same subject)Student with Disability (SWD) Student has an IEPSWD spends less than 40% of time in general education settingPercent SWD in student’s classEnglish Language Learner (ELL)Student is an ELLNYSESLAT scoresPercent ELL in student’s classPovertyStudent poverty indicator (yes/no)Percent poverty in a student’s classThe State Growth Model compares the progress that students make in a year to a group of similar students statewide.In the State Growth Model, similar students are defined by 13 characteristics divided into four categories at the student and classroom level.To be included, a student has to be enrolled for a minimum of 60% of the year.Student scores are weighted by the percent of time they were enrolled in the course and their attendance5 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
41Growth Models: Considerations 41Does not introduce additional work in schools.Enables schools to compare their students’ performance to similar students.Gives teachers credit for the degree to which students exceed predicted growth.Better able to account for unexpected outcomes resulting from unfamiliar, new assessments.Growth model score results are not available until after assessments have been administered (i.e., the following spring/summer).Does not actively facilitate brainstorming and discussions about individual student performance and goals.May be more difficult to explain to schools.BenefitsChallenges5 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
42State Growth Model: Scoring and Results Most teachers scored Effective on the state growth score in The results for NYC were slightly higher than the rest of the state:NYC Teachers (n=10,544)Rest of State (n = 22,585)Highly Effective8%6%Effective76%77%Developing10%IneffectiveNote: SED has stated that it expects the future distribution of ratings to remain similar to what it was in5 minutes
43Citywide Growth ModelIn addition to the State Growth Model, which covers grades 4-8 ELA and Math at this time, the DOE is also producing a Citywide Growth Model.This is available as a growth measurement option for any state or local assessment not covered by the State Growth Model.Like the State Growth Model, the citywide model will control for multiple factors including academic history, English language learner status, disability status, and poverty.3 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
44Citywide Growth Models Process 44DOE Reports Results and ScoresDOE Measures Students’ GrowthAdminister Baseline and End-of-Year AssessmentsFall: Baseline assessments administered to students (not required for all assessments).Spring: End-of-year assessment administered to studentsGrowth models measure students’ growth compared to similar students on the end-of-year assessment.Similar students are determined by baseline performance, academic history, and demographic characteristics.DOE provides student targets, results, and teachers’ scores to teachers and principals after assessments have been administered.5 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
45Goal-Setting Process Administer baseline assessment DOE sends predicted student targetsTeachers review DOE predicted targetsPrincipals approve or adjust targetsAdminister end of year assessmentTeachers’ RatingsBaseline assessment administered (not required for all assessments).Report baseline assessment results.DOE sends predictions for how individual students will perform.Predictions are based on baseline performance, student achievement history, and student demographic characteristics.Teachers may choose to adjust these targets based on additional information about their students.Teachers submit student targets to principal.Principal (or designee) report finalized student targets.10 minutesEnd-of-year assessment administered to students.Teachers’ HEDI ratings are calculated with a conversion chart based on students’ performance on outcome assessments relative to their targets.M3: GrowthGrowth ModelsGoal-SettingPractice
46Goal-Setting Considerations Particularly valuable for teachers/schools with unique student populations or high mobility.Allows teachers and principals to individually tailor student goals.Requires additional time/resourcesTeacher’s rating is based on the percentage of students who meet their target but does not consider the degree to which students fall below or exceed their target.Targets must be set early in the school year, possibly before much diagnostic info is gatheredSetting goals may be challenging if:Teachers are not familiar with the comparability between assessmentsAssessments are new or changingBenefitsChallenges5 minutesM3: GrowthGrowth ModelsGoal-SettingPractice
47Goal-Setting Considerations This is not the same as the goal-setting you may typically see in schools.Goals are scored against a state conversion chart which makes the target-setting process difficult and non-intuitive.Before considering goal-setting, make sure you understand the work and additional training this entails.M3: GrowthGrowth ModelsGoal-SettingPractice
48Goal-Setting – Who Sets Goals Teachers set goals for students in their classroom (target population: individual), but when goals go beyond an individual teacher’s students (target population: grade or school), the principal sets the goal and the superintendent approves.Target PopulationTeachersPrincipalsSuper.Individual LevelBy student; orBy subgroups of students; orWhole class goalSetApprove*Grade LevelWhole grade goalApproveSchool LevelWhole school goal2 minutes- Several goal types by target population, will go into detail in a minute. Most important thing here is the set vs. approve for different target populations. For an “individual” target population teacher sets, discusses and adjusts with principal, principal approves. However, and this is new information, for school and grade target populations, the principal sets the goals and the superintendent approves.* If a teacher fails to submit goals, the principal will set the goals for that teacher. The principal should not simply “approve,” but rather work with the teacher on adjusting targets as necessary.
49All targets must be set as ranges. Goal-Setting – RangesAll targets must be set as ranges.Example #1: A teacher creates goals for each student within a class:Johnny will score between 50 and 75.Jane will score between 60 and 80.Jose will score between 55 and 80.(and so forth…)Example #2: A teacher creates goals for a class of students:All students (i.e., each student) will score between 60 and 85.Note: Given how the SED conversion chart works, this is the same as setting a target such as “Johnny will score 50 or higher.”3 min.- Key points: goals must be set as ranges, not single numbers, not percentages. Ultimately, it is the lower number that counts when deciding if a student “hit” a target per the state scoring chart.
50Goal-Setting – Types of Targets Teachers (with principal approval) can set three types of targets for individual-level MOSL goal-setting*Classroom LevelExample: “All students will score between 50 and 75.”Note: Teachers cannot set targets in the format “85% of students will score 75% or higher.”Subgroup LevelExample: “All students with 8th grade scores of 1.50 to 2.49 will score 55 to 80; all students with 8th grade scores of 2.50 to 3.49 will score 65 to 85, etc.”Note: Only subgroups based on incoming performance are allowed per SED rules.Student LevelExample: “Johnny will score between 55 and 75.”3 minutes- Discuss the three types of goals teachers can set for students in their classroom.*This includes “Individual – Lowest Third” targets, which must be the same target as is set for the broader target population.
51Goal-Setting – Types of Targets Principals (with superintendent approval) can set two types of targets for grade or school target populations.Whole Group Level (grade or school target population)Example: “All students in school/grade will score between 50 and 75.”Subgroup Level (grade or school target population)Example: “All students with 8th grade scores of 1.50 to 2.49 will score 55 to 80.”Note: These goals are set on a slightly different timeline. Principals should use reference tables released on 9/24 in order to draft and submit targets to superintendents by 10/15 for a final approval date of 11/15.3 minutesDiscuss the two types of goals that principals can set for grade or school level goals. Note: Individual student targets are not an option.Also note the difference in timeline for approvalPrincipals should follow DOE suggested target ranges for subgroup and whole school and not adjust too much – this will have implications on what may be approved at the superintendent level (clarify wording of this with ORD).
52From Goals to Ratings: Scoring Teachers’ MOSL subcomponent scores are calculated from the chart below using the following process:Calculate the percent of students who met or exceeded their target.Use the following table to determine the teacher’s HEDI points and rating:Note: 60% of a teacher’s students need to meet or exceed their target to earn a rating above “Ineffective.” If a teacher sets goals above the DOE-predicted student target or the assessment is more difficult than expected, the teacher may be at risk of receiving a low rating.HEDI Conversion Chart% of students meeting or exceeding targetHighly EffectiveEffectiveDevelopingIneffectiveHEDI Pts2019181716151413121110987654321% at Target100-9796-9392-9089-8584-8079-7574-6766-6059-5554-4948-4443-3938-3433-2928-2524-2120-1716-1312-98-54-03 minThis is the scoring chart for the MOSL subcomponent of goal setting. Once scores are returned, the percentage of students hitting the target is calculated and then mapped to this chart to determine rating for this subcomponent of the overall Advance rating.As you can see here, at least 60% of students must meet their target in order to get above an Ineffective on the SED scoring chart.
53DOE Target Suggestions Target suggestions will be provided by the DOE.DOE target suggestions for school and grade target population (for principal use in goal-setting):By September 24 – Reference tables available.Tables with target ranges by assessment administered in each school.Tables with subgroup target ranges by assessment.By October 15 – Principals submit targets to superintendents for review.DOE target suggestions for individual classroom target population (for teacher use in goal-setting):By November 1 – Student-level targets available that reflect any baseline data submitted by October 4.Teachers may also use reference tables mentioned above prior to receiving student-level targets in order to draft goals prior to November 1.All goals must be finalized by November 15.3 minReview differences in timeline and resources for different levels of goals.Explain that they will see an example of a reference table in the upcoming activity.
54Goal-Setting: Fall Implementation Steps By 9/9School Local Measure Committees recommend whether or not to use goal-setting for Local Measure.Principals decide the same for State Measure (where the option exists).SeptemberSchools who choose goal-setting norm on expectations across classrooms.Schools administer baseline assessments.9/24 – DOE target reference tables and goal-setting worksheets released on Intranet.OctoberBy 10/4 – Schools submit results of baseline assessments if they want data included in DOE-suggested targets (otherwise submit by 10/31).By 10/15 – Principals submit grade/school level goals to superintendent for review.Teachers start process of goal-setting using baseline data and reference tables.November11/1 – Schools can access goal-setting worksheets which include baseline results and DOE-suggested targets.11/15 – Principals finalize teacher-set goals. Superintendents finalize school/grade goals.3 minRemind participants that apart from these dates, should consider additional time needed to norm on target setting, gather background data for adjusting goals, printing forms, completing targets, approving targets, etc.
55Turn and Talk: Goal-Setting Implications Given this activity, what support do you think schools will need when setting targets?Resources?Help interpreting reference tables?Help collecting classroom-based evidence?Help adjusting targets?How can you help principals to think through what it means to set grade or school level targets and work with teachers to adjust targets for classroom goals?How will you provide this level of support (coaching sessions, PD, etc.)?10 min- With your tables, reflect on this activity and the supports you can provide schools in your network to help with this process.
56CFN109 Leadership MoSL Training Welcome, Review Agenda, Norms & Goals10:45 – 11:00 a.m.Overview of MoSL Long & Short Term Priorities11:00 – 11:10 a.m.Assessment Types11:10 – 11:30 amBaseline Measures11:30 – 11:35 amGoal Setting vs. Growth Model11:35 – 11:45 amThe Default Option11:45 – 11:55 amReview of Next Steps11:55 am – 12:00 pmQ & A12:00 – 12:10 pmFeedback12:10 – 12:15 pm
57Default MeasuresIf a committee cannot reach decisions about Local Measures, or the principal does not approve the committee’s recommendation, the default option will be put in place.GradeLocal MeasuresAll Teachers in a Building with Grades 4-8Assessment: State ELA & Math Assessments (School)Measurement: Growth Model (SED)All Teachers in a Building without Grades 4-8Assessment: School-wide: All assessments at the school for State MeasuresMeasurement: Student average of school growth measures
58CFN109 Leadership MoSL Training Welcome, Review Agenda, Norms & Goals10:45 – 11:00 a.m.Overview of MoSL Long & Short Term Priorities11:00 – 11:10 a.m.Assessment Types11:10 – 11:30 amBaseline Measures11:30 – 11:35 amGoal Setting vs. Growth Model11:35 – 11:45 amThe Default Option11:45 – 11:55 amReview of Next Steps11:55 am – 12:00 pmQ & A12:00 – 12:10 pmFeedback12:10 – 12:15 pm
59Immediate Next StepsBy September 9 principals must submit decisions in the Web Application regarding measures for teachers in their schools.Before September 9:Principals must select assessments (and, where applicable, baseline assessments), target population, and growth measurements for State Measures for grades/subjects without required measures.*School Local Measures Committees must select assessments, target population, and growth measurements for Local Measures for all grades/subjects.School Local Measures Committees must present their recommendation to the principal (if not on the committee) for approval.Principals must either approve the School Local Measures Committee recommendation or choose the default option.* In some cases, an assessment may be required, but the principal will select a baseline assessment and the growth measurement.OverviewTimelineRoles & Resp.Application