2. Purpose & Context
This work, particularly around MoSL, is very nuanced and complex. There are places where some policy decisions are still being made.
Throughout the day we want you to capture your reflections and thinking to help you process and distinguish between what will become feedback to the DOE and what you need to move the work forward in your schools.
The goal for today is to understand, so that you are ready to lead the work in your schools.
The day is packed with content, but we will be making connections throughout the day and the institute so that you can see how to pull it all together for your schools.
Use reflection sheet

3. DOE Resources and Training
The following resources will assist principals and the School Local Measures Committee throughout the decision-making process and implementation:
Measures of Student Learning Guide: You can find more information on assessment and measurement options, as well as sample selections in the Guide, found at
Interactive Selection Tool: The DOE will release a tool to select assessments and measurements in August.
MOSL Specialists: Experts to help networks and schools implement in the fall.
Additional training opportunities:
MOSL Specialists will be providing additional support to networks, who will in turn support schools, as the school year begins.

4. MoSL Options Template
Required assessments are checked and shaded in gray.
State Measures decisions made by principal are color-coded in orange and Local Measures decisions made by the school committee are color-coded in blue (same as MoSL guides).
For State Measures, teachers who teach both ELA & math must have an ELA and math component for their State Measure, but not their Local Measure. The Local Measure can be any subject they teach. For schools that departmentalize, teachers are only responsible for the content they teach.
Pending policy decisions, for Local Measures, school committees may select up to 2 measures per group of similar teachers, i.e. same subject and grade. (2 measures = 1 assessment with 2 populations, or 2 assessments)
For example:
1 assessment, 2 populations: ELA state exam – school and school lowest third
2 assessments: ELA state exam – school, ELA performance tasks – individual
Use reflection sheet

5. MoSL: Measures of Student Learning (40%)
All teachers will receive:
Two different Measures of Student Learning (40%)
State Measures
Local Measures
Initial planning conference and summative end of year conference to include discussion of student outcomes (Will be covered on Aug. 8th)
Multiple measures provide a more valid, robust picture of teacher performance, providing teachers with multiple sources of feedback

6. State and Local Measures
Although principals make decisions about State Measures and the School Local Measures Committee makes decisions about Local Measures, these two decisions go hand in hand.
There are many reasons why it is important for the two measures to be determined in collaboration:
Teachers will receive the best picture of student growth with multiple measures, each telling a slightly different story of student performance.
For example: Principal reinforces a focus on school-wide learning by selecting a school-wide measure on a State Assessment for a given group of teachers’ State Measure. School Local Measures Committee then decides to base Local Measure on individual teacher performance on a 3rd party assessment (where possible) to include a variety of assessment perspectives.
Per law, teachers may not have the same exact measure for both State and Local (e.g., 4th grade teacher cannot have an individual classroom growth measure on ELA/Math state tests count for 40% of overall rating. Her Local Measure would have to measure another population either the lowest 1/3 or group measures such as school or grade).
The more alike the State and Local Measures are to each other, the more likely a teacher will receive the same rating on each. This may have positive or negative consequences.
Local Measures
State Measures

7. Schools re-select options each year, learning with experience
Long-term Vision
MORE OPTIONS
LEARN WITH EXPERIENCE
More assessment options each year
Schools re-select options each year, learning with experience

8. Terminology – Common Mix Ups
State Measures (20%)
The MOSL category that includes State Assessments or, where there are no State Assessments, a list of allowable assessments that can be used (chosen by principal where there is choice).
Local Measures (20%)
The MOSL category with options chosen by the School Local Measures Committee and submitted to the principal who may accept the recommendation or opt for the default measures.
State assessments can be used for both state and local measures. When we are talking about state tests, it doesn’t always mean we are talking about state measures.
The state made decisions about requirements on both the state and local side. Principals have a role in both as well.
Local assessments – NYC performance assessments – are required for different grades on both the state and local side. Therefore, when we talk about local assessments it does not always mean we are talking about local measures.

9. ELA Lowest 1/3, Grade, School
State
Local
Individual Class
ELA Lowest 1/3, Grade, School
State ELA Grade 4/5

10. Terminology – Common Mix Ups
For each measure, the following three items need to be considered:
Assessment(s)
Student growth is measured based on one or more of the following types of assessments: State Assessments, 3rd Party Assessments, or NYC Performance Assessments.
Target Population
Refers to the students included in the measure – this can include students school-wide, in a particular grade level, or only those students a teacher teaches. If the same assessment is selected for the State and Local Measures, the target population for the Local Measures will be the lowest third of students.
Growth Measurements
The method by which student growth is measured on a given assessment. For MOSL, options include teacher/principal goal-setting or growth models.
Even though there are only 3 main decisions that need to be made for each measure, decisions impact each other and all options are not available in all cases.

11. NYC Assessment Options
Information on assessments available for specific grades/subjects is available in the Measures of Student Learning Guides (
The following parameters guided the selection of assessments available for each grade and subject:
State rules regarding allowable assessments: SED released rules about what assessments are allowed in each grade and subject, including specific rules about 3rd Party and NYC Performance Assessments.
Assessment quality: NYC assessment options were evaluated to ensure instructional value, validity and reliability.
Security and comparability: Assessments must be available across all NYC schools and students cannot have access to them prior to administration. School-created assessments are not allowed for outcome assessments.*
Existing practice in schools: Whenever possible, NYC assessment options build upon existing assessment practices in NYC schools.
*Where baseline selections need to be made, schools may use school-based/created assessments as baselines. Schools may NOT use school-based/created assessments for final outcomes.

12. NYC Assessment Options
Assessment Type*
Target Population Options**
Description
Example (K-5)
State Assessments
Individual
Grade
School
Measure the performance of students based on state-created assessments.
3-8 ELA & Math
4 & 8 Science
NYSESLAT
MS Regents: Integrated Algebra, Living Environment, Earth Science
3rd Party Assessments
Are created by assessment experts. Assessment format varies (multiple choice, performance based, etc.)
Math: K-2 Discovery Math, 3-8 Scantron
ELA: 3rd grade Scantron, 3rd grade Degrees of Reading Power (DRP)
NYC Performance Assessments (developed w/ NYC Teachers)
Authentic tasks (e.g., evidence-based essays), scored against a common rubric. Created by DOE, NYC teachers, curriculum and assessment experts to be used as Measures of Student Learning in teacher evaluation.
K-8 ELA (2 options: tasks & running records)
3rd grade Math
4, 6 & 7 Science
6-8 Social Studies
Integrated Algebra
Living Environment 
NYC
Make sure facilitator references additional bulleted rules on in guides (page 23 of elementary guide).
Only state-approved 3rd Party Assessments can be used. Not all NYC assessments are on this list. A final list will be approved on August 1. **NYC is not using all 3rd Party Assessments that were previously approved for use throughout the state.** NYCDOE is funding 3rd Party Assessments used for teacher evaluation.
3rd Party & NYC Performance Assessments:
Include both pre- and post-tests.
Teachers (per law) can score their own baselines/pre-tests, but CANNOT score their own post-test results.

13. Activity: Preview NYC Performance Task
Review the NYC Performance Task and aligned Rubric sample.
Use the template to guide your reflections and thinking by considering the following prompts:
3 key points you learned from the NYC Performance Task and Rubric.
2 questions or implications of the assessment.
1 message to present to school/local measures committee.

14. Baseline Assessments Assessment Baseline Recommendation
Other Baseline Options
3rd Grade Math/ELA
NYC Performance Assessments
Scantron ELA & Math
Degrees of Reading Power (DRP) ELA
School based
ESL
Prior Year NYSESLAT
LAB-R (where prior year NYSESLAT isn’t available)
4th Grade Science
3rd Grade Math, NYC Science Performance Assessment
School Based
8th Grade Science
7th Grade Math, NYC Living Environment Performance Assessment
A baseline assessment is one given in the beginning of the year to assess student starting points.
NYC Performance Assessments and approved 3rd Party Assessments come with baseline (pre) assessments.
The principal must select a baseline assessment when none is included or where there is a State Assessment without historical data (e.g., 3rd grade).
There are only a few instances where baselines will need to be selected for Local Measures. Where this is the case, the principal, not the committee, selects the baseline.
Schools that select school-based/created baselines must use goal setting for that measure.
FACILITATOR NOTE: In cases where the baseline is not given at the beginning of the year, the first bullet does not apply.

15. Target Population Options
Target population options include:
Individual
only those students that a teacher teaches (only applicable if a teacher administers an assessment to his/her own students).
Grade
students in a given grade level.
School
all students within the school.
Where are they available?
All 3 options – Individual, Grade, and School – are available for the state tests for testing grade teachers. For non-testing grades teachers, the only options available for state tests are grade and school.
The only population available for NYC Performance Assessments is individual.
For 3rd party assessments, the only options are individual and school (this does not mean the assessment must be administered to all students in the school, it only means that the teacher will be evaluated based on everyone in the school who took the assessment).
Nuances:
In some cases these distinctions overlap. For example, for a 4th grade science cluster teacher, her individual student population will also be grade and school if she teachers the entire 4th grade.
Requirements:
For required State and Local Measures, the target population will always be the individual.
Teachers who do not give approved individual classroom-level assessments will use either a grade- or a school-wide measure.
If an assessment is repeated across measures, the target population must differ. If a school wants to use the same target population with the assessment, the Local Measures must focus on the lowest third of performers. You may only select the lowest 1/3 if the larger population was already selected as a measure.
Make sure facilitator references additional bulleted rules on in guides (page 23 of elementary guide).

16. Measurement: Growth or Goal Setting
Growth Model
Goal Setting
Gives teachers credit for the degree to which students exceed growth as compared to similar students.
Benefits & Challenges:
Does not introduce additional work in schools
Better able to account for unexpected outcomes resulting from unfamiliar, new assessments
Growth model score results are not available until after assessments have been administered (i.e., the following spring/summer)
Does not actively facilitate brainstorming and discussions about individual student performance and goals
Must use citywide assessments for baselines to select growth model
Teacher’s rating is based on the percentage of students who meet their target but does not consider the degree to which students fall below or exceed their target.
Particularly valuable for teachers/schools with unique student populations or high mobility
Allows teachers and principals to individually tailor student goals
Requires additional time/resources
Targets must be set early in the school year, possibly before much diagnostic info is gathered
Setting goals may be challenging if:
Teachers are not familiar with the comparability between assessments
Assessments are new or changing
Must use goal setting if using school-based/created baselines

17. Measurement: Growth Model
Calculated by the state for 4-8 ELA & math; calculated by the city for all other assessments including 3rd grade ELA & math, science, NYSESLAT, regents, etc.
Puts students in buckets according to 13 characteristics. Similar students are compared to each other to determine SGP.
Some characteristics describe the student and some describe the classroom.
Controls for similar characteristics as the Progress Report, but in different ways.
The Progress Report controls for classroom characteristics in their peer grouping and comparison. For growth scores, classroom characteristics are embedded in the student comparison “buckets”.
The Progress Report accounts for student characteristics by adding weights to student scores, For growth scores, student characteristics are accounted for by putting students in buckets to compare them only to similar students.
Category
Defining Characteristic of “Similar Students”
Academic History:
Prior year test score, same subject
Prior year test score, different subject
Retained in grade
New to school in year other than entry year
Average prior achievement and range around average prior score in student’s class/course (same subject)
Student with Disability (SWD)  
Student has an IEP
SWD spends less than 40% of time in general education setting
Percent SWD in student’s class
English Language Learner (ELL)
Student is an ELL
NYSESLAT scores
Percent of ELLs in student’s class
Poverty
Student poverty indicator (yes/no)
Percent poverty in a student’s class

18. Measurement: Growth Model
Ms. Smith’s Class
Prior Score
Current Score
SGP
Student A
450
510 45
Student B
470
500 40
Student C
480
525 70
Student D
550 60
Student E
600
650
If we compare student A’s current score to other students who had the same prior score (450), we can measure her growth relative to other students. We describe her growth as a “student growth percentile” (SGP).
Student A’s SGP is the result of a statistical model and in this example is 45, meaning she performed better in the current year than 45% of similar students.
To measure teacher performance, we find the mean growth percentile (MGP) for his or her students. To find an educator’s mean growth percentile, take the average of SGPs in the classroom. In this case:
Step 1: =255
Step /5=51
Ms. Smith’s mean growth percentile (MGP) is 51, meaning on average her students performed better than 51% of similar students.

19. Activity: Growth Model Understanding Check
Turn and Talk to your colleagues and discuss the two scenarios:
A teacher says to you: “I appreciate that the new evaluation system aims to measure growth, but my kids face so many challenges and just don’t grow as fast as other kids. I feel like I’m going to be disadvantaged by this new system.” In 2 minutes or less, explain how using a growth model will address her concern.
A teacher says to you: “I mean, we get these numbers called Student Growth Percentiles (SGPs), and we have no idea what they refer to. If Jose’s SGP is 65, does that mean he got 65% right on the test?” In one minute or less, explain the misconception this teacher has about SGPs, and what this number actually refers to.

20. MGPs and Statistical Confidence87
Confidence Range
Upper Limit
Lower Limit
MGP
5 minutes
NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be provided.
An educator’s confidence range depends on a number of factors, including the number of student scores in their MGP and the variability of student performance in the classroom.

21. State Growth Model: Scoring and Results
Most teachers scored Effective on the state growth score in The results for NYC were slightly higher than the rest of the state:
NYC Teachers (n=10,544)
Rest of State (n = 22,585)
Highly Effective 8% 6%
Effective
76%
77%
Developing
10%
Ineffective
Note: SED has stated that it expects the future distribution of ratings to remain similar to what it was in
5 minutes

22. Goal-Setting Process Administer baseline assessment
DOE sends predicted student targets
Teachers review DOE predicted targets
Principals approve or adjust targets
Administer end of year assessment
Teachers’ Ratings
Baseline assessment administered (not required for all assessments).
Report baseline assessment results.
DOE sends predictions for how individual students will perform.
Predictions are based on baseline performance, student achievement history, and student demographic characteristics.
Teachers may choose to adjust these targets based on additional information about their students.
Teachers submit student targets to principal.
Principal (or designee) report finalized student targets.
10 minutes
End-of-year assessment administered to students.
Teachers’ HEDI ratings are calculated with a conversion chart based on students’ performance on outcome assessments relative to their targets.

23. Goal-Setting Considerations
This is not the same as the goal-setting you may typically see in schools.
Goals are scored against a state conversion chart which makes the target-setting process difficult and non-intuitive.
Before considering goal-setting, make sure you understand the work and additional training this entails.

24. 50% Rule
All teachers who have individual assessments (assessments for their students and subject) available must have State Measures that cover at least 50% of their students. Therefore, you may need to add additional measures to get to 50%.
Begin by accounting for courses that end in state exams that result in state growth scores (4-8 ELA & Math)
Next, add courses that end in state exams but do not lead to state growth scores. You must begin with the largest (highest enrollment) courses first.
Finally, add additional courses beginning with largest (highest enrollment) courses first.
50% of students covered is the minimum, where principals have choice, you may choose to cover a larger percentage.

25. Does not introduce additional work
Default Option
Strategic
Status Quo
School wide
Does not introduce additional work
Accurate results
All Teachers

26. DEFAULT
Default only affects the Local Measure and applies for the entire school.
When you default, all local measure decisions and required assessments (ELA NYC performance assessments) are eliminated and replaced with the default.
The local measure default is:
K-5
SED provided growth scores on 4-5 ELA & Math state tests
 Lowest 1/3 growth scores for teachers who have school-wide measures for their State Measure
K-8 Schools
SED provided growth scores on 4-8 ELA & Math state tests
K-3 & K-2 Schools
An average of all measures on the State side.
Defaulting DOES NOT affect the State Measure. Principal’s are still required to make decisions highlighted in orange. All required assessments for the State Measure are still required.

27.  Pending upcoming policy decisions 
DEFAULT - Nuances
 Pending upcoming policy decisions 
If a committee does not make a decision for a particular grade and subject and it leaves it blank, the default option is automatically filled in and eliminates the required assessments.
For grades without required Local Measure options, the committee may select the default options.
If the committee selects the default options for a grade with required assessments, that means they are doing both the required assessments and the default.

28. Activity: Committee Decision Check
Review the committee’s choices for assessment and target population for errors.
Circle any selections that are not allowable within the guidelines.
Use your graphic organizer to note the error, why it is an error (reference the rule), and what suggestions you have for making changes.
Review the committee’s choices for assessment and target population for broader implications.
Draw a box around any selections that ARE allowable, but might have broader implications that are worth discussing before selections are finalized.
Use your second graphic organizer to note the selection and what broader implications this selection might have.

29. Activity: Problem Solving Scenarios – Part 1
Read and select 1-2 scenarios to discuss with your colleagues. Use the template to guide your conversation. For each scenario:
Describe the problem
Indicate the problem type (technical, cultural, contextual, indecision)
Brainstorm possible problem solving strategies
Identify 3 next steps for the school

30. Activity: State Measures Decisions
Practice using the MoSL Options Template to make your decisions for state measures for your teachers.
Use your Table of Organization:
First, cross off any Pre-K teachers and teachers who do not teach 40% of the time. Make a note that these teachers will be rated S or U.
Second, write the names of your teachers in the appropriate place in the left column of the MoSL Options Template.
Exception: For teachers who do not have 50% or more of their students in a particular grade and subject, make a note to include them on the last page of the template.
Check off the decisions you have made or are considering for your teachers.
Discuss your thinking and reasoning with your colleagues.

31. Activity: Connecting MoSL - Elevator Speech – Take 1
More than ever, the success of the school will be dependent on the clarity of school leaders vision and communication. In light of the new Principal Evaluation System, school leaders ability to clearly and concisely voice their “theory of action” will largely impact their Measures of Leadership Practice.
Over the course of the Summer Institute and throughout the year we will revisit the idea of the elevator speech as a way of working through how to best articulate the work we do.
Now, as a way to synthesize our thinking around MoSL and next steps for our school we will begin to draft our elevator speech regarding assessment in our school.
Use the elevator speech template to draft your thinking around the following prompts:
Goal & Action Steps: We are working on…by….
Data & Purpose: because…so that…..
Monitoring & Next Steps: The evidence we are looking for…. based on this data our next steps will be

32. Activity: Connecting MoSL - Considering our Community
Even though we may hold particular beliefs and a vision for assessment in our schools, depending on how your committee and teachers are processing this work, it may make sense to modify our decisions to honor/account for their perspective.
Turn and Talk to your colleagues and share:
How your teachers are reacting to or may react to this work
Does that align or not with where you are in this work
How might that impact your final decisions

33. Activity: Connecting MoSL - Elevator Speech - Take 2
Based on your considerations for your community, revise your elevator speech in “take 2” on the template.
How might this message impact your…
Meetings with your MoSL Committee
September faculty conferences
Initial planning conferences
Turn, Talk & Share with your colleagues.

34. Activity: Problem Solving Scenarios – Part 2
Use the Consultancy Protocol to discuss a challenge you are having at one of your schools.
Protocol overview:
Presentation: Presenter provides overview of work and 1-3 questions (3 minutes)
Clarifying Questions: Colleagues ask presenter clarifying questions (3 minutes)
Probing Questions: Colleagues ask presenter probing questions (3 minutes)
Non-presenter Discussion: Colleagues discuss the presenters work and questions without the presenter (5 minutes)
Presenter Response and Open Discussion: Presenter responds and discusses work with colleagues (5 minutes)
Reflection/Debrief: Group reflects on the process (5 minutes)