1. District Determined Measures
Waltham Public Schools
Ann Feldman
English Language Learner Program Director
PDRT / November 4, 2014
Waltham High School ELL Staff

2. Goals We will look at the benefits and challenges related to DDMs.
We will complete the sentence frame “A challenge associated in creating/identifying DDMs will be……” as we collaboratively fill out a brainstorm web.
We will complete the sentence frame “ A benefit of creating/identifying DDMs will be …….” as we collaboratively fill out a brainstorm web focusing on the “positive” aspects of DDMs.
We will read about three approaches for measuring growth in jig-saw style and fill out a four-column note-taking template.
We will go through the DDM process step-by-step.
We will discuss considerations for ELLs related to DDMs.

3. Agenda Basic Facts About DDMs
Think-Pair-Share – Benefits and Challenges of DDMs
Measuring Student Learning
Educator Evaluation Framework
Summative Performance Rating
Student Impact Rating
Developing a DDM Step-by-step
Checklist Distribution
Jigsaw Reading: Approaches for Measuring Growth
Rubric and Student Impact Rating
Considerations for ELLs / Fairness and Accommodations
SY Deadlines

4. Major Shifts From what teachers do….. to what students do
From student proficiency ……. to student growth

5. Key Features of a DDM Aligned to Content Provides useful information
Measures growth
Comparability
Definition: District Determined Measures (DDMs) are measures of student learning, growth, and achievement aligned to the MA Curriculum Frameworks that are comparable across grade level or subject district wide.

6. Why DDMs? Teacher impact on student learning
“The ability to improve educator effectiveness relies on having meaningful and accurate information about student progress.”
(ESE, Technical Guide B, page 3)

7. Basic Facts
DDMs must be developed for all licensed educators in all grades and subjects as well as for all administrators
Guidance counselors, nurses, physical education teachers, principals, superintendents, science specialists, business administrator . . .
Measures growth NOT achievement
Need some baseline data
DDMs administered across all schools in the district where the same grade or subject is taught
DDMs should assess learning as directly as possible
Two DDMs must be developed for every educator

8. Second measure must be locally developed
If you teach ELA and/or math in grades 4-8, one DDM must be the appropriate grade level MCAS with associated Student Growth Percentiles (SGP)
Educators who teach ELA and math are not required to use both the ELA and math MCAS
Second measure must be locally developed
Approved commercial assessment
District-developed pre- and post-unit and course assessments
Portfolios
Approved commercial assessments
Capstone projects
ELA and math MCAS in grades 4-8
Writing assessments using rubrics (example: ESL)
Oral assessments using rubrics (example: ESL)

9. Think-Pair-Share
Take a few minutes to INDIVIDUALLY complete the BENEFITS web on your own. Please discuss what you came up with in a group. We will chart our responses.
Take a few minutes to INDIVIDUALLY complete the CHALLENGES web on your own. Discuss in your group and we will share our ideas.

10. How is Student Learning Measured?
#1: Student Growth Percentiles (SGP) from MCAS or ACCESS for ELLs
SGP is a measure of an individual student’s growth as compared to a cohort group of students with similar score history
Look at the comparison of ACCESS scores from 2013 to for at least 20 grade 9 – 12 ELLs.
#2: Measures of growth based on standards-aligned assessments that are comparable across grade level or subject district wide.
How might you pinpoint areas of need for students you work with?
What types of measures might you use?

11. DDMs and Teacher Evaluation

12. DDMs and the Teacher Evaluation System
A Summative Performance Rating is a rating of educator practice and a Student Impact rating is a rating of educator impact on student learning, growth, and/or achievement. These two ratings are independent, but intersect to provide educators and evaluators with a more complete picture of educator effectiveness. This intersection results in a number of opportunities for educator growth and development.

13. Educator Evaluation Framework
Beginning in , educators earn ratings based on two years of performance:
Years 1 and 2: Summative Performance Rating
Exemplary
Proficient
Needs Improvement
Unsatisfactory
Year 2: Student Impact Rating
High
Moderate
Low

14. Student Impact Rating
Separate but complementary to the Summative Performance Rating:
Informed by trends (at least two years) in student learning based on two DDMs
Informed by patterns (at least two measures) in student learning based on two DDMs
Year 1:
Year 2:

15. Student Impact Rating Three Ratings High Moderate Low
Indicates significantly higher than one year’s student learning growth relative to academic peers in grade or subject
Moderate
Indicates one year’s student learning growth relative to academic peers in the grade or subject
Low
Indicates significantly lower than one year’s student learning growth relative to academic peers in the grade or subject

16. Using SGP for Student Impact Rating
66-99
High
35-65
Moderate
1-34
Low

17. Type and Length of Plan: The Summative Performance Rating and Student Impact Rating are used together to determine the type and length of an educator’s Educator Plan. The Summative Performance Rating determines the type of plan and the Student Impact Rating may have a direct impact on the length of the plan. The educator evaluation regulations define four different types of Educator Plans.
The Self-Directed Growth Plan applies to educators who earn Summative Performance Ratings of Proficient or Exemplary. It is developed by the educator. Educators placed on Self-Directed Growth Plans who earn Student Impact Ratings of moderate or high are placed on two-year plans, while educators who earn impact ratings of low are placed on one-year plans.
The Directed Growth Plan applies to educators who earn Summative Performance Ratings of Needs Improvement. It is developed by the educator and evaluator and is one year or less in duration.
The Improvement Plan applies to educators who earn Summative Performance Ratings of Unsatisfactory. It is developed by the evaluator and is 30 days to one year in duration.
The Developing Educator Plan applies to educators without Professional Teacher Status (PTS), administrators in their first three years in a district, or educators in new assignments (at the discretion of their evaluators). It is a one-year plan developed by the educator and evaluator.

18. Two Ratings

19. Step-By-Step Development of the DDM

20. Developing a DDM Step 1: Identify the key content
Is the measure aligned to content?
Does it assess what is most important for students to learn and be able to do?
Does it assess what the educators intend to teach?
Is the measure informative?
Do the results inform educators about curriculum, instruction, and practice?
Does it provide valuable information to educators about their students, helping them identify whether students are making desired progress, falling short, or excelling?
See DDM checklist

21. Developing a DDM
Step 2: Ensure that the change in performance represents student growth
Start with baseline
Measures similar content
Demonstrates what students know and don’t know
Step 3: Select an approach for measuring growth
Pre-test/post-test
Identical measures administered twice or comparable versions
Repeated Measure Design
Some teachers use short measures throughout the year to monitor student growth on a set of skills
Holistic Evaluation
A holistic evaluation of student growth combines aspects of pre- and post- test model with the regularity of repeated measures.
Use a rubric that describes growth over time

22. Step 3: Select an Approach for Measuring Growth - Jigsaw
Read your section, take notes and report to others.

23. Developing a DDM Step 4: Align to grade level standards
Align to content
MA Curriculum Frameworks
Step 5: Decide on a scoring rubric
Raw score, percent score, rubric score
How will growth be determined?
Step 6: Draft a scale for low, moderate, and high impact
High: significantly higher than expected
Moderate: one year’s worth of student growth
Low: below expectation

24. Looking at a Rubric and Drafting a Scale for Low, Moderate, and High Impact

25. 3rd Grade Writing Rubric
Look at the 3rd Grade Rubric for Opinion Writing. Discuss what you think related to:
Expectations for writers at this grade level
Fairness
Components of writing highlighted in the rubric

26. Developing a Scale for Low, Moderate and High Impact
Look at the results of the DDM for the November and March administration. Note that if student score went up, it is in green, no color/no change, if a student score went down, it is in red. Come up with a Student Impact Rating.
Low Student Impact Rating
Moderate Student Impact Rating
High Student Impact Rating

27. Accommodations and Considerations for English Learners Taking District DDMs
See handout: Quick Reference Guide: DDMs and Examples of Instructional Supports

28. For core academic teachers who provide instruction to ELLs, districts are likely to administer the same DDMs for both ELLs and general education students.
This is why we have to pay close attention to the DDM we chose to administer to this population of students

29. DDM Revisions to Accommodate ELLs
Identify language that might cause unnecessary hurdles for ELLs or advantage one group over another
Irrelevant to the content being assessed
Unnecessarily complex language and/or sentence structures
Assumes cultural knowledge or social or school experiences.
Identify language and/or images that may convey stereotypes or bias
Characterizes groups as more or less powerful, advantaged, smart, attractive, successful, etc. than other groups

30. DDM Revisions to Accommodate ELLs
Retain the rigor and content of targeted standards
Retain relevant academic vocabulary
Retain complex sentence structure, if appropriate
When selecting or developing a DDM that will be used with ELLs, involve ESL teachers or other educators with expertise in ELL education
See DDM Implementation Briefs (look specifically at the one for ELLs)

31. Is This Item Unfair? How Might YOU Revise an Unfair Item?
Sofia had 30 brand new pencils and gave 18 of them to her best friend Maria. How many did Sofia have left?
Mark and Carl went to a book sale at the library. Mark got 19 books and Carl bought 24 books. Make a ballpark estimate of how many books they had all together.
Pretend you went to a pet store. Write about which pet you would most like to bring home with you and how you would take care of it.

32. Is This Item Unfair? How Might YOU Revise an Unfair Item?
Sofia had 30 brand new pencils and gave 18 of them to her best friend Maria. How many did Sofia have left?
Sofia had 30 pencils. She gave 18 to Maria. How many pencils did Sofia have let?
Mark and Carl went to a book sale at the library. Mark got 19 books and Carl bought 24 books. Make a ballpark estimate of how many books they had all together.
Mark has 19 books. Carl has 24 books. Estimate how many they have all together.
Pretend you went to a pet store. Write about which pet you would most like to bring home with you and how you would take care of it.
Write about an animal you would like for a pet.
What would you do with this pet?
How would you take care of it?

33. DDM Accommodations Common Other Accommodations
Testing in small groups or separate rooms
Extended time
Directions or questions read aloud in English
Other Accommodations
Additional breaks between sessions
Directions translated/read in students’ L1
Bilingual or English glossaries or dictionaries
Dictation of answers or use of a scribe
Side-by-side bilingual version of the test
Test-taker response in native language

34. Sample Section of a DDM for ELLs
Start planning your DDM. You may target a specific population of students, a specific grade, etc. You may focus on one particular domain that students’ have had difficulty with on the ACCESS.
Consult the DDM checklist for the step-by-step process of developing a DDM.

35. By December, 2014: For All Staff Members
Identify/create two (2) DDMs
Select an approach for measuring growth
Align to grade level standards
Decide on a scoring rubric
Draft a scale for low, moderate and high growth
Complete & upload to Google Drive
DDM Submission Cover Sheet
DDM Checklist
DDM Standards Alignment & Growth Form

36. Helpful Documents Quick Reference Guides:
District-Determined Measures
Considerations for educators of English Language Learners
WIDA Examples of Instructional Supports, Subject- Specific Sensory Supports, and the Use of Graphic Organizers Across Subjects
Northeast Comprehensive Center / West Ed: DDMs: Examining Fairness for English Learners