1. District Evaluation Advisory Committee Office of Accountability Ms. Brenda Patterson, Chief Accountability Officer Mr. Sham Bacchus, Interim Executive Director Dr. Burnie Bristow, Interim Director Dr. Vanessa Serrano, Interim Supervisor Tammy Williams, Program Coordinator Evelyn McLeod, Confidential Secretary DEAC Meeting-April 23, 2013-Dr. Burnie Bristow1

2. Welcome District Evaluation Advisory Committee DEAC Meeting-April 23, 2013-Dr. Burnie Bristow2

3. 3 AGENDA Office of Accountability DEAC Committee Meeting April 23, 2013Facilitator: Dr. Burnie Bristow, NBCT 90 Delaware AvenueCafeteria, 2:30-4:30 pm Thoroughly Trained 2:30-2:40Meet and Greet 2:40-2:50Revisit Roles and Responsibilities of the DEAC Committee 2:50-3:05Steering Committee Updates Sub-Committee Student Information System 3:05AchieveNJ Updates 3:05-3:20Non-Tested Grades and Subjects Developing Student Growth Objectives (Teacher Focus) 3:20-3:35Tested Grades and Subjects Student Growth Percentiles(Teacher Focus) Developing Student Growth Objectives(Teacher Focus) 3:35-3:40Summer Professional Development Opportunities for Teachers 3:40-4:00Exemplary Teacher Training -Dave Roudebush, Ph.D., Curriculum Focal Point 4:00-4:25How will the Paterson Public School District define “THOROUGHLY TRAINED”? 4:25-4:30 Action Items: Next steps Deliverables Next meeting date

4. Roles and Responsibilities of the DEAC Committee Roles and Responsibilities of the DEAC Committee The DEAC committee (formed Dec. 2011) functions as an advisor to the Paterson Public Schools in the implementation of the TEACH NJ Act. Embedded in the TEACHNJ Act is a requirement to revise the Principal and Teacher Evaluation System. Membership on the DEAC Committee provides an opportunity for committee members to provide feedback on the performance rubrics, program development and implementation at the school and community level. The DEAC committee is comprised of teachers, principals, central office staff, school board members and parents. DEAC Meeting-April 23, 2013-Dr. Burnie Bristow4

5. The goal of this legislation is to raise student achievement by improving instruction through the adoption of evaluations that provide specific feedback to educators, inform the provision of aligned professional development, and inform personnel decisions; Teacher Effectiveness and Accountability for the Children of New Jersey (TEACHNJ) Act.” 5

6. Update on the Student Information System Sham Bacchus DEAC Meeting-April 23, 2013-Dr. Burnie Bristow6

7. Current Data Sources/Systems that directly impact new Evaluation Instrument Student Achievement Data Performance Matters Renaissance Learning Scholastic/ Pearson NJSMART Class Gradebooks Student Information Fusion/Keystone Genesis (Eastside/JFK) EasyIEP Staff Information EdumetKronosApplitrackPDProMcRelTeachscape* Financial EdumetAlio Communication/T echnology EmailActive DirectoryParentLink Currently requires human input from schools and central office. (Error Laden) - 17 Currently gets data from other systems. (Preferred) - 5 Brenda Patterson, Chief Accountability Officer, PPS 7

8. DEAC Meeting-April 23, 2013-Dr. Burnie Bristow Data Integration There was a need for an internal review of the programs, processes and technical components required to effectively support the implementation of the new Principal and Teacher Evaluation. This review of our infrastructure necessitated a cross departmental collaboration to integrate various systems. 8

9. 9 Communications, Business, HR Assessment, MIS, Technology Merging platform ALIO and MIS Support to assessment Develop data platform to record and keep track of teacher and school achievement data Teacher E-board / blog /Dropbox Collaborative Teams as Decision Makers: Communications/Technology/MIS, Business, Reform - Assessment/HR/PD, C & I, Special Programs and Accountability

10. STUDENT GROWTH OBJECTIVES Introduction to the Guidebook w/ Sham Bacchus April, 2013

11. 11 Charlotte –Mecklenburg SLO study Key findings: The higher the quality of the SLO, the more likely it is to be attained The quality of SLOs increases significantly in just two years Students whose teachers had higher quality SLOs achieve higher scores on math and ELA tests Students whose teachers met their SLOs achieve higher scores on math and ELA tests

12. 12 Charlotte –Mecklenburg SLO study What principals said about SLOs “The way of doing business in the school is the SLO process. It adds to the spirit of cooperation.” “Developing SLOs forces teachers to target low performing kids and then work harder to pull those kids up.” “We have moved from a priority school to a high growth school, and I give a lot of credit for this change to the SLO process.”

13. 13 Charlotte –Mecklenburg SLO study What mattered most in the SLO process 1)Analyze student baseline data, for use in planning and teaching 2)Set individual student growth targets, for both more informative and accurate assessment of student learning 3)Participate in collegial collaboration in the development of SLOs

14. Shift in Practice Student Growth Objectives Brenda Patterson, Chief Accountability Officer, PPS 14DEAC Meeting-April 23, 2013-Dr. Burnie Bristow

15. 15 What is an SGO? A Student Growth Objective is a long-term academic goal that teachers set for groups of students and must be: Specific and measureable Aligned to New Jersey’s curriculum standards Based on available prior student learning data A measure of student learning between two points in time Ambitious and achievable Page 3

16. 16 Teacher Practice Student Growth Percentile Student Growth Objective Sum. Rating Teacher Evaluation: Student Growth Objectives A Student Growth Objective is a long-term academic goal that teachers set for groups of students. It must be:  Specific and measureable  Focused on the right content and skills  Aligned to state/national standards  Based on available prior student learning data  Based on growth and achievement  Ambitious and feasible 16

17. 17 Type of SGODefinitionExamples Course-level Focused on the entire student population for a given course, which often includes multiple classes. Covers all of the students in a teacher’s Algebra I classes (Ideally the same form of assessment covering all standards in the curriculum) Class-level Focused on the student population in a given class. Covers all of the students in ninth period. Targeted students Focused on a subgroup of students that need specific support. Covers a group of students that scored below 45% on the pre-test. Targeted content Focused on specific skills or content that students must master. Students will all master 80% of CCSS standards related to Quadratic Functions and Modeling 17 NTGS 101: Student Growth Objective Overview Different types of Student Growth Objectives: Student Growth Objectives can take many different forms

18. 18 TypeStudent Growth Objective Targeted studentsThe average score on the post-test for the ten ELL students who scored below 40% on the pre-test will be at least 70%. Targeted standardThe average student score on questions related to Standard 5.1.12.B (scientific thinking and design) will increase from 40% to 80%. 18 Specific Objective: Examples The word “specific objective” would likely be used to identify anything that does not cover your general population or is not a cumulative test NTGS 101: Specific Student Growth Objective

19. 19 Five steps for SGOs 19 Teacher Practice Student Growth Percentile Student Growth Objective Sum. Rating Teacher Evaluation: Student Growth Objectives Step 1: Choose or develop a quality assessment aligned to state standards Step 2: Determine students’ starting points Step 3: Set ambitious and feasible student growth objectives Step 4: Track progress, refine instruction Step 5: Review results and score 19

20. 20 SGOs in Achieve NJ - requirements A teacher develops SGOs in consultation with his or her principal. All teachers who receive an SGP score must set between 1 and 2 SGOs. Teachers who do not receive an SGP score must set 2 SGOs. SGOs must be aligned to NJCCCS or CCSS and measure student achievement and/or growth. SGOs must be specific and measurable and based on available student learning data.

21. 21 Types of SGOs General SGOs are broad in the scope. They include all, or a large proportion of the curriculum, and all, or most of your students. Specific SGOs focus on a particular subgroup of students, or specific content or skill. Page 5

22. 22 NTGS 101: Student Growth Objective Overview Different types of Assessment that Teachers might use: Student Growth Objectives can be set towards many assessments Traditional Assessments National norm-referenced tests e.g. AP exams, DIBELS State-developed tests e.g. EOC exams, NY regents LEA-developed tests e.g. final exams, benchmark tests Portfolio Assessments Examples might Include - Writing and reflection samples (ELA) Laboratory research notebook (sciences) Student project based assessments (all) Portfolio of pieces (art, photography, graphic design) Videos of student performance (performing arts) Log/Comparison of workouts (PE) *Assessment is a key piece of quality student growth objectives

23. 23 NTGS 101: Pilot Districts Student Achievement Goals (SAGs): Pilot Districts were given the recommendation to use SAGs for the 2012 – 2013 school year DEAC Meeting-April 23, 2013-Dr. Burnie Bristow

24. 24 S ETTING Y OUR G OAL Directions: Please develop a SMART goal that indicates what you want your students to achieve over a specified interval of instruction. You should also indicate the interval of instruction and whether the goal is a mastery or growth goal. Goal (example): 90% of Algebra 1 students will master 80% or more course content, as measured by the end of year district-wide, summative Algebra assessment (the interval of instruction is a full year & this is a mastery goal). Interval of Instruction:  Full Year  Semester  Other Type of Goal:  Mastery  Growth NTGS 101: Pilot Districts Student Achievement Goals (SAGs): Pilot Districts were given the recommendation to use SAGs for the 2012 – 2013 school year Rationale How did you choose the goal and why is it an appropriate area of focus? Aligned Standards To what New Jersey or Common Core State Standards does the goal align? Baseline Student Data What do you know about students’ achievement levels at the start of the year? DEAC Meeting-April 23, 2013-Dr. Burnie Bristow

25. 25 Insufficient Attainment of Goal (1): The teacher has demonstrated an insufficient impact on student learning by falling short of the goal. Partial Attainment of Goal (2): The teacher has demonstrated some impact on student learning, but does not meet the goal. Full Attainment of Goal (3): The teacher has demonstrated a considerable impact on student learning by meeting the goal. Exceptional Attainment of Goal (4): The teacher has demonstrated an outstanding impact on student learning by surpassing the goal by a meaningful margin. Achievement Range: 70% of students scored 80% or more on summative assessment Achievement Range: 80% of students scored 80% or more on summative assessment Achievement Range: 90% of students scored 80% or more on summative assessment Achievement Range: 95% of students scored 80% or more on summative assessment M EASURING Y OUR G OAL Directions: After you develop your goal, you will need to determine how you will measure the level to which you met the goal. Please complete the table below to identify the achievement range that defines each of four goal attainment levels. 25 NTGS 101: Pilot Districts Student Achievement Goals (SAGs): Pilot Districts were given the recommendation to use SAGs for the 2012 – 2013 school year DEAC Meeting-April 23, 2013-Dr. Burnie Bristow

26. 26 NTGS 101: SGO – a high school biology example 26 SGO Components: General Timeline Assessment Administer department developed pre-test aligned with 2009 CCCS Baseline Data Determine general preparation level. Group students into three categories (low, medium, high). Determine specific areas of deficit. Growth Objectives General – set an objective for all students at each preparation level covering all content. Specific – set an objective for all students to grow in a particular content standard. August November

27. 27 NTGS 101: SGO – a high school biology example 27 SGO Components: General Timeline Monitor and Adjust Use benchmark assessments to monitor progress and adjust instruction. Measure Administer department-developed standards-based post-test at the end of the year and analyze data. Evaluate Determine to what degree growth objective was met and produce a summative rating. January June

28. 28 June Annual summary conference includes: Available component measures. October Department collects all other component measures for teachers with SGP. November/December NJASK scores released. Department calculates SGP data and sends to districts the SGP and summative ratings of each teacher with a SGP score. January Summative rating added to personnel file. Teacher Evaluation: Summative Rating Timeline 28 At summative conference, all available component scores (i.e. teacher practice, SGO results) will be discussed. SGP data will be available on the following timeline. Teacher Practice Student Growth Percentile Student Growth Objective Sum. Rating

29. 29 Teacher Evaluation: Student Growth Objectives 29 Preparation Level Based on Pre-Test Score Number of Students SGO Based on Predicted Achievement on Post-Test Score Low (0-45%)2170% or above Medium (46-69%)5480% or above High (70-100%)790% or above Teacher: Ms. MuseySubject: Lab Biology Grade: 11 Sections: 4Number of students: 82 29 Tiered SGOs - EXAMPLE Teacher Practice Student Growth Percentile Student Growth Objective Sum. Rating 29

30. 30 Assessing Outcomes – General SGO Teacher Evaluation: Student Growth Objectives Student Preparation Level Target Score Rating based on number of students reaching target score % Highly Effective Effective Partially Effective Ineffective Low 70>1711-175-10<5 Medium 80 >45 26-4510-26<10 High 90>65-62-4<2 2 Score 43 3 30 Tiered SGOs - EXAMPLE Teacher Practice Student Growth Percentile Student Growth Objective Sum. Rating 30

31. 31 5 Steps Step 1: Choose or develop a quality assessment aligned to NJCCCS or CCSS. Step 2: Determine students’ starting points. Step 3: Set ambitious and achievable SGOs with the approval of the principal/supervisor. Step 4: Track progress, refine instruction. Step 5: Review results and score in consultation with your principal/supervisor. Page 8 – Quick Start Guide

32. 32 Page 10

33. 33 Develop assessments collaboratively. Align all assessments with NJCCCS or CCSS. Align all assessments with district, school and department goals. Make sure all the content in your SGO is covered in the assessment. Incorporate test items that vary in levels of difficulty. Include a sufficient number of test items to ensure rigor. Collaboratively determine possible modifications to meet the needs of students. Develop rubrics to assess essay responses. Make sure content- and skill-based rubrics are specific and address multiple levels of proficiency. Page 11

34. 34 Page 26

35. 35 Page 27

36. 36 Page 28

37. 37 Source of Performance Data to Determine Students’ Starting Points Examples and Notes Results from beginning-of-course diagnostic tests or performance tasks  Department-generated pre-assessment  Early course test Results from prior-year tests that assess knowledge and skills that are pre-requisites to the current subject/grade  NJASK for math, LAL and science  DRA for reading  End of course tests assessments, e.g. results on English 9 writing portfolio are used by the English 10 teacher Results from tests in other subjects including both teacher- or school-generated tests and state tests (tests must have assessed pre- requisite knowledge and skills)  A physics teacher uses results of her students’ prior math assessments Students’ grades in previous classes  Teachers should make sure they understand the basis for the grades given by students’ previous teachers Page 13

38. 38 Page 14 Student Portfolio Score (June 2013) Pre-Assessment (Sep 2013) Preparedness Group 18976High 26843Low 37854Medium 48666Medium A 9 th -grade LAL teacher has two sets of data readily available: a department-wide pre-assessment that is based on the content and structure of the final assessment and scores on the portfolio that the students completed the previous year.

39. 39 Attainment of Student Growth Objective Exceptional 4 Full 3 Partial 2 Insufficient 1 Teacher has demonstrated an exceptional impact on learning by exceeding the objective. Teacher has demonstrated a considerable impact on learning by meeting the objective. Teacher has demonstrated some impact on learning but did not meet the objective. Teacher has demonstrated an insufficient impact on learning by falling far short of the objective. Page 15

40. 40 Target ScoreAttainment Level in Meeting Student Growth Objective 80% or Higher on Final Assessment Exceptional 4 Full 3 Partial 2 Insufficient 1 Percent of Students Meeting Target Greater than 84% 70-84%55-69% Less than 55% Page 16

41. 41 Page 17

42. 42 Page 18

43. 43 SGO Guidebook Development Continuous improvement with information from the field – Modification and addition of forms – Clarity of language – Models of SGOs – More guidance on assessments, including portfolios

44. 44 Summer Professional Development 44 Dr. Vanessa Serrano

45. Principal and Teacher Evaluation System Training Discussion Dr. Dave Roudebush, Focal Point April 23, 2013

46. 46 Teacher Evaluation System Performance Standard #1: Preparation for Instruction – Establishing a culture of high expectations for learning and achievement – Uses district adopted curriculum and content knowledge to design coherent lessons – Post aligned lesson objectives and plan for demonstration of learning

47. 47 Teacher Evaluation System Performance Standard #2: Use of Data to Inform Instruction – Focus on improving instruction using data – Use a variety of assessment methods when designing classroom assessments – Involve students in assessing their own learning

48. 48 Teacher Evaluation System Performance Standard #3: Delivers Quality Instruction – Instruct bell to bell – Use a variety of instructional strategies to focus instruction – Engages students in learning – Continually checks for understanding – Deliver rigorous and relevant content – Integrate 21 st Century Skills in instruction – Provides feedback about student proficiency

49. 49 Teacher Evaluation System Performance Standard #4: Interventions to Meet Diverse Needs – Differentiate instruction based on student needs and background – Implements interventions with fidelity and adjusts interventions based on results – Adapt and modify instruction for the unique needs of learners

50. 50 Teacher Evaluation System Performance Standard #5: Classroom Environment – Contribute to a safe and orderly learning environment – Use effective classroom management procedures – Effectively manage student behavior – Foster collaboration and self-regulation in students – Promote positive and respectful rapport

51. 51 Teacher Evaluation System Performance Standard #6: Leadership – Understand their role and responsibility for professional growth and positive leadership – Promote the concept of Professional Learning Communities through collaboration and purposeful involvement – Continue professional growth

52. 52 Teacher Evaluation System Performance Standard #7:Professional Responsibilities – Adhere to Federal laws, state statutes and regulation pertaining to education, Board of Education policies, Memoranda of Understanding and school rules – Demonstrates professionalism – Effectively communicates and solves problems

53. 53 Teacher Evaluation System Professional Development Opportunities Two Major Professional Development Strands – Curriculum Alignment/Instructional Strategies – The Teacher Evaluation Performance Rubric

54. 54 Professional Development Opportunities Aug. 18-19, 2011 * Aug. 25-27, 2011 Sep. 19-21, 2011 * Oct. 8, 2011 Aug. 21-30, 2012 * Sep. 4-5, 2012 Oct. 12, 2012 * Feb. 15, 2013 To Date 2,300 teachers have been trained Approximately 202 teachers need to be trained

55. DEAC Meeting-April 23, 2013-Dr. Burnie Bristow55 Thoroughly Train Teachers: Stop. Reflect and Discuss: Think about what you have seen presented here, How will the Paterson Public School District define “THOROUGHLY TRAINED”? Please write down and discuss your thoughts and ideas at your table. Please be prepared to share your most salient ideas with the committee.

56. DEAC Meeting-April 23, 2013-Dr. Burnie Bristow Next Steps 56

57. DEAC Meeting-April 23, 2013-Dr. Burnie Bristow THANK-YOU 57