NCES Forum TECH Committee February 21, 2011

Panelists Bethann Canada, Virginia Department of Education Charlene Swanson, New York State Education Department Larry Fruth, SIFA Tom Purwin, Jersey City Public Schools (NJ) Mike Hopkins, Rochester Schools (NH) Raymond Yeagley, NWEA

Perspectives The data collector The data reporter The data user

The Data Collector LEA and vendor communications Designing the least-burdensome data collection Balancing data needs with LEA capacity to report Hardware and software infrastructure Data use/misuse FOIA concerns

The Data Reporter Data system capacity to report Non-linked data systems (HR/SIS) Mapping to SCED/NSCED Concern over how the data will be used Reporting regional program/virtual program data Non-licensed providers FOIA concerns Need to report multiple teachers (roles)

Multiple Roles Lead teacher (s) Other teacher whole class most of the time Other teacher whole class some of the time Other teacher few students Long term substitute Contracted private provider Pupil personnel service provider Administrator Aide/Tutor

The Data User Teachers – feedback on classroom outcomes Teacher preparation programs – feedback on recent grads Administrators – evaluations, salary, tenure Program evaluators Researchers Parents and the public The media

Charlene Swanson New York State Education Department

Dimensions to be Considered Teachers One or more teachers Teacher assignment start/end dates Teacher attendance (period and daily) Teacher-student instructional weights Students Course and School Course length (minutes) School calendar State and local assessments Student enrollment start/end dates Student attendance (period and daily) Teacher-student instructional weights

Data Model Teacher-Student Linkage Variables Course length (minutes) Teacher-student linkage start/end dates Number of minutes of student enrollment-teacher assignment linkage for course Number of minutes of student attendance-teacher assignment linkage for course (based on student enrollment, teacher assignment, and period student attendance) Teacher-student instructional weightings Student exclude flag

Policy Decisions Should student inclusion in evaluation be based on: Ratio of student enrollment-teacher assignment to course length; or Ratio of student attendance-teacher assignment to course length? Available VAM covariates: daily student/teacher attendance or student course attendance linkage information Definitions of student-teacher instructional weights and student exclude flags (reported or calculated) The student exclude flag could be based on percent difference between course enrollment and course attendance

Student Management Systems Interactive Reports Roster verification of teacher- student linkages Collection from teachers of student instructional weights and exclude flags (to be verified by principal) Missing Data Collected School calendar Course length (minutes) Assessments Teacher-student instructional weights Teacher daily attendance (to be merged from HR systems) Period-level teacher attendance (not available) Student course enrollment Teacher course assignment Student attendance (daily and course)

Cautions Whatever is collected, must be verified Timing Anticipated deployment in student management systems is during the school year, in time for student scheduling for the school year Daily teacher and student attendance is on the time line for , but there are policy issues that need to be resolved

Larry Fruth Schools Interoperability Framework Association (SIFA)

Gates – CELT TSDL Work The Gates Foundation invited five states—Arkansas, Florida, Georgia, Louisiana, and Ohio —to participate in the TSDL project. Three school districts from each participating state are included in all phases of the project, based on size, use of data, and technology infrastructure. Education Service Agencies (ESAs) in Ohio and Charter Schools in Florida and Georgia were part of the site visits and assessments of current policies and practices.

Teacher- Student Data Link Identify programs that prepare highly qualified and effective teachers Help align student needs with teacher effectiveness characteristics Aid in determining characteristics of effective teachers Provide assistance for teachers needing support early on Identify staff development programs that help create effective teachers Assess the value of non- traditional teacher preparation programs Determine variables that help or hinder teacher effectiveness Determine which teachers help students become college ready and successful

Sample Definitions A Teacher of Record is an educator (or educators in co-teaching assignments) who has been assigned the lead responsibility for a student’s learning in a subject/course section with aligned performance measures. A Contributing Professional is an individual who has been assigned the responsibility to provide additional services that support and increase a student’s learning.

Characteristics The Teacher of Record definition should: Be flexible to cover all grade levels, pre-K through 12. Accommodate teacher assignment changes and turnover during the course of the semester or year. Be supportable by current systems and data collection methods. Be clear and understandable for all stakeholders. Be applicable to all teachers and cover all courses and subjects including virtual (online) courses. Accommodate co- or team- teachers for a given subject/course.

Purpose Support fund/resource allocation formulas. Allow for the identification of the primary teacher(s) for a subject/course and track contribution(s) to achievement. Identify other educators contributing to student’s learning and track their contribution to achievement. Plan and evaluate PD tailored to student outcomes and specific standards, objectives, and pedagogy. Have appropriate student data within evaluation systems.

Purpose Examine teacher prep programs and other program providers using student outcome data. Better evaluate instructional practices and programs and determine their effectiveness Support accountability models including ones based on longitudinal data linking contributions to student outcomes to multiple teachers, programs and schools Support pay for performance programs. Identify highly effective teachers to be role models and coaches in support of other educators.

Tom Purwin Jersey City Public Schools (NJ)

Teacher of Record Data Model Participation Weighting Class Level vs. Student Level Measuring for Compliance (Staff Perspective) vs. Measuring for Pacing/Adjusting for Learning (Learner Perspective)

Class Level Participation Weighting Students in Class XXX Room 127 Period 4 Teacher in Class XXX Room 127 Period 4 Example 1

Class Level Participation Weighting (Cont’d) In the Scheduling Software you link the Course section to the Teacher ID number and the students to the Course Section. Define the room by period and each period has the teacher (staff ID) assigned to the room. [Class Membership and Employee Class Membership] One Teacher is responsible for instruction to a class full of students. Example 1

Class Level Participation Weighting Students in Class XXX Room 127 Period 4 Teacher Inclusion Teacher Class XXX Room 127 Period 4 Example 2

Class Level Participation Weighting (Cont’d) In the Scheduling Software you link the Course section to the Teacher ID number and the students to the Course Section. Define the room by period and each period has the Adults (staff IDs) assigned to the room with a Specified Proportion. A Teacher and an Inclusion Teacher are each responsible for a Specified Proportion of instruction to a class full of students. Example 2

Class Level Participation Weighting Possible Scenarios Multiple Adults in the Room: Teacher Aide L.A./Math Literacy Coaches Student Teacher Others (Tutors, Mentors,…) [ Each Adult is assigned to the room with a Specified Proportion ]

Student Level Participation Weighting Is the Inclusion Teacher impacting on all the students in the class, or a subset? The Inclusion Teacher is being used as a Special Education Teacher providing accommodations for the classified students; The Inclusion Teacher is also being used as “Response to Intervention” (RTI) resource to non-classified students; or, The Inclusion Teacher is actually providing direct instruction to all students in a core subject (Regular Teacher and Inclusion Teacher select core subjects to teach based upon interest or certification). [This can vary in a district from school-to-school or teacher-to- teacher within a school.]

Student Level Participation Weighting The other impacts on a student’s subject mastery: The student participates in After-School remediation in writing; Art, Music, and Phys Ed specialist teachers provide writing assignments twice a week to improve writing skills; Technology Coach provides In-Class support for researching topics for writing assignments and providing web-based writing tools; Library Media Specialist is available as a resource during lunch periods and study periods; Advanced Placement student provides tutoring; and/or, Homeroom Teacher mentors student and contacts parents to verify assignments and give weekly status updates.

Student Level Participation Weighting In the Scheduling Software you go to the student demographic page and collect information that links the Course Topic to the appropriate Staff ID numbers with a Specified Proportion for each Staff ID. The total Specified Proportion for each student’s Course Topic should total 100%.* * = The total can be less that 100% if there are programs that impact on the learning (i.e. sports, new curriculum, …). Each program (Program ID) impact is assigned a Specified Proportion and the total of all Staff IDs and Program IDs = 100%.

Measuring for Compliance (Staff Perspective) The Teacher of Record is identified using the number of adults in a room each having a proportional impact on all of the students. An inclusion teacher is given a proportional impact only on the Special Education Students that are provided modifications. The Teacher Aide is given a proportional impact on all the students in the class. A 1-on-1 Aide is given a proportional impact on the assigned student.

Measuring for Pacing/Adjusting for Learning (Learner Perspective) The Teacher of Record is assigned using the number of adults in a room having a different proportional impact for each student in the room dependant upon other adult/program impacts on each student (regardless of the adult in-room status). An inclusion teacher is given a proportional impact on the Special Education Students that are provided modifications, if appropriate, or is given a proportional impact on all students if used as a co-teacher or responsible for direct instruction of a subject. The Teacher Aide is given a proportional impact on the students in the class serviced by the Teacher Aide.

The Impact of all staff needs to be measured Literacy CoachLibrary MediaArt Teacher Technology CoachGuidance CounselorsBldg/C.O. Administrators

Other impacts need to be measured After School Programs Student Attendance Staff Attendance Curriculum linked to Standards Continuously Enrolled Status Teacher Certification Administrator Certification Teacher Assignments/Stability

District Implementation Process Empower District Data Team Members include Sr. Cabinet, Business Staff, Principals, Testing Staff, Supervisors, Technology Staff Overview of Existing Systems and Data Collection Methods Breakout workgroups to identify data sources, procedures, data quality, etc… Revise as needed. Raise team awareness levels to have the discussions identified on the TEACHER OF RECORD Framework

Teacher of Record Data Model Class Membership Student # Days Absent [# of Days in Attendance # of Days with Instruction/Learning] Employee Class Membership Staff # Days Absent[# of Days in Attendance # of Days providing Instruction/Learning opportunities]

Accountability Student Input Peer Input Self Evaluation Professional Growth Student Achievement Data Attendance and Actual Learning Events Individual Student Growth Models Evaluations School Level Productivity Measures

Mike Hopkins & Raymond Yeagley Rochester Schools (NH) Northwest Evaluation Association (NWEA)

A Model for Studying Student Progress as Related to Instructional Resources and Strategies

Central Assertions In many cases, student learning in a given subject is influenced by more than a single teacher. Identifying most or all contributors to student learning can help with planning and implementation of effective instructional strategies. Some contributors are not teachers. Designation of a single “Teacher of Record” for accountability is seldom, if ever, useful in informing instructional decisions that can influence the rate of learning for an individual student.

Learning Influencers in Schools Teachers – research suggests they are the most important factor in the school setting. Support staff – can help teachers be more efficient and can provide additional support to the student. Strategies – a broad definition may include Textbook selection Program adoption and implementation Individual instructional strategies Teacher/student assignment decisions

The Role of Data Essentially, all models are wrong, but some are useful – George Box, 1987 Statistical significance is not always a necessary precursor of usefulness – Raymond Yeagley, 2011

Student TeacherSchool Aide Test Event Assessment Enrollment Section Ethnicity Strategies The Beginnings of a Data Model

Student: Hamlet Prince Control the Burden for Teachers Teacher of Record: Juliet Capulet Pre-populate as much as possible

Educator Roles Main Teacher (Teacher of Record?) Team Teacher In-class Supplemental Teacher Pull-out Supplemental Teacher After-school Supplemental Teacher In-class Aide Pull-out Aide After-school Aide

Could parts of this model be useful for state data systems? For many students, assigning responsibility and accountability for learning to a single Teacher of Record fosters an illusion that may be politically satisfying, but is educationally weak.

Discussion TECH ideas, opinions, reactions, concerns, and questions.

Contact Information Bethann Canada, Virginia Department of Education Charlene Swanson, New York State Education Department Larry Fruth, SIFA Tom Purwin, Jersey City Public Schools (NJ) Mike Hopkins, Rochester Schools (NH) Raymond Yeagley, NWEA