Ensuring Valid Test Results CGCS Annual Meeting Curriculum and Research Directors July 13, 2012

Session Outline Milwaukee and Wisconsin background Strategies to prevent violations of test security / assessment protocols Actions taken when reported violations are verified Response to unverified allegations of violations

Background: State Assessment Program Annual state test, 3-8 & 10; window opens in the last week of October and remains open until just before Thanksgiving Schools have until the week after Thanksgiving to pack up test boxes and return for scoring Results returned to schools in February; to the district in March State historically did not provide erasure analyses or conduct investigations for reported violations of test security

Preventative Strategies—Written Documentation Developed Assessment Ethics Policy Developed Assessment Procedure Manual, which outlines a. Rationale for standardized assessment protocols b. Links to policies on assessment ethics and employee code of conduct c. Specific do’s and don’ts with state testing d. Appropriate test prep activities

Preventative Strategies -- PD Require annual training of all staff involved in assessment, including those who proctor, distribute, or box up test booklets Require face-to-face training conducted by district staff of all designated School Assessment Coordinators and all School Leaders; they are required to train their staff at the sites (can use video supports CS provides) Require sign-off documentation annually

Preventative Strategies -- Analyses Conduct internal analyses of available data using several years of data ▫ Large gains analysis ▫ Erasure analysis ▫ Omission analysis Analyses are exploratory, not confirmatory but serve to build awareness of issues Schools with higher rates of irregularities were flagged and school leaders (and their supervisors) were notified Analytic procedures will be updated this summer based on information gleaned from KS conference

Actions Taken Various reports are received, mostly calls to the Department of Research/Assessment, by school staff, who question their own practices e.g. “I didn’t cover up the word list on the wall during testing; is that ok?” or “I found a grade 4 test booklet in the old filing cabinet of the retired guidance counselor. How do I return it to you?” Confirmed student cheating results in an invalidated test following established protocols

Actions Taken Staff reports of inappropriate practices by their colleagues result in 1. report of allegation to Wisconsin DPI 2. internal investigation by Administrative Accountability 3. report of results of internal investigation to DPI Depending on results of investigation, various personnel actions are taken with affected staff

Response to Unverified Allegations Internal allegations that are unverified may result in closer scrutiny of testing situation in the following year 2012 allegation of unusual test score gains was largely refuted by an unprecedented collaboration between the CGCS, the district (and other affected member districts), UW- Madison’s Value-Added Research Center, and the Department of Public Instruction

Further Information MPS Assessment website: Main MPS Home Page – Departments – Research & Evaluation – Assessment Or contact: Deb Lindsey, Director Research and Evaluation