1. Developing a Monitoring and Pre-Scoring Plan for the Virginia Grade Level Alternative (VGLA) Adapted from the Virginia Department of Education Division of Student Assessment and School Improvement August 2010

2. Purpose of Presentation To assist schools in developing plans to monitor and pre-score Collections of Evidence (COE) for the Virginia Grade Level Alternative (VGLA)

3. Developing Monitoring and Pre-Scoring Plans What are monitoring and pre-scoring? Why should administrators develop monitoring and pre-scoring plans at their school? What components should a monitoring plan include? What components should a pre-scoring plan include?

4. What are monitoring and pre-scoring?

5. Monitoring A periodic and systematic review of a Collection of Evidence, while it is in the development process, with opportunities for feedback and intervention

6. What Monitoring is Not Simply “checking-in” with a teacher Flipping through a Collection of Evidence Reviewing a Collection of Evidence without providing feedback

7. Pre-Scoring A detailed review of Collections of Evidence to address technical errors (i.e., grading, SEI tags), prior to scoring, with opportunities for feedback and corrections

8. What Pre-Scoring is Not Assigning a rubric score to evidence Altering student evidence or student responses Judging the instructional quality of the evidence

9. Why should administrators develop monitoring and pre-scoring plans at their school?

10. Monitoring Keeps staff apprised of the status of each Collection of Evidence Helps to ensure that teachers are providing instruction based on the Standards of Learning (SOLs) Ensures that instruction is on target with the Division’s pacing guide

11. Monitoring Helps ensure that technical issues (i.e., grading, SEI tags) are appropriately addressed Provides the opportunity to address issues with ample time for correction and/or intervention prior to scoring at the Division level

12. Pre-Scoring Provides a final opportunity before scoring to correct technical errors and omissions that could result in the loss of credit to students Results in the creation of better Collections of Evidence that accurately represent the student’s true knowledge and/or proficiency

13. What components should a monitoring plan include?

14. Critical Planning Information Schools should develop a Master Participation List for each alternate/alternative assessment program that includes the... –Name of student –Grade level of student –School –Content area being assessed by the alternate/alternative assessment –Teacher(s) responsible for instruction in the content area assessed

15. Monitoring Plan Decisions –Who will monitor? –How will monitors be trained? –When will monitoring occur? –Who will receive feedback? –How will feedback be given? –What options are available if corrections are needed?

16. Remember! Monitors will provide training and/or technical guidance to teachers submitting Collections of Evidence. Monitors and/or teachers are not allowed to score their own Collections of Evidence or ones they have monitored during the course of the school year.

17. Who should monitor? Staff with content knowledge and experience with alternate/alternative assessments –Special education or testing specialists –Instructional specialists –LEP specialists –School Test Coordinators –Title I mathematics and reading specialists –Experienced teachers –Building Administrators Note: The Testing Office recommends Principals and/or VGLA Administrators be responsible for monitoring Collections of Evidence at his/her school.

18. When should monitoring occur? Weekly? Monthly? At the end of each grading period? At the same time as benchmark testing? Other times?

19. Who receives feedback? Teachers Principals Alternate/Alternative Assessment Leaders Central Office Staff? Trainers/Technical Assistance?

20. How will feedback be provided? Post-It notes on Collections of Evidence Locally created feedback forms given to teachers, principals, and/or others Meetings with teachers, principals, and/or others about monitoring results Other?

21. What options are available if corrections are needed? Individualized consultation and support Peer or small group training sessions Professional development Access to state and local SOL and/or ASOL assessment resources Other

22. A Sample Monitoring Plan School-based review teams are created in the school. Division Director of Testing, Special Education Director and Content Supervisors will train school-based review teams using implementation manuals, VDOE power points, division pacing guides, and other resources. Teachers submit Collections of Evidence to the school- based review teams at the end of each review cycle.

23. A Sample Monitoring Plan School-based review teams can report their findings on each Collection of Evidence using a locally created form. A copy of the form can be reviewed by both teachers and principals. Central Office may dispatch specialists to schools in need of additional support and training upon request.

24. A Sample Monitoring Plan 1 Select and train school-based review teams 3 If necessary, specialists may be dispatched to schools based on the reports 2 School-based review teams check Collections of Evidence every review cycle and report to school leadership/administration.

25. Sample School-Based Review Team Sample School-Based Review Team 9-Week Monitoring Form – Grade 4 Reading Student Name 100 - 90 % of SOL Evidenced 89 - 70 % of SOL Evidenced 69- 50 % of SOL Evidenced Less than 50% of SOL Evidenced Ted Smith  Kay Blue  Joe Davis  Ann Jones 

26. A Sample Teacher Review Form VGLA 9-Week Review Sheet School-Based Teams Teacher: ____________________ Student: ___________________________ Grade Level:______ Content Area:_________________________________________ REVIEW for: ____ 1st Nine Weeks ____ 3rd Nine Weeks ____ 2nd Nine Weeks ____ 4th Nine Weeks Reviewed By:____________________________ Date:________ Collection Status: Address the following questions: (1) Is there evidence for all of the standards for the nine weeks according to the Pacing Chart? (2) Does the work submitted align with the standards satisfactorily? (2) Does the evidence demonstrate student mastery? (3) Has the student work been graded accurately? (4) Other? Recommendation(s): Be as specific as possible Follow-up review needed for implementation of recommendations: ___Yes ____No Follow –up Review Date: ________________

27. Pre-Scoring Decisions –Who will pre-score? –How will pre-scorers be trained? –When will pre-scoring occur in relation to submission date? –Who will receive feedback? –How will feedback be given? –What options are available if corrections are needed?

28. Who will Pre-Score? Persons selected for pre-scoring cannot serve as scorers for their students’ Collections of Evidence. Pre-scorers should be knowledgeable about all scoring rules and the SOL and/or ASOL content. Pre-scorers should be detail oriented.

29. Remember! VGLA Administrators should provide training and/or technical guidance to teachers submitting Collections of Evidence. Pre-scorers are not allowed to score their own Collections of Evidence.

30. How will Pre-Scorers be trained? Training should include a list of specific issues to look for in each Collection of Evidence – Grading? – Correctly completed SEI tags? – Detailed anecdotal records (Not allowed) – Captioned photographs (Not allowed)

31. Other helpful questions to ask while Pre-Scoring Is the evidence organized according to the scoring worksheet? Are all required state and local forms included and completed?

32. When should Pre-Scoring Occur Select a time to Pre-Score: Give ample time for Collections of Evidence to be complete or near completion. Give an opportunity to return Collections of Evidence to teachers for corrections well in advance of the due date –Collection of Evidence are scheduled for pick up on April 25, 2013

33. Who should receive feedback? Teacher(s) submitting Collections of Evidence Building level principals Others?

34. How will feedback be given? Attach notes to Collections of Evidence Create forms to give to teachers and/or others Meet with teachers and/or others Other ideas? PWCS Testing Office recommends frequent and direct communication with teachers!

35. What options are available if corrections are needed? Return Collections of Evidence to teachers for corrections. Assign staff to assist teacher in corrections such as grading evidence, organizing evidence, completing SEI tags. Some options may simply be limited or non- existent due to time constraints and volume. Plan accordingly so as to not disadvantage the student!

36. Sample Pre-scoring Plan Pre-scoring will be conducted by teachers and building level staff in two stages –Note: Central Office staff may be consulted, if required. Scoring rules should be reviewed with all persons involved in pre-scoring. Checklists are provided in the VGLA Implementation Manual 2013-14.

37. Sample Pre-Scoring Plan Stage 1 – Final Teacher Review –Conducted early to mid April –Submitting teacher reviews COEs using Teacher Checklists and in accordance with scoring rules outlined in the VGLA Implementation Manual 2013-14 Stage 2 – Final School Review –Conducted late April –Building Administrator and/or VGLA Administrator reviews Collections of Evidence using Administrator Checklist and in accordance with scoring rules outlined in the VGLA Implementation Manual 2013-14. NOTE: Collections of Evidence are scheduled to be picked up from schools on April 25 th.

38. Monitoring & Pre-Scoring Results Helps reduce and eliminate Collections of Evidence with: –Missing Evidence –Missing or incorrectly specified SEI tags –Unacceptable evidence (i.e., in violation of scoring rules) –Inaccurate or Ungraded Evidence

39. Monitoring & Pre-Scoring Results Eliminates Surprises –Collections of Evidence not done for students educated outside of the Division –Incomplete Collections of Evidence in the Division –Collections of Evidence not driven by IEP, 504 or LEP Plans

40. Questions Please visit: Testing Office page on the PWCS Intranet Portal VDOE Alternate/Alternative Assessment Page at: http://www.doe.virginia.gov/testing/alternative_assessments/vgla_va_grade_level_alt/index.shtml Additional questions may be directed to the PWCS Testing Office by calling 703.791.8308.