1. QualityCore –Spring 2013 Test Administration Training January 28, 2013 Joe Dell Brasel, Ed.D. Principal Consultant ACT Atlanta Office joedell.brasel@act.org Jennifer Stafford Policy Advisor Kentucky Department of Education Jennifer.stafford@education.ky.gov KDE:OAA 1/23/2013

2. Agenda What’s New Constructed Response Announcement Testing Windows and Sessions Responsibilities and Deadlines Accommodations Materials Group Headers Labels Answer Documents Return Shipping Prohibited/Acceptable Materials Resources KDE:OAA 1/23/2013 1

3. What’s New When students are tested Constructed Response shift to local administration Multiple Choice test format combination KDE/ACT upload of student accounts Window 6 (June 16-July 16) Computer Based Testing Only KDE:OAA 1/23/2013 2

4. When Students are Tested Policy Change: Students are tested at the completion of all coursework. No longer calculate if 55% Example 1Example 2Example 3 Student passes semester 1 Student passes trimester 1 Student passes semester 2Student fails semester 2Student fails trimester 2 Student takes examStudent retakes semester 2Student takes trimester 3 Student takes examStudent retakes trimester 2 Student takes exam KDE:OAA 1/23/2013 3

5. New Features Currently Available Batch printing for student reports (directions shared in January 7 DAC Monday E-mail) Combined MC/MC into one booklet In Development iPad application KDE:OAA 1/23/2013 4

6. End-of-Course - Dual Purpose Instruction Objectives Syllabus Course Outline Instructional Units Formative Item Pool/Benchmark Assessments Accountability Student School District State KDE:OAA 1/23/2013 5

7. Issues with Constructed Response Critical for Instruction Super Scale Score Student Motivation Security of CR Items Logistics State Guidelines Two Separate Scores Paper Based Testing KDE:OAA 1/23/2013 6

8. Local Administration Instructional Value Constructed Response administered locally will render more instructional value. Students, parents, teachers have student work to score and analyze. Teachers can identify strengths and weaknesses in student writing. Instruction can be based on current data. KDE:OAA 1/23/2013 7

9. Constructed Response Shift to Local Administration Constructed Response will not be part of state administered assessment. Students will take multiple choice sections for state accountability. Schools will receive Scale Score (MC/MC). The 2012-13 grading scale aligns more closely to the performance levels (NAPD). Schools can have a better idea of performance levels. KDE:OAA 1/23/2013 8

10. Constructed Response Shift to Local Administration (continued) Constructed Response should be administered at the local level Local administration and scoring Included in student’s final exam grade Instructional Value Encourage schools to revisit EOC policies Review the policy which created the percentage of the EOC into the student’s grade KDE:OAA 1/23/2013 9

11. Constructed Response Shift to Local Administration (continued) Local staff may obtain from various locations Formative Item Bank (Vantage System) Benchmark Assessment (Vantage System) Local Source Text Book Locally Developed Open Source KDE:OAA 1/23/2013 10

12. Constructed Response Shift to Local Administration (continued) Materials Order process will be the same without constructed response. Computer Based testing for EOC would yield no shipment from ACT. Support materials and manuals are available online. For paper based testing, schools will only receive one Answer Sheet and one set of student labels. It will simplify returning materials. For computer-based testing of multiple-choice items, student results are available in the online system within 24 hours. KDE:OAA 1/23/2013 11

13. Constructed Response Implications Window 4 February 1 – March 15 Constructed Response test booklets will be shipped by ACT in early part of window DACs hold at district office Manuals do not reflect change Window 5 & 6 April 15-June 15 & June 16-July 15 Constructed Response test booklets will not be shipped by ACT DACs will not receive CR test booklets Manuals will reflect change KDE:OAA 1/23/2013 12

14. Two Sessions Two separate EOC sessions Multiple Choice 1 (Paper or Computer) Multiple Choice 2 (Paper or Computer) Policy Change: School staff may choose to combine paper and CBT for Multiple Choice. Sessions are 45 minutes each Paper sessions are timed by the test administrator. Computer sessions are timed automatically by the computer. KDE:OAA 1/23/2013 13

15. Testing Sessions Paper Testing: ALL students must be tested at the same time, in the same content area. Computer Testing: Multiple testing sessions can be utilized in a single day. New Policy: The first computer based test must begin at the same time as the paper based test. Note: A 10-15 minute break is allowed between test sessions. KDE:OAA 1/23/2013 14

16. Testing Windows (Includes Paper and Computer Testing) Six EOC test windows per year August 1-September 15 (new) October 1-November 16 December 1-January 15 February 1-March 15 April 15-June 15 June 16-July 15 (Computer Based Only) Within each window, schools choose a 10-day window (consecutive days) to complete the four tests, including make-up sessions. Within the 10-day window, schools choose 1 day per content area. MC1 and MC2 must be given in order on the same day. May schedule multiple content tests on same day. KDE:OAA 1/23/2013 15

17. Non-Traditional Windows (Includes Paper and Computer Testing) Six EOC test windows per year August 1-September 15 (new) October 1-November 16 December 1-January 15 February 1-March 15 April 15-June 15 June 16-July 15 (Computer Based Only) Within each window, schools choose one day per student(s) to complete sessions. MC1 and MC2 must be given in order on the same day. May schedule multiple content tests on same day. KDE:OAA 1/23/2013 16

18. Traditional vs. Non-Traditional? High Schools with only Traditional Courses: Use school’s 10-day window Alternative Schools with only Non-Traditional Courses: Use State’s broader window Schools with both Traditional and Non-Traditional Courses: Use both school’s 10-day and State’s broad window KDE:OAA 1/23/2013 17

19. Steps for Ordering Confirm Unit Usage (DACs/Designee) Create/Upload Users & Teachers (DACs/Designee) Create/Upload Students (KDE/ACT, Local) Create Class Test Rosters (Local) Finalize Class Rosters (Local) Note: Valuable Resources are located at: www.act.org/stateservices/kentucky www.act.org/stateservices/kentucky KDE:OAA 1/23/2013 18

20. Responsibilities District or School Administrator: Conduct Administration Code and Inclusion Training Confirm Unit Usage per school Determine Paper or Computer Testing Estimate Number of Students Testing No automatic overage. DAC/BAC should order additional tests per school. Determine Testing Dates Create/Upload Students (New enrollees) KDE:OAA 1/23/2013 19

21. Create and Upload Students KDE will extract from Infinite Campus (IC) students enrolled in IC at the high school level within each district. ACT will create student accounts at the appropriate school. Student grade levels will reflect current status in IC. Accounts for students enrolled after this date will need to be created by the school. Note: Middle school students taking courses for high school credit will not be included. KDE:OAA 1/23/2013 20

22. Create and Upload Students (continued) KDE/ACT Upload of student accounts Spring—Data pull from IC Dec. 15 in Vantage early February 2012 High School Graduates will remain in the Vantage system for one year. KDE:OAA 1/23/2013 21

23. Responsibilities School Administrator: Create Class Roster Name: Options Name by school, course, teacher, and period i.e. CCHS, Algebra II, C. Greer, period 5 Name by school, course, per teacher i.e. CCHS, Algebra II, C. Greer Name by school, course i.e. CCHS, Algebra II Note: Separate class roster for extended time for computer based (specific timing). KDE:OAA 1/23/2013 22

24. Responsibilities School Administrator or Teacher: Finalize Class Rosters Verify all students to be tested are on roster No roster in SDRR at this time Print or save electronic copy to use as a reference for accountability in SDRR at the end of school year. Create seating chart KDE:OAA 1/23/2013 23

25. Test Form Changes KDE:OAA 1/23/2013 24

26. Target Dates Deadlines for April 15 – June 16 End-of-Course Testing Begin Testing the Week of: April 15April 22April 29May 6May 13May 20May 27June 3June 10 Confirm Unit UsageFebruary 18February 25March 4March 11March 18March 25April 1April 8April 15 Create/Upload Users & TeachersShould be Complete Create/Upload StudentsMarch 4March 11March 18March 25April l1April 8April 15April 22April 29 Create Class Test RostersMarch 11March 18March 25April 1April 8April 15Aprill 22April 29May 6 *Finalize Class Rosters March 15March 22March 30April 5April 12April 19April 26May 3May 10 *Finalize Class Rosters must be completed four (4) weeks plus one day prior to testing. Other dates are targets for workload management. KDE:OAA 1/23/2013 25

27. Target Dates Deadlines for June 17 – July 15 End-of-Course Testing Begin Testing the Week of:June 17June 24July 1July 8July 15 Confirm Unit UsageApril 22April 29May 6May 13May 20 Create/Upload Users & TeachersShould be Complete Create/Upload StudentsMay 6May 13May 20May 27June 3 Create Class Test RostersMay 13May 20May 27June 3June 10 *Finalize Class Rosters May 17May 24May 31June 7June 14 *Finalize Class Rosters must be completed four (4) weeks plus one day prior to testing. Other dates are targets for workload management. KDE:OAA 1/23/2013 26

28. Special Populations Regulation Inclusion of Special Populations in the State-Required Assessment and Accountability Programs No changes to the Inclusion of Special Populations Regulation (February 2009) Accommodations must be identified in a student’s Individualized Education Program (IEP), 504 Plan or Program Services Plan (PSP) to be provided on the End of Course assessment KDE:OAA 1/23/2013 27

29. Accommodated Testing Paper Only R = Reader A = Audio B = Braille LP = Large Print Scribe Paper and Computer Extended Time Paraphrasing Manipulatives Prompting/Cueing Behavior Modification Interpreter (sign language) Dictionaries/Thesauri KDE:OAA 1/23/2013 28

30. Ordering Accommodated Test Materials Navigate to http://www.act.org/stateservices/kentuckyhttp://www.act.org/stateservices/kentucky Select the Resources tab (top right side of screen) Select Test Accommodations (left side of screen) Download and complete the KDE Accommodated Testing Materials Order Form. Once the order form is completed, e- mail it to qualitycorecustomerservice@act.orgqualitycorecustomerservice@act.org Note: This process is for accommodated materials, not test booklets. Paper units must be allocated to the school and students must be on finalized roster for order to be complete. KDE:OAA 1/23/2013 29

31. Ordering Accommodated Test Materials Deadline This includes ordering of reader scripts. Fax Form to QualityCore Customer Services Form available at: http://www.act.org/stateservices/kentucky/accommodations.htmlhttp://www.act.org/stateservices/kentucky/accommodations.html Begin Testing the Week of: April 15April 22April 29May 6May 13May 20May 27June 3June 10 Accommodations Order March 11March 18March 25April 1April 8April 15April 22April 29May 6 Begin Testing the Week of: June 17June 24July 1July 8July 15 Accommodations Order May 13May 20May 27June 3June 10 KDE:OAA 1/23/2013 30

32. Exemption Types Medical Exemption choice requires more detailed information, including Date of Diagnosis and Hospitalization. More detailed descriptions are needed for approval. Extraordinary Circumstance (Other Exemption) choice has open field to enter information. For both types of exemptions, see the associated Medical Exemption and Extraordinary Circumstance forms on the KDE web site and in the Forms section of the DAC/BAC. SDRR will be open for exemption applications in Spring. KDE:OAA 1/23/2013 31

33. Exemption Information Student identifying information Name SSID Grade Gender DOB School Name Test Type Exemption Type Explanation of Medical or Extraordinary issue Relevant dates 1.Keep paper form on-file in the district. 2.Enter information electronically into SDRR. An Exemption Request is NOT automatically approved. KDE:OAA 1/23/2013 32

34. Receiving/Storing Materials February 1 – March 15 Shipment going to DACs will be identified with course name, school and BAC’s name for easy distribution Check shipment contents Testing Boxes Test Booklets Miscellaneous Boxes Two Answer Folders (MC1/MC2 and CR) Pre-ID Labels Reference Sheets--Algebra II Test Supervisor’s Manual (MC) and Room Supervisor’s Manual (one per 25 students (MC)) School Header Group Header Box 1 Contains Packing List Store SECURELY LOCKED storage Discrepancies— Call (866)764-7228 KDE:OAA 1/23/2013 33

35. Group Headers One per class, per roster One for MC1/MC2 Example: If Ms. Jones has 3 Algebra II classes, there will be 3 separate Group Headers. 3 MC Group Headers KDE:OAA 1/23/2013 34

36. B: Test Roster Name D: Vantage Test Assignment Roster ID

37. Information for Group Headers Vantage Test Assignment Roster ID Test Roster Name KDE:OAA 1/23/2013 36

38. Pre-ID Labels February 1 – March 15 Arrive in Miscellaneous Box(s) One per student answer folder MC1/MC2 Errors on label Do not edit Make corrections online within class roster No Label Add new students online KDE:OAA 1/23/2013 37

39. Answer Document and Labels Affix label to answer folder. Do not bubble student information

40. Return Shipping: Group Headers Group Header 1 MC/MC English 10 KDE:OAA 1/23/2013 39

41. Return Shipping Packing Envelopes Used Answer Documents (MC1/MC2) Testing Irregularity Report Instructions on pages 12-17 of Test Supervisor’s Manual Boxes Used and unused tests booklets in the boxes stamped with their course names Unused answer documents and manuals in the boxes stamped “Miscellaneous” Instructions on pages 12-17 of Test Supervisor’s Manual KDE and ACT recommend return shipping from the school. KDE:OAA 1/23/2013 40

42. Packaging and Shipping April 15 - June 15 One document per student per test Test booklet pre-printed with student and teacher name Answer sheet pre-filled with student information Answer sheet bound as perforated sheet into test booklet Reference Sheet for Algebra 2 bound as perforated sheet into test booklet One document per Test Supervisor Test Supervisors Manual Pre-filled School Header Pre-filled Group Rosters All shipped from the printer More information to come on what shipment will look like KDE:OAA 1/23/2013 41

43. Window 6: June 16-July 15 Policy Change: Computer Based Testing Only Return of student scores Guarantee from ACT to provide accurate data Ensures students’ scores will be included in accountability KDE:OAA 1/23/2013 42

44. Supplemental Order Process E-mail: qualitycorecustomerservice@act.orgqualitycorecustomerservice@act.org Quantity needed per course See example below: MASON COUNTY SCHL DIST MASON COUNTY HIGH SCHOOL Biology suborder 8233 10 tests Eng 10suborder 82324 tests May 5—May 16 KDE:OAA 1/23/2013 43

45. Irregularities A disturbance or distraction occurs that could affect one or more students’ scores. A student becomes ill and discontinues testing. A student questions or challenges a test item. A student uses unauthorized testing aids. Complete a Testing Irregularity Report (Test Supervisor’s and Room Supervisor’s Manuals) KDE:OAA 1/23/2013 44

46. Materials Computer Prohibited Notes Foreign language or English dictionary Electronic devices and cell phones Allowed Scratch Paper-Please collect and destroy securely. Calculators Reference Sheets (Algebra II) Pencil/Paper Prohibited Scratch paper Notes Foreign language or English dictionary Electronic devices and cell phones Allowed Scratch work is to be done in the test booklet. Calculators Reference Sheets (Algebra II) KDE:OAA 1/23/2013 45

47. Practice Online Testing Coordinate with District Technology Coordinators to verify workstations are ready for online testing CBT Launch Demo PPT QC CBT QuickStart LaunchTest Teachers/Students may use the Launch Demo to practice the system. KDE:OAA 1/23/2013 46

48. KDE:OAA 1/23/2013 47

49. Test Security Appropriate training for Administration Code and Inclusion Regulations Signatures on Administration Code and Inclusion Training Seating charts that reflect everyone in each testing room (adults and students) KDE:OAA 1/23/2013 48

50. Seating Chart KDE:OAA 1/23/2013 49

51. Resources Kentucky QualityCore Website www.act.org/stateservices/kentucky www.act.org/stateservices/kentucky Kentucky Department of Education http://education.ky.gov/AA/distsupp/Pages/EOC.aspx http://education.ky.gov/AA/distsupp/Pages/EOC.aspx KDE:OAA 1/23/2013 50

52. Contact Information ACT ACT Customer Services: E-mail: qualitycorecustomerservi ce@act.org qualitycorecustomerservi ce@act.org Phone: (866)764-0228 Joe Dell Brasel, Ed.D. E-mail: joedell.brasel@act.org joedell.brasel@act.org KDE Assessment Support and Research: E-mail: dacinfo@education.ky.gov dacinfo@education.ky.gov Phone: (502) 564-4394 Jennifer Stafford E-mail: Jennifer.stafford@education. ky.gov Jennifer.stafford@education. ky.gov KDE:OAA 1/23/2013 51