1 North Carolina National Assessment of Education (NAEP) 2009 LEA Training --Session B December 2008 Iris L. Garner, Ph.D. NAEP State Coordinator North Carolina Department of Public Instruction

Session B--Training Objectives: 1) To illustrate the NAEP sampling and data collection design 2) To describe the corresponding NAEP activities that take place before, during, and after the assessment

NAEP Sampling & Data Collection Design Training Objective #1

NAEP 2009 Assessment Assessment Window: 1/26/09 – 3/6/09 Subjects to be Assessed: Reading, Math, Science Some schools will have a Science Interactive Computer Task (SCIT) or Hands-on Task (HOT) for the science portion of the Field Test.

NAEP 2007 NC Performance Math grade 8: NC average score was (284) in 2007 was higher than that of the nations public schools (280). Reading grade 8: NC average score (259) in 2007 was not significantly different from that of the nations public schools (261).

NAEP 2009 Assessment Results will be reported at the state and national level The assessment will take approximately 90 minutes of student time (two 25 min. subject matter blocks and survey questions). Students take one subject

NAEP Design: Sampling and Data Collection Multi-Stage Stratified Random Sampling selection of schools students within schools are selected randomly List of schools are obtained from the National Center of Education Statistics (NCES) Common Core of Data (CCD)

NAEP Design: Sampling and Data Collection The Common Core of Data (CCD) U.S. Department of Education's National Center for Education Statistics Program Annually collects fiscal and non-fiscal data about all public schools, public school districts and state education agencies in the United States Data are supplied by state education agency officials

NAEP Activities Training Objective #2

NAEP Team Structure NAEP Coordinator (Iris Garner) NAEP Field Manager (Gene Biggerstaff) Field Supervisor Assessment Admin. (AAs) Assessment Coord. (ACs) Schools

Before Assessment

NAEP Activities Preliminary Activity The North Carolina Department of Public Instruction notified the districts of the school that were selected to participate in NAEP. NAEP Data Collection Forms have been completed and submitted on-line via the MyNAEP Web site at NAEP field staff started contacting the school test coordinators on 12/15/08 to set up the Pre- Assessment Visit (PAV).

NAEP Activities Preassessment Visit (PAV) PAV will be scheduled between January 6 and January 23. PAV should take approximately two hours. The visit could take longer if the school test coordinator has not prepared for the PAV. Prior to the PAV you will receive a packet of materials. Please refer to the INSTRUCTIONS FOR THE SCHOOL COORDINATOR.

NAEP Activities Before Assessment Before the Preassessment Visit 1) Distribute questionnaires to staff most knowledgeable about the student Refer to the Guide for Completing SD/ELL Questionnaires Select the method for completing the questionnaire: online or hardcopy

Questionnaire Guidance Student with Disabilities & English Language Learners Students with Disabilities (SD) and English Language Learners (ELL) Questionnaires Questionnaires for each student in the sample that was identified as SD or ELL. The questionnaires need to be filled out and returned to the coordinator prior to the PAV. If you have more students in the sample that are identified as SD/504 or ELL, let the NAEP field staff if additional questionnaires are needed.

Student with Disabilities and English Language Learners If the student participates in the regular EOGs or EOCs with or without accommodation, they should participate in NAEP. If the student can access a paper/pencil assessment, they should participate in NAEP with accommodations. In 2002, NC had the highest exclusion rate in the nation for grade 4 reading. The exclusion rate has reduced in 2003, 2005 and 2007.

Student with Disabilities and English Language Learners The exclusion rates will be recorded after the pre- assessment visit and will be monitored for high rates. NAEP field staff will administer all accommodated sessions. These sessions may have to occur after the standard administration. If a special needs student feels more comfortable with a known test administrator, then the teacher can administer the assessment with the NAEP field staff in the room.

Questionnaire Guidance Student with Disabilities & English Language Learners Questionnaire Guidance Includes: Folder-A Guide For Completing The Student With Disabilities and English Language Learner Questionnaires Frequently Asked Questions Step-by-step directions Accommodations Chart State-specific policies

Questionnaire Guidance Student with Disabilities & English Language Learners Students assessed using the NCEXTEND1 most likely will not be able to be assessed on NAEP. Students assessed using the NCEXTEND2 most likely will be assessed on NAEP since they can access a paper-pencil assessment. Supply of Blank Paper: The NAEP assessment does not allow a supply of blank paper. All notes, explanations, etc. should be written directly in the test booklet.

Questionnaire Guidance Student with Disabilities & English Language Learners All students that take the NAEP assessment are allowed to write in the test booklet. NAEP does have separate answer sheets. Writes in test booklet is not considered an accommodation for NAEP.

Questionnaire Guidance Student with Disabilities & English Language Learners A Spanish version of the NAEP assessments in Mathematics and Science is available. North Carolina at this time does not have translated versions of the state tests. This accommodation is not allowed. Except for the items listed above the North Carolina State Testing Program accommodations can be used with the NAEP assessments. If an accommodation is part of a students Individualized Education Program (IEP) or 504 plan and is used regularly in the classroom, and on the state assessments, it should be able to be used on the NAEP 2009.

NAEP Activities Before Assessment Before the Preassessment Visit 2) Distribute the parent letters

Parent Letter Parent letters will need to be sent home to students selected to participate in the NAEP assessment prior to the PAV. Use school letterhead Include date

Parent Letter Share the Sample Questions Booklet with the parents.

NAEP Activities Before Assessment Before the Preassessment Visit 2) Review and update Student Information Report – List of all participating students and demographic information. Note: The Student Information Report for the National Indian Education Study (NIES) are included in the Important NIES Material envelope.

NAEP Activities Student Information Review Collect any missing demographic information. Review student demographic data for accuracy. Determine if there have been any changes in students identified as SD and/or ELL. Identify which students, if any, have withdrawn or graduated, and who are neither enrolled nor attend the school in the grade to be assessed. Make all corrections directly on the report.

NAEP Activities Before AssessmentBefore the Preassessment Visit 3) Print out a current list of the students in the selected grade 4) Complete logistics questionnaire.

NAEP Activities Before AssessmentDuring the Preassessment Visit At the beginning of the PAV the NAEP field staff will meet with you to collect the information you completed prior to the PAV: Printed list of current students at the selected grade Corrected Student Information Report SD and ELL Questionnaires Copy of the parent letter Logistics Questionnaire

After the Pre-Assessment Visit Notify the students of the assessment. Notify the teachers of the assessment. Place the NAEP Storage Envelope in a safe place.

Supporting NAEP in Your School NAEP has produced a short movie that you can share with your teachers and students to help them understand the importance of NAEP. You can access Introducing NAEP to Teachers at the NCDPI Web site: You can access Introducing NAEP to Students at the NCDPI Web site:

Before Assessment

NAEP Assessment Day Assessment Day The NAEP assessment team will arrive at the school an hour and half prior to the scheduled assessment time. The NAEP assessment team will need a staging area throughout the assessment day. The NAEP team will administer all assessments, including the accommodated sessions.

NAEP Assessment Day Assessment Day NAEP does not need proctors for the assessments. Classroom teachers and/or other school personnel are welcome and encouraged to stay in the room during the NAEP assessment. School personnel will have to sign a confidentiality agreement if they observe the NAEP assessment.

NAEP Activity Assessment Day Your biggest role on assessment day is to make sure all students in the sample arrive at the designated assessment locations. You may need to work with you attendance staff to make sure missing students are really absent that day. If 90% of the students in the sample are not there that day, then a make-up session will need to be scheduled. The 90% does not include students who have withdrawn, students whos parents have refused participation, or students who were excluded.

After Assessment

NAEP Activity After Assessment NAEP representative conducts a short interview with the school coordinator A make up assessment day is scheduled if needed. The school coordinator will securely destroy the contents of the storage envelope on the date listed on front of the envelope. The school coordinator will mail the postcard to NAEP stating that the materials have been securely destroyed.

Thanks You We appreciate all you do; you make the NAEP administrations possible.