Georgia Modification Research Study Spring 2006 Sharron Hunt Melissa Fincher

Georgia’s Student Assessment Program Includes an array of criterion-referenced tests (including an alternate) and a norm-referenced test Built from the ground up to be inclusive – GA includes teachers of special populations in all phases of test development – GA includes all students in pilots and field tests

Georgia’s Student Assessment Program GA provides a menu of traditional accommodation options – Presentation – Response – Setting – Scheduling Accommodations are provided during pilots and field tests

Georgia’s Student Assessment Program Although Georgia is inclusive, some students still struggle to access the tests – Schools sought additional avenues for students who were caught between assessments – neither the general or the alternate were completely appropriate for these students

Georgia’s Student Assessment Program Georgia looked for avenues to investigate how to make assessments more accessible Questions to answer – Who are the students struggling to access the general assessments? – What additional accommodations/modifications might assist in making the test more accessible? About the same time…. – Enhanced Assessment Instruments Grant announced (ultimately pulled back) – Announcement of the 2% flexibility

Georgia’s Student Assessment Program Georgia moved forward with an empirical research study designed to answer those questions The Testing Division worked with the – Division for Exceptional Students – Curriculum and Instructional Services Division

Study Purpose To investigate the effects of certain modifications on the reliability and validity of the state testing programs – Criterion-Referenced Competency Tests (CRCT) – Georgia High School Graduation Tests (GHSGT) Focus on reading and math

Study Purpose This study will help Georgia determine – What works – Who benefits The goal of the study is to help Georgia operationalize additional options for the school year

What is a modification? an alteration in the administration of an assessment that results in a possible change in the content or construct being assessed – typically either through the addition or removal of content

What is an accommodation? an alteration in the administration of an assessment that allows students to participate – how the test is administered or presented – how the student responds Construct (what the test measures) remains the same

Georgia Modification Study Focused on selected grades and content areas – Students in grades 4 and 7 took the Reading and Math CRCT (March 2006) – Students in grade 12 took the ELA and Math GHSGT (April 2006)

Georgia Modification Study For security reasons, off-level testing was used – Grade 4 students took the 3 rd grade Spring 2005 CRCT – Grade 7 students took the 6 th grade Spring 2005 CRCT – Grade 12 students took the Spring 2005 GHSGT

Georgia Modification Study Study included both special education students and regular education students – The number of special education students who took the test last year at each school was identified Tested the number of students identified, if possible – Schools randomly selected an equal number of general education students to participate

Georgia Modification Study Included all special education students, with the following exceptions: – GAA students – Students who require Braille or large print Set a broad net to help determine the characteristics of students assisted by modifications

Georgia Modification Study Each school assigned a condition for this study – All students within the school took the test under the same condition Both special and regular education students

Georgia Modification Study CRCT (grades 4 and 7) – Three conditions – Condition 1: Resource Guides – Condition 2: Read Aloud/Calculator – Condition 3: Traditional Administration

Georgia Modification Study GHSGT (grade 12) – Two conditions – Condition 1: Resource Guide – Condition 2: Traditional Administration

Condition 1 – CRCT & GHSGT Resource Guide – A single sheet (one for each content area) that provides key definitions, examples, multiplication tables etc. – A short practice test was provided to help introduce the resource guides to students; each item related to something on the resource guide Recommended introducing sample tests and guides a couple of days before the administration

Condition 1 – CRCT & GHSGT Resource Guide Development – Committee comprised of Testing, Special Education, and Curriculum specialists reviewed the tests identified concepts students might struggle with and discussed appropriate information for inclusion on the guide

Condition 2 - CRCT Read Aloud (Reading CRCT) – Teacher read the reading passages and questions to students Teacher reads test verbatim Calculator (Math CRCT) – Basic function calculators provided to schools assigned this condition

Condition 3 - CRCT Traditional Administration – Students took tests as they have always done Special education students received the standard accommodations they normally receive Regular education students did not receive accommodations

Condition 2 - GHSGT Traditional Administration – Students took tests as they have always done Special education students received the standard accommodations they normally receive Regular education students did not receive accommodations

Georgia Modification Study A survey was be provided for each special education student – The person most knowledgeable about the student completed the survey – This information will be critical in determining who benefits from the modifications

Georgia Modification Study Survey collected important information about each special education student – Student’s disability/ies – Severity of disability/ies – IEP objectives for reading and math – Time in mainstream/general ed classroom – Area/s served – Instructional services – Type of accommodation, if any

Georgia Modification Study Administration Procedures – Follow all administration protocols as outlined in the Examiner’s Manual – Complete all student demographics on answer documents, including State-Required Codes

Georgia Modification Study Administration Procedures – Followed all administration protocols as outlined in the Examiner’s Manual – Completed all student demographics on answer documents, including State-Required Codes – Complete survey for each special education student

Georgia Modification Study Key Research Questions Are there performance differences for students with disabilities on the CRCT and GHSGT when these assessments are administered under modified conditions? Are there performance differences for regular education students on the CRCT and GHSGT when these assessments are administered under modified conditions?

Georgia Modification Study Key Research Questions How, if at all, do performance differences on the traditional versus modified version of the CRCT or GHSGT differ for students with disabilities compared to regular education students? Are the CRCT and GHSGT reliable and valid assessments when administered under modified conditions?

Georgia Modification Study Currently analyzing data to answer the research questions – Analysis at the item level Are certain items more likely to function differently? – Analysis at the form level How does the test form function under each condition (modified vs traditional) for each group of students?

Georgia Modification Study Sharron Hunt, Director of Testing / Melissa Fincher, Assistant Director of Testing /