National Accessible Reading Assessment Projects Goals of Project NARAP Collaboration General Advisory Committee Project Details (ETS and PARA) Plans for Year 1 Overview

National Accessible Reading Assessment Projects Goals for Project 1.Develop a definition of reading proficiency 2.Research the assessment of reading proficiency 3.Develop research-based principles and guidelines making large-scale reading assessments more accessible for students who have disabilities that affect reading 4.Develop and field trial a prototype reading assessment

National Accessible Reading Assessment Projects NARAP Collaboration Proposal hinted at collaboration and we decided to bring it alive – in several ways: Name for Collaborative Entity Shared Web site Shared Committees (GAC, Definition Panel) Collaboration on Activities

National Accessible Reading Assessment Projects General Advisory Committee RFP Language: Form a general advisory committee on which representation is invited from the National Center for Education Statistics; the National Assessment Governing Board; the National Center on Educational Outcomes; the National Center for Research on Evaluation, Standards, and Student Testing; organizations representing test developers, reading educators, and researchers; disability groups; and other appropriate projects and organizations.

National Accessible Reading Assessment Projects General Advisory Committee RFP Language: The primary purposes of this advisory committee are to review and advise on the general plans of the project and to provide liaison with significant stakeholder groups.

National Accessible Reading Assessment Projects Question Does the Committee have any ideas about how to maximize collaboration between the projects? Or, does the committee have any concerns about the collaboration of the two projects, and suggestions for things to avoid?

National Accessible Reading Assessment Projects Project Details (ETS) Focus on learning disabilities Component approach to assessing reading –Isolate individual components of reading (e.g., comprehension or word recognition) into subtests –Allows for accommodations (e.g. audio presentation) to be specific to subtest without impact validity of entire test

National Accessible Reading Assessment Projects Research Agenda (ETS) 2005 –Focus Groups –Differential Boost –Factor Analysis –Differential Item Functioning

National Accessible Reading Assessment Projects Research Agenda (ETS) 2006 –Focus Groups –Cognitive Labs –Usability Studies –Diagnostic Score Reporting

National Accessible Reading Assessment Projects Project Details (PARA) Partnership for Accessible Reading Assessment Partners = National Center on Educational Outcomes (NCEO) Center for Research on Evaluation, Standards, and Student Testing (CRESST) Westat, Inc. Disability Focus = All disabilities that affect reading (for field test, will have sufficient samples of students with Learning Disabilities, Speech Language Impairments, Mental Retardation, and Deaf/Hearing Impairments)

National Accessible Reading Assessment Projects PARA – Seizing the Opportunity to Build on the Expertise of Others! Reading proficiency is being approached from the lens of students with vastly diverse needs There is a need for expertise and input of reasoned scholars, key national stakeholders, expert advisors, and end users of assessments.

National Accessible Reading Assessment Projects General Advisory Committee Definition Panel Technical Advisory Committee Principles Committee GOAL 1 Formulate a definition of reading proficiency, analyze the definition in relation to state standards, obtain input, and refine the definition GOAL 3 Develop research based principles and guidelines for making large-scale assessment of reading proficiency more accessible GOAL 4 Develop instruments and methods for assessing reading proficiency that are suitable for large scale administration Review and advise the project on general plans and to liaison with significant stakeholder groups Formulate working definitions of key terms for use in collaborative effort with other funded projects GOAL 2 Conduct a program of research on the assessment of reading proficiency to determine the effects of various factors of test development, design and administration Formulate working papers on research based principles and guidelines for collaborative effort Review and provide technical input on the specific research and development plans and activities Collaboration Project Details (PARA)

National Accessible Reading Assessment Projects Research Agenda (PARA) General Questions: 1.Sampling Issues: Which students need which kinds of assistance to access the types of reading assessment that states currently administer? 2.Validity of Accessible Assessments: How can a valid assessment be developed that will permit students with disabilities that affect reading to demonstrate their reading proficiency?

National Accessible Reading Assessment Projects Research Agenda (PARA) General Questions: 3. Universal Design and Individual Student Characteristics: What common forms of assessment are appropriate for group testing that are accessible to students with disabilities that affect reading? 4.Instructional Sensitivity/Opportunity for Use of Access Tools: What opportunities do students with disabilities that affect reading have to access reading through alternatives to print in the classroom?

National Accessible Reading Assessment Projects Plans for Year 1 Develop reading proficiency definition Projects hold meetings with advisory groups (GAC, TACs)

National Accessible Reading Assessment Projects Work of the Definition Panel (DP) Leadership Roles, Work Group, Membership of the DP, Tasks and Timeline

National Accessible Reading Assessment Projects Work Group Group co-leaders: Deborah Dillon, University of Minnesota and John Sabatini, ETS Members: Ann Clapper, NCEO; Laurie Cutting, ETS and Kennedy Krieger Institute; Lee Galda and David OBrien, University of Minnesota

National Accessible Reading Assessment Projects The Role of the Work Group Determine membership of the DP Develop agendas for the DP meetings Outline tasks for the DP Identify the products that will emerge from the work of the DP

National Accessible Reading Assessment Projects Membership of the DP Reviewed the current proposed membership ( 9 members plus 2 from ETS and 3 from the U of MN) Considered new panel members and a rationale for their inclusion Struggled with the need for expertise (re: the task at hand) and the potential for unwieldiness with a large panel

National Accessible Reading Assessment Projects Agendas for DP Meetings Teleconference Meetingoverview tasks and timeline; begin to identify research that we need to review Document Reviewsrequest that panel members review selected reports and drafts of the Synthesis Document Face-to-Face Panel Meeting/sseek panel members consensus on the working definitions of key terms and the definition of reading proficiency

National Accessible Reading Assessment Projects DP Tasks Identify critical issues and terms in addition to reading proficiency needed for the collaborative efforts Identify relevant research literature for the comprehensive review to generate a working definition of reading proficiency and other related terms

National Accessible Reading Assessment Projects DP Taskscont. Draft a Synthesis Documentthis document is comprised of key pieces of research, panel meeting minutes, and a review of current literature and research reports (e.g., RAND; NAEP Reading Framework for 2009); a list of key terms (glossary) will also be documented Identify issues and points in the document that need clarification

National Accessible Reading Assessment Projects Products from the DP and Continued Work Synthesis Document; formulation of a definition of Reading Proficiency Supporting documentation for the definition Focus Groupsindividuals will respond to the work of the DP and offer feedback Reconvene the DP to finalize the definition based on external feedback