1. No Child Left Behind: Considerations for the Assessment of Students with Disabilities Martha Thurlow and Sandra Thompson National Center on Educational Outcomes University of Minnesota

2. No Child Left Behind... a reauthorization of ESEA continuing in the context of the standards-based reform movement...with an emphasis on system accountability

3. Purpose of No Child Left Behind “…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments”

4. NCLB does NOT require student accountability (e.g., graduation exams to get a diploma) NCLB does require SYSTEM level accountability to ensure that all students learn to high levels.

5. State standards for what a child should know in math and reading now, and in science by 2005-06 Test every student's progress toward the standards. Beginning in 2005-06, test in grades 3 through 8 and once in high school in math and reading. Beginning in 2007-08, science achievement must also be tested. Requirements

6. Under NCLB Assessments shall provide for… Participation of all students Reasonable adaptations and accommodations for students with disabilities (IDEA & 504), and alternate assessment Inclusion of limited English proficient students with accommodations, including, if practicable, native-language versions of the assessment Assessment in English of reading/language arts for any student in US for 3 consecutive school years

7. But, isn’t IDEA our law? Why do we have to worry about NCLB?

8. IDEA 1997... a reauthorization created within the context of the standards-based reform movement... with another reauthorization coming in 2003

9. Key Provisions in IDEA 97 Statement of present levels, needs, and how they affect involvement and progress in general curriculum Annual goals and objectives to allow involvement and progress in the general curriculum

10. General educator collaboration Assessment – full integration into standards-based reform Key Provisions in IDEA 97 The key provisions in IDEA 97 really address equity concerns – access to common standards, challenging curriculum, and effective instruction

11. NCLB Requirements IDEA Requirements Together, they require us to address content standards, achievement standards, assessment and access to the general curriculum How do IDEA and NCLB “fit together?”

12. Assumptions that underlie both NCLB and IDEA point to the benefits of standards-based systems for all students All children can learn We need to be responsible for the learning of all children

13. Standards-Based Reform Context --- Everything else is negotiable --- schedules, place, time, structure, curriculum, instructional methods, methods of assessment...

14. This is SO important … “It is not possible to predict which children will be in the top half of the achievement distribution at any given level of general intelligence.” Kevin McGrew (2003)

15. 405060708090100110120130140 150 160 IQ=75 + 5 What should our expectations be for “Child in School” in reading? Child A in School

16. 90100110120130140 BRS 4050607080 Basic Rdg Skills SS (for GIA SS = 70-80 ) Educators’ views of expected reading achievement performance based on IQ of 75

17. 90100110120130140 BRS 4050607080 Basic Rdg Skills SS (for GIA SS = 70-80 ) Where will most of the Child A’s of the world achieve in reading? Regression- based expected score How can this be ?

18. “For most students with below average IQ scores, it is NOT possible to predict individual levels of expected achievement with the degree of accuracy that would be required to deny a student the right to high standards/expectations.” Kevin McGrew (2003) http://www.iapsych.com/iapdirector.htm

19. Data Are Emerging New York Regents Exam, 2001: Number of students with disabilities passing is higher than the number taking in the past Trend data across grades in large southern state – special education population changes over time mask closing of gap between special education and general education students

20. Recent article in the Boston Globe (December 22, 2002) Katie Bartlett has spent all of her 17 years exceeding the expectations the world placed on her when she was born with Down syndrome....Still no one was quite sure what would happen when Bartlett took the MCAS exam, now a requirement for a high school diploma in Massachusetts. This is what happened: She passed

21. Critical Assessment Issues Decision Making Accommodations Universally Designed Assessments Alternate Assessments

22. 3 Ways to Participate in Assessments  Same way as other students  With accommodations  In an Alternate Assessment But, this does not mean that it is simple

23. Decisions and the IEP team But this does not mean that it is simple WHO needs to be part of the discussion? WHAT external opportunities and constraints must be considered? IMPLICATIONS of decisions must be identified and discussed, recorded, and reconsidered each year. The IEP team has authority to make decisions...

24. Accommodations Accommodations are changes in instructional and assessment materials or procedures that allow the student’s knowledge and skills to be developed and assessed. Accommodations provide students with disabilities access to instruction and assessments, so that ALL can have access, participate, and make progress...

25. Good Accommodations Decisions Start with good instructional decisions Systematic questions about accommodations for individual students Collection of data to aid decision making What helps student learn or perform better? What has student or parents told you? What gets in the way of the student showing skills? What has the student been taught to use?

26. Types of Accommodations Response Mark test booklet Word processor Use references Presentation Repeat directions Large print edition Braille edition Setting Study carrel Small group Individualized Timing Extended time Frequent breaks Unlimited time Scheduling Specific time of day Subtests in different order Other Test preparation Out-of-level Motivational cues Across multiple days

27. Report on a Case Study: Out-of-Level Testing in a School District Jane Minnema

28. Research Questions  Student instruction?  Student and teacher perceptions?  Student selection?

29. Data Collection Strategies  IEP reviews (n=14)  Teacher provided student data (n=65)  Face-to-face interviews  Conducted on-site in the schools

30. Conclusions Instructed below grade of enrollment Not reported in accountability programs Test scores not usable for instructional decisions Policy implementation inconsistencies Lacked understanding of future effects

31. Definitional Issues Some accommodations are considered to change what is assessed, and others are viewed as “ok” – But the terms used to describe what is and isn’t “ok” vary across the states and districts. Know what terms mean in your state and district!

32. Stay on top of the literature at: http://education.umn.edu/NCEO/ AccomStudies.htm [NCEO’s online accommodations bibliography with search features]

33. Is the the choice of accommodations appropriate? Aligned with instructional accommodations, but not an excuse not to teach Student needs it to demonstrate knowledge and skills – or to participate in assessment Implications of using this accommodation have been identified and carefully considered Not determined by test publisher, but by student need, what is being measured (construct), and the purpose of the test

34. Legal Cases Are Changing Views About Accommodations Oregon – a new view of accommodations (innocent until proven guilty) California – cannot require a special waiver to use needed accommodations

35. Oregon Case “ Accommodations shall be considered allowable, valid, and scorable if they are used during instruction 1 or classroom assessment and are listed on a student’s IEP or Section 504 plan, unless ODE can show that the accommodation invalidates the score interpretation. 2 Rather than consider all accommodations first invalid until proven valid, ODE shall consider all accommodations valid unless ODE can show that the accommodation would invalidate the score interpretation.” Oregon ASK Settlement Agreement

36. California Case U.S. District Judge Charles R. Breyer said the state may administer its first mandatory statewide high school exit exam this week as planned. But he ruled that special education students can be afforded any testing accommodation that they have been determined to need, including the use of calculators, spell- checkers, or extra time, during the testing set for March 5-7…. The judge has yet to rule on how the state should grade the tests of students with disabilities, as well as other contentious issues. Education Week, March 6, 2002

37. Issues: Tendency to allow too many accommodations, possibly reducing expectations for student learning. Poor decision making about accommodations, reflecting lack of knowledge about instructional accommodations. Use of accommodations as an excuse to exclude students scores on reports or accountability.

38. Promising Practices: States keeping track of how many students use accommodations and which ones. Out-of-the-box thinking about the use of accommodated test scores. Clear decision-making criteria (e.g., alignment to instructional accommodations) and training on how to make decisions. Teaching students about the tests they take and about the accommodations that they need.

39. Develop a process for making decisions about accommodation use Choose accommodations based on individual student needs and preferences Teach students to use selected accommodations routinely in the classroom, at home, and in the community – evaluate effectiveness Recommendations for IEP Teams

40. Know state/district accommodations policies Give students opportunities to use selected accommodations on practice tests Make sure test administrators know about accommodations a student will use Record accommodations use accurately on test booklet (or other form) Recommendations for IEP Teams

41. Individualized accommodation decisions should be linked to what is being assessed, access tools a student uses for learning, and the student’s characteristics FOR EXAMPLE: Purposeful reading – reading to select and apply relevant information for a given task Does this allow different ways of interacting with print? And, what are the implications for accommodations?

42. The “door” we need to go through: What is meant by ‘reading’ What about ‘literacy?’ The answer(s) to these questions will determine the accommodations that can be used when certain content standards are taught and assessed!

43. Ways to interact with print: with examples of accommodations Visual Tactile (feeling print) Auditory (listening to printed messages) Multi-modal (using any combination of the above modalities) Printed text; ASL Article in Braille and Nemeth Codes Listen to taped articles (Radio or TV?) Computer-based “ assistive ” reading/ viewing programs – (e.g., digital talking news)

44. Universally Designed Instruction and Assessments are designed to be accessible and valid for the widest range of students Moving to universally designed instruction and assessment

45. Think about universal design in architecture and tool design  Curb cuts and ramps  Elevators that talk to you  Door handles rather than knobs  Special pen shapes that are easier to hold

46. Elements of Universally Designed Assessments  Inclusive assessment population  Precisely defined constructs  Accessible, non-biased items  Amenable to accommodations

47. Elements of UD Assessments (continued)  Simple, clear, and intuitive instructions and procedures  Maximum readability and comprehensibility  Maximum legibility

48. “Think aloud” Recently interviewed 90 students using think aloud protocol 4 th and 8 th grade Used multiple choice and constructed response items from state math test

49. Student Characteristics Grade 4Grade 8 Learning Disability10 Deaf/Hard of Hearing10 Mild Cognitive Impairment55 English Language Learner10 No disability10

50. Overall Observations Students who were confident of content did not have problems with design Students who had no idea how to solve the problem did not have problems with design Students “in the middle” – not sure of content, some reading difficulty, design made a difference

51. Examples of Student Perceptions Many students didn’t see one of the cities on a map The name of one of the cities was “Independence” - uncommon meaning Box between top and bottom of item – some students did not read entire item Sign for parallel gave away the answer Some students read fraction 3 5/8 as “35 divided by 8” Students unfamiliar with settings – “Glee club does number,” “fitness club”

52. Considerations for Item Review Overall appearance is clean and organized Clear format for text Clear format for pictures and graphics (when essential to item) Concise and readable text Format allows for changes without changing meaning or difficulty Meets criteria for measuring what it is intended to measure

53. Alternate Assessments First introduced in IDEA 97 - for students unable to participate in general state assessments

54. Alternate Assessments Required for school accountability decisions under NCLB Reflect shifting goals for students with significant cognitive disabilities

55. Proposed Regulations (Notice of Proposed Rule Making – NPRM) NPRM: Alternate assessment is for students with the most significant cognitive disabilities who are 3 standard deviations below the mean on intellectual and adaptive behavior measures NPRM: Up to 1% of students above can be held to alternate achievement standards

56. Assessment Decision-Making Process in Standards-based Reform Context

57. Development of Alternate Assessments 1.Stakeholder and policymaker identification of desired student outcomes for the population, reflecting the best understanding of research and practice, thoughtfully aligned to same content expected for all students. 2.Development, testing, and refinement of assessment methods. 3.Scoring of evidence according to professionally accepted standards, against criteria that reflect understanding of desired student outcomes. 4.Standard-setting process to allow use of results in reporting and accountability systems. 5.Continuous improvement of the assessment process.

58. Lara’s IEP Goals Before Alternate Assessment Lara will be fed Lara will be cleaned up Lara will be moved

59. Critical Functions focus on function of standard in enhancing a student’s life Standard: “Student communicates ideas through speaking to various audiences” Critical Function: “Communicate ideas” Alternate Form: “Use augmentative and alternative communication system” »Kleinert & Kearns, 2001

60. Alternate assessments have produced dramatic changes in the education of students with disabilities They raise expectation issues all over again!

61. More information? Visit: http://education.umn.edu/nceo or Search for NCEO Web site includes:  Topic introduction  Frequently Asked Questions  Online and Other Resources