1. Setting Standards on Alternate Assessments and Assessments of English Language Proficiency

2. Assessment Options Participation in a general grade-level assessment.
Participation in a general grade-level assessment with accommodations.
Participation in an alternate assessment based on grade-level academic achievement standards.
Participation in an alternate assessment based on modified academic achievement standards.
Participation in an alternate assessment based on alternate academic achievement standards.

3. Why include Students with Disabilities
Established law
All students benefit instructionally from participation in assessments as part of an accountability system
To secure appropriate resources for continued improvements of achievement of all students – some information is better than none

4. Legislation Individuals with Disabilities Education Act
IDEA, 1997
No Child Left Behind Act
NCLB, 2001
Individuals with Disabilities Education Improvement Act
IDEIA, 2004

5. Inclusion
The goal of inclusion in the regular educational program and regular assessment activities is an important one
Some students, with moderate to severe cognitive, physical, or behavioral/emotional impairments are not able to participate in regular assessments

6. Alternate Assessments
An alternate assessment is an assessment designed for the small number of students with disabilities who are unable to participate in the regular grade-level State assessment, even with appropriate accommodations.

7. What is an Alternate Assessment
There is no typical or single format for an alternate assessment. Some alternate assessments are built on portfolios of student work or activities that demonstrate knowledge through performance of specific tasks. An alternate assessment may include materials collected under a variety of circumstances:
teacher observation of the student;
samples of student work;
standardized performance tasks produced in an “on-demand” setting;
standardized tests with MC and CR items.

8. Alternate Achievement Standards
US Department of Education: Alternate Achievement Standards for Students with the Most Significant Cognitive Disabilities , Non-Regulatory Guidance; August 2005
altguidance_213511_7.doc

9. What about IEPs?
There are at least two reasons why IEP goals or functional life skills are not appropriate achievement measures for AYP purposes.
IEPs are individualized;
As required by Title I, schools are accountable for student achievement only in the content areas of reading/language arts and mathematics

10. An out-of-level assessment may be used only as an alternate assessment based on alternate achievement standards to assess students with the most significant cognitive disabilities, and then only if it meets three key Title I requirements.

11. An alternate achievement standard sets an expectation of performance that differs in complexity from a grade-level achievement standard.

12. In general, alternate achievement standards must be aligned with a State’s academic content standards, promote access to the general curriculum, and reflect professional judgment of the highest achievement standards possible.

13. An alternate assessment based on alternate achievement standards may cover a narrower range of content (e.g., cover fewer objectives under each content standard) and reflect a different set of expectations in the areas of reading/language arts, mathematics, and science than do regular assessments or alternate assessments based on grade-level achievement standards.

14. Who is Eligible?
The Department intended the term “students with the most significant cognitive disabilities” to include that small number of students who are (a) within one or more of the existing categories of disability under the IDEA (e.g., autism, multiple disabilities, traumatic brain injury, etc.); and (b) whose cognitive impairments may prevent them from attaining grade-level achievement standards, even with the very best instruction.

15. 1% Cap
Under the December 9, 2003 regulation, when measuring AYP, States and LEAs have the flexibility to count the proficient scores of students with the most significant cognitive disabilities who take alternate assessments based on alternate achievement standards—so long as the number of those proficient scores does not exceed 1.0 percent of all students in the grades assessed (about 9 percent of students with disabilities) at the LEA and State levels.

16. 1. 0 percent cap does not apply at the school level
1.0 percent cap does not apply at the school level. Some LEAs may deliver special services for students with the most significant cognitive disabilities in one or a few schools.

17. The 1.0 percent cap does not restrict the number of students who may participate in an alternate assessment. It does limit the number of proficient and advanced scores based on alternate achievement standards that may be used in the calculation of AYP.

18. Modified Achievement Standards
US Department of Education: Modified Academic Achievement Standards, Non-Regulatory Guidance; April 2007
Guidance_192442_7.doc
Standard Setting

19. Modified Academic Achievement Standards
An expectation of performance that is challenging for eligible students, but is less difficult than a grade-level academic achievement standard. Modified academic achievement standards must be aligned with a State’s academic content standards for the grade in which a student is enrolled.

20. An assessment based on modified academic achievement standards should cover the same grade-level content as the general assessment. The expectations of content mastery are modified, not the grade-level content standards themselves.

21. Title I regulations permit a State to develop alternate academic achievement standards for students with the most significant cognitive disabilities and to include those students’ proficient and advanced scores on alternate assessments based on alternate academic achievement standards in measuring AYP, subject to a cap of 1.0 percent of all students assessed at the State and district levels.

22. Since those regulations were published, the experiences of many States, as well as recent research, indicate that, in addition to students with the most significant cognitive disabilities, there is a small group of students whose disability has precluded them from achieving grade-level proficiency and whose progress is such that they will not reach grade-level proficiency in the same time frame as other students.

23. These students must take either a grade-level assessment or an alternate assessment based on alternate academic achievement standards. Neither of these options provides an accurate assessment of what these students know and can do.

24. The regulations on modified academic achievement standards permit a State to adopt such standards and to develop an assessment aligned with those standards that is appropriately challenging for this group of students as part of its State assessment and accountability system under Title I of the ESEA.

25. 2% Cap
Under the final regulations on modified academic achievement standards, when measuring AYP, States and LEAs have the flexibility to count--in determining AYP--the proficient and advanced scores of students who take alternate assessments based on modified academic achievement standards – so long as the number of those proficient and advanced scores does not exceed 2.0 percent of all students in the grades assessed (about 20 percent of students with disabilities) at the LEA and State levels.

26. Under no circumstances may the percent of proficient and advanced scores on alternate assessments based on alternate and modified academic achievement standards exceed 3.0 percent at the State level. The vast majority of students with disabilities can and should be assessed based on grade-level achievement standards

27. Setting Standards
The regulations require use of a “documented and validated standards-setting process” to establish modified academic achievement standards. Evidence of the procedures employed by a State must be submitted to the US Department of Education for peer review and approval.

28. Setting Alternate Achievement Standards Marianne Perie (2007) National Center for the Improvement of Educational Assessment

29. PLD Considerations
PLDs can be written to cover a grade span. If content is similar from one grade to the next, states may write one descriptor to cover the performance of up to 3 grades (Grades 3-5).
States must be able to demonstrate that students can still progress across grade levels within a grade span.

30. PLD Considerations
Another option is to adopt more than one set of PLDs.
One set of PLDs could apply to pre-symbolic students and another set could apply to emerging symbolic or symbolic students.
This is most appropriate when the performance tasks are such that the symbolic PLDs make sense.

31. Validating Standards
Recall that we do not validate the cut-score itself, but the inferences and uses of the achievement standards.
Was the standard setting procedure internally valid?
Do the cut scores divide students reasonably in terms of achievement?
Do the effects of the achievement standards match what was intended?

32. English Language Proficiency Assessment in the Nation Jamal Abedi (Ed
English Language Proficiency Assessment in the Nation Jamal Abedi (Ed.) University of California, Davis education.ucdavis.edu/sites/education.ucdavis.edu/files/ELP_Report.pdf

33. ELL Defined
The definition of an ELL [or limited English proficient (LEP)] student, as outlined in The No Child Left Behind (NCLB) Act of 2001, (NCLB, 2002) is: (a) age 3 through 21; (b) enrolled or preparing to enroll in an elementary or secondary school; (c) not born in the United States or whose native language is not English;
Abedi, Chapter 1

34. (d) a Native America, Alaskan Native, or a native resident of the outlying areas; (e) from an environment where a language other than English has had a significant impact on an individual’s level of English language proficiency; (f) migratory and comes from an environment where English is not the dominant language;
Abedi, Chapter 1

35. and (g) has difficulties in speaking, reading, writing, or understanding the English language that may deny the individual the ability to meet the state’s proficient level of achievement and the ability to successfully achieve in classrooms where English is the language of instruction, or to participate fully in society (NCLB, 2002, Title IX).
Abedi, Chapter 1

36. The Many Names of ELL
English language proficiency (ELP) standards, (specifically addresses competency in reading, writing, listening, & speaking English)
English language development (ELD) standards (instruction for ELLs based on developing reading, writing, listening, & speaking skills in English)
English as a second language (ESL) standards (based on instructional models of bilingual ed)
Abedi, Chapter 1

37. NCLB Requirements
develop and implement ELP standards suitable for ELL students’ learning of English as a second language;
implement a single, reliable and valid ELP assessment aligned to ELP standards that annually measures listening, speaking, reading, writing, and comprehension;
Abedi, Chapter 1

38. NCLB Requirements (cont.)
align these tests with the states’ English language development content standards and provide content coverage across three academic topic areas, including: English Language Arts; Math, Science, and Technology; and Social Studies as well as one non-academic topic areas related to school environment, such as extracurricular activities, student health, homework, and classroom management
Abedi, Chapter 1

39. NCLB Requirements (cont.)
establish Annual Measurable Achievement Objectives (AMAOs) for ELL students that explicitly define, measure, and report on the students’ expected progress toward and attainment of ELP goals
Abedi, Chapter 1

40. English Language Proficiency
The Mountain West Assessment Consortium includes
Alaska, Colorado, Idaho, Michigan1, Montana, Nevada, New Mexico, North Dakota, Oregon, Utah, and Wyoming
ELP Domains: reading, writing, listening, speaking
ELP levels: Pre-Emergent, Emergent, Intermediate, Fluent, Advanced
Grades: K-1 (checklist), 1-2, 3-5, 6-8, 9-12
Mathews, Chapter 3

41. ELP Standard Setting
Due to the very challenging nature of proposing cut scores on an ordered item booklet containing all four language skill domains, states should consider:
setting proficiency levels separately by language skill domain
how to combine reading, writing, listening, and speaking results for each student into an overall composite score
Mathews, Chapter 3

42. ELP Standard Setting
Create proficiency-level descriptors that serve multiple grade spans
Set state-specific standards and use empirical data to inform the process
Select participants who have both English language acquisition training and grade-level experience
Mathews, Chapter 3

43. Abedi (2007) Summary
According to Musick (2000), the percentage of students meeting state standards in grade 8 math ranged between 13% in one state and 84% in another state.
When the same two states were compared on their NAEP performance scores, the state with the lowest percentage meeting state standards actually scored higher on NAEP than the state with the highest percentage.
Abedi, Chapter 9

44. Abedi (2007) Summary
Loomis (2001) evaluated the outcome of different standard setting procedures by comparing teachers’ judgments of student performance to the empirical classification of student performance (such as with the contrasting group approach) and to the performance represented in test booklets (such as the Bookmark and Modified Angoff methods). He concluded that there was no certain way to verify the validity of the cut scores.
Abedi, Chapter 9

45. Abedi (2007) Summary
Jaeger (1989) indicated that “at best, the ratio of the largest recommended standard to the smallest recommended standard was 1.00,” indicating identical standards resulting from different methods. At worst, however, “the recommended standard resulting from one method was 42 times as large as that resulting from another method”.
Jaeger recommended that the results from several methods must be used in setting standards.
Abedi, Chapter 9

46. Abedi (2007) Summary
There are inconsistencies between achievement levels set for the different domains of reading, writing, speaking, and listening. When achievement levels are set separately for each of the domains, then discrepancies between such levels across the domains could make interpretation of the results difficult.
Abedi, Chapter 9

47. Abedi (2007) Summary
For example, many students can be classified as proficient or above in one domain but may be classified as below proficient in other domains. How can such issues be resolved? (see for example, Bunch, 2006).
Should achievement levels be set at the whole test level?
If so, then how should the total test score be obtained?
Abedi, Chapter 9

48. Abedi (2007) Summary
the preferred model for NCLB is the conjunctive model in which students should score at the proficient level in each of the four domains to pass AMAO requirements.
Abedi, Chapter 9