1. World-Class Instructional Design and Assessment
Alignment of Standards as a Source of Validity for ACCESS for ELLs®
Margo Gottlieb, Ph.D.
Lead Developer

2. Validity is the superstructure that defines testing.

3. Evidence of validity must be systemic throughout the testing cycle…
Evidence of validity must be systemic throughout the testing cycle….and alignment between the components of the system is one source of validity.

4. Evidence of Alignment in Test Development
1. Second language acquisition theory and research
P D
R E
O V
F E
E L
S O
S P
I M
A E
L N T V A L I D T O N
2. WIDA’s English Language
Proficiency Standards
3. Test Specifications
4. ACCESS for ELLs®

5. Theoretical Background: A Model of English Language Proficiency
Listening Speaking
Contexts of Interaction
Support
Language Complexity
Level of Cognitive Engagement
Reading Writing
(Gottlieb, 2003)

6. The English language proficiency standards are aligned to state academic content standards.
Social and Academic
Language
Concepts
Knowledge
and Skills

7. The English Language Proficiency standards are defined by a hierarchical structure.
Formative and Summative Frameworks f
English Language Proficiency Standards
Language Domains
Grade Level Clusters
Levels of Language Proficiency
Model Performance Indicators

8. WIDA’s English Language Proficiency Standards
Standard 1: English language learners communicate for social and instructional purposes within the school setting.

9. Standard 2: English language learners communicate
information, ideas, and concepts necessary for academic success in the area of language arts.
Standard 3: English language learners communicate
information, ideas, and concepts necessary for academic success in the area of mathematics.
Standard 4: English language learners communicate
information, ideas, and concepts necessary for academic success in the area of science.
Standard 5: English language learners communicate
information, ideas, and concepts necessary for academic success in the area of social studies.

10. WIDA’s English Language Proficiency Standards: The Interaction between Language and Content
The language of
LISTENING
SPEAKING
READING
WRITING
LANGUAGE
ARTS
MATH
SCIENCE
SOCIAL STUDIES

11. English Language Proficiency Standard:
The Matrix
English Language Proficiency Standard:
Grade Level Cluster:
Level 1
Level 2
Level 3
Level 4
Level 5
Listening
Speaking
Reading
Writing
Levels of English Language Proficiency
A Model Performance Indicator
A strand of Model Performance Indicators

12. The Second Language Acquisition Continuum
Level 1
Entering
Level 2
Beginning
Level 3
Developing
Level 4
Expanding
Level 5
Bridging
Level 6
Reaching
Vertical alignment occurs between contiguous levels of English language proficiency to form a strand of Model Performance Indicators.
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

13. Performance Definitions
Movement along the continuum is based on an English language learner’s increasing
Vocabulary Usage- The specificity of words and phrases
Linguistic Complexity- The amount and quality of speech or writing
Language Control- Comprehensibility based on the amount and types of errors.

14. A Strand of Model Performance Indicators
The language of science
Language domain: Speaking or writing
Levels of English Language Proficiency
Level 1
Level 2
Level 3
Level 4
Level 5
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

15. Expectations for English Language Learners
Level 1
Level 2
Level 3
Level 4
Level 5
Relate information about weather from photographs, charts, or graphs (such as, ‘It’s hot.’)
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

16. Level 1
Level 2
Level 3
Level 4
Level 5
Describe weather conditions based on information from photographs, charts, or graphs (such as, ‘I see dark clouds in the sky.’)
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

17. Level 1
Level 2
Level 3
Level 4
Level 5
Compare and contrast weather information from charts, tables, or visually supported text (such as, ‘It is colder today than yesterday.’)
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

18. Level 1
Level 2
Level 3
Level 4
Level 5
Match predictions about weather from graphs, charts, or visually supported text (such as, ‘I think that tomorrow it will’ ….)
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

19. Level 1
Level 2
Level 3
Level 4
Level 5
Make inferences about weather based on modified grade level material (such as, ‘It seems as if it might…’)
The criteria used to determine the proficiency level definitions are couched in terms of the language used in schools to impart content area information. Issues of linguistic complexity and semantic & pragmatic knowledge are brought to bear in formulating the definitions. At the two lower proficiency levels, it is assumed that ELLs would need extralinguistic support via graphic and visual aids in order to carry out language functions. This requirement also motivates the use of graphics for test items at these levels.

20. Assessing Comprehension and Communication in English State to State for English Language Learners
ACCESS for ELLs®

21. ACCESS for ELLs® is built on:
A theoretical model of English language proficiency
Social and academic language proficiency across standards
Five levels of language proficiency
Four language domains
Developmentally appropriateness for each grade level cluster
Second language acquisition research and best practice

22. The purposes of ACCESS for ELLs® … to provide one data source for:
Identifying initial placement of English language learners (ELLs) on the L2 acquisition continuum
(W-APT)
Monitoring annual progress of ELLs toward meeting English language proficiency standards
Marking attainment of ELLs of full English language proficiency
Complying with the requirements of No Child Left Behind

23. Samples of alignment from a string of Model Performance Indicators, representative of a standard, to ACCESS for ELLs® items.

24. Sample Items from a Theme (from the Pilot test)
English Language Proficiency Standard:
The language of science
Language Domain: Listening
Grade Level Cluster: 1-2
Remember that all sample items presented here are themes and items that appeared on the pilot test. These items do not appear on the operational test. The sample items give a good sense of what themes and items are like and how they are aligned to the standards and performance indicators, but for a variety of reasons, they did not survive to become operational. Therefore, criticisms of the content, theme, possible bias, or other observations about these specific items are not particularly useful. However, overall comments and suggestions are always welcome.

25. Model Performance Indicator (PI) for Language Proficiency Level 1
PI: “identify objects according to chemical or physical properties from pictures and oral statements”
SCRIPT: “A seed is small. Find the small seed.”
These slides illustrate the direct relationship between the performance indicators (PIs) from the large-scale assessment framework of the standards and the items. This slide and the following two show how items of contiguous proficiency levels are located within a single theme.

26. Model Performance Indicator for Language Proficiency Level 2
PI: “match objects with their chemical or physical properties from pictures and oral statements”
SCRIPT: “One day the seed will grow into something large, round, and heavy. Find what the seed grows into.”

27. Model Performance Indicator for Language Proficiency Level 3
PI: “identify and group objects according to chemical or physical properties from oral statements”
SCRIPT: “Seeds grow into plants. Find something else that grows.”

28. Validation of the construct of English language proficiency, from theory to standards to specifications to test items…

29. ….is a set of interlocking, interdependent gears within an aligned system.
Academic content
standards
English language proficiency standards
ACCESS for ELLs®