1. Amelia Courts, aadavis@access.k12.wv.us WESTELL Putting it All Together WV Department of Education

2. Amelia Courts, aadavis@access.k12.wv.us Agenda Overview Standards-Based Assessment Design Language Domains Validity Issues Challenges

3. Amelia Courts, aadavis@access.k12.wv.us Overview Collaborative Initiative funded by $1.3 million dollar USDE grant Chief Council of School State Officers (CCSSO), LEP SCASS (State Collaborative on Standards and Assessment), American Institutes for Research (AIR) Census Operational / Field test 2004 Goal of balance: Language proficiency development Psychometric requirements Practical feasibility needs Cost

4. Amelia Courts, aadavis@access.k12.wv.us Standards-based Title III requirements Performance Standards Steering Committee collected and analyzed all ELP standards from member states Rigorous definition of FEP Realistic expectations for beginning LEPs Informative levels between the two Content Standards

5. Amelia Courts, aadavis@access.k12.wv.us Assessment Design Align with Content and Performance Standards Four Content Areas English Language Arts Math / Science / Technology Social Studies Social / School Interactions Four Grade Clusters K-2 * not yet developed 3-5 6-8 9-12

6. Amelia Courts, aadavis@access.k12.wv.us Assessment Design Report Performance Reading Listening Writing Speaking Comprehension RLWS and C scores

7. Amelia Courts, aadavis@access.k12.wv.us Language Domains Listening Focus on comprehension of academic English Authentic, self-contained events 6 Standards (e.g., follow spoken directions, making inferences) 5 Text Types (phrases, extended presentations)

8. Amelia Courts, aadavis@access.k12.wv.us Language Domains Listening 3-56-89-12 Number of Items in final form 50 60 Item Types Multiple choice*Multiple choice Approx. Administration times 60 min. 75 min. Administration type Group

9. Amelia Courts, aadavis@access.k12.wv.us Language Domains Reading 8 Broad Categories (e.g., early reading skills, making inferences) 3 Text Types (150-300 word passages with 5-8 comprehension questions)

10. Amelia Courts, aadavis@access.k12.wv.us Language Domains Reading Specifications 3-56-89-12 Number of Items in final form 50 60 Item Types Multiple choice Approx. Administration times 60 min. 75 min. Administration type Group

11. Amelia Courts, aadavis@access.k12.wv.us Language Domains Writing Constructed Response Comprehensibility and discourse features Short (3-5 per form) Extended (1-2 per form) Multiple Choice Editing and Revising Planning and Organizing

12. Amelia Courts, aadavis@access.k12.wv.us Language Domains Writing Specifications 3-56-89-12 Number of Items in final form 1 Extended CR 3 SCR 15 Multiple Choice 1 Extended CR 3 SCR 15 Multiple Choice 1 Extended CR 4 SCR 15 Multiple Choice Item Types ECR SCR Multiple choice ECR SCR Multiple choice ECR SCR Multiple choice Approx. Administration times 60 min. 65 min. Administration type Group

13. Amelia Courts, aadavis@access.k12.wv.us Language Domains Speaking Task Structure Input Prompt Scaffold Prompt Repeated

14. Amelia Courts, aadavis@access.k12.wv.us Language Domains Speaking Specifications 3-56-89-12 Number of Items in final form 16 Item Types Prompted oral response** Prompted oral response* Approx. Administration times 15 (Plus field test items and instructions) 15 (Plus field test items and instructions) 20 (Plus field test items and instructions) Administration type Group

15. Amelia Courts, aadavis@access.k12.wv.us Psychometrics Score reporting scales Pilot test Spring 2003 12 states, 31 schools, 100 students per cluster Encouraging results Score reliabilities:.77-.92, mostly mid-80s Improve item clarity

16. Amelia Courts, aadavis@access.k12.wv.us Validity Research Agenda Alignment of items with standards Item-level analysis (scoring, difficulty) Test-level analysis (reliability, scaling/equating) Internal structure/construct validity Correlations with external measures

17. Amelia Courts, aadavis@access.k12.wv.us Challenges Spring 2004 Field test Screen and select items Produce 1 operational form Establish item bank Produce student scores for Title III reporting Speaking Component Design to meet intent of law / administrative feasibility Cost

18. You are the key…to putting it all together! WESTELL