1. 26th Annual Management Information Systems [MIS] Conference February 14, 2013 Washington, DC Common Education Data Standards (CEDS) Supporting Assessment Systems Development Copyright © IMS Global and SIF Association

2. Overview Need What is AIF Status Demonstration Prototypes Discussion and Questions Copyright © IMS Global and SIF Association

3. Created by Wordle.net Copyright © IMS Global and SIF Association

4. Need Standards – Confusing Assessment – RTTA and Consortia – Formative – Accountability Inform Instruction Stock photo.

5. Assessment Lifecycle Content Development Content Development Pre-Test Administration Pre-Test Administration Test Administration Test Administration Scoring Reporting Post-Test Administration Post-Test Administration Content and data reviews Test form construction Field testing Item banking & statistics Content exchange / interoperability Planning & blueprinting Item types Content development & universal design Learning standard alignment Administration planning & scheduling Registration, assignment, Form sampling Online infrastructure readiness assessment Pre-session planning (paper / online) & setup Alternate form assignment Test form delivery Platform (paper, online, mobile) presentation Item content & tools Adaptive testing Response collection Proctoring controls Form content security Desktop security Accessibility Testing anomalies Computer scoring Professional scoring Algorithmic (AI) scoring Portfolio scoring Sub test / strand scoring Attemptedness Performance levels Scaling / norming Growth scores Range finding Assessment Life Cycle Assessment Life Cycle Individual reporting Diagnostic reporting Informing & personalizing instruction Performance on standards Dashboard / summary reporting Aggregation / disaggregation Exchanging results / data Psychometric analysis Equating Score tables - scaling, norming Performance levels / cut scores Field test analysis Aligning results with curriculum / instruction Program and teacher effectiveness

6. What is AIF? Systemic view of an assessment system and applications Demonstrates points of interoperability between applications within system Outlines the data model and transport Copyright © IMS Global and SIF Association

7. REGIONAL Systems Assessment Platform Main Components STATE Systems Assessment Creation & Management System (ACMS) Assessment Creation & Management System (ACMS) LOCAL Systems SIF Assessment Delivery System (ADS) Assessment Delivery System (ADS) Assessment Reporting System (ARS) Assessment Reporting System (ARS) APIP SIF APIP SIF APIP Assessment Score Processing System (ASPS) Assessment Score Processing System (ASPS) ? ? APIP

9. Status Use cases outlined Elements complete Public review complete Prototype demonstrations complete Best practice document complete Available on ceds.ed.gov/aif.aspx

10. Demonstration Pilots Logical and Physical Testing Arrow 1 Arrow 10 Arrow 14

11. Arrow 1 Item Bank to Item Bank

12. Arrow 1 Diagram

13. Arrow 1 Test Scenario Scenario Description The purpose of this scenario is to demonstrate assessment items moving from an item authoring system to an item bank or from one item bank to another. This scenario covers only assessment items moving from an item authoring system to an item bank. A consortium has secured item authoring system A. This Item authoring system will not house the items; it is simply a means for item creation. The items will reside in item banking system B. Item authoring system A has created the assessment items and the editing and revision cycle has been complete. Item authoring system A is ready to send the items to item banking system B for storage and assessment instrument creation. Copyright © IMS Global and SIF Association

14. Test Scenario Components Required Item authoring system Item banking system Pre-Condition Assessment content (items, instruments, etc.) is ready to be transferred in the sending item bank. Post-Conditions Content is now ready for use including reviews, edit, extensions, assessment instrument creation, passing of instruments to delivery system, etc. Copyright © IMS Global and SIF Association Arrow 1 Test Scenario Cont’d

15. Copyright © IMS Global and SIF Association Arrow 1 Process Flow

16. Arrow 10 Assessment Registration Copyright © IMS Global and SIF Association

17. Arrow 10 Diagram Copyright © IMS Global and SIF Association

18. Arrow 10 Test Scenario Scenario Description This scenario describes the necessary components for an assessment registration. This scenario covers student demographic, teacher, hierarchy and PNP information. A consortium is ready to give an assessment. The student, school, LEA, PNP and teacher information have already been entered into the SIS or data warehouse. Assessment registration and administration system A pulls over all of the necessary information from the SIS or data warehouse B. The students are then assigned to a specific administration of an assessment. The registration and administration information is then passed from the assessment registration and administration system A to the assessment delivery system C. An alternate scenario would be that the PNP information is entered into the registration system after the information has been pulled over from the SIS or data warehouse. Copyright © IMS Global and SIF Association

19. Arrow 10 Test Scenario Cont’d Test Scenario Components Required Student information system or data warehouse that houses all of the necessary information for registration Assessment registration and administration system Assessment delivery system Pre-Condition All of the student, school, LEA and teacher information have been entered into the SIS and/or Data Warehouse. An administration of an assessment(s) has been identified. Post-Conditions The students are registered for a specific administration(s) of an assessment. Copyright © IMS Global and SIF Association

20. Arrow 10 Process Flow Copyright © IMS Global and SIF Association

21. Arrow 14 Reporting System to Data Warehouse Copyright © IMS Global and SIF Association

22. Arrow 14 Diagram Copyright © IMS Global and SIF Association

23. Arrow 14 Test Scenario Scenario Description The purpose of this scenario is to demonstrate sending assessment results to the SEA from the Consortium administered assessment. The consortium has administered an assessment. The assessment results system A has received and compiled all information based upon the assessment. Assessment results system A compile, packages and disseminates the information to the SEA data warehouse B. Copyright © IMS Global and SIF Association

24. Arrow 14 Test Scenario Cont’d Test Scenario Components Required Assessment results system State data warehouse or reporting system Pre-Condition Assessment results are available and have been collected so that summaries can be produced. Post-Conditions The state system has the results loaded. Copyright © IMS Global and SIF Association

25. Arrow 14 Process Flow Copyright © IMS Global and SIF Association

26. Data Available by Entity Assessment Family Assessment Assessment Form Assessment Form Section Assessment Form Subtest Assessment Subtest Result Assessment Performance Level Assessment Item Assessment Item Possible Response Assessment Item Response Assessment Item Rubric Assessment Participant Session Learner Action Assessment Administration Assessment Registration Assessment Session Organization Registration Accommodation Person Achievement Evidence Achievement Learning Goal Learning Assignment Learning Resource Learning Resource Learner Activity Learning Standard Document Learning Standard Item Competency Item_Competency Set Competency Set Learning Standard Item Grade Level Assessment Levels for Which Designed Assessment Form Subtest Levels for Which Designed Copyright © IMS Global and SIF Association

27. Documentation Assessment Interoperability Framework Definitions and Requirements Use Cases for the Assessment Interoperability Framework Data Elements Best Practices Demonstration Prototype Copyright © IMS Global and SIF Association

28. Work Moving Forward Continue to evolve the technical standard within IMS and SIF respectively and reflect back to CEDS Implementation Additional pilots API and transport Certification Copyright © IMS Global and SIF Association

29. Contact Jill Abbott, CEO Abbott Advisor Group jabbott@abbottadvisorgroup.com Rob Abel, CEO IMS Global Consortium rabel@imsglobal.org Larry Fruth, CEO SIF Association lfruth@sifassociation.org All current documentation can be found at http://ceds.ed.gov/aif.aspx - Assessment Interoperability Framework (AIF)http://ceds.ed.gov/aif.aspx