1. Modeling Units of Assessment for Sharing Assessment Process Information: towards an Assessment Process Specification Yongwu Miao, Peter Sloep, and Rob Koper Educational Technology Expertise Centre Open University of The Netherlands ICWL 2008, 20-22 August 2008, Jinhua, Zhejiang, China

2. Overview Where are we from: OUNL What we have done: EML => IMS LD What is missing in IMS LD: Assessment What is our objective: A New Specification What is the current status: Conceptual Model How can it be used: Olympic Games Summary

3. Where are we from: OUNL

4. The Open University of the Netherlands (OUNL) –Started in 1984; national institute; –Open distance education provider 7 faculties, 24000 students 24 study centres in Netherlands and Belgium education delivered using a variety of technologies (print, cd-rom, internet, face to face contact sessions, practical rooms, etc.) –R&D: Educational Technology Expertise Center

5. What we have done: EML => IMS LD Educational Modelling Language (EML) IMS Learning Design (LD): An e-learning technical specification IMS LD is derived from EML IMS Global Learning Consortium, Inc. OUNL is contributing member (voting rights) of IMS –Also Apple, Blackboard, Microsoft, Cisco, Sun, Texas Instruments, … 1998 * Information model * Best practice guide * XML binding 2003

6. What we have done: EML => IMS LD Basic idea: Develop a ‘language’ to describe the learning/teaching processes with associated content in a formal, semantic way, so that it can be interpreted by computers. Compare: music notation This description should be independent of any delivery platform => can be deployed in any platform that provides the adequate functionality

7. What we have done: EML => IMS LD Learning object: - Book, - Audio & video Which notations are needed to describe a lesson plan?

8. What we have done: EML => IMS LD IMS LD, an education-specific process modelling language, can be used to describe a wide variety of pedagogical strategies or approaches in terms of a play including acts and role-parts (based on metaphor of theatrical play) People act in different roles working towards certain objectives by performing learning and/or support activities within an environment, consisting of learning objects and services used in the performance of the activities. Moving focus from what to learn to how to learn: Lesson plan or learn-flow

9. play Act 1Act 2Act 3Act 4Act 5 Role-part 1 Role-part 2 Role-part 3 Role-part … Role Activity Environment Learning objects Learning services Activity- Description method components What we have done: EML => IMS LD

10. LD authoring tool LD-compatible run-time environment actual or imaged pedagogical processes a formal pedagogical model multiple executions of the same model carried out by different groups the events that can be observed or expected represented in LD (in the form of XML) Run-time Design-time author learner, teacher, and other stakeholder

11. What is missing in IMS LD: Assessment LD can not explicitly support various types of assessment. Assessment components within the Educational Modeling Language were excluded when LD was adopted by IMS, because of the existence of IMS Question and Test Interoperability (IMS QTI). QTI is also an open e-learning technical standard, which describes a data model for the representation of assessment item/test and their results. It defines a set of interaction types which can be used to specify basic question types (e.g., multiple-choice, fill_in_the_blank, and slider) and complicated question types through combination.

12. What is our objective: A New Specification 1.facilitating representation, understanding, communication, comparison, and reuse of a variety of assessment practices at design-time; 2.guiding and scaffolding learner, tutors, and other stakeholders to perform a variety of activities at run-time by providing guidance and awareness information, by configuring a workspace for carrying out prescriptive tasks, by controlling and changing the sequence of activities based on the execution state and circumstantial information, and by orchestrating the efforts made by different roles/participants. Develop a high-level assessment process modeling language for facilitating both experts and practitioners to share assessment process information by extending QTI and LD with an additional layer.

13. What is the current status: Conceptual Model Semantic aggregation model Conceptual structure model Process structure model Developed a conceptual model, the core of the specification, which can be represented as: The first validation study has been conducted through investigating whether the conceptual model of specification meets the requirements of completeness, flexibility, adaptability, and compatibility.

14. What is the current status Semantic aggregation model:

15. What is the current status Conceptual structure model:

16. What is the current status Process structure model

17. What is the current status Restrictions between stages and activities:

18. How can it be used: in Olympic Games It is not necessary to develop a software applications for supporting each assessment

19. How it can be used: in Olympic Games

20. Use existing Run-time Environment Player.1Player.2Player.n internet Dispatcher/Service broker IMS LD engineIMS QTI engine AA Assessment service UoL management and enactment service … an assessment model multiple instances server

21. How can it be used: in Olympic Games An assessment model can be reused and modified to support other assessment

22. Summary IMS LD can support interoperability and reusability of a variety of pedagogical models and systems. IMS LD provides insufficient expressiveness to model integrated learning, teaching and assessment processes. A high-level assessment process modeling language can be developed by extending IMS QTI and IMS LD with an additional layer. The conceptual model of the specification has been developed and validated.

23. Thanks for your attention Questions?