© 2001 By Default! A Free sample background from Slide 1 Interoperability: future- proofing CAA Niall Sclater CETIS Manager, Educational Systems University of Strathclyde Heriot-Watt University 6th February, 2002

© 2001 By Default! A Free sample background from Slide 2 Why do I need to know about QTI? I’m authoring a bank of questions in my subject area I’m authoring a bank of questions in my subject area I’m involved in the selection of a CAA system for my institution I’m involved in the selection of a CAA system for my institution I have a general interest in online learning and assessment I have a general interest in online learning and assessment

© 2001 By Default! A Free sample background from Slide 3 The basics IMS QTI is a means for exchanging assessment information IMS QTI is a means for exchanging assessment information It uses XML It uses XML 2 key parts of the spec: 2 key parts of the spec: –Assessment Section Item –Results Report Also QTI lite Also QTI lite Can be implemented separately Can be implemented separately

© 2001 By Default! A Free sample background from Slide 4

© 2001 By Default! A Free sample background from Slide 5 Response types Multiple choice Multiple choice True false True false Multiple response Multiple response Image hot spot Image hot spot Fill in the blank Fill in the blank Select text Select text Slide Slide Drag object Drag object Drag target Drag target Order objects Order objects Match item Match item Connect the points Connect the points

© 2001 By Default! A Free sample background from Slide 6 The concept of rendering The same item can be “rendered” in different ways to allow for different The same item can be “rendered” in different ways to allow for different –delivery platforms –technical capabilities –participant challenges (eg visual, audio, physical etc)

© 2001 By Default! A Free sample background from Slide 7 Rendering example

© 2001 By Default! A Free sample background from Slide 8 Basic response & render types Response types Response types –logical identifier –xy coordinate –string –number –logical group Render types Render types –choice –hot spot –fill in blank –slider –‘object’

© 2001 By Default! A Free sample background from Slide 9 True false example

© 2001 By Default! A Free sample background from Slide 10 True false example XML This is a simple True/False multiple-choice example. The rendering is a standard radio button style. No response processing is incorporated. Paris is the Capital of France ? True False

© 2001 By Default! A Free sample background from Slide 11 Response processing … T 1 Yes, you are right …

© 2001 By Default! A Free sample background from Slide 12 Multiple choice example

© 2001 By Default! A Free sample background from Slide 13 Multiple choice example XML Which one of the listed standards committees is responsible for developing the token ring specification ? IEEE IEEE IEEE IEEE None of the above.

© 2001 By Default! A Free sample background from Slide 14 Response processing … A 1 Yes, you are right. …

© 2001 By Default! A Free sample background from Slide 15 Multiple response example

© 2001 By Default! A Free sample background from Slide 16 Image hot spot example

© 2001 By Default! A Free sample background from Slide 17 Fill in blank (string) example

© 2001 By Default! A Free sample background from Slide 18 Numerical example

© 2001 By Default! A Free sample background from Slide 19 Composite example

© 2001 By Default! A Free sample background from Slide 20 Results reporting Specific to a single participant Specific to a single participant 4 constructs 4 constructs –summary (eg max score, num attempts) –assessment –section –item Each construct contains a context which specifies details such as name of participant Each construct contains a context which specifies details such as name of participant

© 2001 By Default! A Free sample background from Slide 21 Where does QTI fit in with other specs? IMS Meta-data Specification IMS Meta-data Specification IMS Content Packaging Specification IMS Content Packaging Specification IMS Learner Information Packaging Specification IMS Learner Information Packaging Specification IMS Digital Repositories Interoperability Specification IMS Digital Repositories Interoperability Specification

Elect Drafts Scope IMS Spec Development Process IMS Members Technical Board Public Working Group Final Scope Base Document Public Draft Feedback Approval

© 2001 By Default! A Free sample background from Slide 23 What’s next with QTI? V1.2 currently being approved V1.2 currently being approved –selection and ordering of sections and items –scoring roll-up in sections and assessments (from question scores to test scores) –results reporting Question and test metadata Question and test metadata Harmonisation with other specs Harmonisation with other specs Integration with ADL SCORM Integration with ADL SCORM Implementations now needed Implementations now needed V1.3 early in 2002 V1.3 early in 2002

© 2001 By Default! A Free sample background from Slide 24 UK Interoperability Developments Strong representation in QTI working group Strong representation in QTI working group CETIS QTI SIG CETIS QTI SIG –meetings –online discussions –repository of open source tools –hosting workshops CAA Centre & SCAAN CAA Centre & SCAAN JISC MLE CAA Project JISC MLE CAA Project e3an e3an

© 2001 By Default! A Free sample background from Slide 25 Products which say they are implementing QTI E-Test 3, RIVA Technologies Inc E-Test 3, RIVA Technologies Inc Question Mark Perception Question Mark Perception Can studios Can studios Oracle i-learning Oracle i-learning Blackboard Blackboard WebCT WebCT Ultimate assessment Ultimate assessment E3AN E3AN

© 2001 By Default! A Free sample background from Slide 26 IMS and other initiatives IMS Europe IMS Europe SCORM SCORM Open Knowledge Initiative Open Knowledge Initiative Prometeus Prometeus Questionmark -> Blackboard & WebCT Questionmark -> Blackboard & WebCT

© 2001 By Default! A Free sample background from Slide 27

© 2001 By Default! A Free sample background from Slide 28 QTI page

© 2001 By Default! A Free sample background from Slide 29 QTI: some problems For vendors For vendors –implementing QTI is costly –stops them from protecting their user base Vendors want simple specifications; users want broad, well-defined ones Vendors want simple specifications; users want broad, well-defined ones It is difficult to define specifications which do not have an effect on functionality It is difficult to define specifications which do not have an effect on functionality

© 2001 By Default! A Free sample background from Slide 30 Finally: why bother with QTI? Avoid lock in Avoid lock in –allow data to be moved in standard formats –allow one system to replace another Integrate systems from different vendors; allow choice of best assessment tool Integrate systems from different vendors; allow choice of best assessment tool Facilitate exchange of questions and tests between institutions, allow market to develop Facilitate exchange of questions and tests between institutions, allow market to develop Enable separation of content from presentation Enable separation of content from presentation Standard assessment records will facilitate student mobility Standard assessment records will facilitate student mobility

© 2001 By Default! A Free sample background from Slide 31 Further information