1. Moodle Quiz UI for creating quizzes, from scratch 1.Introduction 2.Moodle 3.Quiz, an activity module 4.Background & Starting points 5.The old UI 6.Quiz creating: Use cases 7.Core problem: Old Quiz UI's forced conceptual model 8.The concrete issues of the old exam creating UI 9.The result of Kesäkoodi 10.Community response Kesäkoodi 2008 - Olli Savolainen ➔ Exams and Electronic Exams

2. 1. Introduction Exams, anyone? Differences: pen&paper vs. electronic exams Core concept: A random question in an exam gets selected randomly from a set of questions: Question 2a Question 2b Question 2c... THE EXAM Question 1c Question 2b Question 3f ➔ Moodle?

3. The leading open source Course Management System Used by schools, universities and commercial training providers worldwide About 5000 installations with 600+ users, over 4 million users total Pedagogical tools: Lesson, Forum, etc. ➔ and Quiz 2.

4. 3. Quiz, an activity module What is it used for? 1. Teachers create quizzes (exams) 2. Students take exams, get live feedback 3. Teachers analyze results 4. Teachers feedback (qualitative/quantitative) Question types: multiple choice, essay,... ➔ What has happened so far?

5. 4.1. Background Winter 2006-2007: Teachers refuse to create exams with Moodle Quiz by themselves Old user interface doesn't support the model, familiar to teachers' from pen&paper exams Discussions (spring & summer 2007): Moodle support staff @ Uni of Tampere Teachers Moodle community Paper prototype testing suggests that the new UI really helps novices ➔ What did we learn during the discussion & research?

6. 4.2. Starting point? Start small. Start small, expand the UI later to cover all features of current UI Moodle Quiz ➔ UI for Creating and managing quizzes ➔ from scratch (ignore pre-existing/imported questions for now) ➔ So what does the old UI look like?

7. ➔ What is the UI for? 5. The old UI for creating and managing quizzes

8. 6. Quiz creating: Some use cases Random questions Use cases we will deal with 1.Add a random question into a quiz (example follows) 2.Add a plain question into a quiz 3.Set the order of the questions 4.Divide the questions in pages 5.Set the grades (max. points) for individual questions and the quiz ➔ What do the teachers need to learn to use the old/new UI?

9. Question bank Categories ➔ Subcategories Questions in categories Quiz's model 7.1. Core problem: Old Quiz UI's forced conceptual model Quiz ➔ Random questions The novice user's model ➔ How does Quiz's conceptual model affect our example use case (”Add a random question into a quiz”?)

10. 1. Create a new question category in question bank 2. Add questions to the category 3. Add the category as a random question into the quiz. 4. Preview the quiz to confirm a question from the category is showing 7.2. Core problem: Old Quiz UI's forced conceptual model (cont.) Quiz ➔ Random questions Question bank Categories ➔ Subcategories Questions in categories The novice user's model Quiz's model Use case: Adding a random question in the old UI:

11. 1. Create a new random question directly into the quiz 2. Add alternatives (questions) into the random question ✔ (No need to preview: the new UI directly shows the questions available for the random question) 7.3. The new UI's conceptual model Quiz ➔ Random questions The novice user's model ➔ How do you actually achieve this in practice? Use case: Adding a random question in the new UI:

12. ”What exactly am I supposed to use this for?” Three major use cases in one screen create user confusion: Quiz content: Questions in this Quiz Paging and ordering Q's Question Bank management 8. The concrete issues of the old exam creating UI ➔ The new UI?

13. 9. The result of Kesäkoodi (graphical and layout details subject to change) 1. 2.

14. 9.2. The result of Kesäkoodi: Quiz creating and managing

15. 9.2. The result of Kesäkoodi: Question ordering and paging

16. 10. Community response Further use cases gathered during discussion Overall attitudes positive: people from several schools and universities worldwide have expressed willingness to do testing. [...] and I think it is a worthwhile experiment. The important thing is to keep some continuity between the way Moodle currently works, and the new interface, [...] Note that Olli's proposal is entirely about user interface, all the behind- the-scenes stuff about how the quiz design is stored in the database does not change at all. Our hope it to find a way for Olli to do this work so it is easy for people to download it and try it out. Then we can decide whether to use it in Moodle 2.0, but I hope we can. Tim Hunt, Quiz module maintainer:

17. Thank you! Any questions?

18. Schedule Already tested with paper prototypes Focus on usability: testing has a high priority Principal goal: Defining the UI such that we positively know that it suits all the use cases

19. Risks & challenges Time line: Usability testing Pressure to take into Moodle core too quickly