2. IWMW Glasgow June 2002 Mark Simpson De Montfort University - Leicester Managed Learning Environment - MLE September 2000 to September 2002 Joint Information Systems Committee -

3. IWMW Glasgow June 2002 User Issues - Formative and Summative Assessment Designing for Usability How did we specify the system? How did we decide on content? How did we verify content? Develop conventions for pages? Usability trade-offs and compromises?

4. IWMW Glasgow June 2002 What is a MLE? University Electronic resources linked together Student record system - QLS Timetabling and Calendar system Library databases Module and Results database Online learning resources - WebCT Social databases Email and discussion pages Other student oriented data resources

5. IWMW Glasgow June 2002 MLE Architecture MLE (Web) Server User Manager Resource Manager MLE Usage Database Student Record System Authentication System Client Application (Web Browser) BIDS EDINA e-Journals etc Remote Authorised Resources Timetabling Finance System Course Management Secure, Authorised Access e-Learning – VLE App Course Notes, Study Guides, Exam Papers Library Diary Discussion Lists, Groupware, email Entertainment, Social outside firewall inside Resources Crossing Firewall

6. IWMW Glasgow June 2002 Usability how hard is it? Everybody has an opinion “I hate …… …… …” “Frames / Flash / PDF are rubbish” Not useful comments Anything new will not satisfy X % of users, until they have used it.

7. IWMW Glasgow June 2002 What is this in your opinion ? RED

8. IWMW Glasgow June 2002 BLUE What colour is this in your opinion?

9. IWMW Glasgow June 2002 Development cycle Web Site Requirements / Specification Develop Content List Preparing Content / Layout Designing initial prototype October 2000 March 2001 September 2001 December 2001 1st user testing Modifying prototype 2nd user testing 4 th Navigation & Logic Testing Correcting Navigation Issues – uportal February 2002 3rd user testing Modifying design April 2002 June 2002 August 2002 Writing Training Material User Log Tracking

10. IWMW Glasgow June 2002 Specification What does the system do? What are the users like What do they want the system to do What can we do from the users want list What are the usability issues with current systems Can the MLE solve these issues Project team produced a list of what they wanted as indicative student users

11. IWMW Glasgow June 2002 User Requirements Ask yourself / project team / steering committee Ask users Research other systems or similar sites What is unique to your project - QLS Produce a list of content Refine list to remove duplication / impossible content Record list with reasons

12. IWMW Glasgow June 2002 Sources of Requirements 1. DMU PC ownership and use survey 1997, 98, 99, & 00 2. Collaboration in Research Support by Academic Libraries in Leicestershire survey [CORSALL] 3. ANGEL user analysis results 4. DMU Nursing School, online course survey 5. DMU Library survey of student information needs 6. DMU Research papers on using the Internet for teaching. 7. DMU Keyskills survey of all first year students 8. Student Experience Unit feedback from students leaving the university 9. DMU Student focus group meeting 10. Usage reports from similar 11. User interviews with users of similar existing systems

13. IWMW Glasgow June 2002 Results of Specification Consistent trend that PC ownership is increasing and that online use is also increasing Users ask for more content than they need Academic users want the MLE to be integrated in university and not isolated Students are unconcerned about security and privacy of online information Navigation must be mouse rather than keyboard based, back arrow used in preference to all other navigation devices Contents List

14. IWMW Glasgow June 2002

16. Dendrogram

17. IWMW Glasgow June 2002

18. MLE Version 2

19. IWMW Glasgow June 2002 Further login trials, testing navigation Login trials - user name and password Simple content location Logging out Interface issues Don’t look at too much each time

20. IWMW Glasgow June 2002 Testing login & photographs

21. IWMW Glasgow June 2002 Login trails with HNC, BA, MA students 12 first year students HNC Housing Part-time 7 second year students BA Accounting and Finance Full-time 26 postgraduates taking MA degrees Full-time Between each test correct problems and develop a new feature for testing

22. IWMW Glasgow June 2002 How many times do you test it, how many subjects do you need? As many times as you can which will yield useful data Between 4 and 10 subjects at a time What are you testing ? what is the question what will you do with the results What resources are available Don’t leave it hoping it will be alright

23. IWMW Glasgow June 2002 MLE Version 3

24. IWMW Glasgow June 2002

25. User tracking data User Logs for six months Logon Data, route through site, User Names Course being studied, Self assessments completed, discussion involvement Didn’t match interview or focus group reports

26. IWMW Glasgow June 2002 Pathways, which way do people go. Track routes taken by users Particular attention to successful students Use data to provide mentoring support to students having problems Use data to provide a list of shortcuts Do people do what they said they wanted in the specification

27. IWMW Glasgow June 2002 Future Usability: Training and Help pages Focus groups – face to face Online Questionnaires Feedback from modules with teaching resources online

28. IWMW Glasgow June 2002 Benefits of the methods Provide a handle on the university administration information issues Publicity and visibility for project Resolve problems as they were discovered Kept in touch with all types of users

29. IWMW Glasgow June 2002 How would we do it differently Have an off-line non development system for testing usability Use external evaluation only in the second half of the project When carrying out usability testing freeze the design

30. IWMW Glasgow June 2002 What should you do? Set-up a steering group or panel Think about what the site / page / system is providing Do testing using a prototype of content and navigation regularly Always have a usability issue to hand Use conventions and standard layouts – what did the users tell you

31. IWMW Glasgow June 2002 Summary 3 Formative and 2 Summative methods of assessment Ask users what they want Test navigation regularly Implement changes and test again Don’t expect to please everybody

32. IWMW Glasgow June 2002 Where am I now ?