1. Visual Analysis of a Moodle-based Software Engineering Course Francisco J. García (fgarcia@usal.es)fgarcia@usal.es Miguel Á. Conde (mconde@usal.es)mconde@usal.es Sergio Bravo (ser@usal.es)ser@usal.es Diego A. Gómez (dialgoag@usal.es)dialgoag@usal.es Roberto Therón (theron@usal.es)theron@usal.es University of Salamanca – GRIAL Research Group Methods and Cases in Computing Education (MCCE-2011) Ciudad Real, June 16 th, 2011 Held at SPDECE-2011. Multidisciplinary symposium on the design and evaluation of digital content for education - Ciudad Real, June 15 th -17 th, 2011

2. Outline Introduction The Problem Visual analytics Case study results Conclusions GRIAL – Universidad de Salamanca

3. Introduction (I) GRIAL – Universidad de Salamanca

4. This evolution involves new tools and necesities LMS as a mean to support and make evolve subjects 100% of the universities have a LMS 79,5% of the big companies use these learning tools LMS don’t provide the expected benefits LMS as spaces to publish courses Oriented to a specific period of time Focused on the course and institution Problems to exploit information of learner activity GRIAL – Universidad de Salamanca Introduction (II)

5. Introduction (and III) LMS must evolve and be complemented with other tools To define what tools are better, learner information activity must be explored Interaction with the LMS Participation in different activities Interaction with other learners and teachers Interest of the learners Visual analytics tools as a way to do this GRIAL – Universidad de Salamanca

6. Outline Introduction The Problem Visual analytics Case study results Conclusions GRIAL – Universidad de Salamanca

7. The Problem (I) How to analyse information about what learner does How to know what are her worries and doubts about the subject? What are main important periods of activity? How are participating learners in activities and how to take it into account? What are main important issues of each subject area? And what are those understood worse? Is it working a specific tool to improve the subject? GRIAL – Universidad de Salamanca

8. The Problem (II) To analyse this GRIAL – Universidad de Salamanca

9. The Problem (III) Extract information Analysis information Take decisions GRIAL – Universidad de Salamanca

10. The Problem (IV) Extract information There are several ways to extract information about a LMS Direct access to database Functions definition to obtain information (hacking the system) Standard ways to obtain information (i.e.: Web Services) Exploit information Data mining Semantic analysis Visual analytics GRIAL – Universidad de Salamanca

11. The Problem (V) Software Engineering Subject Tries to explain a set of principles that are regularly and systematically applied to the development of viable software General Concepts 60 hours (45 theoretical hours and 15 lab hours) Focus on – Lifecycle and process requirement elicitation and documentation – Analysis and design methods and notation – Modularity, software architecture, and software reuse principles Practical part – Small project and 10 hours of workshops to solve different model problems GRIAL – Universidad de Salamanca

12. The Problem (VI) Software Engineering Subject Technological features Institutional LMS of the University of Salamanca (Studium) GRIAL – Universidad de Salamanca

13. The Problem (and VII) Software Engineering Subject 88100 log entries in two academic years It is needed to know Platform activity level during the academic year At what time learners are conected to the platform Activity in the forums Take into acount learners participation during evaluation processes … GRIAL – Universidad de Salamanca

14. Outline Introduction The Problem Visual analytics Case study results Conclusions GRIAL – Universidad de Salamanca

15. Visual analytics (I) Visual analytics Emerging area for supporting analytical reasoning by interactive visual interfaces More that only visualization Visualization, Human Factors and Data Analysis It allows users to detect expected events and also help them to discover the unexpected Surprising anomalies, changes, patterns, relationships … GRIAL – Universidad de Salamanca

16. Visual analytics (II) GRIAL – Universidad de Salamanca Tools to analyse information Semantic Spiral Timeline Considers time dimension Representing the overall use of the LMS From a global to a specific perspective – Course, time, person, activity Patterns of activity

17. Visual analytics (III) GRIAL – Universidad de Salamanca Tools to analyse information Temporal tag-cloud Most popular tags in alphabetical order, and visually weighted by font size Perceiving importance of data Evolution of each tag during the time It can be used at different levels Other functionalities – Exclusion of words – Synonyms – Exportation of forbidden words

18. Visual analytics (and IV) GRIAL – Universidad de Salamanca Tools to analyse information Social Network Graph Massive graph structures of social connectivity Represents relationships, links and and frequency of activities of students and teachers Icon size depends on the quantity of activity users perform Allows exploring participation in the learning system

19. Outline Introduction The problem The Solution Case study results Conclusions GRIAL – Universidad de Salamanca

20. Case Study Results (I) GRIAL – Universidad de Salamanca Case study Academic year 2008-2009 and 2009-2010 88100 log entries 160 students average Four possible sceneries – Monitoring the activity on the platform – Tag-cloud application over subject information – Monitoring a specific theme, forum, discussion or course – Tracking learners’ participation in the subject

21. Case Study Results (II) GRIAL – Universidad de Salamanca Case study Monitoring the activity of the platform SST to check the months greater activity (October, November) Incorporation of support staff during this time New tools to provide help to users Tagcloud application over course information 51 forums, 114 discussion threads, 172 posts, 26979 forum accesses 2907/9332 different words Uses of forum – Informative use (News, Notice, Defence, Summons, Review, Time) – Modelling discussions (Discussion, Entity, Relation, Base, Data, Diagram, Attribute, Model, SET) – Question solving (USAL, week, problems, practices, doubt, form, Web, Office, Delivery Solution)

22. Case Study Results (and III) Case study Monitoring a specific theme, forum, discussion or course Most relevant words – When are used – What kind of action (create, read, update) Most relevant forums and threads by using semantic zoom Tracking learners’ participation in the subject SNG What users are participating more What discussions are more relevant GRIAL – Universidad de Salamanca

23. Outline Introduction The problem The Solution Case study results Conclusions GRIAL – Universidad de Salamanca

24. Conclusions Technology can improve learning processes but is necessary to know what to include and when to do it Visual analytics will provide this information In this way decision taking is easier What tools to use When to include more staff Which issues requires more time or extensive description Take into account user participation in her evaluation Improve the way in which we teach and learn GRIAL – Universidad de Salamanca

25. GRIAL contact http://grial.usal.es http://www.facebook.com/grialusal http://twitter.com/grial_usal http://grial.usal.es

26. Acknowledgments This work is partially supported by the Ministry of Industry, Tourism and Trade of Spain (project IST-020302-2009-35), the Ministry of Education and Science of Spain (project TIN2010-21 695- C02) and the Department of Education of Junta de Castilla y León through the project GR47 GRIAL – Universidad de Salamanca

27. Visual Analysis of a Moodle-based Software Engineering Course Francisco J. García (fgarcia@usal.es)fgarcia@usal.es Miguel Á. Conde (mconde@usal.es)mconde@usal.es Sergio Bravo (ser@usal.es)ser@usal.es Diego A. Gómez (dialgoag@usal.es)dialgoag@usal.es Roberto Therón (theron@usal.es)theron@usal.es University of Salamanca – GRIAL Research Group Methods and Cases in Computing Education (MCCE-2011) Ciudad Real, June 16 th, 2011 Held at SPDECE-2011. Multidisciplinary symposium on the design and evaluation of digital content for education - Ciudad Real, June 15 th -17 th, 2011