Ivan Pribela, Mirjana Ivanović

Table of contents  Overview of Svetovid  The first Svetovid version  The second Svetovid version  Extension of the second version  Overview of Testovid  Integration of Svetovid and Testovid  Overview of Mag  The MILE environment  Conclusions  Further work 2

Table of contents 3

The Svetovid system  Special environment for development and submission of student programming assignments  Implemented purely in Java 4

Background  Before reforms, students were assessed based on their solutions written on paper  After reforms, we concentrate on practical exercises as a form of continual assessment 5

Motivation  Now instructors have to much work Compile each program Run it with some test cases Examine all of the output  Not all students are producing their own solutions Solutions are easily shared Awareness and honesty is on a poor level 6

The goals of Svetovid  Allow students to comfortably develop their programs  Allow access to support literature  Disallow students to share programs and solutions, intentionally or unintentionally  Keep a detailed log of student efforts  Help instructors to mark student solutions  Allow students to test their programs before submission  Be flexible enough and usable for different courses Wide range of programming languages Different project stages: coding, documenting… 7

Table of contents 8

The first version  Command line based  Set of command line tools for assignment retrieval and solution submission  Used virtual student directories for assignment and solution storage 9

Virtual directories John Smith Marc Tyler Fred Carlen Joanna Blue Lazar Hans Lucy May Peter White Assignment 1 Assignment 2 Assignment 3 All students Root directory Assignment 1 Assignment 2 Assignment 3 All students John Smith Marc Tyler Peter White 10

Usage  Students used favorite development tools and submitted solutions at the end  Command line tools are cumbersome Had to type name and password every time  Deemed uncool by the students  Easy to cheat 11

Table of contents 12

The second version  GUI based  Introduced integrated development environment as student client  Introduced instructor control client 13

The student client  IDE allowing the student comfortable work  Prompts a login dialog  The files owned by a student are contained in virtual directory  Content of virtual directories can be saved to local file system or sent via only after assessment 14

The student client 15

The instructor client  Set of configuration dialogs Setting up courses Managing students  Prompts a login dialog  Full control over the conduction of practical exercises Groups Order Timing 16

The instructor client 17

Table of contents 18

Encountered problems  The text editor component should accommodate all programming needs  A third party component was built in Syntax highlight, parenthesis matching…  Main problems No keyboard shortcuts while editing text No automatic indention 19

Goals of the extension  Create a new text editor component  Use of java.swing.text package  Keep all of the previous features and add new ones 20

CodeEditorPane  New textual component  Extends JEditorPane  Main features Syntax highlighting Marking parenthesis pairs Line numbering Indent preservation Measures against cheating Convenience methods for applications 21

Syntax highlighting  Done in a separate object representing the language  Supported languages Java Scheme Modula2 TeX 22

Marking parenthesis pairs  Pairs of parenthesis usually have meaning in program code 23

Line numbering  Provides useful information during debugging 24

Indent preservation  Indentation helps to make code more readable  On line breaks, the indent from the previous line is copied 25

Measures against cheating  Usage of the system clipboard is forbidden  Overriding cut, copy and paste methods  Using a local static clipboard 26

Convenience methods  Status bar Caret row and column Active language  Toolbar Language selection Manual syntax highlight refresh  Line numbering component  Fully set up JPanel with the component 27

Table of contents 28

The Testovid system  Interactive and batch testing system Framework for testing systems  Implemented using Apache Ant 29

Niklaus Wirth 30

The goals of Testovid  Allow students to check their solutions directly form their environment  Allow assessment of all collected solutions after the assignment deadline  Be flexible enough and usable for different courses Wide range of programming languages Diverse set of testable aspects 31

Testing modes  Students can test their solutions at any time  Instructor can test all submitted assignments at once  In both cases detailed reports are generated 32

Interactive test mode report Predmet: Operativni sistemi 1 Zadatak: Obilazak direktorijuma Student: Pera Perić Provera: Direktorijum bez poddirektorijuma Uspeh: 1 poen. Provera: Direktorijum sa poddirektorijumima Neuspeh: Rezultat izvrsavanja se ne slaze sa ocekivanim. Provera: Opsti slucaj direktorijuma Neuspeh: Rezultat izvrsavanja se ne slaze sa ocekivanim Ukupno: 1 poen. 33

Batch test mode report Predmet: Operativni sistemi 1 Zadatak: Obilazak direktorijuma Bajcetic Vladimir3 poena Baranovski Nenad0 poena Barjaktarovic Marko1 poen Blagojevic Katarina2 poena Damjanovic Jelena0 poena Djurkovic Velimir3 poena Jovanovic Jelena1 poen Kovacevic Miroslav2 poena Krickovic Valentina1 poen... 34

Table of contents 35

Integration  One of Svetovid goals was automatic assessment Not implemented directly Delegated to another system  Testovid was designed with integration in mind Internal logic is the same Directory structure is compatible  Integration was straightforward 36

Table of contents 37

The Mag system  Tutoring system for distance learning  Provides three types of activities Tutoring Quiz and feedback Online programming 38

The goals of Mag  Be platform independent  Create useful reports for instructors  Have rapid development tools for creating basic elements of tutoring system 39

The user interface 40

Table of contents 41

The MILE  Multifunctional Integrated Learning Environment  Modular design Learning environment (Mag) Submission system (Svetovid) Testing system (Testovid) 42

The goals of MILE  Provide adequate tutorials to students  Allow students to test their understanding of new material  Provide online programming support  Provide students with automated feedback  Allow final exams via internet or in computer laboratory 43

The environment provides  Communication Student-student Student-instructor  Tools to simplify maintenance  Student web pages 44

Main problems  Mag was designed as a Java programming tutor  Svetovid and Testovid are designed for computer laboratories  Great difference between structures of student modeling 45

Table of contents 46

Svetovid  Svetovid goals are fulfilled Less time spent on submission Less effort spent on assessment  Has been successfully used in many courses Operating systems Programming languages Computer graphics Compiler construction 47

Text editor component  Goals are fulfilled Support for all existing features Added new features Flexible syntax highlight system Simple to build in into existing applications  Should be used from this year All courses using Svetovid 48

Testovid  Testovid goals are fulfilled Less effort spent on assessment Assessment available at any time  Has been successfully used Operating systems 49

MILE  MILE goals are fulfilled Showed itself as a modern learning tool Student satisfaction was the same as with the manual method Less effort spent on assessment  Has been used by first year students As a complement to classroom teaching 50

Published work  Svetovid – Special submission environment for student’s assessment Ivan Pribela, Nataša Ibrajter, Mirjana Ivanović Proceedings of 2nd Balkan Conference in Informatics, BCI’2006 Ohrid, Macedonia, November 17-19, 2005, pages  Testing almost any aspect of student’s assignments Ivan Pribela, Mirjana Ivanović, Zoran Budimac Proceedings of 3rd Balkan Conference in Informatics, BCI’2007 Sofia, Bulgaria, September 27-29, 2007, pages  Svetovid—Interactive development and submission system with prevention of academic collusion in computer programming Ivan Pribela, Mirjana Ivanović, Zoran Budimac British Journal of Educational Technology (in print)  MILE – Multifunctional Integrated Learning Environment Boban Vesin, Mirjana Ivanović, Zoran Budimac, Ivan Pribela Proceedings of e-learning 2008, IADIS Multi Conference on Computer Science and Information Systems Amsterdam, Netherlands, July 22-27, 2008, pages

Table of contents 52

Continual improvement  Continue to improve usability and user experience of GUI clients Adding new features and tools Adding support documentation and manuals  Leveraging effort of test creation Creating a solid base of reusable test modules for multitude of aspects Creation of a test module generator  Adding plagiarism detection Just in case 53

54

55