José Paulo Leal | Ricardo Queirós CRACS & INESC-Porto LA Faculdade de Ciências, Universidade do Porto Rua do Campo Alegre, Porto PORTUGAL PExIL Programming Exercises Interoperability Language
Outline 1. Introduction 2. Learning Objects Standards 3. PExIL definition Textual elements Specification elements Program elements 4. PExIL validation Generating a IMS CC package Describing crimsonHex programming exercises 5. Conclusions and Future Work
1. Introduction Learning Objects (LO) container with educational resources and metadata fundamental for producing, sharing and reusing content may include exercises to be automatically evaluated uses IMS QTI to describe questions and tests a LOM extension ideal for simple exercises insufficient for complex exercises IMS CP package imsmanifest.xml Metadata (LOM) Resources (LOM+ QTI) exercise assets
1. Introduction Complex exercises (programming exercises) require specialized content description collection of resources test cases, solution programs, exercise descriptions, feedback configuration data compilation and execution lines, time and memory limits communication with heterogeneous systems Learning Object Repositories (LOR) Evaluation Engines (EE) Integrated Development Environments (IDE)
1. Introduction Challenges of programming exercises as Learning Objects interdependent resources processed in different moments of the exercises’ life-cycle manual creation using standards is time-consuming error-prone creation selection presentation resolution evaluation
assets XML manifest XML manifest 2. Learning Objects Standards Learning Objects Standards levels: packaging – IMS CP, ADL SCORM, IMS CC metadata – IEEE LOM, Dublin Core exercises – IMS QTI metadata resources resource Exercise description (IMS QTI) PDF, HTML, SWF files
3. PExIL definition PExIL Programming Exercise Interoperability Language consolidate all the data required in the programming exercise life-cycle XML dialect formalised using a XML Schema organized in three groups of elements: textual elements specification elements program elements
3. PExIL definition Textual elements Textual information (e.g. title, date, challenge) Used for Presentation selection
3. PExIL definition Specification elements set of restrictions used for generating specialized resources test cases (input and respective output file) feedback exercise statement (part)
3. PExIL definition Example: Given 3 numbers verify that the last number is between the first 2 <feedback when=”$num1>$num2”> Numbers that limit the range can be given in descending order True False INPUTINPUT OUTPUTOUTPUT
3. PExIL definition Program elements references to programs as external resources solution program correctors metadata about those resources compilation and execution line Hints used for test case generation, evaluation
3. PExIL definition Interdependence TypeElement Phases SelectionPresentationResolutionEvaluation Textual titlexx creation/authors/authorxx creation/datexx creation/purposexx challengex contextx skeletonxx Specificationinput/specificationxxx output/specificationxxx Program solutionxx correctorx hintsxx
4. PExIL validation We validate the PExIL definition according to: its usefulness input of a set of programming exercise life-cycle tools generation of a IMS CC learning object package its expressiveness ability to define a set of programming exercises in a repository extracted from the UVA on-line judge (EduJudge)
4. PExIL validation The Generation tool: uses as input: a valid PExIL instance a program solution file and generates 1. an exercise description 2. a set of test cases and feedback files 3. a valid IMS CC manifest file. 4. an IMS CC LO package 4 4
4. PExIL validation 1. Exercise description generation...
4. PExIL validation 2. Test cases and feedback generation the test values: cover the range limits of all variables – T1-T8 (2 3 = 8) are constrained by the feedback (at least one) – T5, T6 are generated Randomly – T9, …Tn $num2”>Numbers that … Var.T1T2T3T4T5T6T7T8T9Tn n RR n RR n RR
4. PExIL validation 3. Manifest generation - metadata IMS CC (Common Cartridge) manifest PExIL mapped to IEEE LOM using templates Data TypeSchemaElement path Title LOMlomcc:general/lomcc:title PExILexercise/title Date LOMlomcc:lifecycle/lomcc:contribute[lom:role=’Author’]/lom:date PExILexercise/creation/date Author LOMlomcc:lifecycle/lomcc:contribute[lom:role=’Author’]/lom:entity PExILexercise/creation/authors/author/v:VCard/v:fn Purpose LOMlomcc:general/lomcc:coverage PExILexercise/creation/purpose
4. PExIL validation 3. Manifest generation – resources resource elements references generated assets of the IMS CC package IMS CC manifest resources element
4. PExIL validation UVA on-line judge (EduJudge) programming exercises validate the expressiveness of the PExIL definition cover the requirements (input/output constraints of the exercise) random subset of repository (crimsonHex) 24 exercises – (1% of a total of 2393 exercises)
5. Conclusion and future work PExIL – a XML dialect for programming exercises Used in programming exercises’ life-cycle: to present the exercise description in different formats to regenerate test cases to produce feedback to the student PExIL schema is available for download and testavailable
5. Conclusion and future work Our plans are to finish the development of the generator deployable as an IDE plug-in command line to support PExIL in different classes of tools learning object repositories - crimsonHex contest management systems – Mooshak development support tools – Eclipse, Visual Studio Express learning management systems – Moodle, Sakai
Questions? Authors José Paulo Leal Ricardo Queirós Thanks!