1. Meta Keijzer- de Ruijter
Campuswide implementation of MapleTA
Meta Keijzer- de Ruijter
ICTE-adviser TU Delft

2. Introduction

3. Delft University of Technology
Facts and Figures
TU Delft founded in 1842
8 faculties
Architecture
Applied Sciences
Aerospace Engineering
Mechanical, Maritime & Materials Engineering
Technology, Policy & Management
Civil Engineering & Geosciences
Industrial Design Engineering
Electrical Engineering, Mathematics & Computer Sciences
Delft Extension School (2014)

4. Online education DelftX Massive Open Online Courses (MOOCs)
Learning Activities & Course Materials
Free
Enrolled students only, massive numbers
Bachelors level
Certificate of Completion
Massive Open Online Courses (MOOCs)
Open Course Ware (OCW)
Online Distance Education
On Campus education
OpenCourseWare (OCW)
Course Materials
Free
Big Exposure, Worldwide audience
Both Bachelor and Master level
No interaction with faculty
No accredited certificate
On Campus Education
Direct access to education
Research facilities
World Class Library
Student social tissue
Student projects
Online Distance Education
Learning Activities & Course Materials
Paid enrollment
Enrolled students only, limited numbers
Master level
Accredited Course Certificate
Full Master Degree
Open Course Ware review: “After MIT, the best provider of open courseware is TU Delft”. Visitors from over 170 countries log in.

5. Delft Extension School
Mission
Delft Extension School for open & online education
supports the mission of the Roadmap 2020 to realize a permanent position among leading universities in engineering, science & design by:
Growing academic output:
More students & academic staff
Better reputation & visibility of academic niches
Higher & better quality international enrollments
New learning & research networks
Improving quality of both campus & online education
Higher quality courses
Better progress of students in full-time degree programs (study succes)
Better fit with future student expectations

6. Digital Testing @ TU Delft

7. Digital Testing at TU Delft
Short history (MapleTA)
< 2008
Use of MapleTA in Math course(s) a very small scale
2008
Selection of centrally supported test system
jan 2009
First digital exam
Jan 2012
Large scale digital exams 3mE faculty (bachelor)
Summer 2012
Building large scale facilities
Oct 2012
First exam in new facilities (500 seats, 2 locations)
Sept 2013
Curriculum review all bachelor programs
2014
Start development Extension School
Jan 2014
Evaluation digital testing
June 2014
Proposal for improvement and expansion facilities digital testing (800 seats, 2 locations)

8. Implementation approach
Software selection
Basic structure of MapleTA
Vinkje bij aanmelding
Kort procedure

9. MapleTA Configuration
MapleTA (formative setting)

10. Implementation approach
Software selection
Basic structure of MapleTA
Set up support
Teacher training
Technical application management (centralized)
Security (exam)
Exam rooms and proctor training
Functional application management
Logistical support for exams
focus on quality of assessment (specially for adaptive questions and use of advanced features)
Vinkje bij aanmelding
Kort procedure

11. Growth Homework Assignments in MapleTA
Number of classes and students taking a test

12. Growth digital exams in MapleTA
Number of classes and students taking a test

13. Scenarios
For
Testing

14. Scenarios for testing Formative testing
In order to improve the study succes rate for students. Formative testing is becoming more important leading to an increase of MapleTA usage.
Structured homework (Statics)
16 assignments (2 per week). Limited availability.
4 attempts to reach 70% score » obtain access to next assignment automatically (through adaptive release in Bb)
Language test
Placement tests English (fill in the blanks).
Text with 100 blanks (automated grading) + 2 essay questions (manual grading) to assess performance level prior to start of course

15. Scenarios for testing Summative
Growing numbers of students, creation of large computer exam rooms and focus on quality stimulates the use of digital exams.
Multiple choice pre-tests
Financial Management. First mc questions, minimum score required in order to proceed to assignment with essay questions.
Exams with open (essay) questions want to use digital exams Students prefer typing over writing.
Using additional software for calculation purposes and have the responses automatically graded in MapleTA

16. Question
Examples

17. Examples Question types Adaptive Question Designer Randomisation
Strength of Materials. Breakdown of main question into sub questions. Better comprehension of student error and more opportunity for the student to get credit for sub steps.
Randomisation
Production Technology. Use of text and image randomisation.
Enlarging the itembank
Question Designer
Mechatronics. Use of Maple code to grade student response.
Compensation for prior error
Sketching
Coastal Dynamics – use of Java in MapleTA 9.5
use of Math App in Maple TA 10

18. Statics/Strength of Materials
Adaptive Question

19. Subquestion 1

20. Subquestion 2

21. Subquestion 3 (same as main)

22. Randomisation
Text /image

23. Randomisatie
Afbeelding

24. Maple Graded Taking into account previous error
Calculate the answer to Sub A: e.g. Calculate the diameter of a circle with a surface area (A) of 𝐴=𝜋 𝑟 2 𝑜𝑟 (𝜋∗ 𝑑 2 )/4 (correct response = 10)
lets say you answered 5: confusing r and d)
Calculate the answer to Sub B:
What would be the volume of a sphere with that same diameter? 𝑉= 4 3 ∗𝜋∗ 𝑟 3 (correct response = 523.6)
Feedback to response Sub B:
Your answer to sub b is correct if you type 10 or if you use your answer to Sub A to calculate the volume:
𝑉= 4 3 ∗𝜋∗ 5 3

25. Vraagtypes
Sketching

26. Integration of Digital Testing existing Policies on Assessment

27. TU Delft Policy
The director of Education is responsible for all assessments is (digital testing is no exception)
Digital testing is to be integrated in general policies
Digital testing is a central facility
MapleTA is centrally supported
Vinkje bij aanmelding
Kort procedure

28. Impact of digital testing on general policies
Introduction of digital testing sharpened some general procedures:
Academic Integrity
More strict admittance to the exam room due to
Limited seat capacity and
More administration in the exam room
Vinkje bij aanmelding
Kort procedure

29. Academic Integrity Access control (rightful participation)
Authentication
Proctoring
No communication possible
No illegit help (books, sheets, software, etc)
Clear roles for both teaching staff, proctors and exam committees
Vinkje bij aanmelding
Kort procedure

30. Developments Advanced Assessment Module for teaching staff,
Itembanking and test matrices
Rules for automated feedback
Open questions and online grading
Moving from mc to a variety of digital question types
Use and quality of adaptive questions
Growing capacity in exam rooms (stepwise)
Testing at a distance (Remote Proctor Now)

31. De Business Case

32. Break even Investment into large capacity of computer rooms for exams
In digital exams were taken (not just MapleTA)
This is about 10% of all exams taken
This number is expected to grow to 20%
It is expected to grow to 18-20% in the next 2 years exceeding the break even point