1. Graphics Annotation Usability in eLearning Applications Dorian Gorgan, Teodor Ştefănuţ Computer Science Department Technical University of Cluj-Napoca dorian.gorgan@cs.utcluj.ro

2. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20072 Contents  Objectives  Graphics annotation techniques  eTrace platform  Knowledge assessment  Usability evaluation  Experiments  Measurements  Results  Conclusions

3. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20073 Objectives – I-Trace Project  Study the requirements and specifications for graphics pen annotation based eLearning environments User requirements (i.e. professor, students), eLearning environment functional specifications, usability requirements, lesson structure, user interaction techniques.  Adapt and integrate graphical annotation capabilities in eTrace eLearning framework Develop eTrace eLearning environment, design and implement the client-server architecture, resource management, security, annotation model, annotation persistence.  Integrate graphical interaction techniques in teaching materials Develop lessons in various domains including Computer Science, Medicine, Physics, Mathematics, Algorithms, Computer Graphics.  Usability evaluation for graphics annotation techniques Design and produce evaluation instruments for pen and mouse based graphics annotation; Develop test cases for graphics annotation according with usability requirements and specifications. Usability measurements, analyse data, usability evaluation.  Disseminate the results on usability tests and lesson development Conference (PLT2007, CBMS2007, RoCHI2007), Web sites (I-TRACE, eTrace), technical reports.

4. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20074 Objectives – Usability evaluation  Design and produce evaluation instruments for pen and mouse based graphics annotation  Develop test cases for graphics annotation according with usability requirements and specifications  Usability measurements  Data visualization and analysis  Usability evaluation

5. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20075 Graphics annotation techniques  2D graphical annotation techniques on text, images, and documents  2D graphical annotation techniques for 3D objects  3D graphical annotation techniques on 3D objects  E-learning framework based on 2D and 3D annotation techniques

6. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20076 eTrace Platform  eTrace – eLearning Environment based on graphics annotation  Developed at the Technical University of Cluj-Napoca (UNI-CLUJ)  Developed through the I-TRACE Project “Interactive Tracing and Graphical Annotation in Pen-based e-learning”, 223434-CP-I-2005-IT-Minerva-M (2005-2007) ITrace project: http://users.utcluj.ro/~gorgan/res/cgis/itrace/  eTrace references: eTrace eLearning Environment: http://dataserver.mediogrid.utcluj.ro/adnotare/ eTrace presentation: http://users.utcluj.ro/~gorgan/res/cgis/itrace/

7. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20077 eTrace Platform  Experiment the annotation techniques on 2D and 3D scenes  Support annotation independence against document type (text, image), format and 3D scene description  Annotation modeling  User interaction techniques  Annotation description and communication  Synchronization between object and annotation  Annotation processing and interpretation  Persistence among working sessions and users  Visualization  Security  Integration into e-learning lessons  Implementation and experimentation of the annotation techniques in e-learning applications

8. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20078 eTrace – Graphics Annotation eLearning Environment

9. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 20079 eTrace – Graphics Annotation Based Lessons

10. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200710 eTrace – Graphics Annotation Based Lessons

11. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200711 eTrace – Graphics Annotation Based Lessons

12. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200712 Usability evaluation Evaluation methodology 1. Usability attributes efficiency of the pen related annotation, number of errors by drawing a symbol, precision, and graphics quality 2. Scenario user actions, the objects and the text which are subject of the annotation 3. Measurement method parameters which are to be measured (e.g. execution time), the measurement and recording approach (e.g. automatically) 4. Evaluation criteria assessment approach, and the parameter value range for normal, accepted and critical cases

13. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200713 Experiments  17 exercises of eTrace lessons 2D graphics annotation in the context of eTrace lessons  Group of 20 users women and men, 22-52 years, 12-20 experimented users  Measurements: Runtime: Execution Time Off-line: Aspect, Number of Errors  Usability estimation: Usability ← Aspect / Time x Number_of_Errors)  Experiment cases: Pen vs mouse Selection Drawing Handwriting...

14. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200714 Experiments - Exercises Other exercises (17): http://users.utcluj.ro/~gorgan/res/cgis/itrace/http://users.utcluj.ro/~gorgan/res/cgis/itrace/

15. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200715 Measurements

16. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200716 Measurements and Results Usability measurements for individual selection by contour (Exercise 14). User Time [sec] Omission ErrorsTime x Errors mousepenmousepenmousepen 1278.10265.1915969.5239.78 2301.25283.562111105.4451.99 3370.56319.57151792.6490.54 4312.61202.602723140.6877.66 5333.80232.96161189.0142.71 6534.07431.922716240.33115.18 7578.97462.051216115.79123.21 8164.55195.3071119.2035.81 9200.45141.9515850.1118.93 10156.55155.8814136.532.60 11258.11205.822515107.5551.46 12304.97262.8941020.3343.81 13206.41200.077324.0810.00 14160.4361.67101226.7412.33 15167.60120.0611830.7316.01 16149.13108.4015937.2816.26 17137.75111.9114332.145.60 18114.33122.447713.3414.28 19141.13119.549621.1711.95 20138.61138.8812327.726.94

17. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200717 Measurements and Results Usability measurements for group selection by contour (Exercise 4). User Time [sec] Wrong inclusionTime x Errors mousepenmousepenmousepen 19.4317.706356.6053.11 29.697.958477.5231.80 39.5611.3820.119.121.14 410.2616.656461.5466.59 58.068.61148.0634.43 611.569.084646.2554.49 711.1012.5310.111.101.25 815.3024.430.1 1.532.44 911.8712.291311.8736.87 1014.3822.2430.143.132.22 1127.8815.918537.6629.71 1231.4525.5510.131.452.55 1314.5510.781214.5521.55 1416.0720.420.111.6120.42 1511.2412.391211.2424.77 168.2910.440.110.8310.44 1719.2117.090.111.9217.09 188.8612.23138.8636.70 1913.2010.036379.2030.09 2020.3214.091220.3228.17

18. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200718 Measurements and Results Usability measurements for gesture drawing (Exercise 1). User Time [sec]ErrorsAspect Aspect / (Time x Errors) mousepenmousepenmousepenmousepen 14.996.5632350.200.38 22.501.7724240.400.56 34.233.5362150.040.71 44.557.6284120.030.07 53.423.0364230.100.25 65.995.6531240.110.71 74.414.0154240.090.25 86.636.8722450.300.36 97.598.2745240.070.10 1010.9111.5412450.370.22 119.606.5024440.210.15 1213.3812.3035430.100.05 135.836.6251350.100.76 147.307.0911450.550.71 153.693.9143340.200.34 162.783.7053540.36 178.927.3411550.560.68 182.562.8052340.230.71 193.554.8142250.140.52 204.734.2832440.280.47

19. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200719 Conclusions  Free forms of expression  New types of questions and answers in the eLearning applications  Visual free form answer provides support for creativity, flexibility, imagination, and artistic ability  Annotation based interaction techniques must be designed according with the characteristics of each interaction device  The assessment of the annotation quality has a significant impact on the quality of the answer  Automatically evaluation of the annotation based answer

20. 15 January, 2007 Pen-based Learning Technologies Workshop, Catania, 24-25 May, 200720 Future work  Usability of the 3D annotation techniques  Develop automatically techniques for knowledge assessment in graphics annotation based lessons  Multi user sessions  Real time communication  Develop graphics annotation lessons in various domains  Propose technical specifications for standards concerning with 1. Graphical annotation model 2. User interaction techniques 3. Automatically knowledge evaluation

21. Questions Thanks Dorian Gorgan, Teodor Ştefănuţ Computer Science Department Technical University of Cluj-Napoca dorian.gorgan@cs.utcluj.ro