1. EURIDICE project Evaluation of image database use in online learning environment 11/10 2006 pieta.eklund@hb.se elena.maceviciute@hb.se maria.lindh@hb.se wilsontd@gmail.com

2. Outline Background – the EURIDICE project Role of SSLIS in the project Preparation for the evaluation –Preparation of instruments –Training workshop Implementation Current status Conclusion

3. Background EURIDICE – European Recommended Materials for Distance Learning Courses for Educators http://www.euridice-edu.org 10 Partners from 7 countries: –Austria, Belgium, Greece, Italy, Poland, Spain and Sweden

4. Background The idea –is to offer to the educational institutions a wide source of mostly unpublished image content from different providers (commercial, public) The service –is based on combination of technical instruments, contents, networking platforms and community of users and providers –contents will be accessible through a multilingual web-based interface, within a client/sever architecture

5. Warehouse in Torun, 17th century

6. Kaiser’s permission

8. SSLIS in the project A survey of e-learning methodology and image use practice at the Univ. College of Borås A market analysis of similar services in Sweden Evaluation of the service Dissemination of information about the project and EURIDICE service

9. SSLIS in the project Responsibility for the development of the evaluation methodology and monitoring of the assessment process. –created a set of principles and instruments to be used for the evaluation of the service –conducted a workshop for the evaluators in Borås –monitored the pilot sites

10. Evaluation process Practical product/service evaluation Based on: –from the customers’ point of view –end users' evaluation of various aspects of the service Students associated with courses and appropriate for testing the resource base Teachers and their perception of the training needs

11. Evaluation process Specific elements of evaluation: –technical validation from the customer’s point of view, –use of the resources in e-learning, –the content of the databases, –search engine, indexing, etc. –interface.

12. Evaluation process in SSLIS Pilot site - validating the EURIDICE service –1 teacher designing a course –2 teachers evaluating the service from the point of view of information professional –7 teachers evaluating the service as potential users of the content –23 students from a variety of departments following a common course and evaluating the usage of images in the course and the service

13. Training for evaluation (workshop in Borås in February) Discussion of main evaluation problems Designing common data collection, coding and analysis instruments Hands-on training of the actual evaluators Additionally an evaluator from e-ISOTIS trained during a monitoring visit One partner did not receive any training and as a consequence the data from one pilot site is not comparable to others

14. Implementation In Borås: 1.Adaptation of a course (February-March) 2.Ensuring the access to EURIDICE service 3.Running the course (April-May) 4.Training students to use the platform (May) 5.Observation and interviewing students (May) 6.Training packages for the teachers (May) 7.Interviewing teachers (June)

15. Current status Evaluation data from all pilot and demonstration sites (8 partners) Increasing need for usage of images in teaching and learning 2 questions: –Service acceptance, satisfaction of users, usability, completeness, accessibility –Service sustainability (capability to satisfy user needs)

16. Assessment results Answers: –service requires significant improvements to be accepted and capable to satisfy users’ needs in terms of usability of the search/retrieval tools; completeness of the offered resources, indexing and description quality –neither the process of retrieval and usage of images from the EURIDICE service (efficiency), nor the relevance of the service and also the effectiveness (outcome) of its usage is sufficient for the intended customers and end users for e-learning purposes. The validation phase has provided a set of recommendations that should be implemented to meet the needs of the segment of the market and to be competitive in many respects.

17. Conclusions Image databases can be effectively integrated with e- learning platforms. Integration with platforms other than used for the project will require rather intensive efforts and work on the part of educational users Evaluation by educational partners has revealed some significant issues of design and retrieval system which will need attention before the service can be fully developed on commercial basis SSLIS staff acquired skills and knowledge about running EU projects and types of new online resources for e-learning