1. What Current Research May Mean for the Future of Quality Management Systems LoD Meeting Zurich, 16.02.2004 Dr. Sabine Seufert

2. LoD Zurich 16.02.2004 Page 2 SCIL Activity Fields Innovator Change Engine Promotor Quality Catalyst Initiator Research Lab Moderator Community Platform Trainer SCIL Academy

3. LoD Zurich 16.02.2004 Page 3 Quality? Ihre Durchfallquote ist besorgnis- erregend! Die Qualität der Mensa sollte überprüft werden!

4. LoD Zurich 16.02.2004 Page 4 What is „Quality“? 4 Main Questions eLearning - Quality What are the targeted objects to evaluate? ObjectsGoals Why to evaluate? What are the norms?

5. LoD Zurich 16.02.2004 Page 5 Meso-Level Programmes, Courses Certificates Micro-Level Products: eMedia, „LearningWare“ Aggregation Organisation, Universities, Enterprises, Business Units Macro-Level Context, Conditions What is „Quality“?: Objects

6. LoD Zurich 16.02.2004 Page 6 What is „Quality“? 4 Main Questions eLearning - Quality Process How to evaluate? What are the methods? What are the targeted objects to evaluate? ObjectsGoals Why to evaluate? What are the norms?

7. LoD Zurich 16.02.2004 Page 7 Prozess Input What is „Quality“: Process Output (Outcome) Qualification of tutors, teachers Feedback to students Evaluation of the tutor support Process of a Quality Management System: Evaluation - formative evaluation: action use (initiate changes) - summative evaluation: conceptual use (decisions) e.g.

8. LoD Zurich 16.02.2004 Page 8 What is „Quality“? 4 Main Questions eLearning - Quality Process How to evaluate? What are the methods? What are the targeted objects to evaluate? ObjectsGoals Why to evaluate? What are the norms? People Who are the (internal, external) evaluators?

9. LoD Zurich 16.02.2004 Page 9 What is the Quality Initiative „ELIP“? Cooperation with efmd (European Foundation for Management Development) Accreditation Institute in Brussels: EQUIS: Accreditation for Universities CLIP: Accreditation for Corporate Universities ELIP (eLearning Quality Improvement Programme)

10. LoD Zurich 16.02.2004 Page 10 Goals Why to evaluate? What are the norms? People Who are the (internal, external) evaluators? ELIP Self-Assessment, External Evaluation: Auditor Team What is ELIP? Process How to evaluate? What are the methods? What are the targeted objects to evaluate? Objects ELIP eLearning Programmes for Management Education ELIP Input-, Process- Output-Qualities Auditing Process Why to evaluate? What are the norms? ELIP Quality standards based on research, learner perspective eLearning - Quality

11. LoD Zurich 16.02.2004 Page 11 Why „ELIP“? Marketing Strategy to promote advanced quality of eLearning (transparency for „end-customers“) Implementing quality standards From a customer perspective: No systematic approach for programmes with focus on quality improvement (facing multi-dimensional challenges) From a market perspective: Urge for theoretically sound quality framework Supported by empirical evidence From a research perspective:

12. LoD Zurich 16.02.2004 Page 12 Why ELIP?

13. LoD Zurich 16.02.2004 Page 13 How does ELIP look like? Strategy Development Implementing, Quality Development Organisation Culture Technology Resources Pedagogy Quality Management-Approach Quality Framework and Criteria: - Expert Interviews (25 Experts) - Delphi-Study (37 Experts, 2 Rounds) Results: 47 Quality Criteria Self-Assessment Guide and Review Guide for Auditor Teams Focus: Quality and Sustainability Based on:

14. LoD Zurich 16.02.2004 Page 14 How does ELIP look like? Strategy Development Implementing, Quality Development Organisation Culture Technology Resources Pedagogy What to do? The strategic (added) value of e-Learning within the programme is explained; clear understanding about the contribution e-Learning is to provide for the success of the programme. Examples How to do? Students are provided with programme information (e.g. objectives; structure; estimated study time; required technology and technical competence).

15. LoD Zurich 16.02.2004 Page 15 How does ELIP look like? Strategy Development Implementing, Quality Development Organisation Culture Technology Resources Pedagogy

16. LoD Zurich 16.02.2004 Page 16 Strategy Development Implementation, Quality Development Organisation Culture Technology Resources Pedagogy ELIP Quality Criteria Pedagogy Ultimate goal “Sustainable Learning Outcome“ Pedagogic added value of eLearning Feedback to student questions and assignments Presentation of contents Construction of assessments Integration of eLearning components Examples

17. LoD Zurich 16.02.2004 Page 17 ELIP Quality Criteria Strategieentwicklung: Zielsystem Implementierung, Qualitätsentwicklung Organisation Culture Technology Resources Pedagogy Technology Usefulness of eLearning System Easyness of Use, Usability-Criteria Technology standards Ultimate Goal „ stable and problem-adequate functionality“ Examples

18. LoD Zurich 16.02.2004 Page 18 ELIP Quality Criteria Strategieentwicklung: Zielsystem Implementierung, Qualitätsentwicklung Organisation Culture Technology Resources Pedagogy Resources Balance between running and advancing the program Allocation of resources reflect the priorities according to the programme objectives Ultimate Goal „ Efficient and effective use of resources“ Examples

19. LoD Zurich 16.02.2004 Page 19 ELIP Quality Criteria Strategieentwicklung: Zielsystem Implementierung, Qualitätsentwicklung Organisation Culture Technology Resources Pedagogy Organisation Clearly defined roles and responsibilities Support structures for students and staff Processes for student complaints Efficient programme coordination Ultimate Goal „Flexibility and Efficiency of Structures and Processes “ Examples

20. LoD Zurich 16.02.2004 Page 20 ELIP Quality Criteria Strategieentwicklung: Zielsystem Implementierung, Qualitätsentwicklung Organisation Culture Technologie Resources Pedagogy Culture Enhancing the acceptance of the programme promoting a motivating environment Incentives for the staff and students Commitment of the management Ultimate goal „Readiness / willingness for innovation and change, self-organisation” Examples

21. LoD Zurich 16.02.2004 Page 21 Outlook Pilotphase: 2 Universities, 1 Company for testing the ELIP Certification Process Validation of the Quality Criteria (Phd contribution): Theoretical basis and empirical proof in practice Systematic, Multi-dimensional approaches, considering contextual conditions Shift from (static) Quality Assurance towards (dynamic) Quality Development General Outlook Quality Management Systems: ELIP Outlook:

22. LoD Zurich 16.02.2004 Page 22 Thank you very much for your attention! Dr. Sabine Seufert SCIL - Swiss Centre for Innovations in Learning sabine.seufert@unisg.ch www.scil.ch