Presentation on theme: "Quality & Effectiveness of E-Learning Courses – Some Experiences from Singapore Lim Kin Chew 16 – 17 Dec 2010 7 th International E-Learning Conference."— Presentation transcript:
Quality & Effectiveness of E-Learning Courses – Some Experiences from Singapore Lim Kin Chew 16 – 17 Dec th International E-Learning Conference for a Knowledge-Based Society Srisakdi Charmonman IT Center Assumption University Bangkok, Thailand
Contents Introduction What is E-Learning Quality? What is E-Learning Effectiveness? Quality initiatives in Singapore What more can we do to improve E-Learning Quality? Conclusion
This is my blog on Quality Matters in E-Learning. I started this blog some time in 2007 mainly to share my experiences and information on learning objects and e-learning in general. Introduction
Request for Consultancy Received request from a government bodys e- learning Team: 1.E-learning features that support the pedagogical design of e-learning content / instructional objectives 2.Criteria for e-learning effectiveness 3.Evaluation of the various teaching & learning domains, i.e. facilitation; transference; retention of learning, etc 4.Key competencies which e-learning aims to equip learners
Some deductions Quality and effectiveness of e-learning or e- training are becoming important issues, not just the development of contents or the use of LMSs
What is E-Learning Quality? The predominant view is that quality relates to obtaining the best learning achievements (50%). Together with 'something that is excellent in performance' (19%), this primarily pedagogical understanding was more widespread than options related to best value for money or marketing. (from Quality in e-Learning – use and dissemination of quality approaches in European e-Learning – A study by the European Observatory –
Business Times, Monday, 3 Sep 2001 BizIT Section Business Times is a newspaper in Singapore that deals mainly with business matters.
Quotation from BT journalist, Chua Hui Min: Effectiveness, rather than technology, is the key, and defining this is perhaps both the most difficult and important step in deciding how to invest in e-learning.
Entities where e-learning is implemented widely ICT Masterplan for schools Polytechnics and Universities Corporate bodies Singapore Armed Forces Civil Service College
The ICT Masterplan Journey - 4
The ICT Masterplan Journey - 1 Managed by the Ministry of Education (www.moe.gov.sg) E-learning was given a major boost with the implementation of the following IT Masterplans for Education: – 1 st Masterplan for IT in Education (1997 – 2002) – MP1 – 2 nd Masterplan for IT in Education (2003 – 2008) – MP2 – 3 rd Masterplan for IT in Education (2009 – 2014) – MP3 1 st Masterplan for IT in Education: – Build the strong foundation for schools to harness ICT – Provide basic ICT infrastructure – Equip teachers with a basic level of ICT integration & competency – 363 schools had fundamental building blocks in place to use ICT in the curriculum – 30% of ICT usage in curriculum
The ICT Masterplan Journey - 2 The Second Information Technology in Education Study (or SITES 2) conducted by the International Association for the Evaluation of Educational Achievement (or IEA) in 1999 reflected this paradigm change – accepting ICT as a tool in education. It reported that Singapore stood out favourably in terms of the level of teachers knowledge of ICT-based instructional practice. They had more avenues to develop their ICT skills than their counterparts in most other countries. Singapore principals were very positive towards ICT in schools. They had an attitude score of 90 on a scale of 0 to100. Mental barriers of old mindsets had been successfully breached.
The ICT Masterplan Journey nd IT Masterplan for Education: – built on foundation from MP1 to strive for an effective and pervasive use of ICT in education by, for example, strengthening the integration of ICT into the curriculum, establishing baseline ICT standards for students, and seeding innovative use of ICT among schools.
The ICT Masterplan Journey - 5
The ICT Masterplan Journey - 6
The ICT Masterplan Journey - 7 West Zone Sharing of Resources Project, WeSHARE in short, is such an example of a first digital repository project developed for teachers by teachers in The project has now been expanded to every zone as part of the Inter-cluster Sharing of Resources project, or iSHARE. To date, a total of 12,450 teachers have gone into iSHARE both to upload and use the more than 66,000 resources in the repository. These resources have been metatagged using the CEMS (Content Exchange Metadata Standard) that is based on Dublin Core Metadata Set.
The ICT Masterplan Journey - 8 Lessons learnt: A gap continues to exist between familiarity with ICT and translating this into effective teaching. Teachers need to still base effective outcomes on sound pedagogical principles when they use ICT tools to bring out a learning point.
The ICT Masterplan Journey - 9 Wrong use of technology: Students were asked to use the Internet to search for information. However, they were given only 15 minutes to discuss what they have found before they put up a PowerPoint presentation. Whats more important? Discussion or using technology? This use of ICT neither transforms nor enhances the learning experience. Better use of technology: There were well constructed lessons where teachers require their primary students to use simple technological tools like discussion forums to create and write stories that teachers and their classmates could critique.
The ICT Masterplan Journey - 10 Masterplan for ICT in Education 3 Learning Outcomes: – Strengthen competencies for self-directed learning – Tailor learning experiences according to the way that each student learns best – Encourage students to go deeper and advance their learning – Learn anywhere Source: 5/opening-address-by-dr-ng-eng-h-1.php
Polytechnics & Universities polytechnics in Singapore: – Singapore Polytechnic – Ngee Ann Polytechnic – Temasek Polytechnic – Nanyang Polytechnic – Republic Polytechnic All polytechnics and the Institute of Technical Education use e-learning widely. Every polytechnic has a dedicated department to look after the design and development of e-learning content and use of other ICT technologies like video conferencing. No centralization of e-learning contents.
Polytechnics and Universities - 2 Universities: – National University of Singapore (NUS) – Nanyang Technological University (NTU) – Singapore Management University (SMU) – SIM University (UniSIM) – Private University – Singapore University of Technology and Design (SUTD) Quality of e-learning is only one component in the Quality Assurance Framework for Universities (QAFU). All universities must go through this QAFU audit. QAFU was set up in Responsibility of e-Learning rests with the individual universitys department responsible for e-learning. There is very little sharing or collaboration amongst all the universities. Partly this is because of rivalry. Source:
Corporate Bodies -1 E-LEAP – E-Learning Early Adopters Programme: – Introduced by the IDA in – Purpose is to encourage companies and corporate bodies to use e-Learning for staff training – Funding provided for non-governmental companies to implement e-Learning E-Learning Competency Centre (ECC) helped to ascertain the courseware quality
Corporate Bodies - 2 Courseware Quality
Corporate Bodies - 3 Source: International Journal of The Computer, the Internet and Management Vol. 12 No. 2 (May - August, 2004) pp
Corporate Bodies , the ECC developed some Courseware Evaluation Guidelines for the E- LEAP. Guidelines are on gathering evaluation opinions on 10 criteria
Singapore Armed Forces – SPOT-ON – Introduced the Self-Paced On Time On Need (SPOT-ON) programme Intention is to tailor as much training as possible in accordance to individual needs, rate of learning and time of convenience Initially the move was more of distance learning packages using manuals and CD-ROMs When the Internet became popular, e-Learning becomes an extension of SPOT-ON.
Singapore Armed Forces – SPOT-ON the Battalion Tactics Course (BTC) went online – the National Service Command and Staff Course (NSCSC) went online. The Instructional Technology Centre at SAFTI developed a five-point management indicator to analyze the following: course results, internal validation reports, focus group surveys, interviews and the instructors feedback.
Singapore Armed Forces – SPOT-ON - 4 The PsExplanation PurposeThe effect of e-Learning on meeting the organization's aim and course objectives PlayersThe effect of e-Learning on the students and instructors PedagogyThe effect of e-Learning on adult education ProcessThe effect of e-Learning on training system processes ranging from curriculum design to delivery PerformanceThe quality of the end-product
Civil Service of the Singapore Government Some of the e-learning programmes for the civil servants are organized and offered by the Civil Service College of Singapore (http://www.ipamonline.com)http://www.ipamonline.com They also managed the quality & effectiveness of all the e- learning courses. Ministries and statutory boards can choose from over 200 titles in the comprehensive library of e-Learning courses. Public organisations implementing e-Learning with Open Academy can now enjoy a 50% development grant on courses.!
CIVIL SERVICE COLLEGE - IPAM OPEN ACADEMY
Model used in SIM University Quality & Effectiveness Model for UniSIM
Use of Kirkpatricks 4-level Model (done by ASTD in 1996) Level 1 (Reaction)92% Level 2 (Learning)34% Level 3 (Behaviour)11% Level 4 (Results)2% Most of the e-learning courses do have some sort of evaluations done by the learners but these tend to be at Level 1 of the Kirkpatricks Model. There is no available information on surveys done on the effectiveness.
What more can we do to improve e- Learning quality? Promote the awareness of research work on quality of e-learning e.g. the ISO/IEC JTC1 SC36 Working Group 5 on Quality assurance and descriptive frameworks Create collaborative projects involving quality and learning measurements Disseminate case studies on quality and learning measurements Share resources on quality and learning measurements Carry out quality audit of e-learning courses
What more can we do to improve e-Learning quality? Move from activity measures to performance measure Source: f/2/110303mgt-h%5b1%5d.pdf
Conclusion - 1 All organizations doing e-learning know the importance and value of checking the quality of e-learning. However, there is no consensus of how to go about doing it in a systematic way. Some organizations, especially the government bodies, are very interested in measuring the performance of participants who have completed e-learning courses. Schools are more interested in the pedagogical values that the students have attained. Quality checking and/or measurements of e-learning courses in the corporate bodies depend on the initiatives of their individual Human Resource Department. Much work need to be done in raising the need to monitoring the quality of e-learning programmes and also to measure it.
Conclusion - 2 From Robert Wager (1997): To rise from zero To big campus hero To answer these questions youll strive to answer the following questions: Where am I going, How shall I get there, and How will I know Ive arrived?