Characteristics of on-line formation courses. Criteria for their pedagogical evaluation Catalina Martínez Mediano, Department of Research Methods and Diagnosis in Education I UNED

2 Contents 1. Characteristic of the on-line courses 2. Users of on-line courses 3. Components of the on-line formation courses’ instructional design 4. Communication possibilities 5. Evaluation of the on-line formation courses by mean of survey technique

3 1. Characteristic of the on-line courses To be in mind the wide range of users and intentions, on-line professional formative courses, should be characterized by: Clear information to students about objectives and contents, prior knowledge and abilities required, learning methodology, evaluation criteria and computer characteristics. Quality of contents, developed in progression of difficult, with clarity in language, and activities and projects adapted to the objectives and contents.

4 1. Characteristic of the on-line courses Tools easily usable for administrators, professors, tutors and students, for registration, carry out learning activities, consulting, follow-up and recording of student’s results. Communication among all the users, to facilitate one collaborative learning environment. Access to learning resources. Criteria for evaluating the pedagogical and technological qualities, by means of rigorous instruments of easy application.

5 2. Users of on-line courses Users Duty Check the system operation, facilitate the access right to administrative staff, professors / tutors and students, carry out the pursuit of the activities and offer periodical results reports for decisions making of improvement. Educational institution Responsible for the courses offered, whose Government Council gets agreements on normative for instructional design and, for all. Programs, instructional design, follow-up and guidance learning activities, provide formative assessment and evaluation. Select the course that better adapts to its training necessities, study, carry out learning activities, consult and contribute to the courses. System administrators Professors and tutors Students

6 3. Components of the on-line formation courses’ instructional design The IDC should be dynamic in sense of being active to carry out a pursuit of the student's learning through activities: Offering information about their characteristics, duration, terms for their enrolment and realization. Informing about the course objectives, contents, learning methodology, level of achievement and evaluation criteria of performance. Develop the courses in connection with objectives and learning contents, with clarity, difficult in progression and activities and projects adapted to the objectives and contents.

7 3. Components of the on-line formation courses’ instructional design Instruments and means for checking learning progress, by means of self-evaluation performance tests. Student guidance, by means of confirmation of their advances in self-evaluation, by tutor. Orientation to the study by ‘Learning Guides’. Tools of consultation, to confirm the carried out of their activities, achievement in the course, and reorientation of the study and activities, if necessary.

8 3. Components of the on-line formation courses’ instructional design Communication tools among students, professor / tutor by means of electronic mail, forums, chats and learning communities. The platform should possess good characteristics in connection with the design of the interface, navigability, interactivity, and communication.

9 4. Communication possibilities Communication tools are essential for motivation and overcome of isolation’s student feelings. They facilitate to student: To carry out consultations on what is studying, to professor / tutor, through the electronic mail, forums and chats, as well as To carry out contributions on questions outlined by others in the forums and works in group, what will be able to be valued by the professor / tutor and to contribute for a complementary punctuation.

10 4. Communication possibilities Communication tools also facilitate to professor / tutor: To send messages to all the students through forums and electronic mail, participate in the forums, answer the consultations and focus the students’ learning. To incorporate complementary files, if necessary. To inform about the course objectives, contents, learning way, level of achievement and evaluation criteria of performance.

11 4. Communication possibilities Communication tools offer possibility to: The administrator, to study the information, contributing with reports on how goes the course, so that professors / tutors consider to introduce changes and improvements in the course in time. The study of different reports will allow to those responsible for the courses to redraw them by the light of the information and date analysis.

12 5. Evaluation of the on-line formation courses by survey technique The survey technique makes possible: to arrive to population's wide sample, embracing a great quantity and quality of opinions and valuations at one time, on the study topic because of: the relationship of short questions, its answer easiness, consistent in the valuation of the question in a range, here used, of 1 to 5, minimum to maximum agreement, the inclusion of open questions so that they can express their opinions about that they consider important and has not been asked.

13 According with that said, we elaborate a questionnaire With this six blocks: 5. Evaluation of the on-line formation courses by survey technique Information received about the course Course development, study orientation and evaluation Coordination, tutorial and guidance to the student Platform (interface, navigability, interactivity) General valuation of the course Proposals for possible future courses

14 The questionnaire 1.Information received by student for the course’s election 1.1. Were the objectives of the course clearly formulated? 1.2. Were the previous knowledge for the realization of the course specified? 1.3. Were the abilities to carry out the course specified? 1.4. Were the computer’s characteristics required for the course specified? 1.5. Was the course learning’s methodology specified? 1.6. Were the course evaluation’s criteria specified? 1.7. In connection with the course’s previous information, what would you add?

15 The questionnaire 2. Course development, learning orientation and assesstment The course’s contents were presented in a clear way? Are the contents of the course enough? Is the contents difficulty adapted to the course’s objectives? Is the course’s contents difficulty progression appropriate ? Did the course provide opportunities to carry out works in group? Are the instructions for practices realization in a comprehensible way? Did the course provide enough opportunities to carry out practices? Are the course’s evaluations coherent with the course’s evaluation criteria? Has the qualification obtained in the course fair? Has the qualification obtained in the course fair? Has the final work achieved the objective of knowledge’s consolidation in a concrete topic?

16 The questionnaire 3. Coordination, tutorial and guidance to the student On administration: Could you get guidance when you needed it? Has it been assisted in a satisfactory way? On coordinator and learning material’s delivery: Could you get guidance when you needed it? Has it been assisted in a satisfactory way On Tutor: Could you get guidance when you needed it? Has it been assisted in a satisfactory way? On tutorial media: Did you have opportunity to write messages by electronic mail? Was your electronic mail answered in a time interval appropriate? Was your electronic mail answered in a satisfactory way? Were face to face sessions carried out? Were video-conferences carried out? Did video-conferences satisfy your expectations?

17 The questionnaire 4. Platform characteristic (interface, navigability and interactivity) 4.1. Is information presented in an attractive way in the screen? 4.2. Are the icons easy to identify? 4.3. Is the information quantity presented in each screen suitable? 4.4. Do images adapt to the characteristics of your screen? 4.5. Do graphics adapt to the characteristics of your screen? 4.6. Is the screen flow speed appropriate? 4.7. Is it easy to choose the route that you want to take? 4.8. Is clear to know where are you in the course? 4.9. Does the system facilitate the group work among partners? Did it have learning documents? Were the learning resources accessible?

18 The questionnaire 5. General course valuation Has the course fulfilled your initials expectations? What global valuation would you give to the course? Do you find useful the time spent in answering this survey as it is allowing you to analyze the global result of the course? Did you enjoy the course? Would you enrol in another similar course? Would you recommend it to a friend?

19 The questionnaire 6. Proposals for possible future courses 6.1. Enumerate your topics of interest for possible future courses: 6.1. Enumerate your topics of interest for possible future courses: Any more comment that you believe important for the future of the course and don't have been asked?