InfoSkills2 FOLIO course Developing objectives and learning outcomes for the evaluation of learning InfoSkills2 FOLIO course

Introduction… In the last briefing, ‘Evaluating learning’ (http://www.nelh.nhs.uk/folio/infoskills2/learning.htm), we saw that one way to evaluate learning is by setting assignments for trainees To aid the evaluation of learning, it is useful to match the assessment criteria by which you will mark assignments with the objectives and learning outcomes of the training course Objectives, learning outcomes and assessment criteria should be created during the design of information skills training courses. For more information about designing training courses, see the ‘Designing information skills training courses’ briefing from the previous InfoSkills course at: http://www.nelh.nhs.uk/folio/infoskills/methods.htm

Objectives and learning outcomes So… what are objectives and learning outcomes and how do we create them?

What are objectives? Objectives are… “An expression in concrete and specific terms of what will happen on the course. The specific activities which participants will undertake on the course in order to achieve the stated aims” (1). “Note: not what will be learned, that comes next, but what will be done” (1). Objectives usually start with the phrase “By the end of this training course, participants will be familiar with…” (or similar)

Examples of objectives… For a literature searching course (2)… By the end of this session, participants will be familiar with: the names, coverage and content of the core health and health-related databases including Medline, CINAHL and Embase the types of information that can be identified through interrogation of these databases the process of developing a research question into an effective search strategy including the identification of keywords, concepts and synonyms the techniques and skills required to conduct a database search including the use of a controlled language thesaurus, truncation, boolean operators and explosion

Examples of objectives… For a ‘searching for health information on the internet’ course (2)… By the end of this session, participants will be familiar with: the various components of the Internet, such as electronic mail and the World Wide Web the basic structure of, and methods of navigating, the World Wide Web the types of information which are available through the internet the strengths and weaknesses of the Internet as an information resource criteria that can be used to evaluate the quality of internet-based information examples of resources which can be used to find evidence to support research and practice the techniques and skills needed to find resources on the internet; including the use of information gateways, search engines and resource lists

Examples of objectives… For a library induction By the end of this session, participants will be familiar with: the opening hours of the library the resources held by the library (books, journals, statistics etc) and how to locate them (including an introduction to the NLM classification scheme) their borrowing rights the services offered by the library (literature searching training, inter-library loans, enquiry desk etc) the contact details of the library (including email address, telephone number, web address)

What are learning outcomes? Learning outcomes (also known as key learning activities) are… “What will be learned. What the participants will be expected to achieve at the end of the course - knowledge, skills, and attitudes. Specifically what achieving the course objectives should have enabled the participants to learn” (1)

Writing learning outcomes (1) Learning outcomes should specify the minimum acceptable standards for students to pass the training course (3) Therefore, it is important that learning outcomes are expressed in terms of the essential learning for the information skills training course (3) Each information skills training course should have between four and eight key learning outcomes (3)

Writing learning outcomes (2) Learning outcomes should begin with the following phrase (or similar) (3): On successful completion of this training course, students will be able to…

Writing learning outcomes (3) Each learning outcome should be preceded by an action verb so that students are able to demonstrate that they have learned or achieved the outcome (3) Verbs relating to knowledge outcomes such as- ‘know’, ‘understand’, ‘appreciate’- tend to be rather vague and focus on the process that students have gone through (3) Instead, where possible, action verbs such as- ‘solve’, evaluate’, ‘use’, ‘analyse’- should be used to indicate how students can demonstrate their knowledge (3)

Writing learning outcomes (4) Bloom’s Taxonomy of 1956 is a good aid for writing learning outcomes (3) Bloom identified six categories of learning (3): 1. Knowledge 2. Comprehension 3. Application 4. Analysis 5. Synthesis 6. Evaluation The first two relate to knowledge and understanding, while the remaining for involve intellectual skills It may be tempting to concentrate on the first two categories for your information skills training session, but it is important to introduce the higher level activities where possible

Examples of learning outcomes: Stage 1… For an ‘information skills for medical students’ training course… Upon successful completion of this training course, participants will be able to:

Examples of learning outcomes: Stage 2… For an ‘information skills for medical students’ training course… Upon successful completion of this training course, participants will be able to: Understand Develop Conduct Evaluate and select Use Understand and evaluate

Examples of learning outcomes: Stage 3… For an ‘information skills for medical students’ training course [Based on (4)]… Upon successful completion of this training course, participants will be able to: Understand the information seeking process and its role in research Develop a search strategy for your research topic(s) Conduct subject searches and other search tactics Evaluate and select information Use traditional library sources Use on-line databases Use the WWW for healthcare/medical information Understand and evaluate the range of primary healthcare resources available to you

What are the benefits of learning outcomes? Courses which are designed using learning outcomes are student-centred (2) Designing courses in this way marks a shift from the content of a module or course (i.e. what trainers teach) towards its outcome (i.e. what the students are able to do upon successful completion of the course) (2)

What are the benefits of learning outcomes? Learning outcomes can… Help to guide students in their learning because they explain what is expected of them (3). Help trainers to focus on what they want students to achieve as a result of the training course provide a useful guide to inform potential students and employers about the knowledge and understanding that a graduate of the training course will possess (3)

What about assessment criteria? Now that we know what objectives and outcomes are, we need an understanding of assessment criteria…

Assessment criteria “Assessment criteria describe how well a student has to be able to achieve the learning outcome in order to be awarded a particular grade” (5). “Assessment criteria should test, assess or relate to the learning that is mentioned in the learning outcomes” (6) Assessment criteria are important because they ensure that marking is fair and standard as all students work is marked to the same criteria (5) For an example of assessment criteria for an information literacy course, see: http://www.nelh.nhs.uk/folio/infoskills2/criteria.doc For detailed information about how to create assessment criteria, see ‘Designing and using assessment criteria’ (Oxford Brookes University) at: http://www.brookes.ac.uk/services/ocsd/firstwords/fw24.html

Linking objectives, learning outcomes and assessment criteria to the evaluation of learning When setting assignments or tests for trainees, the tasks that you set should have a clear alignment with the objectives, learning outcomes and the assessment criteria for the course (3) This enables you to design appropriate tasks and to deliver the training course in a way that enables students to reach the required outcomes (3) It also enables you to ensure that there is coherence in your information skills training session (3) The alignment between objectives, learning outcomes, assessment tasks and assessment criteria should be made transparent to the students to help them to successfully attain the required knowledge and skills (3)

References West Midlands Modular Trainers Course. 2004. Curriculum. http://www.trainer.org.uk/members/theory/planning/curriculum.htm Alison Hicks. ‘Developing information skills training for National Health Service personnel: experiences at the Trent Institute for Health Services Research’. Program, 1998. Vol 32, No. 2; pp 123-136 University of Central England in Birmingham. 2004. Guide to learning outcomes. http://lmu.uce.ac.uk/outcomes/UCE%20Guide%20to%20Learning%20Outcomes.pdf

References University of Strathclyde. 2004. GAELS. http://gaels.lib.strath.ac.uk/info_skills Oxford Brookes University. 2001. 2.4 Designing and using assessment criteria. http://www.brookes.ac.uk/services/ocsd/firstwords/fw24.html Jenny Moon. Linking Levels. Learning Outcomes and Assessment Criteria. http://www.liv.ac.uk/~cll/files/LinkingLevelsplusasscrit.doc Exeter University

Further reading Chris Butcher. 2004. Bloom’s Taxonomy. SDDU University of Leeds. http://www.leeds.ac.uk/sddu/online/bloom.htm E.A. Hesketh. 2004. Aims and objectives. NHS Education for Scotland. http://www.nes.scot.nhs.uk/courses/ti/AimsObjectives.pdf Janet Peters. 2004.Learning outcomes and information literacy. SCONUL University of Aberdeen. 1997. Guidance on aims and objectives for teaching and learning. http://www.abdn.ac.uk/secretariat/aimsobs.hti University of Portsmouth. 2004. Aims and objectives explained: getting the most out of units. http://www.port.ac.uk/departments/academic/edam/resources/intranet/studyguide/filetodownload,3626,en.pdf