1. King Saud University, College of Science Workshop: Programme accreditation and quality assurance Riyadh, June 13-14, 2009 II.2 From educational objectives to learning outcomes www.asiin.de

2. Content 1.Principles and definitions 2.Reference frameworks for learning outcomes 3.From educational objectives to learning outcomes 4.Exercise 1: Defining programme learning outcomes 5.Exercise 2: Defining module learning outcomes 2

3. Learning outcomes: Definition Learning outcomes are statements of what a student should know, understand or be able to do at the end of a learning activity.  The learning activity could be a lecture, a lab, a seminar, a module (combining different forms of teaching and learning) or an entire programme.  Learning outcomes must be simply and clearly described.  Learning outcomes must reflect the level of capability as well as the range.  Learning outcomes must be capable of being validly assessed. Learning outcomes are statements of what a student should know, understand or be able to do at the end of a learning activity.  The learning activity could be a lecture, a lab, a seminar, a module (combining different forms of teaching and learning) or an entire programme.  Learning outcomes must be simply and clearly described.  Learning outcomes must reflect the level of capability as well as the range.  Learning outcomes must be capable of being validly assessed. 3

4. Learning outcomes: Definition There is a distinction between  intended learning outcomes (ILOs), i.e. written statements in a course/programe syllabus, and  achieved learning outcomes (ALOs), i.e. those results that students actually have (and can be demonstrated to have) achieved. There is a distinction between  intended learning outcomes (ILOs), i.e. written statements in a course/programe syllabus, and  achieved learning outcomes (ALOs), i.e. those results that students actually have (and can be demonstrated to have) achieved. 4

5. Correspondence of ILOs and ALOs ILOs learning outcomes / competence profile knowledge, skills, competencies job / career profiles occupational area(s) work environment(s) ILOs learning outcomes / competence profile knowledge, skills, competencies job / career profiles occupational area(s) work environment(s) Input of HEI support processes e.g. student services, staff resources, infrastructure, programme structure, curriculum, didactic concept, quality assurance Input of HEI support processes e.g. student services, staff resources, infrastructure, programme structure, curriculum, didactic concept, quality assurance ALOs correspondence of educational objectives and learning outcomes results of outcomes assessment and internal/external evaluation ALOs correspondence of educational objectives and learning outcomes results of outcomes assessment and internal/external evaluation Assessment of the process: coherence of goals, input + outcomes 5

6. Content 1.Principles and definitions 2.Reference frameworks for learning outcomes 3.From educational objectives to learning outcomes 4.Exercise 1: Defining programme learning outcomes 5.Exercise 2: Defining module learning outcomes 6

7. Learning outcomes: Reference Frameworks Learning outcomes for degree programmes and modules/courses should refer to established qualification frameworks to ensure compatibility, for example:  European Qualification framework for Higher Education (“Dublin Descriptors”).  European Qualification framework for Chemistry (“Budapest Descriptors”, ECTNA Eurobachelor/Euromaster).  ASIIN e. V. qualification framework for degree programmes in engineering, informatics, the natural sciences and mathematics.  GCC qualification framework FN. FN  … Learning outcomes for degree programmes and modules/courses should refer to established qualification frameworks to ensure compatibility, for example:  European Qualification framework for Higher Education (“Dublin Descriptors”).  European Qualification framework for Chemistry (“Budapest Descriptors”, ECTNA Eurobachelor/Euromaster).  ASIIN e. V. qualification framework for degree programmes in engineering, informatics, the natural sciences and mathematics.  GCC qualification framework FN. FN  … 7

8. The Bologna-Process: System of qualification frameworks Learning outcomes - Programme level LO:s COURSE/MODULE LEVEL LO:s TASKLEVEL LO:s TASKLEVEL LO:s TASKLEVEL LO:s TASKLEVEL LO:s TASKLEVEL LO:s TASKLEVEL Learning outcomes - National level National qualification descriptors e. g.: Knowledge and understanding Skills and competences Learning outcomes - European level Dublin Descriptors 8

9. Learning outcomes and mobility Country A Country B 9

10. Professional and academic mobility – A European approach Completion of the European Higher Education Area until 2020, in which citizens can choose from a wide and transparent order of high quality courses and benefit from smooth recognition procedures Goals: European knowledge society characterized by high mobility and permeability, lifelong learning Promotion of comparability and compatibility of degrees: Two cycle study system, ECTS, adoption of a system of easily readable and comparable degrees, Diploma Supplement, permeability of study structures Promotion of the European Dimension in Higher Education Recognition of degrees Transnational cooperation in QA Common European Market Goals: freedom of movement of workers, right of establishment, freedom of services. Declaration of Lisbon, Barcelona: „to make Europe the most competitive and dynamic knowledge-based economy in the world.“ The European Directive on the recognition of professional qualifications : Goal: „ A clear, secure and quick system for the recognition of qualifications in the field of the regulated professions is required to ensure free movement.“ The EC institutions and member states should facilitate employment and the provision of services through wholesale consolidation of the existing regimes of professional recognition in the regulated professions. Paris (1998) Bologna (1999) Prag (2001) Berlin (2003) Bergen (2005) London (2007) Leuven (2009) Promotion of comparability and compatibility of degrees: Three cycle study system, ECTS, Diploma Supplements, Permeability of study structures Common European Market Goals: freedom of movement of workers, right of establishment, freedom of services. Declaration of Lisbon, Barcelona: „to make Europe the most competitive and dynamic knowledge-based economy in the world.“ The European Directive on the recognition of professional qualifications : Goal: „ A clear, secure and quick system for the recognition of qualifications in the field of the regulated professions is required to ensure free movement.“ Professional Mobility 10 Completion of the EHEA until 2020: choice of a wide variety of high quality courses; benefits of smooth recognition procedures Goals: European knowledge society characterized by high mobility and permeability, lifelong learning Academic Mobility

11. 11 ASIIN e. V. qualification framework: Definitions “Qualification”: Formal outcome of an assessment and validation process. “Learning outcomes”: Statements of what a learner knows, understands and is able to do on completion of a learning process, defined in terms of knowledge, skills and competences. “Knowledge”: Outcome of the assimilation of information through learning (theoretical and / or factual knowledge). “Skills”: Ability to apply knowledge to complete tasks and solve problems (cognitive skills such as logical, intuitive and creative thinking and practical skills such as manual dexterity and the use of methods, materials, tools and instruments). “Competence”: Proven ability to use knowledge, skills and personal, social and / or methodological abilities, in work or study situations and in professional and / or personal development. “Qualification”: Formal outcome of an assessment and validation process. “Learning outcomes”: Statements of what a learner knows, understands and is able to do on completion of a learning process, defined in terms of knowledge, skills and competences. “Knowledge”: Outcome of the assimilation of information through learning (theoretical and / or factual knowledge). “Skills”: Ability to apply knowledge to complete tasks and solve problems (cognitive skills such as logical, intuitive and creative thinking and practical skills such as manual dexterity and the use of methods, materials, tools and instruments). “Competence”: Proven ability to use knowledge, skills and personal, social and / or methodological abilities, in work or study situations and in professional and / or personal development. Based on the European Qualification Framework

12. 12 ASIIN e. V. QF: Bachelor graduates...

13. 13 ASIIN e. V. QF: Bachelor graduates...

14. 14 ASIIN e. V. QF: Master graduates have...

15. 15 ASIIN e. V. QF: Master graduates have...

16. Benefits of learning outcomes Help to explain more clearly to students what is expected of them and thus help to guide them in their studies. Help teachers to focus more clearly on what exactly they want students to achieve in terms of knowledge and skills. Help teachers to define the assessment criteria more effectively. Help to provide guidance to employers about the knowledge and understanding possessed by graduates of programmes. Important instruments in achieving comparability and transparency of qualifications and facilitating mobility (of students and graduates/professionals). Help to explain more clearly to students what is expected of them and thus help to guide them in their studies. Help teachers to focus more clearly on what exactly they want students to achieve in terms of knowledge and skills. Help teachers to define the assessment criteria more effectively. Help to provide guidance to employers about the knowledge and understanding possessed by graduates of programmes. Important instruments in achieving comparability and transparency of qualifications and facilitating mobility (of students and graduates/professionals). 16

17. Potential problems with learning outcomes Could limit learning if learning outcomes written within a very narrow framework – lack of intellectual challenge to learners. Danger of assessment-driven curriculum if learning outcomes are too confined. Could give rise to confusion among students and staff if guidelines not adhered to when drawing up learning outcomes, etc. Must reflect established national and cultural standards and practices in order to be accepted and useful. Could limit learning if learning outcomes written within a very narrow framework – lack of intellectual challenge to learners. Danger of assessment-driven curriculum if learning outcomes are too confined. Could give rise to confusion among students and staff if guidelines not adhered to when drawing up learning outcomes, etc. Must reflect established national and cultural standards and practices in order to be accepted and useful. 17

18. Content 1.Principles and definitions 2.Reference frameworks for learning outcomes 3.From educational objectives to learning outcomes 4.Exercise 1: Defining programme learning outcomes 5.Exercise 2: Defining module learning outcomes 18

19. Student-centred approach Learning outcomes place an emphasis…  on the learner, and  on the learner’s ability to do something. Learning outcomes place an emphasis…  on the learner, and  on the learner’s ability to do something. Traditional approach: focus on teaching – description of aims and objectives and use of terms like “know”, “understand”, “be familiar with”. Traditional approach: focus on teaching – description of aims and objectives and use of terms like “know”, “understand”, “be familiar with”. Learning outcomes approach: Focus on what you want the student to be able to – use of terms like “define”, “list”, “name”, “recall”, “analyse”, “calculate”, “design”, etc. Learning outcomes approach: Focus on what you want the student to be able to – use of terms like “define”, “list”, “name”, “recall”, “analyse”, “calculate”, “design”, etc. 19

20. 20 From Educational Objectives to Learning Outcomes Correspondence of educational objectives, programme learning outcomes and module learning outcomes. What is characteristic for graduates, what will they do in which environment? How do the objectives translate into a detailed competency profile of the graduates? How does the curriculum support learning outcomes and programme objectives? 20

21. From programme objectives... The overall objective of the undergraduate programme “Chemical and Metallurgical Process” is to educate professionals who are able to plan, control and monitor processes as well as to develop and operate chemical and metallurgical processes. In particular they should supervise processes, design and develop laboratory tests, identify and analyse problems and create and implement solutions. They should manage resources effectively and work in teams in an effective and creative manner. Students should be committed to quality, environ- mental protection and safety as well as to ethics. The overall objective of the undergraduate programme “Chemical and Metallurgical Process” is to educate professionals who are able to plan, control and monitor processes as well as to develop and operate chemical and metallurgical processes. In particular they should supervise processes, design and develop laboratory tests, identify and analyse problems and create and implement solutions. They should manage resources effectively and work in teams in an effective and creative manner. Students should be committed to quality, environ- mental protection and safety as well as to ethics. 21

22. ...to programme learning outcomes (1) Graduates should be able to operate and control chemical and metallurgical processes based on their knowledge of chemistry and of unit operations. They also should be able to apply their knowledge of chemistry, physics, mathematics and technology in chemical and metallurgical processes, to perform chemical analyses and metallurgical tests, and interpret results in order to improve processes. They should also be capable of designing chemical and metallurgical processes and of identifying, analysing and solving technical problems. Graduates should be able to operate and control chemical and metallurgical processes based on their knowledge of chemistry and of unit operations. They also should be able to apply their knowledge of chemistry, physics, mathematics and technology in chemical and metallurgical processes, to perform chemical analyses and metallurgical tests, and interpret results in order to improve processes. They should also be capable of designing chemical and metallurgical processes and of identifying, analysing and solving technical problems. 22

23. ...to programme learning outcomes (2) Furthermore, graduates should be able to work in teams, to know contemporary aspects of their profession and be prepared for lifelong learning. They should also apply environmental protection standards and demonstrate an ethical awareness towards their work. Furthermore, they should be able to manage both material and human resources effectively. Furthermore, graduates should be able to work in teams, to know contemporary aspects of their profession and be prepared for lifelong learning. They should also apply environmental protection standards and demonstrate an ethical awareness towards their work. Furthermore, they should be able to manage both material and human resources effectively. 23

24. From Educational Objectives to Learning Outcomes: Example Educational Objective: Graduates are team oriented Programme Learning Outcome: Graduates are able to function effectively as member of a team that may be composed of different genders or cultural backgrounds Module Learning Outcome (e.g. for a module “Team Project”) : Knowledge on: Methods and instruments of information research supported by electronic media; presentation and moderation techniques and their application; project planning tools. Skills: They are able to… -gather information on a specific problem relevant for their project team and -communicate this information effectively; -assume the designed function within a team and independently fulfil the responsibilities associated with it; -accept the function and the personalities of other team members; -actively share their knowledge with other team members; -listen to other team members. 24

25. From Educational Objectives to Learning Outcomes: Example 25 Educational Objective: Graduates take legal and ethical considerations into account when taking decisions Programme Learning Outcome: Graduates are aware of the social and ethical responsibilities that underpin their actions, and of the professional ethical principles and standards that apply to their chosen discipline, and able to apply them. Module Learning Outcome (e.g. as part of a module “Software Engineering”): Knowledge on: Codes of ethics, laws, standards and guidelines relevant to software engineering; positions in the current discussion on software engineering and its impact on society. Skills: They are able to… -make ethical decisions when faced with ethical dilemmas, with reference to general principles of ethics as well as codes of ethics for software engineering; -describe and apply the laws that affect software engineers, including laws regarding copyright, patents, and other intellectual property; -describe the effect of software engineering decisions on society, the economy, the environment, their customers, their management, their peers, and themselves; -understand the role of standards and standards-making bodies in software engineering.

26. The task of writing learning outcomes Learning outcomes should be clearly written so that they are understood by students, colleagues and other relevant stakeholders. When writing learning outcomes it may be helpful to focus on what you expect students to be able to demonstrate upon completion of the module or programme. Focus on “specialist skills” as well as on “social skills”. Avoid complicated sentences. If necessary use more than one sentence to ensure clarity. Learning outcomes should be clearly written so that they are understood by students, colleagues and other relevant stakeholders. When writing learning outcomes it may be helpful to focus on what you expect students to be able to demonstrate upon completion of the module or programme. Focus on “specialist skills” as well as on “social skills”. Avoid complicated sentences. If necessary use more than one sentence to ensure clarity. 26

27. The task of writing learning outcomes List the learning outcomes using a phrase like “On successful completion of this module, students should be able to: … [list of learning outcomes].” General recommendation: 5 to 8 learning outcomes per module. Avoid certain words……….  The key word is DO and the key need in drafting learning outcomes is to use active verbs.  Avoid ambiguous verbs like “understand”, “be familiar with”, “be exposed to“, “be aware”, “appreciate”… List the learning outcomes using a phrase like “On successful completion of this module, students should be able to: … [list of learning outcomes].” General recommendation: 5 to 8 learning outcomes per module. Avoid certain words……….  The key word is DO and the key need in drafting learning outcomes is to use active verbs.  Avoid ambiguous verbs like “understand”, “be familiar with”, “be exposed to“, “be aware”, “appreciate”… 27

28. Content 1.Principles and definitions 2.Reference frameworks for learning outcomes 3.From educational objectives to learning outcomes 4.Exercise 1: Defining programme learning outcomes 5.Exercise 2: Defining module learning outcomes 28

29. 29 From Educational Objectives to Learning Outcomes Correspondence of educational objectives, programme learning outcomes and module learning outcomes. What is characteristic for graduates, what will they do in which environment? How do the objectives translate into a detailed competency profile of the graduates? How does the curriculum support learning outcomes and programme objectives? 29

30. 30 Exercise 1: Programme learning outcomes a)Group: Chose one of your department’s undergraduate programmes and write down the educational objectives. (10 minutes). b)Individual: Formulate 4 key learning outcomes for the selected programme, based on its educational objectives (adhesive labels). (10 minutes) c)Group: The group members collect their proposals on a joint sheet (5 minutes). d)Group: Jointly sort the proposals and group them by content/topic (10 minutes). e)Group: Formulate one programme learning outcome for each group of proposals and thus create a joint list (10 minutes). Please be prepared to briefly present your result to the plenary. 45 minutes group workTime available:

31. 31 You have formulated programme learning outcomes based on its educational objectives. You have experienced a typical situation of a programme coordinator who practices outcome- oriented curriculum development. Objectives: Exercise 1: Programme learning outcomes

32. 32 - Break -

33. Content 1.Principles and definitions 2.Reference frameworks for learning outcomes 3.From educational objectives to learning outcomes 4.Exercise 1: Defining programme learning outcomes 5.Exercise 2: Defining module learning outcomes 33

34. 34 From Educational Objectives to Learning Outcomes Correspondence of educational objectives, programme learning outcomes and module learning outcomes. What is characteristic for graduates, what will they do in which environment? How do the objectives translate into a detailed competency profile of the graduates? How does the curriculum support learning outcomes and programme objectives? 34

35. 35 Exercise 2: Module/course learning outcomes a)Group: Chose one of the programme learning outcomes you have just formulated and follow the instructions b) to e) b)Individual: Formulate knowledge and skills that describe the chosen learning outcome in more detail (adhesive labels). Question: What does one have to be able to do to demonstrate that the learning outcome has been achieved? (10 minutes) c)Group: The group members collect their proposals on a joint sheet (5 minutes). d)Group: Jointly sort the proposals and group them by content/topic (10 minutes). e)Group: Formulate/sum up one skill for each group of proposals and thus create a joint list of skills (15 minutes). Please be prepared to briefly present your result to the plenary. 40 minutes group workTime available:

36. 36 Exercise 2: Module/course learning outcomes You have transformed a learning outcome at the programme level into concrete and measurable learning outcomes at the level of a specific teaching module. You have experienced a typical situation of a programme coordinator who practices outcome- oriented curriculum development. Objectives:

37. 37 Exercise 2: Module/course learning outcomes Did you find these exercises challenging? In how far?