1. Outcomes-Based Programme Development and Quality Assurance at Masaryk University1 Outcomes-Based Programme Development and Quality Assurance at Masaryk University Soňa Basovníková Jiří Nantl DECOWE International Conference 24 – 26 September 2009, Ljubljana, Slovenia

2. Outcomes-Based Programme Development and Quality Assurance at Masaryk University2 Trends in Tertiary Education: Why do we introduce Qualifications Framework and Learning Outcomes? overall move towards universal higher education changes in students structure changes in structure of higher education study programmes various perceptions on purposes and content of higher education in general growing demands of public authorities on accountability and possibility to assess and measure higher education‘ s outcomes internationalization and mobility

3. Outcomes-Based Programme Development and Quality Assurance at Masaryk University3 Qualification, qualification frameworks, learning outcomes: Qualification - a formal certification of learning outcomes achieved. Qualification frameworks – generic description of competencies/learning outcomes that a student must demonstrate in order to reach a qualification. learning outcomes - statements of what a learner is expected to know, understand and/or be able to demonstrate after a completion of a process of learning (e.g. course, module, programme) Adelman C. Bologna Process For US Eyes. 2009

4. Outcomes-Based Programme Development and Quality Assurance at Masaryk University4 Qualifications framework A „warranty“ of higher education (C. Adelman): Students must demonstrate to the HEI that they have achied the intended learning outcomes Institutions must demonstrate to the government that their students have demonstrated the achievement of intended learning outcomes It is designed not only for HE institutions, academic staff, and students, but also the government and general public A tool for understanding what the higher education is meant to be about

5. Outcomes-Based Programme Development and Quality Assurance at Masaryk University5 Role of academic staff in the use of qualification frameworks Academic staff has an irreplaceable responsibility for creation, development and assessment of study programmes: they formulate objectives of the study programme and translate them into measurable learning outcomes they choose corresponding teaching methods and they assess students‘ performance

6. Outcomes-Based Programme Development and Quality Assurance at Masaryk University6 Programme‘s objectives/aims vs. learning outcomes (1/2) Programme‘s objectives are broad general statements comprising: philosophy, values, and theoretical-methodological background of the discipline; identification of those students for which study programme is suitable; graduates profile and their main characteristics; possibilities of employment Programme‘s objectives should be formulated in language comprehensible for prospective students and their parents. Usually their are not measurable.

7. Outcomes-Based Programme Development and Quality Assurance at Masaryk University7 Programme‘s objectives/aims vs. learning outcomes (2/2) Learning outcomes (LOs) represent operationalization of programme‘s objectives/aims They are formulated with „active verbs“ (Bloom‘s taxonomy) LOs are measurable There is an alignment among LOs, teaching and leaning methods, and assessment Intended LOs refer to minimum threshold requirements on students (how well student fulfil these requirement is reflected in assessment criteria and grading scale)

8. Outcomes-Based Programme Development and Quality Assurance at Masaryk University8 Advantages of learning outcomes: help students to reflect their own learning and therefore learn more effectively help teachers to tell students more precisely what is expected from them provide information to prospective students and employers about key graduates competencies facilitate comparison of individual study programmes for purposes of international recognition together with the criterion of workload enable to determine credit value of a given period of study clearly define study programme for purposes of accreditation and evaluation

9. Outcomes-Based Programme Development and Quality Assurance at Masaryk University9 Formulating learning outcomes (1/4) They should be formulated as a result of an explicit agreement of concerned academic community and their idea about study content and typical graduate They should take into consideration trends in a given discipline and expectations of the society Decision on what won‘t be included in the curriculum is as much important as a decisions on what will be the content

10. Outcomes-Based Programme Development and Quality Assurance at Masaryk University10 Formulating learning outcomes(2/4) LOs represent measurable knowledge, understanding or abilities which are the object of assessment LOs are designed on two levels: programme and course LOs do not have to (and cannot) cover everything what student will learn during period of study Each compulsory course should contribute to one of study programme‘ s learning outcome

11. Outcomes-Based Programme Development and Quality Assurance at Masaryk University11 How formulate learning outcomes (3/4) Begin each LO with an action verb, followed by the object of the verb followed by a phrase that gives the context. Use only one verb per LO, avoid complicated sentences. Avoid vague terms like know, understand, learn… Ensure that the LOs of the course relate to the overall outcomes of the programme. LOs must be observable and measurable and capable of being assessed. Ask yourself if it is realistic to achieve the LOs within the time and resources available. Before finalising the LOs, ask your colleagues and possibly former students if the learning outcomes make sense to them. Kennedy, Hyland, Ryan: "Writing and Using Learning Outcomes: a Practical Guide"

12. Outcomes-Based Programme Development and Quality Assurance at Masaryk University12 Alignment of the study programme‘s objectives and course‘s structure: Formulation of study programme‘s objectives Formulation of learning outcomes Choose appropriate teaching and assessment methods Alignment to the structure of courses

13. Outcomes-Based Programme Development and Quality Assurance at Masaryk University13 Study programme‘s assessment at Masaryk University: Study programme‘s assessment is part of student- centered approach Programme level assessment covers not only academic quality but also sustainability of the programme, and its contribution to the profile and mission of the university Part of the university strategy since 2005, pilot projects in 2008, a university-wide approach since 2009 A continual process involving all the academic staff (no bureaucratic exercise)

14. Outcomes-Based Programme Development and Quality Assurance at Masaryk University14 Principles of study programme‘s assessment Formative assessment – each programme should be given feedback so as to enable improvement. There should be no „ranking“ or „rating“ of programmes within university. Non-bureaucratic approach – quality assurance and monitoring are an integral part of academic staff job description (under university regulations since 2008) Balanced assessment – the self assessment report by the programme is reviewed by a group of external examiners (graduates, employers, and academic colleagues)

15. Outcomes-Based Programme Development and Quality Assurance at Masaryk University15 Process of study programme evaluation (1/5) Self-evaluation report contains: Strategic goals: strategy of study programmes‘ s development for next 5 years (human and material resources, competitiveness, etc.) Educational goals: what a graduate is expected to know, understand and/or be able to demonstrate after a completion of a process of learning Self-assessment report is to reflect strenghts, weaknesses, opportunities and threats to the programme, using the SWOT analysis technique

16. Outcomes-Based Programme Development and Quality Assurance at Masaryk University16 Process of study programme evaluation(4/5) Self-assessment report is subject to a review by group of external examiners: An academic colleague outside the university (or, exceptionally, from outside the faculty involved) A graduate (or, exceptionally, a student) An employer A representative of the university leadership (vice rector, or a member of the Study Programmes Board)

17. Outcomes-Based Programme Development and Quality Assurance at Masaryk University17 Process of study programme evaluation (5/5) At the end of the assessment procedure, a joint meeting of the self-assessment team and external examiner enables the fair exchange of views Pilot projects in 2008 showed that many academic staff sought support from external examiners views to foster curricular and didactic changes unpopular with the rest of the faculty Final recommendation submitted to the faculty management should be, as far as possible, based upon understanding between the self-assessment team and external examiner

18. Outcomes-Based Programme Development and Quality Assurance at Masaryk University18 Benefits of study programme`s assesment Increased transparency of the study programme: helps students to follow learning process and academic staff to clear up their aims Verification of study programme‘s consistency and logics – identification of possible gaps or overlaps Setting strategic goals for study programmes Not a ranking but focus on reflection, recommendations for development and improvement

19. Outcomes-Based Programme Development and Quality Assurance at Masaryk University19 Thank you for your attention Soňa Basovníková Head, Office for Strategy basovnikova@rect.muni.cz Jiří Nantl Chief Legal and Policy Officer nantl@rect.muni.cz