1. Assessment of Learning Outcomes in Engineering - an International Perspective Mona Dahms mona@plan.aau.dk UCPBL, Aalborg University

2. aloe conference 20082 Presentation Outline 1.Curriculum Development and Assessment 2.An International Perspective 3.The Situation in Denmark

3. aloe conference 20083 Definitions Learning outcomes: ”sets of competences, expressing what the student will know, understand or be able to do after completion of a process of learning” (ECTS Guide 2005); describes the end product of learning Assessment: Judgement of students’ performance, i.e. judgement of whether students have achieved the learning outcomes Evaluation: Judgement of quality of study programs and activities

4. aloe conference 20084 Curriculum Development (CD) – a sequential process Design of Nth sem. Competence profile of graduate Design of study program Design of 1st sem. Design of 2 nd sem. Design of 3rd sem. Design of 4th sem. Study activity 1 Study activity 2 Study activity 3 Study activity 4 Study activity 5

5. 5 Teaching, learning and assessment Competence profile of graduate Design of study program Design of semester program Design of study activity Evaluation of study activity Evaluation of semester program Evaluation of study program Evaluation of graduate competences This is where assessment is carried out!!! CD – a top-down process

6. aloe conference 20086 Cowan, 2004 = learning outcomes The Cowan Curriculum Development model

7. aloe conference 20087 Constructive alignment Learning outcomes Assessment Learning/teaching “Alignment is about trust in the relationship between student and teacher” (Derick)

8. 8 What is assessment? setting criteria (LOs) collecting evidence making judgement and possibly providing feed-back for improvement  Example : The customer tastes the food served and...... setting criteria collecting evidence making judgement and always providing feed-forward for further improvement Example : The chef tastes the food to be served and......... FormativeSummative

9. aloe conference 20089 Assessment Important questions to consider are: –WHO sets criteria? –WHEN and WHERE is evidence prepared? –HOW is evidence presented? –WHO makes the judgement? –WHO provides feed-forward/feed-back?

10. aloe conference 200810 Changing competences… remembering understanding applying analysing evaluating creating Lower level competences Higher level competences Declared learning outcomes Learning outcomes assessed …necessitates changes in assessment

11. aloe conference 200811 FailPass Fail Pass Exam Traditional assessment Jakobsen and Rump, 1998 Good understanding Failed examination: 0 % Good understanding Passed examination: 25 % Poor understanding Failed examination: 25 % Poor understanding Passed examination: 50 % !!!! Understanding

12. aloe conference 200812 The International Perspective Malaysia (2006?) and South Africa (1997) have both introduced Outcomes Based Education (OBE) at all levels

13. aloe conference 200813 Malaysian Qualification Framework (MQF) Learning Outcomes 1. knowledge of discipline areas 2. practical skills 3. social skills and responsibilities 4. value, attitude and professionalism 5. communication, leadership and team skills 6. problem solving and scientific skills 7. information management and life-long learning skills 8. managerial and entrepreneurial skills “The challenge to us is in drafting assignments, projects and exams as means of evaluating the MQF learning outcomes.”

14. aloe conference 200814 South African Critical Cross-Field Learning Outcomes 1.Identify and solve problems and make decisions using critical and creative thinking 2.Work effectively with others as members of a team, group, organisation and community 3.Organise and manage themselves and their activities responsibly and effectively 4.Collect, analyse, organise and critically evaluate information 5.Communicate effectively using visual, symbolic, and/or language skills in various modes 6.Use science and technology effectively and critically showing responsibility towards the environments and the health of others 7.Demonstrate an understanding of the world as a set of related systems by recognising that problem-solving contexts do not exist in isolation

15. aloe conference 200815 Achievements in SA ? A study of critical thinking abilities of 1st year education students documented deficient critical thinking abilities Possible reasons: Teaching practice; educational change; socio-cultural environment; language ability (Lombard & Grosser 2008)

16. aloe conference 200816

17. aloe conference 200817 Skills Kolmos and Holgaard 2007

18. aloe conference 200818 Skills Kolmos and Holgaard 2007

19. aloe conference 200819 Skills Kolmos and Holgaard 2007

20. aloe conference 200820 Skills Kolmos and Holgaard 2007

21. aloe conference 200821 Statements on team based exams “ I dread a transition to an individual exam because I believe that this would result in a significant decline in the academic level. Besides I am of the opinion that the team based exam is the type of exam which best relate to the working environment which the students will be in later on. ” (External examiner) “ I think it is important that the members of the group are tested in each others ’ co-presence and in a collaborative process. It opens the possibility for partially evaluating their team play and co-operation, ability to stand out in a group, ability to follow up and to have an effect on a discourse. ” (External examiner) Kolmos and Holgaard 2007

22. aloe conference 200822 Conclusion Individual assessment does not test complex knowledge and collaborative competences (Kolmos and Holgaard 2007)

23. Thank you for your attention! Comments and questions are very welcome at: mona@plan.aau.dk