Roles and types of examination ITU, October 8, 2010 Hanne Leth Andersen, RUC

“Assessment methods [….] probably have a greater influence on how and what students learn than any other single factor.” (Boud 1988) “Students can avoid bad teaching: they can’t avoid bad assessment.” (Boud 1994)

Framework Exam systems and national policies Institution based or externally controlled exams Authority: testing agency or ministry Assurance of independency and criterion based evaluation Assessment and censorship Oral and written exams: finance and reliability

Exam typology: Parameters 1.Medium: oral – written – combination 2.Control: set questions – free – combination 3.Approach: problem based – syllabus based 4.Materials allowed: none, certain, notes, all 5.Preparation: without – with (+ time) 6.Examination time 7.Number of students: individual – group 8.Evaluator: teacher – co-examiner – student 9.Focus: process/product/presentation orientation

Exam types Oral, written and practice related assignments (one or several) Participation in teaching, courses, seminars,... Combinations of 1-2 Practice University act, Examination order, §3,2

Criteria for choosing exam forms Relevance – The goal is to judge if and to what extend student qualifications are in accordance with objectives, competences and academic requirements established in the Ministerial Order on Education, the study program etc. (Ministerial order on Examinations, §2) Validity – Alignment between formulated objectives, measurable operational demands at exams and assessment criteria Reliability – Precise and consistent (fair) assessment (oral vs. written) Economy

Ressources Mono-dimensional exam forms: basic skills and knowledge Poly-dimensional exam forms: overall (often multi disciplinary) objectives (analysis, synthesis) Combined exam forms Uddannelsesstyrelsen 2004

Competences as learning objectives Critical approach Independence and autonomy Collaboration Structuring and planning Problem solving and project work Oral and written presentation Creativity Peer and self evaluation

Alignment Unaligned Course Aligned Course Teacher’s intention Student’s activity Exam assessment Teacher’s intention Student’s activity Exam assessment  e.g. - explain - relate - prove - apply e.g. - memorize - describe e.g. - memorize From Claus Brabrand, ITU

Relevant, modern, authentic exam forms Portfolio Synopsis Case Project Group preparation Practice based exams Peer Assessment

Portfolio: collection, reflection, selection Systematic collection of works -> expression of development and learning Variation in materials and assignments Electronic portfolio Process orientation Reflexion, consciousness about learning Coverage

Portfolio and feedback ”As an indicator of achievement, portfolios were designed specifically as an opportunity to provide feedback, assuming an active, even steering role of the learner in the assessment task” Smith & Tillema, 2003, s. 626

Oral exam with synopsis Research question or problem formulation Problems, discussion of materials, theories and methods Conclusions Literature Perspectives NB: Reliability and resources

Case exams: authenticity Point of departure: a “real” situation Available data and materials: “real” Preparation and evaluation Objective: identify, analyse and solve problems Use of theories while working with the case Case based exam forms useful to test problem solving competences in specific contexts

Case: Complexity dimensions Analytic dimension Questions Problem definition and formulation Type of analysis needed One or several solutions Complexity Which theories and models can be used – and how difficult are they? Do they belong to the same discipline? To which degree must several disciplines be integrated? Presentation level Is relevant information accessible? Which important information is not included? Is information structured?

Project exams Problem based learning Within a restricted area be able to formulate a problem that makes sense Beyond declarative knowledge Focus on functional knowledge Specific knowledge directed towards practice Autonomy Explicitly include development of personal and social competences Project management -> demanding work form

Examination skills Dialogue at oral exams Collaboration and question typologies Good questions Level adaptation Written exam questions Censor’s role

Training Collegial supervision as a tool Conversation analysis Question typologies Video + reflection Censor’s observations Exam testing with colleagues or students

References Andersen, H.L. & J. Tofteskov, Eksamen og eksamensformer. Betydning og bedømmelse, Samfundslitteratur. Biggs, J Student Approaches to Learning and Studying, Hawthorn, Victoria: Australian Council for Educational Research. Bloom, B.S. (ed.) Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain, David McKay, New York. Brown, S. Rust, C. & Gibbs, G Strategies for Diversifying Assessment in Higher Education. Oxford Centre for Staff Development. Dysthe, O. & K.S. Engelsen “Variations in higher education portfolio assessment. A nationwide survey in Norway across institutions and disciplines”. Norgesuniversitetets konferanse om e-portfolio. Uddannelsesstyrelsen 2004, Modernisering af prøver, eksamener og karakterer.