Presentation on theme: "Investigating the oral viva as a form of assessment in business and management education Dr Gabor Lukacs Oxford Brookes University Business School"— Presentation transcript:
Investigating the oral viva as a form of assessment in business and management education Dr Gabor Lukacs Oxford Brookes University Business School
22 The context Advanced level UG module, 70 students Assessment structure: oral viva + student-led session formative viva Research objectives: Understand more about the dialogic process and the nature of the preparation and the interactions Understand how students come to understand what 'good communication' is through the process of doing vivas Understand the impact as opposed to written assignments Research method: focus group interviews, anonymous questionnaires with students, data from the feedback and discussion session in class Brief results of prior research and literature review MORE
3 How students prepared for the viva Learning in small groups, practice with friends – mock viva Formative viva – learning through formative feedback Learning through examples: did not have to memorise models, rather understand Additional research: looking up journals, do a lot of extra reading Write down answers for each question MORE
4 Difference from other types of assessment Requires wider knowledge of all aspects Read more, understand better An open dialogue, a discussion, a debate Knowledge becomes more cemented – becomes long term knowledge Random aspect and uncertainty Requires quick thought, on the spot aspect Individual preference, style MORE
5 Skills developed Communication, discussion In-depth analysis and understanding of the reading Builds confidence Recall things quickly Interviews Real life situation for business students MORE
6 Benefits Learning: Promotes wider reading, timely preparation, extra research Promotes learning, better understanding, deeper knowledge Links different issues together – more holistic Comprehensive and offers feedback during assessment Skills: Prepares for the real business world Communication, interviewing, discussion Confidence, stress management Deliver a concise understanding of the subject and explain it to others (tacit-explicit) Style: Conversational, debate, discussion Open, informal, friendly New and novel
7 Problems Amount of material: Lot to cover and remember, too many questions Not enough time to say everything & think about the answer Learning: Difficult to explain what you know & get the core out of the theory Requires a lot of reading, revision, extra research Had to memorise answers Skills: Nerve wracking, lack of confidence, first time experience On the spot – puts more pressure on you Unfair: Harder to standardise marking For introvert people or non native speakers Not everybody gets the same questions Not all of the questions are asked 50% weight is too much for 15 minutes
8 Discussion points Group viva as summative assessment? How to evaluate the effectiveness of different forms of assessment? Is it innovative? University should prepare students (and tutors) from the first year – programme level assessment strategy Implications for Brookes?
9 Bibliography Biggs J. and Tang C. (2007) Teaching for Quality Learning at University 3rd ed., McGraw-Hill Chamorro-Premuzic, T. (2006) Creativity Versus Conscientiousness: Which is a Better Predictor of Students Performance? Applied Cognitive Psychology Vol 20., pp Fry, H., Ketteridge, S. and Marshall, S. (editors) (2009) A Handbook for Teaching and Learning in Higher Education 3rd ed., London: Routledge Furnham, A., Christopher, A., Garwood, J. and Martin N. G. (2008) Ability, Demography, Learning Style, and Personality Trait Correlates of Student Preference for Assessment Method Educational Psychology Vol. 28., No. 1, pp Joughin, G. (1998) Dimensions of Oral Assessment Assessment and Evaluation in Higher Education Vol. 23, No. 4., pp Morley, L., Leonard, D. and David, M. (2002) Variations in Vivas: Quality and Equality in British PhD Assessments Studies in Higher Education Vol. 27, No. 3, pp Race, P. (2007) The Lecturers Toolkit: A Practical Guide to Assessment, Learning and Teaching London: Routledge Superville, L. K. (2001) Oral Assessment as a Tool for Enhancing Students Written Expression in Social Studies The Social Studies May/June, pp Thornburn, M. and Collins, D. (2006) Accuracy and Authenticity of Oral and Written Assessments in High-Stakes School Examinations The Curriculum Journal Vol. 17, No. 1, pp Woods, C. (2001) Bridging The Creative and the Critical Educational Studies in Language and Literature No. 1, pp
10 Brief results of previous research Written forms of assessment dominate summative assessment Race (2007), Joughin (1998), and Superville (2001): Advantages: Borderline cases Explore ideas, think out loud Discuss alternative explanations Revise thinking in a supportive environment Practice for workplace Disadvantages: Some students will underperform in oral exams Not anonymous Intimidated by figures of authority
11 How students prepared for the viva And then I found it was useful to practice with friends. So one person would start off answering it and then wed all contribute our ideas and then I learnt quite a lot that way as well, other things that people have found. I wouldnt have prepared this way for any other module. I have some friends that were taking knowledge management together and at times wed just sit down and talk like normal stuff, and the next thing, Oh thats like an example of tacit to explicit knowledge. And wed try to joke around and everything but unconsciously we were getting into it.
12 How students prepared for the viva I was memorising it through actually real life examples and then associating it back instead of going the other way was my method. I found almost a trigger in my mind, if I could think of an example. I had to do a lot of reading, and re-read, compared with normally Id just skim-read an article or just yeah a lot shallower revision.
13 Difference from other types of assessment …it looks really bad if they ask you a question and you cant answer it. So I think that you do have to have a wide knowledge of basically everything covered. …you didn't know what was going to come up … requires you to cover a much wider berth of knowledge, whereas exams are very prescriptive. Theres a degree of unknown and uncertainty about a viva, which I think has its plus points.
14 Difference from other types of assessment For the viva you have to read wide Yeah it did make me learn the stuff that I needed to learn – I didn't learn everything but if I hadn't done the viva I probably wouldnt have known half the stuff that I did for the viva. …if youre talking around something you actually need to be able to understand it in more depth, which I think the viva promotes a lot of it was seeing if I actually knew the thing not from memory, if I actually got to grips with some of the models that he asked me. I mean I quite enjoyed the viva.
15 Difference from other types of assessment it was sort of an open dialog, it wasnt really like set questions it was just talking to me and then, Could you explain this? - it came across I think a lot more friendly I didnt recognise until after talking to other people that there was three distinct questions. - Mine was just a dialog. Some of the questions left it a bit open to go where you wanted to go with it, rather than have to completely follow exam guidelines like a structured question in an exam.
16 Difference from other types of assessment theres stuff where Ive done written exams and then give it a week and its gone out the other way because Ive had to focus on something else. Whereby with the viva Ive still got Nonakas name in the back of my head, and Ill never forget him unfortunately. But yeah, just because its something that you had to do so much reading around … I had a clear understanding I think. I think if anyone wakes me up now and starts asking me questions about knowledge management … So I think it does, well in my mind its kind of a bit more cemented in my head
17 Difference from other types of assessment youre sitting there and someone asks you a question its bam, its on the spot. And I went in with that flow diagram in my head: definition, application, critique and its hard because youre always thinking to actually keep to that. … I think its a very good method of examination. you get one shot, its so quick youre asked the question, whilst an exam youve got the question in front of you, you can think about it and you can analyse and take your time over it, whilst with a viva you're there and then, which I think is more difficult but I think its quite a good skill… - thats a really good skill to have.
18 Skills developed …when you go into organisations, if they ask you for your opinion you dont sit there and write an essay for them and hand it over, you have to … there and then off the top of your head have an understanding. I did a years placement and I found that working life was very different to university life - a lot of it wasnt around could I write a good assignment and how did I structure that? It was much more viva-based where a manager may ask for my opinion and I had to give that opinion around a topic and be concise and give examples. … So I think, yeah if I was going to promote it [the viva] to an individual Id say that it helps you gain a better understanding of a module and its much better for real life situations.