Presentation on theme: "Faculty of Medicine and Health Sciences Exploring new approaches to assessing attitudes and behaviours Dr Sandra Gibson Dr John Winpenny 6 th February."— Presentation transcript:
Faculty of Medicine and Health Sciences Exploring new approaches to assessing attitudes and behaviours Dr Sandra Gibson Dr John Winpenny 6 th February 2012
Faculty of Medicine and Health Sciences The plan Introductions Discussion of attitudes and behaviours Item types used in medical training Transferability Lunch Item writing Future plans Closing discussion
Faculty of Medicine and Health Sciences Outcomes To discuss the theory and application of assessing attitudes and behaviours To discuss the methods of assessing behaviours used in medical training and their transferability to other disciplines To try writing assessment items in chosen disciplines To develop a plan of action to follow up questions/ideas raised during the workshop
Faculty of Medicine and Health Sciences Why are attitudes and behaviours important to you? Take a minute and write down why measuring attitudes and behaviours is of interest/importance to you
Faculty of Medicine and Health Sciences What do we mean by attitude? a psychological tendency that is expressed by evaluating a particular entity with some degree of favour or disfavour Eagly and Chaiken (1993)
Faculty of Medicine and Health Sciences Attitudes How an attitude is expressed (or not) can be influenced by –Context (specificity) –Reinforcement (behavioural control) –External drivers- guidelines (subjective norm) –Reflection (Schön) Attitudes can change
Faculty of Medicine and Health Sciences Attitudes vs. Behaviours A student I supervised repeatedly mentioned she held very strong personal and religious views against the termination of pregnancy. I knew she had just been placed in a family planning clinic.
Faculty of Medicine and Health Sciences Our approach Attitudes do not determine behaviour Students who hold suitable attitudes will experience less cognitive dissonance Seletion procedures are increasingly incorporating the assessment to try and assess attitudes/behaviours.
Faculty of Medicine and Health Sciences Tommorows Doctors The graduate will be able to behave according to ethical and legal principles. The graduate will be able to: (c) Be polite, considerate, trustworthy and honest, act with integrity, maintain confidentiality, respect patients dignity and privacy, and understand the importance of appropriate consent. (d) Respect all patients, colleagues and others regardless of their age, colour, culture, disability, ethnic or national origin, gender, lifestyle, marital or parental status, race, religion or beliefs, sex, sexual orientation, or social or economic status.
Faculty of Medicine and Health Sciences Why are attitudes and behaviours important to you? Before I describe how we measure behaviours, lets revisit your interest in measuring them attitudes and behaviours.
Faculty of Medicine and Health Sciences Medicine Fostering professional attitudes Measuring Professional behaviours Observed Structured Clinical Exams (OSCEs) Situation Judgment tests Script concordance tests
Faculty of Medicine and Health Sciences Dealing with uncertainty Studies with Foundation Year Doctors are consistently finding that dealing with uncertainty is an issue that they are concerned about. Use of these item types can be helpful in preparing and reassuring students about their behaviour in situations where decision making is not clear cut.
Faculty of Medicine and Health Sciences Observed Structured Clinical Exam (OSCE) Sit back and enjoy the movie. osce.mp4
Faculty of Medicine and Health Sciences Observed Structured Clinical Exam (OSCE) Greater reliability that traditional clinical assessments. The more realistic the better Feasibility can be an an issue Generalisibility Analysis used to optimise design
Faculty of Medicine and Health Sciences Observed Structured Clinical Exam (OSCE) The mark scheme –Global judgment used to set boundary between minimum competence and non competence –Can be used to assess professional behaviour Dealing with difficulty patients Honesty Dealing with uncertainty/ambiguous situations –Multiple mini interview
Faculty of Medicine and Health Sciences Situation judgment tests Another assessment. For further examples of SJTs see the Medical School Council- Assessment Alliance Project page Improving Selection to the Foundation Programme http://www.isfp.org.uk/SJT/Pages/Example- questions.aspxhttp://www.isfp.org.uk/SJT/Pages/Example- questions.aspx
Faculty of Medicine and Health Sciences Situation Judgment Tests Vary widely in format and application Format can have an impact on what they test. Scoring be complex Experts can score similarly to novices, but with smaller standard deviation. Faking
Faculty of Medicine and Health Sciences Situation judgment Tests Used in General Practitioner (GP) selection Will be used in Foundation Year selection in the near future May be introduced into MBBS selection (UKCAT)
Faculty of Medicine and Health Sciences Script concordance Another test to try.
Faculty of Medicine and Health Sciences Script concordance Based on script theory Ddesigned to assess decision making in ambiguous situations. Studies confirm acceptable reliability and validity Novices consistently less well than experts Case specificity and generalisabiity need to be considered Importance of reference group
Faculty of Medicine and Health Sciences Script concordance Currently used in Masters in Oncoplastic Breast Surgery on a formative basis Plan to introduce in undergraduate course for formative assessment
Faculty of Medicine and Health Sciences Questions
Faculty of Medicine and Health Sciences Writing workshop Ask group what they want to do Split into groups each try each item type, or groups doing different types? Spend half time facilitating writing, then presenting items to other groups for review.
Faculty of Medicine and Health Sciences The critical incident To help identify potential areas to write items in, no matter what the format, it can help to start with a critical incident/significant event. Take a few minutes and jot down an event were you felt that your judgment felt uncertain.
Faculty of Medicine and Health Sciences Planning What have you got out of the workshop? Are there any areas that we could take forward as a group/subgroup? Do you have any suggestions for future workshops?
Faculty of Medicine and Health Sciences Thank you Thank you to the HEA for supporting this workshop Feedback forms will be sent to you individually after the session. Contact details –firstname.lastname@example.org@uea.ac.uk –email@example.com@uea.ac.uk
Faculty of Medicine and Health Sciences References Bland AC, Kreiter G & Joel A (2005). The psychometric properties of five scoring methods applied to the script concordance test. Academic Medicine 80:395-9 Charlin B, et al. (2000). The script concordance test: a tool to assess the reflective clinician. Teach Learn Med 12: 189-95 Creighton P, Scott N. (2006). An introduction to situational judgement inventories. Sel Dev Rev 22:3-6 Eagly HA & Chaiken S, (1993). The Psychology of Attitudes, Fort Worth, TX: Harcourt Feltovich, PJ.et al (1984). LCS: The role and development of medical knowledge in diagnostic expertise. In W.J. Clancey & E.H. Shortliffe (Eds.), Readings in medical artificial intelligence: The first decade (pp. 275-319). Reading, MA: Addison Wesley Fournier JP, et al (2008). Dcript Concordance Testis: Guidelines for construction BMC Midecal Informatics and Decision Making 8:18 Gagnon R (2005). Assessmet in context of uncertainty: how many members are needed on the panel of reference of a script concordance test? Med Educ 39: 28 Humbert AJ, Besinger B & Edward JM (2011). Assessing clinical reasoning skills in scenarios of uncertainty: convergent validity for a script concordance test in an emergency medicine clerkship and residency. Academic Emergency Medicine 18: 627-634 Mc Daniel MA, & Nguyen NT (2001). Situational Judgment Tests: A Review of Practice and Constructs Assessed. International Journal of Selection and Assesment 9(1-2): 105-113 McDaniel MA, Hartman NS & Grubb W L (2003). Situational Judgment Tests, Knowledge, Behavioral Tendency, and Validity: A Meta-Analysis. Paper presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology. Orlando. McDaniel MA, & Whetzel DL (2005). Situation Judgment Tests, An IPMAAC Workshop. Motowidlo S. Borman W & Schmit M (1997). A theory of individual differences in task and contextual performance. Human Performance 10(2): 71-83 Schön D. (1983) The reflective practitioner: how professionals think in action. New York: Basic Books
Faculty of Medicine and Health Sciences Exploring new approaches to assessing attitudes and behaviours Dr Sandra Gibson Dr John Winpenny