Assessment Tools

Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions

Educational Triangle Needs / Objectives/Outcomes & Contents Strategies Assessment Curriculum

Assessment Formative v/s Summative

Formative vs Summative It is continues assessmentAt the end of a course/ program Feedback is an essential partUsually NO feedback session Usually NO gradsMarks OR grades are essential There is no pass-fail decisionThere should be a pass-fail decision

Assessment V/S Evaluation

Evaluation: is more focused on making a judgment or determination concerning the quality of a performance, work product or use of skills against a set of standards. Assessment is more focused on measuring a performance, work product, or skill in order to offer feedback to document strengths and growth and to provide directives for improving future performance. Assessments are nonjudgmental and are designed and intended to be helpful to produce improvement.

Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Cognition/ Knowledge Skills & Behaviour Assessment

Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Knowledge Competence Performance Action Assessment

One method can’t do it all –One measure is no measure –We need a mixture of methods to cover the entire pyramid –We can choose different assessment tools

Criteria for different assessment methods 1.Validity: 2.Reliability 3.Educational impact 4.Cost effectiveness 5.Acceptability/ Practicability

Criteria / Requirements 1.Validity:Does it measure what it is supposed to measure? 2.Reliability:Does it produce consistent results? 3.Practicability:Is it practical in term of times & resources.

Educational Impact Assessment may have facilitative or adversive effect on learning Emphasizes the important learning outcomes Provide feedback about the success of study methods

What Should Be Tested ? Important topics are weighted heavily The sample of items reflect the instructional goals and the process.

Assessment

Written Exam

Written Written tests can assess knowledge acquisition and reasoning ability, They cannot so easily measure skills

…all students from (all classes) are submitted to the same test…… Progress test

End of course assessment Multiple Choice Open ended questions Combination MCQ/OEQ Modified Essay Questions Computer assessment Assignments Construct a patient information folder Design a Poster Integration essay Prepare a forum discussion Write a research proposal

Clinical 1.Short cases 2.Long cases 3.OSCE 4.Simulated Clinics 5.Video consultation

Clinical (cont’) 6. Slides Interpretation 7. Lab Data Interpretation 8. Portfolios 9.SOE (simulated oral exam) 10.Oral

Computerized Case-based Testing Computer is used to increase effectiveness in: –Cost –Test administration –Scoring –Authenticity (e.g. by multimedia) –Written, practical, clinical,…….

“Long case" and “Short case" The basic requirements for reliability and validity have not always been achieved in traditional long & short cases

Observing candidates' interaction with patients

OSCE Objective structured clinical examination

Observing Candidates' Interaction with Patients

Oral

Work Place-Based Assessment The oral examination is traditionally an unstructured face to face session with the examiners Direct observation: Clinical work sampling measures: Mini-CEX, Mini-Clinical Evaluation Exercise DOPS, Directly Observed Procedural Skills Directly ObservedDirectly Observed P-MEX, The Professionalism Mini-Evaluation Exercise Global performance measures Multi-Source Feedback (MSF or 360) Aggregation and reflection measures Logbook Portfolio

observable behaviour behaviouroutside valuesnormsinsideattitude professional behaviour= Assessment of professional behavior

PEER EVALUATION The oral examination is traditionally an unstructured face to face session with the examiners Peer Assessment & Self Assessment

 Regular feedback of performance  For all the students  Helping students to grow as a professional  Starting from the beginning of the studies Assessment of professional behavior

Professional behaviour consists of three aspects Dealing with workDealing with othersDealing with oneself Assessment of professional behavior

Student gathers evidence about aspects of his competence of his competence Criteria to meet the different aspects of competence are described by faculty results of assessment (longtudinal or tailored) reports, video projects peer assessment staff assessment articles, presenting papers Student reflects on the different aspects of competence (written statements) Portfolio

Student proves his competence On several aspects described by faculty Role as expert results of assessment reports projects peer assessment staff assessment Student reflects on the different aspects of competence (written statements) Student discusses his competence and reflection on it with the mentor Mentor = coach Portfolio assessment

1. Contineous Assessment 40% Assignment 2. End of the Course Assessment TOTAL a. MCQ % b. Written Short Answer Questions 40% 100%

Conclusion

The oral examination is traditionally an unstructured face to face session with the examiners Comprehensive Assessment Different methods to be applied Standardized / structured Not to depend only on few methods

Knows Shows how Knows how Does Knows Knows how MCQ, SAQ, Short essy, MEQs….Simple recall questions

Knows Shows how Knows how Does Knows howShows how Simulations requiring behavioural performance OSCE, OSPE, Simulated Clinics,

Knows Shows how Knows how Does Shows how Does Assessing performance Work-based assessment: Clinical work-sampling, Mini-CEX, Portfolio, practice visits, case orals….

Knows Shows how Knows how Does Self-, peer, co-assessment, portfolio, multisource feedback, learning process evaluations……