Brian Jolly Centre for Medical and Health Sciences Education Monash University Train the Trainer Workshop October 31, 2003 Hong Kong International Consortium.

Slides:

Advertisements

Similar presentations

Performance Assessment

Advertisements

A Strategy to Enhance Assessment and Feedback Sheila Oliver Director of Assessment School of Dentistry.

“The GMC aims to encourage a culture where the patient and public perspective is sought and recognised across the spectrum of medical education” Paragraph.

Standard 22A Curricular Structure HT Accredited Curriculum.

Update on Goals 1 and 2 Curricular Domain Curricular Domain – accomplishments to date Developed baseline information about current level of faculty.

Introduction to simulation and debriefing Role of simulation in Emergency Department DKA Guidelines Shahzad B. Waheed, M.B.B.S, FRCPC, FAAP Assistant Professor.

General Information --- What is the purpose of the test? For what population is the designed? Is this population relevant to the people who will take your.

 1 – Course Plan – all 7 modules  2 – Module Plan for each module  3 – Session Plan for each session.

Instructional System Design.  The purpose of instructional design is to maximize the value of instruction for the learner especially the learner's time.

Chapter 4 Validity.

An overview of Assessment. Aim of the presentation Define and conceptualise assessment Consider the purposes of assessment Describe the key elements of.

An overview of Assessment. Aim of the presentation Define and conceptualise assessment Consider the purposes of assessment Describe the key elements of.

Constructing a test. Aims To consider issues of: Writing assessments Blueprinting.

PRINCIPLES OF SYSTEMATIC COURSE DESIGN PROF. TREVOR GIBBS.

Assessment of Clinical Competence in Health Professionals Education

Comparison: Traditional vs. Outcome Project Evaluative Processes Craig McClure, MD Educational Outcomes Service Group University of Arizona December 2004.

Assessing Students’ Skills in Science & Technology Hong Kong University – May 19 th 2006 Graham Orpwood York/Seneca Institute for Mathematics, Science.

Training the OSCE Examiners

Grade 12 Subject Specific Ministry Training Sessions

NANDA International Investigating the Diagnostic Language of Nursing Practice.

Introduction to Standard 2: Partnering with consumers Advice Centre Network Meeting Nicola Dunbar October 2012.

Proposal Writing.

Hollis Day, MD, MS Susan Meyer, PhD.  Four domains for effective practice outlined in the Interprofessional Education Collaborative’s “Core Competencies.

Critical Appraisal of Clinical Practice Guidelines

Principles of Assessment

Assessment Tools. Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions.

1 DEVELOPING ASSESSMENT TOOLS FOR ESL Liz Davidson & Nadia Casarotto CMM General Studies and Further Education.

Instructional System Design

Introduction: Teaching and Testing/Assessment

Foundations of Educating Healthcare Providers

Teaching and assessment of clinical Hauora Māori in 4 th year medicine at the FMHS, Auckland Dr Sue Crengle 3rd December 2009.

Assessing Students Learning Outcomes Senia Terzieva

ENGAGING LEADERS FOR CHANGE AND INNOVATION ADEA CCI 2011 Summer Liaison Meeting San Diego, CA June 27-29, 2011 Janet M. Guthmiller, DDS, PhD University.

Building Effective Assessments. Agenda  Brief overview of Assess2Know content development  Assessment building pre-planning  Cognitive factors  Building.

Ensuring State Assessments Match the Rigor, Depth and Breadth of College- and Career- Ready Standards Student Achievement Partners Spring 2014.

Standards-Based Science Instruction. Ohio’s Science Cognitive Demands Science is more than a body of knowledge. It must not be misperceived as lists of.

Comp 20 - Training & Instructional Design Unit 6 - Assessment This material was developed by Columbia University, funded by the Department of Health and.

Writing Clear Learning Objectives. Help trainers focus on “need to know” content and eliminate unnecessary content. Guide trainers in choosing appropriate.

1 Issues in Assessment in Higher Education: Science Higher Education Forum on Scientific Competencies Medellin-Colombia Nov 2-4, 2005 Dr Hans Wagemaker.

Understanding Meaning and Importance of Competency Based Assessment

Standard 4: Medication Safety Advice Centre Network Meeting Margaret Duguid Pharmaceutical Advisor February 2013.

DECEMBER 2013 Dynamic Learning Maps: Claims and Conceptual Areas and Common Core Essential Elements Update The present publication was developed under.

Traditional vs. Alternative Assessment

Results Student Engagement : Students generally found logbooks easy to use and practical in the hospital setting. Purpose : There appeared to be a perceived.

Assessment Tools.

Planning Curriculum Hand in Hand with Assessment and Evaluation.

TUSK Competency Framework Project November 20, 2008.

Component 8 Installation and Maintenance of Health IT Systems Unit 4 Structured Systems Analysis and Design This material was developed by Duke University,

Participate with SWOT. Achievement Standard

Assessment Information from multiple sources that describes a student’s level of achievement Used to make educational decisions about students Gives feedback.

EXERCISE AFTER STROKE Specialist Instructor Training Course T11 Generic Risk and Risk Management Systems (EAP’s) J. Dennis/Bex Townley.

SINGING FROM THE SAME HYMN SHEET Address to SATS Study Day 29 June 2013 Dr Sue Armstrong.

Standards-Based Science Assessment. Ohio’s Science Cognitive Demands Science is more than a body of knowledge. It must not be misperceived as lists of.

Assessing Learners The Teaching Center Department of Pediatrics UNC School of Medicine The Teaching Center.

Rubrics Staff development workshop 19/9/2014 Dr Ruth Fazakerley.

BOC Role Delineation Workshop Nina Walker Jan C. Lauer 2009 BOC Athletic Trainer Regulatory Conference.

MRCGP The Clinical Skills Assessment January 2013.

Curriculum Implementation Support Program (CISP).

PGCE Evaluation of Assessment Methods. Why do we assess? Diagnosis: establish entry behaviour, diagnose learning needs/difficulties. Diagnosis: establish.

Dr. Marciano B. Melchor University of Ha’il, KINGDOM OF SAUDI ARABIA May 2013.

Assessment of Learning 1

MRCGP The Clinical Skills Assessment January 2013.

Assessment Framework and Test Blueprint

Training and Development

Clinical Assessment Dr. H

Principles of Assessment & Criteria of good assessment

Setting Writing Goals in Science The Living Environment

Standards-Based Assessment and Rating of Learning Outcomes

Providing feedback to learners

Presentation transcript:

Brian Jolly Centre for Medical and Health Sciences Education Monash University Train the Trainer Workshop October 31, 2003 Hong Kong International Consortium Blueprinting

Hong Kong International Consortium Objectives Describe basic ideas behind blueprinting Develop 2 examples Show how this does/does not articulate with IDEAL database

Hong Kong International Consortium Principle The PURPOSE of the assessment will define the content, strategy, & technique used.

Hong Kong International Consortium Curriculum and Assessment: ‘Ideal’

Hong Kong International Consortium Curriculum and Assessment: Real

Hong Kong International Consortium H. C. Curriculum and Assessment: Real

Hong Kong International Consortium Curriculum and Assessment: Blueprinting

Hong Kong International Consortium  Face validity  Content  Construct  Criterion  Effects on learning Aspects of Validity - Constructs

Hong Kong International Consortium  Face validity  Content  Construct  Criterion  Effects on learning Aspects of Validity - Constructs

Hong Kong International Consortium  Face validity  Content  Construct  Criterion  Effects on learning Aspects of Validity - Constructs

Hong Kong International Consortium  Define what is to be tested  Clinical problems at some resolution  Tasks for each problem  Blueprint  Select test methods  Authenticity appropriate to clinical tasks  Clinical tasks determine the method  Titrate against practical constraints Content Steps

Hong Kong International Consortium Competence Categories CVSRS…GI…..MSSEndo/ Metab Haem / Oncology …. History Taking Physical Exam Tests and Procedures ….. Management Communication and Patient Education Simple Blueprint

Hong Kong International Consortium The Hospital SettingThe Community Setting Competence Categories CVSRS…GI…..CVSRS…GI History Taking Physical Exam xx Tests and Procedures xx ….. Other Parameters - Hierarchical

Hong Kong International Consortium InpatientOutpatient/Ambulatory Competence Categories CVSRS…GI…..CVSRS…GI History Taking Physical Exam xx Tests and Procedures xx ….. Hierarchical - IDEAL ® Related

Hong Kong International Consortium The ChildThe Elderly Competence Categories CVSRS…GI…..CVSRS…GI History Taking xx Physical Exam Tests and Procedures ….. Other Dimensions

Hong Kong International Consortium The ChildThe Elderly Cognitive /Affective Categories CVSRS…GI…..CVSRS…GI Recall xx Reasoning Affect xx Skill ….. Other Dimensions - Response

Hong Kong International Consortium The ChildThe Elderly Cognitive /Affective Categories CVSRS…GI…..CVSRS…GI Recall KFI Reasoning Affect CISP Skill ….. Test – Response Matrix

Hong Kong International Consortium Steps: A plausible test of Content Validity   Chart distribution of questions by level & topic   Compare this distribution with weighting of topics in curriculum or course documents   Note to establish content validity we need a demonstrated relationship Jolly 2000, Medical Education,

Hong Kong International Consortium  Some researchers use judgments of clinicians or successful test-takers  Dimensions of blueprint not always easy to define c.f. the items  How clinical problems, tasks or other 'concrete examples' relate to content often taken for granted in blueprinting Jolly 2000, Medical Education, Steps: A plausible test of Content Validity

Hong Kong International Consortium Summary Before selecting evaluation tools, need to create an evaluation “blueprint” The blueprint = matrix that outlines parameters for the assessment in 3 major areas: domains to be assessed, content & assessment instruments

Hong Kong International Consortium Blueprint matrix ensures balance across assessment formats - conversely Multiple methods usually required to achieve blueprint objectives Different assessment instruments allow balance of strengths and weaknesses of each method Summary

Hong Kong International Consortium  Content  In Australia RACS has sole responsibility for goals, training delivery & standards  See CANMEDS, Duties of a Doctor,  Investigate ways of addressing these issues  Get profession & public to assist  Responsibility for the ‘hard to measure’  RACS blueprint – are we a surgeon first & a specialist second? Blueprinting Issues: e.g., RACS Assessment

Hong Kong International Consortium  Judgement (80%)  Literature on judgement  Measures of judgement (as yet undeveloped)  Manual dexterity (20%)  Simulation and testing in training  Reznick, Paisley vs. Beard et al RACS Assessment - Content