Presentation is loading. Please wait.

Presentation is loading. Please wait.

Review of Year 1 Immunology/Virology Course occurs in the winter term of Year 1 Course Directors – Charles Sentman, Ph.D. David Bzik, Ph.D. Course has.

Similar presentations


Presentation on theme: "Review of Year 1 Immunology/Virology Course occurs in the winter term of Year 1 Course Directors – Charles Sentman, Ph.D. David Bzik, Ph.D. Course has."— Presentation transcript:

1 Review of Year 1 Immunology/Virology Course occurs in the winter term of Year 1 Course Directors – Charles Sentman, Ph.D. David Bzik, Ph.D. Course has 44 curricular hours Course was last reviewed in May 2012

2 Recommendations from 2012 Review Incorporate more active modes of learning/new methods of pedagogy into the course (i.e. reduce % of lectures) Rewrite/create learning objectives for each session to clearly articulate expectations to students Revise assessment questions; verify that they link back to major course objectives Faculty development is needed for some faculty that are not meeting expectations with regard to teaching

3 Recommendations from 2012 Review Reorganize course to create a better flow for the material; consider matching order of topics used in the text (specific to Immunology) Identify at risk students early and provide support, e.g. review sessions (specific to Immunology) Update/edit notes; ensure lecture content and notes are in agreement (specific to Virology) Provide an opportunity where biologic data are observed, measured and analyzed (specific to Virology)

4 Course Objectives – Immunology There are 17 course objectives that fulfill Geisel competencies as follows: 13 address specific knowledge in the preclinical domain, and seem appropriate 4 address communication skills 3 address components of professionalism, although only 1 is mapped to competency 4 in Ilios 1 addresses habit of inquiry/personal improvement Currently no course objectives are mapped to Geisel competencies 2 (clinical skills) and 6 (health care systems).

5 Course Objectives – Virology could map to 4

6 Course Objectives – Virology There are 19 course objectives listed in Ilios that fulfill Geisel competencies as follows: 19 address specific knowledge in the preclinical domain, and seem appropriate 1 addresses habit of inquiry/personal improvement, but is currently not mapped to competency 5 Currently no course objectives are mapped to Geisel competencies 2, 3, 4 and 6.

7 could map to 5b

8 Course Objectives Course objectives are provided to the students in the syllabus for both parts of the course, and are clearly written using measurable verbs Course objectives correlate well with the content in the Step 1 brochure Did not find a published list of objectives by a national organization in the field

9 Session Objectives Recommendation 2012: Rewrite/create learning objectives for each session to clearly articulate expectations to students Session objectives are provided at the top of each set of notes (all except sessions on Herpes viruses, Tumor viruses, Poliovirus) Most session objectives are written clearly using measurable verbs (all except several immunology sessions that use words like “know” and “understand”)

10 Objectives addressing LCME topics Health of Populations: Course includes discussions of TB and RA, and how they affect the health care system and society; course discusses vaccines in the context of societal needs and problems. Basic and Ethical Principles of Clinical and Translational Research: Course includes discussions of cases that occurred prior to the emergence of more modern views regarding basic scientific and ethical principle of clinical and translational research, including ways in which it is conducted, evaluated, explained to patients, and applied to patient care.

11 Objectives addressing LCME topics Gender and Cultural Biases in Students Themselves: Access to vaccination, access to diagnosis, access to treatments, and risk of virus diseases are strongly influenced by gender, racial/ethnic, and cultural biases. Instruction in Medical Ethics and Human Values: Not included in the course as this time.

12 Assessing integration in the curriculum The curriculum database was used to evaluate the integration of major topics in the course across the curriculum (e.g. Immunodeficiency, Rheumatoid Arthritis, HIV, Viral Hepatitis, Influenza) In the past our curriculum has been weak in issues related to the elderly, so we also assessed whether the course discussed changes of immune function with age

13 Immunodeficiency present in two Year 1 courses (Immunology/Virology and General Pathology) present in three Year 2 courses (Organ-based Pharm, SBM Infectious Diseases, SBM Neurology) present in two Year 4 courses (Advanced Medical Science, Clinical Pharmacology and Therapeutics)

14 Lymphoma present in one Year 1 course (Microbiology) present in one Year 2 course (SBM Hematology)

15 Rheumatoid Arthritis present in one Year 1 course (General Pathology) present in two Year 2 courses (SBM Bone and CT, On Doctoring)

16 HIV present in two Year 1 courses (Immunology/Virology and General Pathology) present in three Year 2 courses (Organ-based Pharm, SBM Infectious Diseases, SBM Neurology) present in two Year 4 courses (Advanced Medical Science, Clinical Pharmacology and Therapeutics)

17 Viral Hepatitis present in one Year 1 course (Immunology/Virology) present in one Year 2 course (SBM GI)

18 Influenza present in four Year 1 courses (Immunology/Virology, Microbiology, General Pathology and HCDS) present in one Year 2 course (Organ-based Pharmacology)

19 Age-related changes in Immunity searched using “geriatrics”, “elderly”, “aging”, “age process” nothing mapped to Year 1 Immunology/Virology course directors should consider including at least one of these words in one of their objectives so that this material is detected in a search

20 Summary regarding Objectives Course objective listed in ILIOS are provided to the students; several opportunities exist for mapping existing objectives to additional Geisel competencies Most sessions now have objectives that are provided to students; several sessions still need objectives, and several sessions need to rewrite their objectives using measurable verbs Searches in ILIOS regarding major topics in the course demonstrated some redundancy with material in other courses; much of this redundancy is planned and appropriate, however the course directors should verify this with other course directors

21 Course Learning Opportunities Recommendation 2012: Incorporate more active modes of learning/new methods of pedagogy into the course (i.e. reduce % of lectures) Immunology Lecture 21 hrs. (some are interactive) – 88% Discussion groups 3 hrs. – 12% Virology Lecture 20 hrs. (some are interactive) – 100% Addition of a web-based laboratory Pedagogy for AY is basically the same as that for AY

22 Course Learning Opportunities Recommendation 2012: Identify at risk students early and provide support, e.g. review sessions (specific to Immunology) Optional review sessions are offered to the class Course director contacts students who are struggling

23 Course Learning Opportunities Recommendation 2012: Provide an opportunity where biologic data are observed, measured and analyzed (specific to Virology) A web-based laboratory was added to the Virology course

24 Summary regarding Pedagogy The percentage of traditional lectures in the course is higher than is desired (goal 40-50% of course hours); the course director should consider ways to incorporate more active pedagogies into the course Opportunities exist for students to discuss unclear material with the faculty (review sessions) A web-based laboratory provides an opportunity for students to observe, measure and analyze biologic data

25 Assessment Course is divided into Immunology and Virology components Immunology (IMM) component – Two Quizzes (each 10% of course grade) Each covers 12 sessions Multiple choice questions each – Small Group Discussion (5% of course grade) 3 session, each student presents in one session Presentation by randomly-selected group of 4-5 students Graded by faculty preceptor with written feedback to students

26 Assessment Virology (VIR) component – Two Quizzes (each 12.5% of course grade) Each covers 8 sessions (2 additional sessions assessed only on final exam) Multiple choice, 20 questions each Final Exam (50% of course grade) – Covers immunology and virology (equal distribution) – Multiple choice, 80 questions Overall Assessment Distribution – Quizzes (4)45% – Final Exam50% – Small Group Discussion5%

27 Assessment – IMM Question Formats Immunology Quizzes (Q1 and Q2) All single best answer with 4 or 5 options In general, questions are clearly worded Only 1 of 42 items used a clinical scenario to stage the question Three of 42 items used “negative” formats (All of the following EXCEPT) Many of the questions used lists of T/F statements, rather than one best answer lists – A small number of items (3/42) were stated T/F questions (“Which of the following statements are correct?”)

28 Assessment – IMM Question Content Immunology Quizzes (Q1 and Q2) Most questions assessed recall of facts, and only a few questions required application of knowledge: Quiz 1 (IMM): ~18/22 assessed recall Quiz 2 (IMM): ~14/20 assessed recall No apparent integration between multiple session objectives Coverage is distributed evenly across the sections

29 Assessment – IMM Correlation with objectives Immunology Quizzes (Q1 and Q2) Questions evaluated correlated well with specific session objectives Ex: Quiz 1 (11 sessions + 1 small group), 22 Questions – 21 of 22 questions clearly map to session objectives – 68 session objectives, ~29 covered on quiz – Several questions had more than 1 mapped objective, although no cross-over between sessions/topics – Minimal overlap (only ~2-3 objectives mapped to more than 1 question)

30 Assessment – VIR Question Formats Virology Quizzes (Q3 and Q4) All single best answer with 4 or 5 options In general, questions are clearly worded 14 of 40 items used a clinical scenario to stage the question No questions used “negative” formats (All of the following EXCEPT) Many of the questions used lists of T/F statements, rather than one best answer lists – A small number of items (2/40) were stated T/F questions (“Which of the following statements are correct?”)

31 Assessment – VIR Question Content Virology Quizzes (Q3 and Q4) Many questions assessed recall of facts; a few questions required application of knowledge: Quiz 3 (VIR): ~12/20 assessed recall Quiz 4 (VIR): ~17/20 assessed recall No apparent integration between multiple session objectives Coverage is distributed evenly across the sections

32 Assessment – VIR Correlation with objectives Virology Quizzes (Q3 and Q4) Questions evaluated correlated well with specific session objectives Ex: Quiz 3 (4 x 2hr sessions), 20 Questions – 20 of 20 questions clearly map to session objectives – 38 session objectives, ~24 covered on quiz – Few questions had more than 1 mapped objective – Minimal overlap (only ~1-2 objectives mapped to more than 1 question)

33 Assessment – Exam Question Formats Final exam (includes both IMM and VIR) All single best answer with 4 or 5 options Some items were statements (i.e., a symptom, followed by a list of potential causative agents) without an explicit question Only 4 of 80 items used a clinical vignette to introduce the item Four of 80 items used “negative” formats (All of the following EXCEPT) Many items used lists of T/F statements, rather than one best answer lists

34 Assessment – Exam Question Content Final exam (includes both IMM and VIR) Many of questions assessed recall of facts: ~75% assessed recall ~25% assessed application of knowledge No apparent integration between multiple session objectives Coverage is distributed evenly across the disciplines (IMM and VIR) and sections

35 Assessment – Correlation with objectives Balanced treatment of topics (i.e., equal division between IMM and VIR, and items maps across the COs) IMM portion of exam (40 items) – ~ 45 session objectives covered (out of ~144) – Several questions had more than 1 mapped objective, although no cross- over between sessions/topics – Minimal overlap (only ~2 objectives mapped to more than 1 question) VIR portion of exam (40 items) – ~ 46 session objectives covered (out of ~80) – Several questions had more than 1 mapped objective, although no cross- over between sessions/topics – Minimal overlap (only ~2 objectives mapped to more than 1 question)

36 Assessment – Small Group Presentations Students worked in groups (4-5, randomly selected) to research an assigned immune-related disease condition and lead oral presentations to fellow students Format: ~30 min of presentation by group, followed by ~25 min for discussion and questions, driven by a) suggested discussion questions provided to the presenters, and b) questions from their peers Students are completely responsible for conducting the sessions

37 Assessment – Small Group Presentations Presentations are scored by faculty, based on the following criteria: – Knowledge of topic – Coverage of immune mechanisms, pathophysiology, epidemiology, diagnosis, and treatment – Quality of presentation – Professionalism – Response to questions from peers – Teamwork Students in the audience provide written feedback to the groups (de-identified and presented in aggregate) This activity accounts for 10% of the IMM grade (5% of final course grade) All members of the group receive the same grade

38 Summary regarding Assessment Quizzes and exam questions were clearly written All quizzes and the final exam use MCQ – a fair number of T/F or “except” type questions – Most items assessed recall, although application questions were more prevalent on the exam – Very few questions used clinical vignette stems Students are provided answer keys and access to the quizzes, including detailed explanations of the answers Exam provided broad coverage of both IMM and VIR topics with equal distribution There was no overlap between IMM and VIR content in the questions; within disciplines, majority of the questions covered a single LO (no integration of topics) Small groups provide opportunity for students to develop presentation and discussion skills with feedback from faculty and peers

39 Measur es of Quality – AAMC GQ BASIC SCIENCES Geisel mean 2008 Geisel mean 2009 Geisel mean 2010 Geisel mean 2011 Geisel mean 2012 All schools means 2012 Behavioral Science Biochemistry Biostatistics/Epidemiology Genetics Gross anatomy/Embryology Histology Immunology Microbiology Neuroscience On Doctoring Pathology Pathophysiology of Disease Pharmacology Physiology Measures of Quality – AAMC GQ “Indicate how well you think that instruction in Immunology/Microbiology prepared you for clinical clerkships and electives.” [1=poor; 2=fair; 3=good; 4=excellent]

40 Measures of Quality – Step I 2009*2010*2011*2012* Means TRADITIONAL CORE DISCIPLINES Biochemistry Biostatistics/Epidemiology Genetics Gross anatomy/Embryology Histology/Cell Biology Microbiology/Immunology Pathology Pharmacology Physiology *values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

41 scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent] Measures of Quality – Course Reviews Immunol 2011 (35%)* Immunol 2012 (96%)* Virology 2011 (23%)* Virology 2012 (96%)* Overall satisfaction of course Overall usefulness of lectures Overall usefulness of small groups N/A Overall usefulness of course materials Congruence of assessment questions to material emphasized in course *student participation rate on course evaluation

42 Measures of Quality – IMM Course Reviews Strengths: sample comments: “Dr. Mullin's was the highlight of this course. His lectures were dynamic and clarified the material. I also found the review session led by Dr. Mullins to be very helpful. This was the strongest aspect of the course.” “What is GREAT about this course is that you provided page numbers of textbooks and board review questions that correlated with each lecture. Every other course in the school could benefit from this approach.” “I really enjoyed how the lecture material mirrored the chapters from the Abbas text book. When I was confused during lecture, the corresponding readings from the book were essential.” “In addition to Levinson and the USMLE-type questions in the back of the text…provided most of the material we need to master in a digestible format.”

43 Measures of Quality – IMM Course Reviews Suggestions for Improvement: sample comments: “Extra information is like overeating. It stresses the system without providing added nutrient benefit. The right amount of food and information causes the best absorption.” “Wade's lectures were very difficult to follow and his notes are long-winded and repetitive. In comparison, Dr. Mullins's notes are clear and concise and his lectures were interesting, easy to follow, and seemed to highlight the most important material. “…I often got more confused while reading his lecture notes than if I'd read the textbook.” “The material stressed did not feel like the material that was tested.” “The most frustrating part of the course was the lack of consistency between the exams and the information presented within the lecture.”

44 Summary Immunology Measures of Quality Dr. Mullins: Engaging lectures, organized presentation of materials, concise notes with a summary, unambiguous expectations for testing Many students found the hypersensitivity lecture to be very good Students appreciated Dr. Mullin’s engaging lecture style accompanied by his well organized, succinct notes Most students discussed the lack of consistency between professors in their lectures and materials provided Students found the textbooks very useful, the questions provided by the Levinson book and the content in Abbas – This is a double edged sword. Students praised Dr. Mullins notes for being concise, highlighting key points, and not needing outside resources to understand the material Students felt that the material tested did not match the material emphasized in class Students would benefit from a summary at the end of each lecture that reiterated the key points Students appreciated that lecture objectives, slides, and textbooks were at times mapped together, so that when looking at a slide you saw the objective that pertained to that slide or when reading lecture notes there was a list of slides that corresponded to that paragraph of notes

45 Measures of Quality – VIR Course Reviews Strengths: sample comments: “Dr. Bzik's lecture notes - the bold and underlined words and the concise notes, made for a very efficient study tool.” “Summaries given out in the end of the course was helpful in organizing different viruses in my head.” “The subject matter is very interesting to me and I liked that the professors attempted to match their questions to the types of questions asked on the National Boards exam.” “I also really appreciated the inclusion of ethical, global health and historical issues into the curriculum.”… “Instead of memorizing all the time you are actually piecing together whys and hows during the process.” The material was taught clearly, tested fairly, and was straightforward…I felt that the exams were representative of the material, and that I could get the information easily by watching lecture and using the excel spreadsheet of viruses.”

46 Measures of Quality – VIR Course Reviews Suggestions for Improvement: sample comments: “The quiz questions were really frustrating. Some of them had multiple right answers, some of them we really never covered in class. I really didn't think these quiz questions were a good representation of what I learned in class.” “It was difficult to grasp which details were important and which were not important to know.” “…Dr. Witters practice exams, as a readiness blackboard quiz with detailed explanations of why something is right AND why it is wrong, would help tremendously in focusing in on thinking and rationalizing through questions correctly.” “…if we need to read the Levinson material because it will not be taught to us in class but it is quizzable then let us know at the beginning of the class not a couple of days before a quiz.” “Not until after I read a few chapters of Levinson, did I realize there are about 10 things we need to know for every virus: viral structure, replication cycle, disease, pathogenesis, transmission, epidemiology, detection, treatment, vaccines. If each lecturer could make an effort to address each of these aspects of the virus in their lecture material, it would make studying for exam much easier.” Dr. Leib’s “..slides were very disorganized and haphazard and with no notes to accompany them, there was no way to piece the information together. Some viruses had incubation times, other didn't. Some had treatments, others didn't.”

47 There was an incongruity between the expectations of quizzable material versus supplemental material and lectures – “…For each of the three lectures, there was a different emphasis placed on the value of slides, notes, and the required textbook.” Lack of understanding as to what are the key points – Students would appreciate a summary of each lecture of the most important ideas Lecture notes that were not proof-read and were out of date, i.e. didn’t match lecture slides Scheduling lab assignments in a different week from quizzes Proof reading quizzes more thoroughly to match class objectives Lecture notes with slides Summary Virology Measures of Quality

48 Summary of Recommendations Course objectives are clearly written and correlate well with objectives in the Step I brochure Some session objectives need to be rewritten; session objectives need to be provided for every activity in the course The issue of redundancy regarding common diseases should be explored by the course directors to ensure that the redundancy is planned The course is heavily lecture-based; the course directors need to introduce more engaged/active forms of pedagogy into the course

49 Summary of Recommendations Assessment questions have been improved since the last review of the course, however further revision is needed; students continue to cite frustration with the content on assessments (i.e. the material assessed does not match what is emphasized in the course) Expectations regarding the course need to be clearly communicated to students; some improvement is needed in the clarity of course materials


Download ppt "Review of Year 1 Immunology/Virology Course occurs in the winter term of Year 1 Course Directors – Charles Sentman, Ph.D. David Bzik, Ph.D. Course has."

Similar presentations


Ads by Google