Review of Year 1 Biostatistics Epidemiology Course occurs in the spring term of Year 1 Course Director – Judy Rees, BM, BCh, MPH, PhD Course has 29 curricular hours

Course Objectives – Content Review There are 13 course objectives that fulfill Geisel competencies as follows: 13 address specific knowledge in the preclinical domain 1 addresses clinical skills 3 address communication skills 1 addresses components of professionalism 13 address personal improvement 6 address health care systems

Biostatistics Epidemiology Course Objectives

Course Objectives – Content Review Course objectives are provided in the syllabus on the first page and seem appropriate for the course Course objectives are written in the correct format

Objectives: Step I Brochure Content of the course objectives correlates well with the content in the Step I Brochure for the category “Quantitative methods”

Objectives: National Organization The course directors are not aware of a national discipline-oriented organization in their field that has published course objectives, thus no comparison was made

Session Objectives Session objectives are provided in the course materials for each lecture and its corresponding small group, however these session objectives do not match the session objectives on Ilios and some do not use measurable verbs The course director clarified that the session objectives in the course materials were the old set, and the objectives in the Ilios system were the correct objectives The session objectives on Ilios are written in the correct format using measurable verbs; some use higher level verbs such as contrast, estimate, interpret, and apply All session objectives map back to one or more course objectives

Redundancy with other courses Significant redundancy of course content with other courses was not found in this course Searches in Ilios for terms such as “population statistics” and “epidemiologic research” did not reveal any overlap Searches for terms such as “diagnostic testing” and “confounding” revealed that these concepts were introduced in this course, then appropriately expanded upon in Y2 (e.g. “Describe the appropriate use of diagnostic testing in patients with disorders of the reproductive system”; “Explain the concept of confounding variables in smoking and lung cancer”)

Summary regarding Objectives Course objectives are well-written and fulfill components of all six Geisel competencies Two sets of session objectives exist – one in the course materials and one on Ilios; the most current set of session objectives (on Ilios) is well-written All session objectives map back to at least one course objective Unplanned and unnecessary redundancy was not found associated with this course

Course Learning Opportunities Lecture 15 hrs. (52%) Small group conferences 12 hrs. (41%) Review session for final exam 2 hrs. (7%)

Course Learning Opportunities Innovative pedagogy for small groups Course offers two formats for conferences: a standard option where students and faculty work together to solve problems during the conferences, and a flipped option where students do the problems before coming to conferences and then they are discussed Students choose the format they prefer for the small group at the start of the course Attendance is required for small groups, and students are encouraged to participate (i.e. some students are asked to project their work for the group to view)

Summary regarding Pedagogy The percentage of traditional lectures in the course is appropriate Small group conferences offer opportunities to apply the material Two methods of pedagogy are offered for conferences allowing students to choose the method that best suits their learning style

Assessment Two written Quizzes - 40% of course grade Final Exam - 60% of course grade Conferences do not count towards the course grade, however attendance/participation in conferences can help a weak student, i.e. syllabus states: “If a student’s final grade is borderline pass-fail, good participation in small groups may benefit the student.”

Assessment – Quizzes and Exams Quizzes consist of approximately multiple choice questions; the final exam consisted of 30 multiple choice/short answer questions All questions were written in correct formats Some questions required students to apply their knowledge (statistical calculations) Questions correlate well with the session objectives; all questions were linked to at least one objective

Assessment - Conferences Currently conference attendance and participation is not assessed Students do not receive feedback regarding their conference performance, however this may not “fit” with the activities for the small groups in the course (i.e. students are solving problems and answers are discussed – they are not doing activities such as physical exam where performance can be critiqued)

Summary regarding Assessment There are no major problems with the assessments in the course – questions are written in correct format and they correlate well with the session objectives Conferences provide opportunities for students to apply their knowledge and discuss the material further with a faculty facilitator

Measur es of Quality – AAMC GQ BASIC SCIENCES Geisel mean 2008 Geisel mean 2009 Geisel mean 2010 Geisel mean 2011 Geisel mean 2012 All schools means 2012 Behavioral Science Biochemistry Biostatistics/Epidemiology Genetics Gross anatomy/Embryology Histology Immunology Microbiology Neuroscience On Doctoring Pathology Pathophysiology of Disease Pharmacology Physiology Measures of Quality – AAMC GQ “Indicate how well you think that instruction in Biostatistics/Epidemiology prepared you for clinical clerkships and electives.” [1=poor; 2=fair; 3=good; 4=excellent]

Measures of Quality – Step I 2009*2010*2011*2012* Means TRADITIONAL CORE DISCIPLINES Biochemistry Biostatistics/Epidemiology Genetics Gross anatomy/Embryology Histology/Cell Biology Microbiology/Immunology Pathology Pharmacology Physiology *values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

Measur es of Quality – AAMC GQ Year 1 courses Overall Satisfaction AY Human Anatomy and Embryology II 4.57 Human Anatomy and Embryology I 4.33 Physiology-Renal 4.32 Biostatistics and Epidemiology 4.28 Metabolic Basis of Disease 4.23 Biochemical and Genetic Basis of Medicine 3.97 Neuroscience 3.90 Physiology-Endocrine 3.81 CTO 3.79 Basic Science of Microbial Disease 3.78 Virology 3.71 General Pathology 3.46 Physiology-Cardiovascular 3.45 Physiology-Respiration 3.38 Immunology 2.35 Measures of Quality – Course Reviews scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality – Course Reviews Biostats 2010 (54%)* Biostats 2011 (23%)* Biostats 2012 (96%)* Biostats 2013 (71%)* Overall satisfaction of course Overall usefulness of lectures N/A Overall usefulness of small groups N/A Overall usefulness of course materials N/A Congruence of assessment questions to material emphasized in course *student participation rate on course evaluation

Measures of Quality – Student Comments Summary of Student Comments Particularly Well Done Overall organization, relevance, and clarity of course Small groups, including ability to choose between preparing ahead and working in class Course notes, option to receive a binder Could Have Been Better Small groups could be more efficient Notes could correlate better with lectures

Measures of Quality – Student Comments Suggestions Final lecture (screening) should not be required Make lecture slides available before class Utilize iPads to write on and project rather than chalkboard Visibility on Echo360 Utilize more screencasts Provide only one facilitator per group Train facilitators more

Measures of Quality – Student Comments Discrepancy between those with a statistics background and those without “I have never taken a stats class before and I found that the notebook text was written for someone with a little bit more understanding than a beginner. There were a few concepts that were not fully explained (power, confidence intervals) that I figured out with the help of my small group professor and Khan videos. I think that this course should be for people who haven't taken stats, and it should be taught at that level. The class wasn't particularly difficult, it was just hard to be with a group of peers who have an extensive knowledge of the topic and to try to learn at a speed that worked for both of us.” “I did not learn very much from this course as I've already taken stats and healthcare research design. I know multiple students have similar or more background and it might be worth restructuring sections or allowing students to test out, rather than having us complete what feels like "busy work" for a full term.”

Measures of Quality – Student Comments Discrepancy between those with a statistics background and those without Heavy background ● Too slow, especially at beginning ● Too much review ● Not difficult enough ● Small groups helpful, but should be more efficient, shorter, or optional No background ● Appreciated slow pace ● Liked beginning with the basics ● Felt some things needed more explanation ● Small groups were extremely helpful Suggestions ●Sort groups based on previous stats experience (2) ●Make small groups optional, especially for students with stats background (4) ●Allow students to test out of course (4)

Summary regarding Measures of Quality This course is consistently well-received by students and rated in the “very good” to “excellent” range – the course director and her faculty are to be commended for their efforts Students do very well on Step I in this discipline Students suggest minor adjustments to make the course even better The major issue that the course director should address is how to meet the needs of two groups of students – those with a no prior knowledge of the material, and those with significant knowledge

Recommendations The session objectives in Ilios and those distributed to the students need to be consistent – the “old” set in the course materials needs to be replaced with the correct version that is in Ilios (it would also be a good idea for the course director to take this opportunity to review these objectives to make sure they still reflect the content that is covered in the course) The course director should consider the student suggestions and make changes where appropriate; one suggestion in particular (providing materials before class) should be addressed to comply with Y1 policy (i.e. class materials should be posted at least 24 hours before the scheduled class session)

Recommendations The course director should consider how to meet the needs of two different populations of students that take the course. Multiple ideas were discussed during the subcommittee meeting and we are confident there is a viable solution.

Action Plan (1) 1.Update objectives in the course materials that are not matched to those in Ilios 2.Upgrade lecture/notes in Lecture on Public Health Emergencies & Population Epidemiology 3.Request lecturers provide slides >=24 hours before 4.Ask lecturers not to use chalk board but learn how to “write” on slides at podium 5.Revise power/sample size notes for clarity

Action Plan (2) Consider / discuss: a)Adding a self-selected option for small group for students with background in the subject – journal club, discussion of a paper and its methods b)Optional v mandatory small groups c)Format & attendance at screening class which was experimental format last year