Presentation on theme: "Teaching the Art of Literature Analysis"— Presentation transcript:
1 Teaching the Art of Literature Analysis Evan Williams, PharmD, BCPSAssistant Professor of Pharmacy PracticeHusson University School of Pharmacy
2 Overview Why literature evaluation is important and why people hate it How to make literature less terrible for students and preceptorsWhat parts of the study are important and what is beyond essential
3 Pre Test QuestionsStudents should focus on the statistical methods used as it relates to internal validity of the findings more so than the external validity of the study when discussing journal articles.True or False
4 Pre Test Questions Journal Article Presentations should Focus on an area of practice relevant to the practice siteTie into some other project or problem at the practice siteBe flexible enough to fit into the amount of time a practice site or preceptor can devote to the assignmentAll of the above
5 Pre Test QuestionsThe following study for gorillastatin produced the following results after 5 years of follow up. Based on the results in the table, about how many patients do I need to treat with gorillastatin for 5 years to prevent one CAD event?a) 5 b) 8 c) 12 d) 16
6 ObjectivesDiscuss reasons literature evaluation is an essential skill for all pharmacistsIdentify components of a quality “journal club” formatDescribe ways to make student literature review useful to your practiceBriefly distinguish between useful and confounding statistical methods and demonstrate ability to calculate NNT and NNH
8 Literature Evaluation is Important! Essential skill for any clinicianEveryone wants to follow “evidence-based medicine”How do you know if recommended treatment is evidence based?Needed to keep up with emerging therapiesNew drugsNew guidelinesIf you don’t do this, the next generation of pharmacists will be followers instead of leaders regarding drug therapy
9 Why People Hate Literature Dry, boring reading50 Shades of Primary Literature?Time considerationsPerceived lack of utilityThis is all summarized in Pharmacists’ Letter anywayFear of statisticsWhat the devil is the difference between a t-test, f- test, and a z-test, anyway?
10 Why Students Hate “Journal Club” There is nothing “clubby” about itWe force them to present some random article and belittle them for not knowing stuff we never taught themIt is detached from other learning outcomes“Pick a recent article that has something to do with your internal medicine rotation.”They pick terrible articles because of thisThey lack good examples of what we are expecting
11 How to Make This Better Decide what the desired learning outcome is Presentation skills?A refined, grammatically perfect handout?Identify all of the trial’s strengths and weaknesses?Distinguish important clinical implications of the trial presented?Once the outcome is identified, make sure this is communicated to the student
12 How to Make Lit Eval Better Selection of ArticlesStudent vs. Preceptor SelectionEnsure that the article relates to something happening on rotation!This will make it relevant and usefulType of ArticleOriginal ResearchReview ArticleGuideline Evaluation
13 How to Make Lit Eval Better Selection of ArticlesGood vs. Bad ArticlesClinical vs. Surrogate OutcomesSketchy statisticsPrimary outcomes vs. Secondary outcomesNew vs. OldOlder articles aren’t bad or irrelevant!!If the student doesn’t know it, it is a learning opportunity
14 How to Make Lit Eval Better Identify what you’d like the student to presentBackgroundDo you need this?Needs to be quickMethodsBriefThese are fairly standardBreeze through the stats
15 How to Make Lit Eval Better Identify what you’d like the student to presentResultsBest presented in a table formatIdentify primary vs secondary outcomesDistinguish between clinical and statistical significanceDoes power matter?Calculate NNT and NNH
16 How to Make Lit Eval Better Identify what you’d like the student to presentDiscussion should take the most timeIs the outcome valid based on the test used?ConfoundersAre they looking at the right groups of patients?Influence of uncontrolled variables?Statistical vs. clinical significance?Does this change my practice?ConclusionsDo you agree with the author’s conclusion?
17 How to Make Lit Eval Better Presentation SettingBig time presentation and roast?Conversational?Panel Discussion?CE style?Do what makes sense for your practice site!Make this useful to you
18 Other Points For Improvement Give students enough timeCheck in to make sure progress is being madeSuggest addressing questions prior to the presentation to improve the quality of the presentationRead the article!Show the student a clear example of a good handout to followMany are available, so pick one that you likeStudents tend to do better with bullet pointsMake them create their own tables
19 What You Need to KnowIn order to properly evaluate the student’s critique, you need to have identified the key points that should have pointed outWhat are the results?Are the results valid?How can I use the results in my practice?
20 What Are the Results? Any statistically significant differences? p-value vs confidence intervalsLittle p-values don’t make results more significantConfidence intervals can give an impression of how precise the results areConfidence intervals that cross unity are not significantClinical significance?
21 What Are the Results? Number Needed to Treat Good way to quantify clinical significance1/difference in the event rate between groups25 of 500 people in the placebo group died - (5%)10 of 550 people in the drug group died – (1.8%)The difference between the groups is 3.2%, or 0.0321/0.032 = ~ 31If I treat 31 people with the drug, I will prevent one deathCompare this to the Number Needed to Harm
22 What Are the Results? Number Needed to Harm Same calculation, but for adverse events10 of 500 people had major bleeding in the placebo group – (2%)22 of 550 people had major bleeding in the drug group – (4%)The difference between the groups is 2%, or 0.021/0.02 = 50If I give 50 people the drug, I will cause 1 major bleedDoes the risk vs benefit ratio seem reasonable?
23 Are the Results Valid?Was the primary endpoint clearly stated and reported?Was it relevant?Surrogate vs clinical endpointsComposite endpointsWas the designed to reveal the primary endpoint?Powered enough to detect a difference if one wasn’t found?Does the discussion focus on subgroup analyses?
24 Are the Results Valid? Where the right people included? Intent to Treat (ITT) or Per Protocol Analysis?Inclusion and exclusion criteriaDo they control for confounders?Do they limit external validity?Was bias effectively dealt with?BlindingValid control groupsPlacebo vs active control?
25 How Can I Apply the Results? NNT vs NNHAre the patients studied similar to my patients?How does this evidence jive with other studies regarding the same subject?Is this study important enough to shift clinical practice?
26 Make This Useful to You New Drugs New Guidelines What studies got this approved?Who was looked at?Side effect profiles?New GuidelinesCheck the referencesThings that have strong/low evidenceThings you don’t know/always get questions aboutWhen is aspirin indicated?Evidence that vaccines don’t cause autism?Does LDL really correlate with MI?Why do we still use warfarin?
27 Summary Pharmacists need to be able to evaluate literature We should be showing students why this is importantMake sure journal article presentations focus on preparing pharmacists that can critically evaluate literaturePresentation style, format, handouts, length can all vary and should be adapted to the practice siteFocus on the key issues within the studies presentedDon’t get bogged down in statisticsConsider the clinical impactMake the exercise beneficial for you and the student
28 Post Test QuestionsStudents should focus on the statistical methods used as it relates to internal validity of the findings more so than the external validity of the study when discussing journal articles.True or False
29 Post Test Questions Journal Article Presentations should Focus on an area of practice relevant to the practice siteTie into some other project or problem at the practice siteBe flexible enough to fit into the amount of time a practice site or preceptor can devote to the assignmentAll of the above
30 Post Test QuestionsThe following study for gorillastatin produced the following results after 5 years of follow up. Based on the results in the table, about how many patients do I need to treat with gorillastatin for 5 years to prevent one CAD event?a) 5 b) 8 c) 12 d) 16