1. Teaching the Art of Literature Analysis
Evan Williams, PharmD, BCPS
Assistant Professor of Pharmacy Practice
Husson University School of Pharmacy

2. Overview Why literature evaluation is important and why people hate it
How to make literature less terrible for students and preceptors
What parts of the study are important and what is beyond essential

3. Pre Test Questions
Students should focus on the statistical methods used as it relates to internal validity of the findings more so than the external validity of the study when discussing journal articles.
True or False

4. Pre Test Questions Journal Article Presentations should
Focus on an area of practice relevant to the practice site
Tie into some other project or problem at the practice site
Be flexible enough to fit into the amount of time a practice site or preceptor can devote to the assignment
All of the above

5. Pre Test Questions
The following study for gorillastatin produced the following results after 5 years of follow up. Based on the results in the table, about how many patients do I need to treat with gorillastatin for 5 years to prevent one CAD event?
a) 5 b) 8 c) 12 d) 16

6. Objectives
Discuss reasons literature evaluation is an essential skill for all pharmacists
Identify components of a quality “journal club” format
Describe ways to make student literature review useful to your practice
Briefly distinguish between useful and confounding statistical methods and demonstrate ability to calculate NNT and NNH

7. Are your students this last guy?

8. Literature Evaluation is Important!
Essential skill for any clinician
Everyone wants to follow “evidence-based medicine”
How do you know if recommended treatment is evidence based?
Needed to keep up with emerging therapies
New drugs
New guidelines
If you don’t do this, the next generation of pharmacists will be followers instead of leaders regarding drug therapy

9. Why People Hate Literature
Dry, boring reading
50 Shades of Primary Literature?
Time considerations
Perceived lack of utility
This is all summarized in Pharmacists’ Letter anyway
Fear of statistics
What the devil is the difference between a t-test, f- test, and a z-test, anyway?

10. Why Students Hate “Journal Club”
There is nothing “clubby” about it
We force them to present some random article and belittle them for not knowing stuff we never taught them
It is detached from other learning outcomes
“Pick a recent article that has something to do with your internal medicine rotation.”
They pick terrible articles because of this
They lack good examples of what we are expecting

11. How to Make This Better Decide what the desired learning outcome is
Presentation skills?
A refined, grammatically perfect handout?
Identify all of the trial’s strengths and weaknesses?
Distinguish important clinical implications of the trial presented?
Once the outcome is identified, make sure this is communicated to the student

12. How to Make Lit Eval Better
Selection of Articles
Student vs. Preceptor Selection
Ensure that the article relates to something happening on rotation!
This will make it relevant and useful
Type of Article
Original Research
Review Article
Guideline Evaluation

13. How to Make Lit Eval Better
Selection of Articles
Good vs. Bad Articles
Clinical vs. Surrogate Outcomes
Sketchy statistics
Primary outcomes vs. Secondary outcomes
New vs. Old
Older articles aren’t bad or irrelevant!!
If the student doesn’t know it, it is a learning opportunity

14. How to Make Lit Eval Better
Identify what you’d like the student to present
Background
Do you need this?
Needs to be quick
Methods
Brief
These are fairly standard
Breeze through the stats

15. How to Make Lit Eval Better
Identify what you’d like the student to present
Results
Best presented in a table format
Identify primary vs secondary outcomes
Distinguish between clinical and statistical significance
Does power matter?
Calculate NNT and NNH

16. How to Make Lit Eval Better
Identify what you’d like the student to present
Discussion should take the most time
Is the outcome valid based on the test used?
Confounders
Are they looking at the right groups of patients?
Influence of uncontrolled variables?
Statistical vs. clinical significance?
Does this change my practice?
Conclusions
Do you agree with the author’s conclusion?

17. How to Make Lit Eval Better
Presentation Setting
Big time presentation and roast?
Conversational?
Panel Discussion?
CE style?
Do what makes sense for your practice site!
Make this useful to you

18. Other Points For Improvement
Give students enough time
Check in to make sure progress is being made
Suggest addressing questions prior to the presentation to improve the quality of the presentation
Read the article!
Show the student a clear example of a good handout to follow
Many are available, so pick one that you like
Students tend to do better with bullet points
Make them create their own tables

19. What You Need to Know
In order to properly evaluate the student’s critique, you need to have identified the key points that should have pointed out
What are the results?
Are the results valid?
How can I use the results in my practice?

20. What Are the Results? Any statistically significant differences?
p-value vs confidence intervals
Little p-values don’t make results more significant
Confidence intervals can give an impression of how precise the results are
Confidence intervals that cross unity are not significant
Clinical significance?

21. What Are the Results? Number Needed to Treat
Good way to quantify clinical significance
1/difference in the event rate between groups
25 of 500 people in the placebo group died - (5%)
10 of 550 people in the drug group died – (1.8%)
The difference between the groups is 3.2%, or 0.032
1/0.032 = ~ 31
If I treat 31 people with the drug, I will prevent one death
Compare this to the Number Needed to Harm

22. What Are the Results? Number Needed to Harm
Same calculation, but for adverse events
10 of 500 people had major bleeding in the placebo group – (2%)
22 of 550 people had major bleeding in the drug group – (4%)
The difference between the groups is 2%, or 0.02
1/0.02 = 50
If I give 50 people the drug, I will cause 1 major bleed
Does the risk vs benefit ratio seem reasonable?

23. Are the Results Valid?
Was the primary endpoint clearly stated and reported?
Was it relevant?
Surrogate vs clinical endpoints
Composite endpoints
Was the designed to reveal the primary endpoint?
Powered enough to detect a difference if one wasn’t found?
Does the discussion focus on subgroup analyses?

24. Are the Results Valid? Where the right people included?
Intent to Treat (ITT) or Per Protocol Analysis?
Inclusion and exclusion criteria
Do they control for confounders?
Do they limit external validity?
Was bias effectively dealt with?
Blinding
Valid control groups
Placebo vs active control?

25. How Can I Apply the Results?
NNT vs NNH
Are the patients studied similar to my patients?
How does this evidence jive with other studies regarding the same subject?
Is this study important enough to shift clinical practice?

26. Make This Useful to You New Drugs New Guidelines
What studies got this approved?
Who was looked at?
Side effect profiles?
New Guidelines
Check the references
Things that have strong/low evidence
Things you don’t know/always get questions about
When is aspirin indicated?
Evidence that vaccines don’t cause autism?
Does LDL really correlate with MI?
Why do we still use warfarin?

27. Summary Pharmacists need to be able to evaluate literature
We should be showing students why this is important
Make sure journal article presentations focus on preparing pharmacists that can critically evaluate literature
Presentation style, format, handouts, length can all vary and should be adapted to the practice site
Focus on the key issues within the studies presented
Don’t get bogged down in statistics
Consider the clinical impact
Make the exercise beneficial for you and the student

28. Post Test Questions
Students should focus on the statistical methods used as it relates to internal validity of the findings more so than the external validity of the study when discussing journal articles.
True or False

29. Post Test Questions Journal Article Presentations should
Focus on an area of practice relevant to the practice site
Tie into some other project or problem at the practice site
Be flexible enough to fit into the amount of time a practice site or preceptor can devote to the assignment
All of the above

30. Post Test Questions
The following study for gorillastatin produced the following results after 5 years of follow up. Based on the results in the table, about how many patients do I need to treat with gorillastatin for 5 years to prevent one CAD event?
a) 5 b) 8 c) 12 d) 16

31. Questions?
Thank you!