1. Critical Appraisal of: Systematic Review: Bisphosphanates and Osteonecrosis of the Jaw
Basil Al-Saigh
August 2006

2. What is Systematic Review?
Based on protocol
Involves critical appraisal
Synthesis of the data
Meta-analysis:
Statistical combination of data

3. Systematic vs. Narrative Reviews
Feature
Narrative
Systematic
Question
Broad; not stated
Focused
Search
Unknown
Comprehensive
Article selection
Not stated
Criterion-based
Uniformly applied
Appraisal
Variable
Rigorous; critical
Inferences
May be evidence-based
Evidence-based
Advantages
Breadth; traditional; integrative
Resolve discord

4. Why do a Systematic Review?
To examine the quality of the evidence
To summarize the treatment benefit in a given therapeutic area – provide the best estimate of its direction and magnitude
To resolve discordance between trials
To investigate reasons for the discordance

6. Formulate the question
Who are the subjects?
Diagnostic criteria :
The end point was jaw disease in association with use of bisphosphanates
Population :
Patients with cancers such as MM and metastatic CA taking IV Bisphosphanates were the target population.
Setting :
Case series and reports analyzed retrospectively using internet searches on MEDLINE and PubMed.

7. Formulate the question
What is the intervention?
Nature, Timing, Route and Placebo :
Since these are case reports and series, the above does not apply when appraising this review
Otherwise, must be taken into account

8. Formulate the question
What are the outcomes?
Mortality :
No case reports or series reported mortality
Morbidity :
Defined as osteonecrosis of the Jaw, irrespective of location (mandible vs maxilla) in patients taking bisphosphanates

9. Formulate the question
Conflict of interest?
No funding received for this review
One reviewer receives grants from Novartis
What type of studies?
Case reports and case series
Included if showed acceptable documentation of disease and use of bisphosphanates irrespective of including information of sex of patient, site of lesion and type of bisphosphanate used

10. Locate Studies
Systematic approach to avoid selection and publication bias :
Several authors published more than 1 paper describing the same patients with osteonecrosis related to bisphosphanate use
Such authors were contacted for confirmation and when this occurred, only date from the larger more recent publication were used

11. Locate Studies Use of multiple sources : Electronic: MEDLINE, PubMed
Could have also used Embase, Cinahl
Manual :
Other references obtained from citations from retrieved articles.
Otherwise, the same search words were used to search abstracts from meetings of the American Society of Clinical Oncology

12. Locate Studies Document the search strategy :
Case studies and reports with search words that crossed :
Osteonecrosis – Avascular Necrosis – Phosphorous Necrosis – Bisphosphanates – Diphosphanates
Jaw diseases – Myeloma – Breast Cancer – Metastatic Cancer
Search words do not include Osteoporosis.
Under-representation of the effect of oral BPn on this condition?
Uses of Reference Librarians :
No mention of having used that

13. Select Studies Consider language restriction :
English and foreign language articles used
Increased the number of case reports and series
Since this is a review of case reports, sample size is important for any meaningful initial conclusions

14. Select Studies Follow explicit inclusion and exclusion criteria :
Inclusion criteria defined in the search words
No search word for Osteoporosis
Acceptable reports are those that documented disease AND use of the BPn, regardless of whether the sex of the patient, the site of lesion of the specific BPn used

15. Select Studies
Consensus of two reviewers examining the study abstracts :
3 reviewers sited
Rules to settle disagreements :
None sited

16. Assess Study Validity
Two reviewers independently review all studies and rate them re :
Allocation concealment – no one guesses randomization
Double blinding – neither patient nor investigator knows the treatment received
Adequate follow-up – avoid attrition bias
Blind outcome assessment – avoid detection bias
Intention-to-treat analysis
Not applicable here, since these were reviews of case reports and series

17. Analyze the Results Consider the clinical homogeneity
Consider the statistical homogeneity
Cochrane’s q test for heterogeneity
If sufficiently homogeneous, conduct a Meta-analysis

18. Meta-analysis
Statistical combination of the results of multiple studies, weighed by the inverse of the variance of the estimate in each study (larger sample sizes receive more weight)

19. Sensitivity Analysis
How sensitive is the review to the specific methods used?
Change the inclusion criteria :
Add Search word of Osteoporosis?
Exclude foreign language/unpublished/poor quality studies :
No mention of who did the translation of foreign studies
Important in this review, as it would increase sample size and give more meaningful conclusions

20. Interpretation and Recommendation
Scientific :
Limitations of the review & Identify areas of deficient knowledge requiring further research :
Are alternate dosing strategies helpful in preventing this condition yet providing the therapeutic effect of the drug?
What is the role of monitoring bone turnover in patients at risk?

21. Interpretation and Recommendation
Need for a staging system to guide treatment?
Criteria for dx of early changes that precede and predict bone exposure are desirable
Studies needed to elucidate additional RF for the development of this condition
RCT’s to determine the most effective treatment protocols for patients with this condition

22. Interpretation and Recommendation
Clinical :
Place results in the context of other evidence
Illustrate how the results might be used clinically
Incorporate into clinical recommendations and guidelines

23. Interpretation and Recommendation
Management recommendations :
Review of Expert panel recommendations of 2004, reviewers own experience and that of colleagues
Group 1 – About to begin IV BPn therapy / Patients with OP and taking oral medications
Group 2 – Patients receiving BPn therapy with no ON of the Jaw
Group 3 – Patients with ON of the Jaw

24. Interpretation and Recommendation
Recommendations on the discontinuation of BPn therapy :
From the context of evidence of the long half life of BPn, d/c of therapy might not be beneficial after development of ON of the Jaw
From the context of anecdotal reports of resolution of ON of the Jaw after several months of therapy cessation, d/c of the medication may be beneficial a few weeks before and after dental surgery as well as after development of ON of the Jaw

25. Interpretation and Recommendation
Many patients have OP
Review relevant because it highlights a dose dependant effect of BPn use and development of ON the Jaw
Helps to reassure patients
Highlights RF ex. Dental procedures to guide management
Precipitates multiple questions for research avenues

26. Critical Appraisal of: Systematic Review: Bisphosphanates and Osteonecrosis of the Jaw
Basil Al-Saigh
August 2006