Critical Appraisal of: Systematic Review: Bisphosphanates and Osteonecrosis of the Jaw Basil Al-Saigh August 2006

What is Systematic Review? Based on protocol Involves critical appraisal Synthesis of the data Meta-analysis: Statistical combination of data

Systematic vs. Narrative Reviews Feature Narrative Systematic Question Broad; not stated Focused Search Unknown Comprehensive Article selection Not stated Criterion-based Uniformly applied Appraisal Variable Rigorous; critical Inferences May be evidence-based Evidence-based Advantages Breadth; traditional; integrative Resolve discord

Why do a Systematic Review? To examine the quality of the evidence To summarize the treatment benefit in a given therapeutic area – provide the best estimate of its direction and magnitude To resolve discordance between trials To investigate reasons for the discordance

Formulate the question Who are the subjects? Diagnostic criteria : The end point was jaw disease in association with use of bisphosphanates Population : Patients with cancers such as MM and metastatic CA taking IV Bisphosphanates were the target population. Setting : Case series and reports analyzed retrospectively using internet searches on MEDLINE and PubMed.

Formulate the question What is the intervention? Nature, Timing, Route and Placebo : Since these are case reports and series, the above does not apply when appraising this review Otherwise, must be taken into account

Formulate the question What are the outcomes? Mortality : No case reports or series reported mortality Morbidity : Defined as osteonecrosis of the Jaw, irrespective of location (mandible vs maxilla) in patients taking bisphosphanates

Formulate the question Conflict of interest? No funding received for this review One reviewer receives grants from Novartis What type of studies? Case reports and case series Included if showed acceptable documentation of disease and use of bisphosphanates irrespective of including information of sex of patient, site of lesion and type of bisphosphanate used

Locate Studies Systematic approach to avoid selection and publication bias : Several authors published more than 1 paper describing the same patients with osteonecrosis related to bisphosphanate use Such authors were contacted for confirmation and when this occurred, only date from the larger more recent publication were used

Locate Studies Use of multiple sources : Electronic: MEDLINE, PubMed Could have also used Embase, Cinahl Manual : Other references obtained from citations from retrieved articles. Otherwise, the same search words were used to search abstracts from meetings of the American Society of Clinical Oncology

Locate Studies Document the search strategy : Case studies and reports with search words that crossed : Osteonecrosis – Avascular Necrosis – Phosphorous Necrosis – Bisphosphanates – Diphosphanates Jaw diseases – Myeloma – Breast Cancer – Metastatic Cancer Search words do not include Osteoporosis. Under-representation of the effect of oral BPn on this condition? Uses of Reference Librarians : No mention of having used that

Select Studies Consider language restriction : English and foreign language articles used Increased the number of case reports and series Since this is a review of case reports, sample size is important for any meaningful initial conclusions

Select Studies Follow explicit inclusion and exclusion criteria : Inclusion criteria defined in the search words No search word for Osteoporosis Acceptable reports are those that documented disease AND use of the BPn, regardless of whether the sex of the patient, the site of lesion of the specific BPn used

Select Studies Consensus of two reviewers examining the study abstracts : 3 reviewers sited Rules to settle disagreements : None sited

Assess Study Validity Two reviewers independently review all studies and rate them re : Allocation concealment – no one guesses randomization Double blinding – neither patient nor investigator knows the treatment received Adequate follow-up – avoid attrition bias Blind outcome assessment – avoid detection bias Intention-to-treat analysis Not applicable here, since these were reviews of case reports and series

Analyze the Results Consider the clinical homogeneity Consider the statistical homogeneity Cochrane’s q test for heterogeneity If sufficiently homogeneous, conduct a Meta-analysis

Meta-analysis Statistical combination of the results of multiple studies, weighed by the inverse of the variance of the estimate in each study (larger sample sizes receive more weight)

Sensitivity Analysis How sensitive is the review to the specific methods used? Change the inclusion criteria : Add Search word of Osteoporosis? Exclude foreign language/unpublished/poor quality studies : No mention of who did the translation of foreign studies Important in this review, as it would increase sample size and give more meaningful conclusions

Interpretation and Recommendation Scientific : Limitations of the review & Identify areas of deficient knowledge requiring further research : Are alternate dosing strategies helpful in preventing this condition yet providing the therapeutic effect of the drug? What is the role of monitoring bone turnover in patients at risk?

Interpretation and Recommendation Need for a staging system to guide treatment? Criteria for dx of early changes that precede and predict bone exposure are desirable Studies needed to elucidate additional RF for the development of this condition RCT’s to determine the most effective treatment protocols for patients with this condition

Interpretation and Recommendation Clinical : Place results in the context of other evidence Illustrate how the results might be used clinically Incorporate into clinical recommendations and guidelines

Interpretation and Recommendation Management recommendations : Review of Expert panel recommendations of 2004, reviewers own experience and that of colleagues Group 1 – About to begin IV BPn therapy / Patients with OP and taking oral medications Group 2 – Patients receiving BPn therapy with no ON of the Jaw Group 3 – Patients with ON of the Jaw

Interpretation and Recommendation Recommendations on the discontinuation of BPn therapy : From the context of evidence of the long half life of BPn, d/c of therapy might not be beneficial after development of ON of the Jaw From the context of anecdotal reports of resolution of ON of the Jaw after several months of therapy cessation, d/c of the medication may be beneficial a few weeks before and after dental surgery as well as after development of ON of the Jaw

Interpretation and Recommendation Many patients have OP Review relevant because it highlights a dose dependant effect of BPn use and development of ON the Jaw Helps to reassure patients Highlights RF ex. Dental procedures to guide management Precipitates multiple questions for research avenues

Critical Appraisal of: Systematic Review: Bisphosphanates and Osteonecrosis of the Jaw Basil Al-Saigh August 2006