Evidence-based Dental Practice Developing guidelines or clinical recommendations Slide #1 This lecture follows the previous online lecture on evidence based dental practice. The focus of this lecture is on how to practice evidence-based dentistry (or EBD). The key to implementing EBD practice is to develop concise, evidence-based, clinical protocols that practitioners can follow in provision of patient care. The EBD clinical protocols, as well as practice guidelines, should be based on comprehensive and critical analysis of the evidence to answer clinical question(s) related to a targeted subject area.

Outline The practice of EBD at a community health center Define clinical questions Locate and critique systematic reviews If no reviews are available, locate and critique primary research Develop clinical protocols Implement the protocols in practice Slide #2 The objective of this lecture is to describe how to practice EBD at a community health center. In this lecture, the steps in developing EBD clinical protocols will be described. These steps include defining clinical questions; locating and critiquing systematic reviews or in the absence of such reviews, locating and critiquing primary research studies; developing clinical protocols; and implementing the protocols in clinical practice.

What is EBD practice? [EBD] is a set of principles and methods intended to ensure that, to the greatest extent possible, medical [dental] decisions, guidelines, and other types of policies are based on and consistent with good evidence of effectiveness and benefit. Eddy M. Health Affairs 2005:24:9-17 Slide #3 EBD practice is defined as a set of principles and methods intended to ensure that to the greatest extent possible, medical [dental] decisions, guidelines, and other types of policies are based on and consistent with good evidence of effectiveness and benefit. Scientific scrutiny and information should be the basis for healthcare decisions. At each stage of the dental care process, there are questions that can be answered using the scientific method. For example, the conventional dental exam relies on the use of explorers to detect caries. A scientific analysis of whether explorers increase the accuracy of detection compared with only visual examination can be conducted. The decision may change how explorers are used or not used. Another example may include the type of cavity preparation for different carious lesions. The scientific method can be used to define the best design that achieves longer preservation of tooth structure and retention of restorations.

EBD Practice at a Community Health Center Step 1: Identify procedures that require standardization or reduction in variability Review of procedures provided, “word-of-mouth” Review of cost variation Review of charts Determine the benefits and harms of different approaches Slide #4 The first step in the practice of EBD is to identify procedures or decisions that require standardization or reduction in variability among dental providers. This task can be accomplished via review of practice patterns or why some providers are outliers in provision of care or diagnosis. Chart audits as well as “word of mouth” can also reveal variations among providers. Cost of providing care is another variable that can be used to assess the need for standardization. Some care provided at the clinic may cause harm. It is important to be vigilant in your quality assurance program about decisions that lead to overtreatment, inappropriate care, and low quality care. EBD practice is a tool to reduce harm to patients.

EBD Practice at a Community Health Center Step 2: Share the data with the practitioners (dentists and dental hygienists) and decide on the need for developing clinical protocols (guidelines), or standards Guidelines are statements that should be followed, but can be adapted based on patients’ needs and conditions. Clinical protocols define in steps how guidelines are implemented in clinical care. Standards: are statements and policies that must be followed without modification. Slide #5 After you have identified one or more area for improvement (preferably one at a time) , share the data with your providers. Discuss and encourage the development of clinical protocols, and sometimes, standards for care (policies that should all follow with no variation). Guidelines are statements that should be followed but can be adapted based on patients’ needs and conditions. Clinical protocols define, in steps, how guidelines are implemented in clinical care. Standards are statements and policies that must be followed without modification (step-by-step implementation of guidelines).

EBD Practice at a Community Health Center Step 2: Form a committee to steer the process Representatives from different groups (dentists, hygienists, medical and allied health) Content consultants from dental schools Other consultants Methods Search for evidence EBD experts Slide #6 After definition of the problem area and agreement to develop guidelines or protocols or standards, a committee should be formed to steer the process. The committee should have representatives from the different clinical teams, content consultants, and other consultants such as methods experts, librarians, and individuals with expertise in the EBD process. The committee can meet online because many times local consultants are not available.

EBD Practice at a Community Health Center Step 3: Decide on the clinical question(s) which, if answered, can resolve the clinical issue defined in the first step. Use a modified PICO standard: Population Intervention Conditions or disease state Outcomes Slide #7 The working team should next define the clinical question or questions, which, if answered, can resolve the clinical issue defined in the first step. The team should define the population characteristics, intervention, conditions or disease state and the desired outcomes. For example, one area with some different practices in your clinic may be the use of local intraoral antibiotic chips in patients with moderate periodontitis may an area. The PICO question may be as follows: In adult patients (P) with moderate periodontitis (C) the use of locally-delivered antibiotics as adjunct to scaling and root planing (I) improves the control of progression of loss of periodontal attachment (O).

EBD Practice at a Community Health Center Step 4: Using PICO, define inclusion and exclusion criteria for guidelines or standards Conduct a systematic review of all guidelines or standards relevant to the question Retrieve and review the documents using the inclusion and exclusion criteria Select a preliminary set for analysis Prepare a summary table of standards or guidelines using the PICO format Slide #8 Using the PICO question, the team should define the inclusion and exclusion criteria of evidence. For example, the question does not specify whether the patients have moderating chronic conditions such as diabetes. The team should decide whether to include or exclude studies where the intervention was applied in patients with chronic medical conditions. Using standard filters for searching for systematic reviews, and with the help of an experienced librarian, the team should search for all guidelines or standards relevant to the question. From the list of identified systematic review or guidelines, the team should select relevant documents based on the inclusion/exclusion criteria defined before the start of the search. The team then selects a preliminary set for in-depth analysis. After a review and critique of the preliminary set a final set should be selected and summarized in a table that all team members and providers at the clinic can review.

EBD Practice at a Community Health Center Step 4: Rate the quality of the guidelines using the GRADE system Guyatt GH, Oxman AD, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ, for the GRADE Working Group. Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ Slide #9 Each set of guidelines or protocols should be rated for quality using the GRADE system. The GRADE system is available on the website of this lecture.

The Grade System Slide #10 The GRADE system evaluates the quality of evidence used to develop the guideline, the uncertainty about the balance between desirable and undesirable effects, uncertainty or variability in values and preferences; and uncertainty about whether the intervention represents a wise use of resources.

The GRADE system Slide 11 Quality of evidence is important in developing quality recommendations. The quality and consistency of evidence impacts the strength of the recommendation. The Grade system evaluates both these characteristics and considers as well the strength of the evidence, health harms and benefits, and net cost benefits. The system assigns a level of strength to recommendations based on the structured appraisal described in the article attached to this lecture.

The GRADE system Slide #12 The Grade system defines the quality of evidence based on whether further research will be needed or can impact the recommendation. High quality evidence is unlikely to change the confidence in the recommendation whereas very low quality evidence is based on very uncertain estimate of effect of the recommendation.

EBD Practice at a Community Health Center Step 6: Decide whether the current guidelines or standards are adequate for your institution or you need to revise them The only change that can be made is not in the evidence, but rather in the conditions which dictate implementation of the guidelines (The C in PICO) E.g. Your organization may decide to apply topical fluorides based on the risk status of adult patients, if the original evidence supporting the application is based on expert opinion. Slide #13 After rating the quality of the guidelines, the next step is to decide whether the current guidelines are adequate for your institution. The only change that should be made is in the conditions which dictate implementation of the guidelines. Evidence is dictated on the status of the information found in the review of literature conducted prior to defining the guideline. Hence, to change the guidelines will require reviewing and analyzing the supporting scientific evidence.

EBD Practice at a Community Health Center Step 6: Disseminate the guideline Train the practitioners to apply the guidelines Conduct calibration exercises using case summaries or patients Monitor the implementation using reviews and chart audits Review and disseminate the findings Slide #14 After the initial phases are completed and a guideline or protocol is developed, the team should disseminate the guideline and initiate training of practitioners to understand and apply the question. Calibration exercises may use case studies or live patients. Consistency in applying the guideline should be evaluated using chart audits or review of cases. The findings of chart audits or examination of patients should be disseminated to the practitioners. Consistent feedback to practitioners is an important step to ensure compliance.

EBD Practice at a Community Health Center Step 4 Revised: If no standards or guidelines are available to answer your PICO question, then search for systematic reviews or the most appropriate evidence to answer the question This process is more time consuming (i.e. more costly) and requires the help of experienced reviewers Slide #15 If no standards or guidelines are available to answer your PICO question, then search for systematic reviews or the most appropriate evidence to answer the question. This process is more time consuming (i.e. more costly) and requires the help of experienced reviewers.

EBD Practice at a Community Health Center Step 5 Revised: Select systematic reviews or primary studies for consideration for further analysis Agree (committee decision) on which studies or reviews will be included in the development of guidelines Summarize the evidence using the PICO format Rate the quality of each bit of evidence using the tools in the ADA EBD website Slide #16 Using the same steps as before, the team should select systematic reviews or primary studies using pre-defined inclusion/exclusion criteria for further analysis. The committee should agree on which studies should be included and why studies should be excluded. The included studies should be summarized in a table and all studies should be rated for quality.

Appraisal Tools in the EBD Website Slide #17 There are different systems for rating the quality of studies. Quality criteria vary based on the study designs. Links to appraisal tools are available in the ADA EBD websites.

EBD Practice at a Community Health Center While each study should be critiqued for its strengths and weaknesses, decisions on best practices should rely on the best available evidence. In addition to the GRADE system, there are several other classification systems The most applicable to community health centers, in my opinion, is the one used in the UK National Health Service. Slide #18 Decisions on best practices should rely on the best available evidence. For many questions in dental practice, the best evidence is no more than “expert opinions.” The team can decide on what is the best approach when high quality evidence from studies is not available. In other words, the team forms an expert panel. In addition to the GRADE system for developing and evaluating guidelines, the UK National Health Service has an excellent system.

EBD Practice at a Community Health Center UK National Health Service Level A: Consistent randomized controlled clinical trial, cohort trial, cohort study, … clinical decision rule validated in different populations. Level B: Consistent Retrospective Cohort, Exploratory Cohort, Ecological Study, Outcomes Research, case-control study or extrapolations from level A studies. Level C: Case-series study or extrapolations from level B studies. Level D: Expert opinion without explicit critical appraisal, or based on physiology (or microbiology, biology), bench research.... Slide #19 The UK system has the following levels or grades of evidence: Level A: Consistent randomized controlled clinical trial, cohort trial, cohort study, … clinical decision rule validated in different populations. Level B: Consistent Retrospective Cohort, Exploratory Cohort, Ecological Study, Outcomes Research, case-control study or extrapolations from level A studies. Level C: Case-series study or extrapolations from level B studies. Level D: Expert opinion without explicit critical appraisal, or based on physiology (or microbiology, biology), bench research....

EBD Practice at a Community Health Center Step 6 After summarizing and grading the evidence, the panel should decide on how to answer the PICO question: State the guidance by population group or specific targets State what are the key points (what, when, and how) for each intervention Identify conditions when the guidance should be applied (risk factors, patient factors, operator factors) Define the anticipated outcomes that should be collected, analyzed and followed Slide #20 After summarizing and grading the evidence, the panel should decide on how to answer the PICO question: State the guidance by population group or specific targets State what are the key points (what, when, and how) for using each intervention Identify conditions when the guidance should be applied (risk factors, patient factors, operator factors) Define the anticipated outcomes that should be collected, analyzed and followed

EBD Practice at a Community Health Center Step 7 Disseminate the guidelines, seek comments, and edit the text to clarify it Prepare training of staff to implement the guidelines Define targets for follow-up in implementation Calibrate the providers Assess process and final outcomes Be vigilant for new evidence and modify the guidelines, if necessary Review the application and usability of the guidelines at least once year Slide #21 The team should prepare plans to: Disseminate the guidelines, seek comments, and edit to clarify the text Prepare training of staff to implement the guidelines Define targets for follow-up in implementation Calibrate the providers. Calibration refers to training to ensure consistent application of the guidelines by an individual provider as well as among a group of providers Assess process and final outcomes Be vigilant for new evidence and modify the guidelines, if necessary Review the application and usability of the guidelines at least once year

EBD Practice as an Individual Dentist or Dental Hygienist Clinical question Has your institution developed EBD guidelines to answer the question? If no, search for published guidelines and relevant evidence Critique the evidence using the tools referenced in the previous slides Make a decision on how best to answer the clinical question and strength of evidence supporting your decision Communicate all information: evidence for/against, risks and benefits to the patient Training and calibration of the providers Quality assurance Slide #22 In summary, applying EBD principles in practice requires the following steps: Definition of a clear clinical question A search for EBD guidelines developed by your institution; if no guidelines have been developed, then a search for published guidelines or relevant evidence. Critique of the guidelines, reviews, or primary research Selection and rating of the best evidence Drafting the guidelines Dissemination of the drafted guidelines for comments Preparing the final version of the guidelines Dissemination of the guidelines Training and calibration of the providers to use the guidelines in practice Finally, the EBD guidelines should be incorporated in quality assurance programs to ensure that practitioners are provide the best care for the right patient and at the right time