1. UNDERSTANDINGCLINICAL DECISION RULES Department of Emergency Medicine Stony Brook University Adam J Singer, MD Professor and Vice Chairman for Research

2. LEARNING OBJECTIVES/QUESTIONS –What are decision rules? –Why do we need decision rules? –How are decision rules developed? –How do I know if a decision rule is ready for primetime? –Decision rule for chest pain in young adults

3. HOW WE MAKE DECISIONS

4. WHAT ARE CLINICAL DECISION RULES? A Clinical Decision Rule is another term for Clinical Decision Support. By true definition, this means “Clinical decision support (CDS) provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care. CDS encompasses a variety of tools to enhance decision-making in the clinical workflow. These tools include computerized alerts and reminders to care providers and patients; clinical guidelines; condition-specific order sets; focused patient data reports and summaries; documentation templates; diagnostic support, and contextually relevant reference information, among other tools.” -The Office of the National Coordinator for Health Information Technology

5. A tool that helps clinicians make diagnostic and therapeutic decisions at the bedside Derived from original research and incorporate 3 or more variables from –History –Physical examination –Simple tests Attempts to reduce uncertainty in medical decision making DEFINITION OF CLINICAL DECISION RULE/AID

6. Why do we need decision rules? –Simplify complex decisions –Standardize decision making –Reduce uncertainty –Improve accuracy –Opportunity to reduce costs w/o compromising care Examples –Sore throat –PE –Depression

7. Non-educational –Relative advantage (better than old?) –Compatibility (represents existing beliefs) –Complexity –HCP characteristics (age, training, experience) –Practice setting –Incentives –Regulations –Patient factors BARRIERS TO IMPLEMENTATION OF CDR AND GUIDELINES

8. Educational strategies –Formal CME –Mailings –Opinion leaders –Audit with formal feedback –System reminders –Educational outreach –Strategies involving more than one intervention

9. DERIVATION –Creation or derivation of a rule VALIDATION –Prospective assessment of accuracy, reliability and potential impact of rule IMPLEMENTATION –Assessment of rule’s impact on patient care STAGES OF DEVELOPING CLINICAL DECISION RULES

11. HOW TO DEVLOP A CLINICAL DECISION RULE Identify condition in need of CDR –Common, variable practice, wasteful Construct list of potential predictors of outcome Typically includes –History, physical exam, basic lab tests –Obtained from prior studies or common sense Examine group of patients with suspected outcome to determine presence of predictors Determine presence or absence of outcome

12. Preform statistical analysis to determine which predictors best predict outcome and which can be omitted Logistic regression –mathematical equations Binary recursive partitioning –Builds a tree in which patients are split into smaller and smaller categories based on risk factors Neural networks

13. LOGISTIC REGRESSION

14. RECURSIVE PARTITIONING

15. PREDICTION TREE FOR TBI IN CHILDREN. Lancet 2009;374;1160

16. NEURAL NETWORKS A computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs

17. Simply derived rules are not ready for primetime Need to be validated in separate, independent settings May reflect associations between predictors and outcomes by chance alone Predictors may be idiosyncratic to specific population Clinicians may fail to implement rule comprehensively or accurately IS A DERIVED CLINICAL DECISION RULE READY FOR PRIMETIME

18. Were all important predictors included? Were important predictors present in a significant proportion of patients? Were all outcomes and predictors clearly defined? Were those assessing outcome blinded to predictors? Was sample size adequate? Were there enough patients with outcome of interest? Does the rule make sense? JUDGING THE QUALITY OF A DERIVED RULE

19. Note the most important predictors and consider them in your practice Consider giving less importance to variable found not to be predictive VALUE OF DERIVED RULE

20. Repeated application leads to similar results Applied prospectively to new population with different prevalence and spectrum of disease Rule performs similarly in a variety of settings and clinicians Rule work well when consciously applying it IMPORTANCE OF VALIDATION STUDIES

21. Were patients chosen in unbiased manner and do they represent wide spectrum of disease? Was there a blinded assessment of criterion standard for all patients? Was there an explicit and accurate interpretation of the predictor variables and the actual rule w/o knowledge of the outcome? Was there 100% follow up of those enrolled? JUDGING THE QUALITY OF A VALIDATED RULE

22. Usefulness of rule depends on predictive power Sensitivity and specificity of rule with 95% CI –OAR: 100% sensitive, 40% specific –All patients with fx were positive, only 40% of those w/o fx negative –If clinicians order XR only on positive rule: no misses and test avoided in 40% HOW TO INTERPRET THE RESULTS

23. Probability of target condition based on particular result of rule –Probability of PE based on Wells criteria –Low (3.4%; 95%CI, 2.2-5%), intermediate (28%, 95%CI, 23.4-32.2%), high (78%, 95%CI, 69.2-86%) Likelihood ratios –Detection of alcoholism based on CAGE scores –CAGE 0: 0.14, CAGE 1: 1.5, CAGE 2: 4.5; CAGE 4: 100 –Probability of alcoholism depends on prevalence and CAGE score

24. CDR demands time and energy Only warranted if they change physician behavior and improve outcomes or reduce costs Accurate rules that are not used or do not improve practice are worthless IMPACT ANALYSIS

25. Intuitive estimates (Gestalt) may be just as good Calculations may be too cumbersome Practical barriers to using CDR –Fear of litigation WHY EVEN ACCURATE RULES MAY NOT MATTER

26. Highest level of evidence that a rule is good Rarely done IMPACT STUDIES

28. A number of rules developed for CP –Poor methodology Patients older than 24 with CP from 3 sites Of 2,718 enrolled, 336 (12%) experienced cardiac event within 90 days –6% AMI, 10% revascularization, 0.2% death CDR developed with 100% sensitivity (95%CI, 97.2-100) and 20.9% specificity (95% CI, 16.9- 24.9) NORTH AMERICAN CHEST PAIN RULE

30. Were all important predictors included? Were important predictors present in a significant proportion of patients? Were all outcomes and predictors clearly defined? Were those assessing outcome blinded to predictors? Was sample size adequate? Were there enough patients with outcome of interest? Does the rule make sense? JUDGING THE QUALITY OF A DERIVED RULE