UNDERSTANDINGCLINICAL DECISION RULES Department of Emergency Medicine Stony Brook University Adam J Singer, MD Professor and Vice Chairman for Research

LEARNING OBJECTIVES/QUESTIONS –What are decision rules? –Why do we need decision rules? –How are decision rules developed? –How do I know if a decision rule is ready for primetime? –Decision rule for chest pain in young adults

HOW WE MAKE DECISIONS

WHAT ARE CLINICAL DECISION RULES? A Clinical Decision Rule is another term for Clinical Decision Support. By true definition, this means “Clinical decision support (CDS) provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care. CDS encompasses a variety of tools to enhance decision-making in the clinical workflow. These tools include computerized alerts and reminders to care providers and patients; clinical guidelines; condition-specific order sets; focused patient data reports and summaries; documentation templates; diagnostic support, and contextually relevant reference information, among other tools.” -The Office of the National Coordinator for Health Information Technology

A tool that helps clinicians make diagnostic and therapeutic decisions at the bedside Derived from original research and incorporate 3 or more variables from –History –Physical examination –Simple tests Attempts to reduce uncertainty in medical decision making DEFINITION OF CLINICAL DECISION RULE/AID

Why do we need decision rules? –Simplify complex decisions –Standardize decision making –Reduce uncertainty –Improve accuracy –Opportunity to reduce costs w/o compromising care Examples –Sore throat –PE –Depression

Non-educational –Relative advantage (better than old?) –Compatibility (represents existing beliefs) –Complexity –HCP characteristics (age, training, experience) –Practice setting –Incentives –Regulations –Patient factors BARRIERS TO IMPLEMENTATION OF CDR AND GUIDELINES

Educational strategies –Formal CME –Mailings –Opinion leaders –Audit with formal feedback –System reminders –Educational outreach –Strategies involving more than one intervention

DERIVATION –Creation or derivation of a rule VALIDATION –Prospective assessment of accuracy, reliability and potential impact of rule IMPLEMENTATION –Assessment of rule’s impact on patient care STAGES OF DEVELOPING CLINICAL DECISION RULES

HOW TO DEVLOP A CLINICAL DECISION RULE Identify condition in need of CDR –Common, variable practice, wasteful Construct list of potential predictors of outcome Typically includes –History, physical exam, basic lab tests –Obtained from prior studies or common sense Examine group of patients with suspected outcome to determine presence of predictors Determine presence or absence of outcome

Preform statistical analysis to determine which predictors best predict outcome and which can be omitted Logistic regression –mathematical equations Binary recursive partitioning –Builds a tree in which patients are split into smaller and smaller categories based on risk factors Neural networks

LOGISTIC REGRESSION

RECURSIVE PARTITIONING

PREDICTION TREE FOR TBI IN CHILDREN. Lancet 2009;374;1160

NEURAL NETWORKS A computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs

Simply derived rules are not ready for primetime Need to be validated in separate, independent settings May reflect associations between predictors and outcomes by chance alone Predictors may be idiosyncratic to specific population Clinicians may fail to implement rule comprehensively or accurately IS A DERIVED CLINICAL DECISION RULE READY FOR PRIMETIME

Were all important predictors included? Were important predictors present in a significant proportion of patients? Were all outcomes and predictors clearly defined? Were those assessing outcome blinded to predictors? Was sample size adequate? Were there enough patients with outcome of interest? Does the rule make sense? JUDGING THE QUALITY OF A DERIVED RULE

Note the most important predictors and consider them in your practice Consider giving less importance to variable found not to be predictive VALUE OF DERIVED RULE

Repeated application leads to similar results Applied prospectively to new population with different prevalence and spectrum of disease Rule performs similarly in a variety of settings and clinicians Rule work well when consciously applying it IMPORTANCE OF VALIDATION STUDIES

Were patients chosen in unbiased manner and do they represent wide spectrum of disease? Was there a blinded assessment of criterion standard for all patients? Was there an explicit and accurate interpretation of the predictor variables and the actual rule w/o knowledge of the outcome? Was there 100% follow up of those enrolled? JUDGING THE QUALITY OF A VALIDATED RULE

Usefulness of rule depends on predictive power Sensitivity and specificity of rule with 95% CI –OAR: 100% sensitive, 40% specific –All patients with fx were positive, only 40% of those w/o fx negative –If clinicians order XR only on positive rule: no misses and test avoided in 40% HOW TO INTERPRET THE RESULTS

Probability of target condition based on particular result of rule –Probability of PE based on Wells criteria –Low (3.4%; 95%CI, 2.2-5%), intermediate (28%, 95%CI, %), high (78%, 95%CI, %) Likelihood ratios –Detection of alcoholism based on CAGE scores –CAGE 0: 0.14, CAGE 1: 1.5, CAGE 2: 4.5; CAGE 4: 100 –Probability of alcoholism depends on prevalence and CAGE score

CDR demands time and energy Only warranted if they change physician behavior and improve outcomes or reduce costs Accurate rules that are not used or do not improve practice are worthless IMPACT ANALYSIS

Intuitive estimates (Gestalt) may be just as good Calculations may be too cumbersome Practical barriers to using CDR –Fear of litigation WHY EVEN ACCURATE RULES MAY NOT MATTER

Highest level of evidence that a rule is good Rarely done IMPACT STUDIES

A number of rules developed for CP –Poor methodology Patients older than 24 with CP from 3 sites Of 2,718 enrolled, 336 (12%) experienced cardiac event within 90 days –6% AMI, 10% revascularization, 0.2% death CDR developed with 100% sensitivity (95%CI, ) and 20.9% specificity (95% CI, ) NORTH AMERICAN CHEST PAIN RULE

Were all important predictors included? Were important predictors present in a significant proportion of patients? Were all outcomes and predictors clearly defined? Were those assessing outcome blinded to predictors? Was sample size adequate? Were there enough patients with outcome of interest? Does the rule make sense? JUDGING THE QUALITY OF A DERIVED RULE