1. Ambiguity and Vagueness in Clinical Practice Guidelines Shlomi Codish, MD Richard Shiffman, MD, MCIS Yale Center for Medical Informatics

2. Overview Definitions and background Definitions and background Classification model Classification model Implications for authoring and implementation Implications for authoring and implementation

3. Linguistic definitions Vague: lacking clear borders, having borderline cases Vague: lacking clear borders, having borderline cases “Dan is tall.” “Dan is tall.” Ambiguous: capable of being understood in two or more possible senses or ways Ambiguous: capable of being understood in two or more possible senses or ways “I will meet you at the bank.” “I will meet you at the bank.”

4. Introduction Focus: human users of CPGs Focus: human users of CPGs Clinicians – reduced adherence or accuracy Clinicians – reduced adherence or accuracy Grol (1998), Shekelle (2000) Grol (1998), Shekelle (2000) Implementers Implementers Ambiguity is often used to describe vagueness Ambiguity is often used to describe vagueness “disambiguation” has been used for the resolution of both ambiguity and vagueness “disambiguation” has been used for the resolution of both ambiguity and vagueness

5. Research on vague terms Kong et al. NEJM 1986 “qualitative expressions of probability” “qualitative expressions of probability” Large variability in interpretation Large variability in interpretation Comparable ranking as in previous study for 7 terms despite variable interpretation : Comparable ranking as in previous study for 7 terms despite variable interpretation : certain > probable > likely > not unreasonable > possible > unlikely > never certain > probable > likely > not unreasonable > possible > unlikely > never

6. Other factors affecting interpretation Mapes (1979) – importance of context Compare interpretation of rare in: Compare interpretation of rare in: “…in the case of this beta blocker drug side effects are rare” “…in the case of this beta blocker drug side effects are rare” “…in the case of this antihistaminic drug side effects are rare” “…in the case of this antihistaminic drug side effects are rare” Kahneman and Tversky – importance of framing Decision making affected by framing Decision making affected by framing Options presented in terms of lives saved or lives lost Options presented in terms of lives saved or lives lost

7. ambiguity ≠ uncertainty NJ judge to rule on nude beach Stolen painting found by tree Stolen painting found by tree Prostitutes appeal to governor

8. Overview Introduction Introduction Classification model for ambiguity and vagueness Classification model for ambiguity and vagueness Implications for authoring and implementation Implications for authoring and implementation

9. Development methodology MEDLINE and Web of Science databases MEDLINE and Web of Science databases Review of the linguistics literature Review of the linguistics literature Lists of terms and categories Lists of terms and categories Validation with random collection of 100 CPG Validation with random collection of 100 CPG

10. Model Axes Axis I: Ambiguity or vagueness? Axis I: Ambiguity or vagueness? Sub classification of ambiguity Sub classification of ambiguity Sub classification of vagueness Sub classification of vagueness Axis II: Rationale Axis II: Rationale Deliberate or inadvertent Deliberate or inadvertent Causes of deliberate vagueness Causes of deliberate vagueness Axis III: Implementers Axis III: Implementers Logical component affected Logical component affected If then because If then because

11. Axis 1 - Ambiguity Syntactic ambiguity: structure or syntax Syntactic ambiguity: structure or syntax …mother with untreated disease or disease that has been treated for 2 or more weeks and is who is judged to be noncontagious at the time of delivery… …mother with untreated disease or disease that has been treated for 2 or more weeks and is who is judged to be noncontagious at the time of delivery… Semantic ambiguity: “classic” ambiguity Semantic ambiguity: “classic” ambiguity Pragmatic ambiguity: reader and writer have different interpretations Pragmatic ambiguity: reader and writer have different interpretations Example: Disclosure of adolescent HIV status should legally require the consent of the adolescent. Example: Disclosure of adolescent HIV status should legally require the consent of the adolescent. ( ( ) )

12. Pragmatic ambiguity 3 cm 4 cm x 3. Find x. Here it is Ocular Trauma – by Wade Clarke © 2005

13. Axis 1 - Vagueness Underspecification is a form of vagueness that occurs when terms are used with insufficient details for interpretation. Underspecification is a form of vagueness that occurs when terms are used with insufficient details for interpretation. moderate, severe, elderly, adult moderate, severe, elderly, adult …should be treated promptly and aggressively. …should be treated promptly and aggressively. Ordinal scales of underspecified words Ordinal scales of underspecified words Temporal scale: Probabilistic scale: Quantitative scale: Never Always Impossible None Certain All rare sometimes common unlikely possibleprobable fewsomemany

14. Axis I - Vagueness Strength qualifiers (a.k.a. hedges) Strength qualifiers (a.k.a. hedges) It is generally recommended that the antibiotic beads be removed 2 weeks… It is generally recommended that the antibiotic beads be removed 2 weeks… It is prudent to… It is prudent to… Passive voice Passive voice Memory complaints should be evaluated and the individual followed to assess progression Memory complaints should be evaluated and the individual followed to assess progression

15. Axis II - rationale Only the authors know the rationale Only the authors know the rationale Inadvertent ambiguity or vagueness is a failure of authoring and editing Inadvertent ambiguity or vagueness is a failure of authoring and editing It might also result from a failure to recognize the problems ambiguity and vagueness create: It might also result from a failure to recognize the problems ambiguity and vagueness create: “…please appreciate that we will not necessarily have full agreement on the fine points of wording as each of us has his or her own writing style…” “…please appreciate that we will not necessarily have full agreement on the fine points of wording as each of us has his or her own writing style…”

16. Axis II – deliberate vagueness Insufficient scientific evidence Insufficient scientific evidence Lack of consensus among authors Lack of consensus among authors Other considerations Other considerations Legal Legal Economic Economic Ethical Ethical Religious Religious

17. Axis III – logical component The logical part affected by vagueness impacts implementation: The logical part affected by vagueness impacts implementation: affects decidability affects decidability impacts executability impacts executability does not seriously impact implementability does not seriously impact implementability

18. Axis I VaguenessAmbiguity Semantic Pragmatic Syntactic Strength qualifiers Underspecification Passive voice ProbabilisticQuantitativeTemporal Non-ordinal underspecification Axis II DeliberateInadvertent Ethical/Religious Legal Insufficient Scientific background Economic Axis III Action (executability) Explanation Condition (decidability) Ambiguity or vagueness in CPG recommendation

19. Overview Introduction Introduction Classification model Classification model Implications for authoring and implementation Implications for authoring and implementation

20. Remedies – axis 1 Numerical values (or ranges) for terms Numerical values (or ranges) for terms Ranking words relative to one another Ranking words relative to one another Lists of ‘good’ and ‘bad’ terms Lists of ‘good’ and ‘bad’ terms Controlled vocabulary Controlled vocabulary BI-RADS “Suspicious Abnormality—Biopsy Should Be Considered” BI-RADS “Suspicious Abnormality—Biopsy Should Be Considered” Fuzzy logic Fuzzy logic

21. Remedies – axis 2 Reason should be documented Reason should be documented …activated protein C should not be routinely used… …activated protein C should not be routinely used… …activated protein C should not be routinely used because of its high cost… …activated protein C should not be routinely used because of its high cost… Selecting a vague term should consider the audience Selecting a vague term should consider the audience The range of interpretation by the intended audience The range of interpretation by the intended audience Other terms with similar meaning Other terms with similar meaning Avoid terms with wide range of interpretation Avoid terms with wide range of interpretation Limit the scope of vagueness Limit the scope of vagueness Cigarette smoking may be hazardous to your health Cigarette smoking may be hazardous to your health Smoking causes lung cancer, heart disease, emphysema and may complicate pregnancy Smoking causes lung cancer, heart disease, emphysema and may complicate pregnancy Clarification of deliberate vagueness should be done with author input Clarification of deliberate vagueness should be done with author input

23. Summary Vagueness is a common problem in CPG recommendations Vagueness is a common problem in CPG recommendations “Classic” (i.e. semantic) ambiguity is uncommon “Classic” (i.e. semantic) ambiguity is uncommon Distinct forms of vagueness and ambiguity require specific remedies Distinct forms of vagueness and ambiguity require specific remedies When deliberate vagueness is deemed necessary, end-user uncertainty should be minimized When deliberate vagueness is deemed necessary, end-user uncertainty should be minimized Clarification of deliberate vagueness must consider the rationale for the vagueness Clarification of deliberate vagueness must consider the rationale for the vagueness

24. Everything is vague to a degree you do not realize till you have tried to make it precise. Bertrand Russell, 1918

25. Acknowledgements Prof. Laurence Horn Yale Linguistics Prof. Laurence Horn Yale Linguistics National Library of Medicine National Library of Medicine