1. Sharing guidelines knowledge: can the dream come true? Medinfo panel Cape Town, September 15, 2010

2. Motivation  The vision of sharing executable clinical knowledge can be achieved only if we: Standardize platforms for deploying scalable knowledge based services Ensure services are mutually compatible and interoperable and free of institution-specific details Develop reusable content and service components Support automated cross-verification for quality and safety Establish communities of practice who share, maintain, update, and improve content 2

3. Objectives  Raise awareness of the practical challenges involved in maintaining repositories of sharable executable clinical knowledge Challenges with maintaining a repository Defining what knowledge can be shared and how Challenges in piecing together knowledge into a care plan and integrating it with EHR data If the knowledge if free, what’s the business model and incentives for contributing knowledge? 3

4. Panel participants  John Fox, Department of Engineering Science, University of Oxford, UK  Robert Greenes, Ira A. Fulton Chair of the Department of Biomedical Informatics, Arizona State Univercity, Phoenix  Sheizaf Rafaeli, Head of the Graduate School of Management and Sagy Internet Research Center, University of Haifa, Israel  Mor Peleg, Head of the Department of Information Systems at the University of Haifa 4

5. What shall we discuss?  Life-cycle approach for sharable knowledge-based patient-care services  Methodology for distilling sharable knowledge from business/ implementation considerations  Methods for weaving medical knowledge services into an application and for mapping clinical abstractions into EHRs  Incentives and business models for a knowledge-sharing community 5 John Fox Bob Greenes Mor Peleg Sheizaf Rafaeli

6. John Fox 6 Options for addressing open-source publishing of medical knowledge, drawing on lessons learned in the OpenClinical project 20th Anniversary Gold Medal Award

7. OpenClinical: Open Source? John Fox University of Oxford (Engineering Science) UCL (Oncology, Royal Free Hospital) www.cossac.org

8. www.OpenClinical.org Goal: To promote awareness and use of decision support, clinical workflow and other knowledge management technologies for improving quality and safety of patient care and clinical research. A resource and portal for technologists, clinicians, healthcare providers and suppliers Currently about 200,000 visitors a year (80% growth in 2010)

10. www.OpenClinical.net Experimental project to explore how to develop content for high quality clinical decision support and workflow services at the point of care Goal is to build a community of users, researchers and content providers who are willing to contribute to the development of a repository of open content, including applications and application components

11. OpenClinical.net test site pro tem: modx.openclinical.net

12. Content development lifecycle Prototype development model for open source content repository on www.OpenClinical.net www.OpenClinical.net Currently limited to PROforma decision and process modelling language Intended to eventually multiple representations (e.g. GLIF, ASBRU, GELLO, OWL...)

13. Load from, save to repository

14. Download tools www.cossac.org/tallis

15. Web publishing (“publets”)

16. Integrate and Deploy

17. Key questions for open content Quality and Safety –Quality lifecycles, safety culture, who is liable? Reusability and interoperability –Open technical standards, who is developing them? Functioning community (Sheizaf Rafaelli) –What will sustain the open source ethic? Facilitating infrastructure (Bob Greenes) –Three organisations; too little? too much? Sustainable business models –How do the proprietary/open source worlds coexist?

18. Sustainable business models (1) Traditional standalone apps? Issues of integration and localisation Likes fragmentation; hates interoperability Pay per patient (analogous to pay per view) Who would/should actually pay? No-one pays for Adjuvant! Online

19. Sustainable business models (2) Standard medical publishing model Commercially viable on a publishing model? (Clinical Evidence) Discussion on www.berkerynoyes.com/ www.berkerynoyes.com/ pages/innovations_in_evidence_based_medicine.aspx Open Source with value-adding services? (c.f. Linux model) Attractive model but how can we achieve critical mass of a content development community?

20. Towards an open content lifecycle? Ioannis Chronakis Vivek Patkar Richard Thomson Matt South Ali Rahmanzadeh

21. Thank you

22. Robert Greenes 22 Morris Collen Award Morningside Initiative Sharing medical knowledge involves separation between the medical content and the business/applications considerations MUMPS

23. Toward sharing of clinical decision support knowledge Robert A. Greenes, MD, PhD Arizona State University Phoenix, AZ, USA  A focus on rules

24. Purpose of this talk Identify key challenges to CDS adoption with focus on rules –Expressed in terms of 3 hypotheses: 1.Sharing is key to widespread adoption of CDS 2.Sharing of rules is difficult 3.Sharing can be facilitated by a formal approach to rule refinement

25. Hypothesis 1: Sharing is key to widespread adoption of CDS We know how to do CDS! –Over 40 years of study and experiments Many evaluations showing effectiveness

26. Rules as a central focus Importance of rules –Can serve as alerts, reminders, recommendations –Can be run in background as well as interactively –Can fire at point of need –Same logic can be used in multiple contexts e.g., drug-lab interaction rule can fire in CPOE, as lab alert, or as part of ADE monitoring –Can invoke actions such as orders, scheduling, routing of information, as well as notifications Relation to guidelines –Function as executable components when GLs are integrated with clinical systems Poised for huge expansion –Knowledge explosion – genomics, new technologies, new tests, new treatments –Emphasis on quality measurement and reporting

27. Yet beyond basics, there is very little use of CDS Positive experience not replicated and disseminated widely –Largely in academic centers –<30% penetration –Much less in small offices –Pace of adoption barely changing Only scratching surface of potential uses –drug dose & interaction checks –simple alerts and reminders –personalized order sets –Narrative infobuttons, guidelines

28. Adoption challenges Possible reasons 1.Users don’t want it 2.Bad implementations Time-consuming, inappropriate Disruptive 3.Adoption is difficult Finding knowledge sources Adapting to platform Adapting to workflow and setting Managing and updating knowledge But new incentives and initiatives rewarding quality over volume can address #1 –Health care reform, efforts to reduce cost while preserving and enhancing safety and quality And #2 AND #3 can be addressed by sharing of best practices knowledge –Including workflow adaptation experience

29. Hypothesis 2: Sharing of rules is difficult Rules knowledge seems deceptively simple: –ON lab result serum K+ –IF K+ > 5.0 mEq/L –THEN Notify physician Even complex logic has similar Event- Condition-Action (ECA) form –ON Medication Order Entry Captopril –IF Existing Med = Dyazide AND proposed Med = Captopril AND serum K+ > 5.0 –THEN page MD

30. Why is sharing not done? Perception of proprietary value –Users, vendors don’t want to share –Non-uptake even with: Standards like Arden Syntax for 15 years, GELLO for 5 years Knowledge sources such as open rules library from Columbia since 1995, and guidelines.gov, Cochrane, EPCs, etc., - most not in computable form Failure of initiatives such as IMKI in 2001 Lack of robust knowledge management –To track variations, updates, interactions, multiple uses Same basic rule logic in different contexts Beyond capabilities of smaller organizations and practices to undertake Embeddedness –In non-portable, non-standard formats & platforms –in clinical setting –in application –in workflow –in business processes

31. Example of difficulty in sharing Consider simple medical rules, e.g., –If Diabetic, then check HbA1c every 6 months –If HbA1c > 6.5% then Notify Multiple translations –Based on how triggered, how/when interact, what thresholds set, how notify –Actual form incorporates site-specific thresholds, modes of interaction, and workflow

32. Multiple rules have similar intent Differences relate to how triggered, how delivered, thresholds, process/workflow integration Challenge is to identify core medical knowledge and to develop a taxonomy to capture types of implementation differences

33. Setting-specific factors (“SSFs”) Triggering/identification modes –Registry, encounter, periodic panel search, patient list for day, … –Inclusions, exclusions Interaction modes, users, settings Data mappings & definitions, e.g., –What is diabetes - code sets, value sets, constraint logic? –What is serum HbA1c procedure? Data availability/entry requirements –Thresholds, constraints Logic/operations approaches –Advance, late, due now, … Exceptions –Refusal, lost to follow up, … Actions/notifications –Message, pop-up, to do list, order, schedule, notation in chart, requirement for acknowledgment, escalation, alternate. …

34. Hypothesis 3: Sharing can be facilitated by a formal approach to rule refinement Develop an Implementers’ Workbench Start with EBM statement Progress through codification and incorporation of SSFs Output in a form that is consumable “directly” by the implementer site or vendor

35. Life Cycle of Rule Refinement Start with EBM statement Stage 1.Identify key elements and logic – who, when, what to be done –Structured headers, unstructured content –Medically specific 2.Formalize definitions and logic conditions –Structured headers, structured content (terms, code sets, etc.) –Medically specific 3.Specify adaptations for execution –Taxonomy of possible workflow scenarios and operational considerations –Selected particular workflow- and setting- specific attributes for particular sites 4.Convert to target representation, platform, for particular implementation –Host language (Drools, Java, Arden Syntax, …) –Host architecture: rules engine, SOA, other –Ready for execution

36. Four current projects addressing this challenge EBM statement 1.Identify key elements and logic – who, when, what to be done 2.Formalize definitions and logic conditions 3.Identify possible workflow scenarios – model rules, defining classes of operation 4.Convert to target representation, platform, for particular implementation Idealized life cycle / Morningside / KMR / AHRQ SCRCDS/ SHARP 2B

37. What we hope to accomplish Implementers’ Workbench (IW) Taxonomy of SSFs Knowledge base of rules Approach –Vendor, implementer, other project input, buy-in, collaboration –Taxonomy as amalgam of NQF expert panel, Morningside/SHARP/Advancing-CDS workflow studies, SCRCDS implementation considerations –Diabetes, USPS Task Force prevention and screening A&B recommendations, and Meaningful Use eMeasures converted to eRecommendations as initial foci –Prototyping, testing, and iterative refinement of IW

38. What we expect to share Experience/know-how Knowledge content Methods/tools Standards/models

39. Representation Data model/code sets Definitions Templates Taxonomies Transformation processes

40. Where CDS should go from here? Need for coordination –Multiple efforts underway –Need to coalesce and align these Need sustainable process –Multi-stakeholder buy-in, participation, support, commitment to use Need to demonstrate success –Small-scale trials –Larger-scale deployment built on success Expansion to other kinds of CDS

41. Comments? Questions?

42. Mor Peleg 42 GLIF3 New Investigator Award Process Mining Biomedical Ontologies KDOM Data K Weaving medical knowledge services into applications. Using a mapping ontology to map medical knowledge into institutional data

43. Implementing decision- support systems by piecing sharable knowledge components Mor Peleg, University of Haifa Medinfo panel, Cape Town, September 15, 2010

44. Motivation  Computerized guidelines have shown positive impacts on clinicians but they take time to develop  Solution: Share executable GL components, stored in Medical Knowledge Repository  Assemble computerized GLs from components  Map the GL’s medical terms into institutional EHR fields

45. Examples of medical resources that could be shared and assembled  Medical calculators  Risk-assessment tools  Drug databases  Controlled terminologies (e.g. SNOMED)  Authoring, validation, and execution tools for computer-interpretable GLs

46. Component interface Peleg, Fox, et al. (2005) LNCS 3581 pp.156-160.

47. The interface can be used for  Sharing components  Indexing and searching for components Using the attributes: clinical sub-domain, relevant authoring stages, and goals  Assembling components into a GL Specifying the guideline's skeleton language (e.g., GLIF, PROforma) into which components can be integrated

48. Example: providing advice on regimens for treating breast cancer Get patient Data Is patient eligible for evaluating therapy choices? Adjuvant's life- expectancy calculator Filter out non- beneficial and contraindicated therapies Present choices to user Choose option Calculate regimen Prescribe regimen

49. Using Standards  Skeleton can be any GL formalism  Eligibility criteria expressed in GELLO standard  Referring to the HL7-RIM patient data model

50. Integrating assembled guideline with EHR data 50  Encode once but link to different EMRs Global-as-View Mapping Ontology + SQL Query Generator RIM Peleg et al., JBI 2008 41(1):180-201

51. Knowledge-Data Ontology Mapper (KDOM) 51 anyEMR Query result: RIM view KDOM mapping instances SQL query generator Guideline Expression (need not use EMR’s terms) GELLO interpreter Evaluated expression KDOM mapping classes: Direct, Hierarchical, Logical, Temporal “Breast Mass = true” Patient has Palpable Breast Mass or Hard_Breast_Mass Palpable Breast Mass is-a Breast Mass. Palpabale Breast Mass is stored in the Problems table Observation of Breast Mass true SQL query

52. Summary  A repository of tested executable medical knowledge components that would be published on the Web  Framework for specifying the interface of components so that they could be searched for and integrated within a Computerized GL specification  KDOM used to integrate the medical knowledge with institutional EMRs 52

53. Thanks! morpeleg@is.haifa.ac.il 53 Hope to see you at AIME 2011, July 2-6, 2011, Bled, Slovenia

54. Sheizaf Rafaeli 54 survey and contrast social, technical, hierarchical and market-based models for motivating and maintaining the sharing of information and processing tools

55. Sharing Guidelines Knowledge: can the dream come true? Sheizaf Rafaeli sheizaf@rafaeli.net http://rafaeli.net MedInfo 2010

56. sheizaf@rafaeli.net, http://rafaeli.net 56 Bits Replacing Atoms

57. Moore, Gilder, Metcalfe, Reed sheizaf@rafaeli.net, http://rafaeli.net 57 utility users

58. Information Overload Economics of Scarcity vs. Economics of Abundance? sheizaf@rafaeli.net, http://rafaeli.net 58

59. What’s really new? Access has become widespread Information as a commodity; IT as a commodity “Does IT matter “? Transmission has been solved Information is an experience good The impossible ease of copying Disintermediation Free information has become commonplace, normative, expected. Both free and for-fee information occupy the same net sheizaf@rafaeli.net, http://rafaeli.net 59

60. New Rules for the New Economy : 10 Radical Strategies for a Connected World by Kevin KellyKevin Kelly “Information Rules : A Strategic Guide to the Network Economy by Carl Shapiro, Hal R. VarianCarl ShapiroHal R. Varian sheizaf@rafaeli.net, http://rafaeli.net 60 “Free” as in free speech, or as in free beer?

61. sheizaf@rafaeli.net, http://rafaeli.net 61

62. How is UGC motivated? sheizaf@rafaeli.net, http://rafaeli.net 62

63. The Value of Information Public source Commodity Overload History Technology Psychology? Private source Uniqueness Timing Presentation Tailoring Technology Network effects sheizaf@rafaeli.net, http://rafaeli.net 63 Emphasis on distinction between Private and Public Suggesting the Subjective Value of Info

64. כלים מוזרים לתיגמול Wiki “barnstars” Wikipedia: a system that shouldn’t work, but does. Participation Power Laws and Long Tail

65. sheizaf@rafaeli.net, http://rafaeli.net 65 Web 2.0 UGC and Co- production

66. Further personal stakes in info value Information markets http://answers.google.comhttp://answers.google.com Online Scientific Journals http://jcmc.indiana.eduhttp://jcmc.indiana.edu Citizens’ Advice Bureaus http://shil.infohttp://shil.info Wikis http://misbook.yeda.infohttp://misbook.yeda.info Online Higher Ed systems http://qsia.org http://qsia.org Games and Serious Games sheizaf@rafaeli.net, http://rafaeli.net 66

67. sheizaf@rafaeli.net, http://rafaeli.net 67

68. sheizaf@rafaeli.net, http://rafaeli.net 68

69. SHIL שרות יעוץ לאזרח ( שי " ל ) Citizen Advice Bureaux (CABs) Established 1957 55 “Brick and Mortar” offices Telephone hot line & Internet web site, operated at the Univ. of Haifa Sagy Center Operated by Volunteers, coordinated and funded by the Israeli Ministry of Social Affairs and Social Services in collaboration with municipalities.

70. Ownership… Legal Perspective Vs. Open Source, Peer-to-Peer, UGC, Web 2.0, etc. Apply 19th century property law to 21st century reality? Legality: "fair use" "first sale" "prior art" doctrines Open Innovation sheizaf@rafaeli.net, http://rafaeli.net 70

73. Discussion Still LOTS to study and learn… Interactivity and Social Motivations seem to be king A high (too high?) overall subjective value for information. As predicted by the Endowment Effect theory, WTA for information was significantly larger than WTP for information This predicts undertrading. Implications for system design sheizaf@rafaeli.net, http://rafaeli.net 73

74. Discussion (2) Information is a commodity. Nevertheless, information is still easier to duplicate, easy to share, and ownership of it proves more difficult to enforce Society has not yet adjusted its information consumption patterns to the present situation of information abundance Scoring and Governance Rules! sheizaf@rafaeli.net, http://rafaeli.net 74

75. Thank you sheizaf@rafaeli.net http://rafaeli.net

76. sheizaf@rafaeli.net, http://rafaeli.net 76

77. Provocative statements 77

78. Statement 1  A national or international effort can be put together to create a repository of implementable knowledge. 78

79. Statement 2  Guideline sharing could be achieved within 10 years 79

80. Statement 3  Guideline sharing at the implementation level requires separation into component steps that can be individually implemented, because of differences in process/work flow that prevent the guideline from being adopted in its entirety 80

81. Statement 4  True sharing of executable medical knowledge could never be achieved because knowledge could not be separated from institutional adaptations 81

82. Statement 5  Guideline formalization activities do not typically address implementation settings and requirements 82

83. Statement 6  The benefits of formalizing and sharing clinical knowledge are beyond dispute: the challenge now is to establish principles of safe deployment and use in clinical service design 83

84. Statement 7  As in so many other fields of engineering, one of the keys to effective and safe deployment will be open technical standards (covering medical concepts, clinical vocabulary, task models for example) 84

85. Statement 8  Adoption of standards will be necessary but will not be sufficient for success: another vital challenge is to persuade the commercial world of medical IT, publishing, etc. to develop business models that accept and build on open standards 85

86. Statement 9  If information “wants to be free” why discuss incentives for sharing anyway? 86

87. Statement 10  The only types of incentives for sharing are material, social, or ego- oriented. 87

88. Statement 11  Which of these incentives is more available (material, social, or ego-oriented)  Which is more likely to generate results (material, social, or ego-oriented)  Which has more leverage for potential participating scientists? (material, social, or ego-oriented) 88

89. Statement 12  Ever since Fred Brook’s “Mythical Man-Month” vs. Eric Raymond’s “The Cathedral and the Bazaar”, we’ve seen a conflict between orderly design and sharing. Following Brook’s recent “Design of Design”, should the notions of iterative design be applied to sharing; or is the Open Code approach the way to go? 89

90. Discussion 90

91. Thank you! 91

92. Rafaeli, S. and Raban, D. (2003), The Subjective Value of Information: An experimental comparison of willingness to purchase or sell information, JAIS: The Journal of the Association for Information Systems (AIS). Vol. 4:5 pp. 119-139 Rafaeli, S. & Raban, D.R. (2003 ) The Subjective Value of Information : Trading expertise vs. content, copies vs. originals in E-Business, The Third International Conference on Electronic Business (ICEB 2003), pp. 451-455. Rafaeli, S. and Raban, D.R. (2005) Information Sharing Online: A Research Challenge, in the International Journal of Knowledge and Learning, (inaugural issue), Vol. 1, Issue 1-2, pp. 62-80., Raban, D.R. and Rafaeli, S. (2006), The Effect of Source Nature and StatusThe Effect of Source Nature and Status on the Subjective Value of Informationon the Subjective Value of Information, Journal of the American Society for Information Science and Technology ( JASIST ), Volume 57, Issue 3 (p 321- 329)

93. Rafaeli, S., Raban, R.D., & Ravid, G., (2005). Social and Economic Incentives in Google Answers. ACM Group 2005 conference, Sanibel Island, Florida, November 2005. http://jellis.net/research/group2005/papers/RafaeliRabanRavidGoogleAnswers Group05.pdf http://jellis.net/research/group2005/papers/RafaeliRabanRavidGoogleAnswers Group05.pdf M. Harper, D. Raban, S. Rafaeli, J. Konstan, Predictors of Answer Quality in Online Q&A Sites. CHI 2008. D. Raban, M. Harper, Motivations for Answering Questions Online. Book chapter in New Media and Innovative Technologies (Caspi, D., Azran, T. eds.), 2007. Rafaeli, S., Raban, D.R. and Ravid, G. (2007) 'How social motivation enhances economic activity and incentives in the Google Answers knowledge sharing market', International Journal of Knowledge and Learning ( IJKL ), Vol. 3, No. 1, pp.1-11.