A Multiple Ontology, Concept-Based, Context-Sensitive Search and Retrieval Robert Moskovitch and Prof. Yuval Shahar Medical Informatics Research Center Ben Gurion University, Israel

Clinical Guidelines Clinical practice guidelines (CPGs) and protocols are a powerful method for standardizing the quality of medical careClinical practice guidelines (CPGs) and protocols are a powerful method for standardizing the quality of medical care The main challenge is providing easy access to CPGs at the point of careThe main challenge is providing easy access to CPGs at the point of care Access involves representation of the guidelines and easy, accurate retrieval of relevant guidelinesAccess involves representation of the guidelines and easy, accurate retrieval of relevant guidelines

The DEGEL Framework Ben Gurion University’s Digital Electronic Guidelines Library (DeGeL) is an architecture and a Web-based set of computational tools for: Ben Gurion University’s Digital Electronic Guidelines Library (DeGeL) is an architecture and a Web-based set of computational tools for:  Authoring  markup (semi-structuring and structuring)  Retrieval  browsing  Runtime application of clinical guidelines  Retrospective assessment of the quality of the application

The Goal Build a search and retrieval tool to retrieve CPGs, to support the challenge of accurately retrieving CPGs at the point of careBuild a search and retrieval tool to retrieve CPGs, to support the challenge of accurately retrieving CPGs at the point of care Enable concept-based search, which supports querying using an existing set of semantic classification indicesEnable concept-based search, which supports querying using an existing set of semantic classification indices Support context-sensitive search, which supports querying for a term only within a particular knowledge role (e.g., eligibility conditions)Support context-sensitive search, which supports querying for a term only within a particular knowledge role (e.g., eligibility conditions)

Classification and Concept Based Search DeGeL uses seven semantic axes (or aspects) that can categorize CGPs (e.g., diagnosis type, therapy type)DeGeL uses seven semantic axes (or aspects) that can categorize CGPs (e.g., diagnosis type, therapy type) Each axis is implemented as a treeEach axis is implemented as a tree Each Guideline can be classified under zero, one, or more indices from each axisEach Guideline can be classified under zero, one, or more indices from each axis

Example Markup, Using The Asbru Ontology Plan … The markup process gradually converts a free-text-based CPG to a semi-structured, then fully structured one, maintaining all formats in parallel (a hybrid architecture) Conditions Filter condition Setup condition Intentions Outcome intentions Process intentions This guideline is intended only for women who are pregnant and who are at high risk for gestational diabetes and who had a glucose-tolerance test… The main goal is reduction of potential hypertension… The guideline uses mainly dietary measures… If a need for insulin develops, use a guideline for using short-acting insulin…

Context-Sensitive Search Within Knowledge Roles of Ontologies Several ontologies such as Asbru, GEM, and GLIF were developed to represent CPGs in a structured fashion in order to provide automated support for their useSeveral ontologies such as Asbru, GEM, and GLIF were developed to represent CPGs in a structured fashion in order to provide automated support for their use Context-Based Search exploits the existence of certain terms within semantically meaningful segments of the text, or knowledge rolesContext-Based Search exploits the existence of certain terms within semantically meaningful segments of the text, or knowledge roles Example: searching within articles summarizing clinical studies [G.Purcell, 1996]. According to Purcell, a context defines a semantically meaningful region of the document for searching, and thus facilitates precise retrieval of information from the medical literature

The Information Retrieval Task in the DEGEL Framework Document CollectionDocument Collection Content IndexingContent Indexing Document RepresentationDocument Representation Query FormulationQuery Formulation Matching ProcessMatching Process Matchin g Process Retrieved Documents Content Indexin g Document Representation Document Collection Query Formulatio n Query Representation Information Need ? GLSGLM N : 1 Vaidurya Query Language - Free Text - Text Value - Text Multiple Value - Int - DateVaidurya Query Language - Free Text - Text Value - Text Multiple Value - Int - Date Source OntologyMarkup Ontology

Representing Ontolgies for Search Purposes To implement the Concept-Based Search and the Context-Sensitive Search, two properties for each element in a guideline representation ontology were defined, Search Type and Search Scope. These properties, or aspects, define how an element will be indexed, queried and retrieved.

Search Type Relevance measure Querying OptionsDescriptionSearch Type MetricKeywords with disjunction or conjunction logic operator. An element containing a free text content. Free Text BooleanRequested string values with disjunction being the only possible relation. An element that may contain only a single fixed string value. Text Value BooleanRequested string values with conjunction, disjunction relations. An element containing one or more fixed string values. Text Multiple Value MetricA date constraint using operators such as ‘>’ or ‘>=’ etc. An element that its content represents a calendar date. Date MetricAn integer constraint using operators such as ‘>’ or ‘>=’ etc.. An element that its content represents an integer value. Integer BooleanRequested concepts using conjunction, disjunction operators between indices. An element represents the conceptual classification of the guideline Semantic Index Not relevant.No query.An element that doesn’t have content or its content is irrelevant for search. Unsearchable

Search Scope DescriptionSearch Scope No search at that element nor at its descendents - elements that don’t contain any content, and their descendents' contents aren’t relevant to them. None Search the element without descendentsSearch-Self No search at that element, search only its descendents.Only-Children Search both that element and its descendents.Children-Included

Query Interface

Results Interface

Evaluation The evaluation goals were, to examine the contribution of the concept search and the context sensitive to the traditional full text search. Test sets:  TREC  NGC CPGs collection

Concept-based and Context- sensitive evaluation NGC CPGs collection  1136 CPGs stored in a GEM based ontology  Classified along two MeSH taxonomies: Disease/Condition and Treatment/Intervention.  Each taxonomy contains ~2500 concepts, in some regions the concepts are 10 levels deep but averages 4-6 levels.

Queries and Judgments In order to evaluate an IR system Queries and judgments should be created. We created a set of 15 daily queries created by 5 physicians ( E&C and Stanford ) Each Physician was asked to label the relevant CPGs, for each query, in the collection. Each query had three formats:  Full Text  Concept Query in 2 nd and 3 rd level  Context Query in 3 elements

Evaluation Measures PRECISION = RECALL = Number of Relevant Documents Retrieved Total Number of Documents Retrieved Number of Relevant Documents Retrieved Number of Relevant Documents in the Document Collection

Evaluation Hypotheses Hypothesis 1 Retrieval performance will be increased as more context elements are queried, also in addition to full-text search. Hypothesis 2 Retrieval performance will be increased as concept based queries will be used in addition to full text search.

Results – Contextual search

Context Sensitive in addition to Full Text search

Results – Concept based search in addition to three contexts

Results – Concept based search in addition to full-text search

Results – Concept based search in addition to single context

Discussion Concept based search increased the retrieval performance in any of the cases. Improvement observed when deeper queries used using conjunctive relation. Context sensitive search improves performance as more contexts participate in the query.

Questions ?