1. www.landc.be 1/58Werner Ceusters CTO www.landc.be Understanding the Message: Linking Aristotelian Realism to Linguistic Functionalism W. Ceusters *, B. Smith **, M. Dos Santos *, J. Simon **, M. O’Donnell *, M. Fielding *,*** * Language & Computing nv, Zonnegem, Belgium ** Institute for Formal Ontology and Medical Information Science (IFOMIS), Leipzig, Germany *** Catholic University of Leuven, Leuven, Belgium

2. www.landc.be 2/58Werner Ceusters CTO www.landc.be Presentation overview Problem description: patient eligibility for clinical trial Meaning theories Required technology for natural language understanding Implementation of a realist ontology for medical natural language understanding Conclusions If enough time: a guided tour of LinkFactory

3. www.landc.be 3/58Werner Ceusters CTO www.landc.be The Medical Informatics Dogma Everything should be structured Fact: computers can only deal with structured representations of reality: –structured data: relational databases, spreadsheets –structured information: XML simulates context –structured knowledge: rule-based knowledge systems Typical conclusion (Dogma?): –there is a need for structured data, hence … –… there is a need for structured data entry

4. www.landc.be 4/58Werner Ceusters CTO www.landc.be Structured data entry Current technical solutions: –rigid data entry forms –coding and classification systems But: –the description of biological variability requires the flexibility of natural language and it is generally desirable not to interfere with the traditional manner of medical recording (Wiederhold, 1980) –Initiatives to facilitate the entry of narrative data have focused on the control rather than the ease of data entry (Tanghe, 1997)

5. www.landc.be 5/58Werner Ceusters CTO www.landc.be Drawbacks of structured data entry Loss of information –qualitatively limited expressiveness and inherent defects of coding and classification systems, controlled vocabularies, and “traditional” medical terminologies use of purpose oriented systems –don’t use data for another purpose than originally foreseen (J VDL) –quantitatively too time-consuming to code all information manually Speech recognition and forms for structured data entry are not best friends

6. www.landc.be 6/58Werner Ceusters CTO www.landc.be Areas for application of medical natural language understanding Coding patient data Structured information extraction from unstructured clinical notes Clinical protocols and guidelines Assessing patient eligibility for clinical trial entry Triggering and alerts Linking case descriptions to scientific literature Easy access to content... towards a medical semantic web

7. www.landc.be 7/58Werner Ceusters CTO www.landc.be Clinical history description Mr. Kovács is an 83-year-old man with a past medical history of hypertension, congestive heart failure, atrial fibrillation, hypercholesterolemia, and a history of CVA who presented himself to Budapest Emergency Room on April 25 with primary complaint of right-sided chest pain since April 24. The patient was in his usual state of health until April 24 when he experienced right-sided chest pain after 10 minutes of bicycling exercise at the YMCA. He described the chest pain as a dull ache in the right side of his chest radiating posteriorly to the right scapular area. He rated the intensity as 7 out of 10. The chest pain lasted about 3 minutes and resolved with rest. That same night, the patient once again experienced right-sided chest pain while lying in bed just before he went to sleep. He describes the pain as right-sided chest pain with same radiation to posterior at an intensity of 6-7 out of 10. The chest pain lasted about 10 minutes and resolved spontaneously.

8. www.landc.be 8/58Werner Ceusters CTO www.landc.be Inclusion criteria of the INVEST study 1. Male or female 2. Age 50 to no upper limit 3. a) Hypertension documented as according to the 6th report of the Joint National Committee on Detection and Evaluation of the treatment of high BP (JNC VI), b) and the need for drug therapy (previously documented hypertension in patients currently taking antihypertensive agents is acceptable) 4. Documented CAD (e.g., classic angina pectoris; stable angina pectoris; Heberden angina pectoris), myocardial infarction three or more months ago, abnormal coronary angiography, or concordant abnormalities on two different types of stress tests 5. Willingness to sign informed consent

9. www.landc.be 9/58Werner Ceusters CTO www.landc.be Do they match ? Mr. Kovács is … an 83- year-old man with past medical history of hypertension, congestive heart failure, atrial fibrillation, hypercholesterolemia, history of CVA who presented to Budapest Emergency Room on April 25 with chief complaint of right-sided chest pain since April 24. The patient was in his usual state of health until April 24 when he experienced right-sided chest pain after 10 minutes of bicycling exercise at YMCA. He described the chest pain as a dull ache in the right side of his chest radiating posteriorly to the right scapular area. He rated the intensity as 7 out of 10. The chest pain lasted about 3 minutes and resolved with rest. That same night, the patient once again experienced right-sided chest pain while lying in bed right before he went to sleep. He describes the pain as right-sided chest pain with same radiation to posterior at an intensity of 6-7 out of 10. The chest pain lasted about 10 minutes and resolved spontaneously. 1. Male or female 2. Age 50 to no upper limit 3. Hypertension documented according to the 6th report of the Joint National Committee on Detection and Evaluation of the treatment of high BP (JNC VI) and the need for drug therapy (previously documented hypertension in patients currently taking antihypertensive agents is acceptable) 4. Documented CAD (e.g., classic angina pectoris (stable angina pectoris; Heberden angina pectoris), myocardial infarction three or more months ago, abnormal coronary angiography, or concordant abnormalities on two different types of stress tests) 5. Willingness to sign informed consent 

10. www.landc.be 10/58Werner Ceusters CTO www.landc.be If the computer is to make this deduction... Mr. Kovács is … an 83-year-old man with past medical history of hypertension, congestive heart failure, atrial fibrillation, hypercholesterolemia, history of CVA who presented to Budapest Emergency Room on April 25 with chief complaint of right-sided chest pain since April 24. The patient was in his usual state of health until April 24 when he experienced right-sided chest pain after 10 minutes of bicycling exercise at YMCA. He described the chest pain as a dull ache in the right side of his chest radiating posteriorly to the right scapular area. He rated the intensity as 7 out of 10. The chest pain lasted about 3 minutes and resolved with rest. That same night, the patient once again experienced right-sided chest pain while lying in bed right before he went to sleep. He describes the pain as right- sided chest pain with same radiation to posterior at an intensity of 6-7 out of 10. The chest pain lasted about 10 minutes and resolved spontaneously. 1. Male or female 2. Age 50 to no upper limit 3. Hypertension documented according to the 6th report of the Joint National Committee on Detection and Evaluation of the treatment of high BP (JNC VI) and the need for drug therapy (previously documented hypertension in patients currently taking antihypertensive agents is acceptable) 4. Documented CAD (e.g., classic angina pectoris (stable angina pectoris; Heberden angina pectoris), myocardial infarction three or more months ago, abnormal coronary angiography, or concordant abnormalities on two different types of stress tests) 5. Willingness to sign informed consent... it must be able to understand !

11. www.landc.be 11/58Werner Ceusters CTO www.landc.be What is understanding ? To understand something is to know what its significance is. What 'knowing significance' amounts to may be very different in different contexts: thus understanding a piece of music requires different things of us than understanding a sentence in a language we are learning, for instance. It would be useful, then, for theorists to look at the different kinds of understanding that there are, and examine them in detail and without prejudice, rather than looking for the essence of understanding. ( Tim Crane, philosopher of mind ) The significance of a single sentence, too, can vary from context to context.

12. www.landc.be 12/58Werner Ceusters CTO www.landc.be The etymology of “understanding” “understanding”  Latin “substare” –literally: “to stand under” Websters Dictionary (1961) understanding = the power to render experience intelligible by bringing perceived particulars under appropriate concepts. “particulars” = what is NOT SAID of a subject (Aristotle) –substances: this patient, that tumor,... –qualities: the red of that patient’s skin, his body temperature, blood pressure,... –processes: that incision made by that surgeon, the rise of that patient’s temperature,... “concepts”: may be taken in the above definition as Aristotle’s “universals” = what is SAID OF a subject –Substantial concepts: patient, tumor,... –Quality concepts: white, temperature –...

13. www.landc.be 13/58Werner Ceusters CTO www.landc.be What is natural language understanding? NLU is constructing meaning from “written” language by which the degree of understanding involves a multifaceted meaning-making process that depends on knowledge about language and knowledge about the world. ( cf. “reading comprehension” by humans. ) But then: what is “meaning”

14. www.landc.be 14/58Werner Ceusters CTO www.landc.be Dyadic models of “meaning” Saussure ( language philosopher ): –signe / signifiant(sign/concept) Ron Stamper ( information scientist ): –thing-A STANDS-FOR thing-B Major drawback: –excludes the “referent” from the model, i.e. that what the sign/symbol/word/... denotes

15. www.landc.be 15/58Werner Ceusters CTO www.landc.be Triadic models of meaning: The Semiotic/Semantic triangle Sign: Language/ Term/ Symbol Referent: Reality/ Object Reference: Concept / Sense / Model / View / Partition

16. www.landc.be 16/58Werner Ceusters CTO www.landc.be Aristotle’s triadic meaning model semeia gramma/ phoné pragma pathema Words spoken are signs or symbols (symbola) of affections or impressions (pathemata) of the soul (psyche); written words (graphomena) are the signs of words spoken (phoné). As writing (grammatta), so also is speech not the same for all races of men. But the mental affections themselves, of which these words are primarily signs (semeia), are the same for the whole of mankind, as are also the objects (pragmata) of which those affections are representations or likenesses, images, copies (homoiomata). Aristotle, 'On Interpretation', 1.16.a.4-9, Translated by Cooke & Tredennick, Loeb Classical Library, William Heinemann, London, UK, 1938.

17. www.landc.be 17/58Werner Ceusters CTO www.landc.be Richards’ semantic triangle Reference (“concept”): “indicates the realm of memory where recollections of past experiences and contexts occur”. Hence: as with Aristotle, the reference is “mind- related”: thought. But: not “the same for all”, rather individual mind-related symbolreferent reference understandingmy your understanding

18. www.landc.be 18/58Werner Ceusters CTO www.landc.be Don’t confuse with homonymy ! “mole” mole “animal” R1 mole “unit” R2 mole “skin lesion” R3

19. www.landc.be 19/58Werner Ceusters CTO www.landc.be Different thoughts Homonymy “ mole ” mole “ animal ” R1 mole “ unit ” R2 mole “ skin lesion ” R3 symbol referent understanding One concept of x understanding of y

20. www.landc.be 20/58Werner Ceusters CTO www.landc.be And by the way, synonymy... the Aristotelian viewRichards’ view “perspiration” “sweat” “perspiration”

21. www.landc.be 21/58Werner Ceusters CTO www.landc.be Frege’s view “sense” is an objective feature of how words are used and not a thought or concept in somebody’s head 2 names with the same reference can have different senses 2 names with the same sense have the same reference (synonyms) a name with a sense does not need to have a reference (“Beethoven’s 10 th symphony”) reference (=referent) sense name

22. www.landc.be 22/58Werner Ceusters CTO www.landc.be Tetrahedric extensions conception actor representationreferent concept termreferent definition FRISCO model (information science) CEN/TC251 ENV 12264

23. www.landc.be 23/58Werner Ceusters CTO www.landc.be Requirements for NLU 1.Knowledge about terms and how they are used in valid constructions within natural language; 2.Knowledge about the world, i.e. how the referents denoted by the terms interrelate in reality and in given types of context; 3.An algorithm that : a.is able to calculate a language user’s representation of that part of the world described in the utterances that are the subject of the analysis. b.can track the ways in which people express what does NOT represent anything in reality (eg for medico-legal reasons) Only a realist ontology (and not an ontology that deals with “alternative realities”) permits correct disambiguation between 3a and 3b.

24. www.landc.be 24/58Werner Ceusters CTO www.landc.be As such... The “things at the top of all the triangles seen so far” are dynamic algorithmic things, and not fixed structures. Additional support comes from: –P.F.Strawson: “we must stop trying to locate meaning in some invariant relation in which phrases and words stand to the world” –But what then ? “meaning is use” (Quine, Wittgenstein, Davidson, McDowell,...) A theory of meaning taking context and referents into account Conclusion: who concentrates on the “concepts” is doomed to fail. to be or not to be structured

25. www.landc.be 25/58Werner Ceusters CTO www.landc.be Why are concepts not enough? Why must our theory address also the referents in reality? –Because referents are observable fixed points in relation to which we can work out how the concepts used by different communities relate to each other ; –Because only by looking at referents can we establish the degree to which concepts are good for their purpose.

26. www.landc.be 26/58Werner Ceusters CTO www.landc.be But why then this fixation on normative “concepts” in Medical Informatics (standards) ? CEN/TC251 ENV 12264 : –This ENV is applicable to the description of the categorial structure of systems of concepts supporting computer-based terminological systems, including coding systems, for health-care. –concept : “unit of thought constituted through abstraction on the basis of properties common to a set of one or more referents” BUT THEY NEVER IN FACT LOOK AT THE REFERENTS AT ALL! ISO/TC215/N142: Health informatics —Vocabulary of terminology –The purpose of this International Standard is to define a set of basic concepts required to describe and discuss formal representation of concepts and characteristics, for use especially in formal computer based concept representation systems. –concept: “unit of knowledge created by a unique combination of characteristics” THEY ARE ALREADY TWO LEVELS REMOVED FROM THE REFERENT!

27. www.landc.be 27/58Werner Ceusters CTO www.landc.be What is part of the real world, and what isn’t ? CEN/TC251 ENV 12264 : – “In this model, the elements in the base plane of the pyramid are all concrete or abstract phenomena of the real world (or so established as items of knowledge that they are considered real, e.g. "perpetuum mobile"), or expressions in a language. These phenomena are called objects or, better, referents. Concepts situated at the top of the pyramid are mental constructs. To talk about them, we have to use established expressions, such as names, terms in special subject fields, or longer verbal definitions”. the real world

28. www.landc.be 28/58Werner Ceusters CTO www.landc.be “Ontology” In Information Science: –“An ontology is a description (like a formal specification of a program) of the concepts and relationships that can exist for an agent or a community of agents.”(Tom Gruber) In Philosophy: –“Ontology is the science of what is, of the kinds and structures of objects, properties, events, processes and relations in every area of reality.” (Barry Smith)

29. www.landc.be 29/58Werner Ceusters CTO www.landc.be "Where there is the sound of a blow, there is respect” (Pashtun proverb) “I repeatedly get confused by the (in my opinion structurally confusing) terminology of those people (like John Sowa) who try to do ontology but end up just studying concepts.” (Barry Smith, pers. comm.)

30. www.landc.be 30/58Werner Ceusters CTO www.landc.be From buzz-word to the “O-word” “An ontology is a classification methodology for formalizing a subject's knowledge or belief system in a structured way. Dictionaries and encyclopedias are examples of ontologies.” (X1) “A terminology (or classification) is a kind of ontology by definition and it should preserve (and "understand") the relationships between the 1,000s of terms in it or else it would become a mere dictionary (or at best a thesaurus).” (X2) “Ontologies are Web pages that contain a mystical unifying force that gives differing labels common meaning.” (X3)

31. www.landc.be 31/58Werner Ceusters CTO www.landc.be Why existing “ontologies” don’t match OUR needs for a “core” ontology MeSH: inconsistency in hierarchical relationships MedDRA: no difference between concepts and terms UMLS: integrates various source terminologies without taking different meanings of terms, different structures, different purposes, etc... into account SNOMED: formal system, but lacks sufficient depth of the ontology GALEN: very detailed ontology for some parts of healthcare but very poor coverage over healthcare as a whole. The ontology is independent from language as medium of communication (the ontology does not accept language as part of reality)... Most important: all of them deal with alternative realities or possible worlds and none is focused on the referents in THIS world !

32. www.landc.be 32/58Werner Ceusters CTO www.landc.be Another known problem: Intentionality in the semiotic triangle “The physician wanted to give the patient an injection” The physician gave the injection (= referent), and because of that, the patient died from a side- effect. Hence: “giving the injection” = “killing the patient” (= two references) Hence??? –“the physician wanted to kill the patient”

33. www.landc.be 33/58Werner Ceusters CTO www.landc.be Our approach concept system languagereferents definitions medical+linguistic ontology (data + algorithms) languagereferents concept system the standard viewour view the real world

34. www.landc.be 34/58Werner Ceusters CTO www.landc.be Exploit the relationships along the vertices language referents Baboons and humans have different cut-off points for discerning "same" objects because our verbal expression for "same" makes the idea of "same" more restrictive.” Fagot and Wasserman (Centre for Research in Cognitive Neuroscience in Marseille) Meaning is located in the interaction between living beings and the environment James J. Gibson, Ecological Realism in Psychology The structures of language are partially determined by our conceptualisation of the world. Halliday No mental representation without language Fodor concept Halliday’s systemic functional grammar Aristotelian realism

35. www.landc.be 35/58Werner Ceusters CTO www.landc.be The current picture linguistic ontologies (per language) medical+linguistic ontology normative concept system(s) Realist ontology

36. www.landc.be 36/58Werner Ceusters CTO www.landc.be The possible final picture BFO/MedO “validates”

37. www.landc.be 37/58Werner Ceusters CTO www.landc.be Why BFO might match our needs BFO = Basic Formal Ontology (Barry Smith, draft 0.0005 (3.7.02) BFO should provide a theory of formal categories for entities of all types, including substances, qualities/ roles/ functions/ dispositions, and processes then for each of a series of what we might think of as flat domains, starting with medicine (chemistry, genetics…= MedO), BFO should provide the basis for a theory of the categories of entities in those domains for non-flat domains like language we will need three components: –1. BFO applied to language itself –2. BFO applied to the world (the referents) –3. an ontological theory of the relations between 1. and 2.

38. www.landc.be 38/58Werner Ceusters CTO www.landc.be BFO/SNAP: Entities existing in toto at a time

39. www.landc.be 39/58Werner Ceusters CTO www.landc.be

40. www.landc.be 40/58Werner Ceusters CTO www.landc.be

41. www.landc.be 41/58Werner Ceusters CTO www.landc.be

42. www.landc.be 42/58Werner Ceusters CTO www.landc.be SPAN: Entities extended in time (1)

43. www.landc.be 43/58Werner Ceusters CTO www.landc.be SPAN: Entities extended in time (2)

44. www.landc.be 44/58Werner Ceusters CTO www.landc.be An integrated approach Data structure and function library for language understanding Medical and linguistic knowledge required for language understanding NLU enabling tools for knowledge supported data-entry and -retrieval

45. www.landc.be 45/58Werner Ceusters CTO www.landc.be Medico-linguistic ontology Formal Domain Ontology Lexicon Grammar Language A Lexicon Grammar Language B Cassandra Linguistic Ontology MEDDRA ICD SNOMED ICPC Others... Proprietary Terminologies

46. www.landc.be 46/58Werner Ceusters CTO www.landc.be Based on formal ontology HAS- PARTIAL- SPATIAL- OVERLAP IS- TOPO- INSIDE- OF IS-GEO- INSIDE- OF IS- INSIDE- CONVEX- HULL-OF IS-PARTLY- IN-CONVEX- HULL-OF IS- OUTSIDE- CONVEX- HULL-OF HAS- DISCONNECTED- REGION HAS- EXTERNAL- CONNECTING- REGION HAS-DISCRETED- REGION HAS- TANG.- SPAT.- PART HAS-NON- TANG.- SPAT.- PART IS- SPAT.- EQUIV.- OF IS- TANG.- SPAT.- PART-OF IS-NON- TANG.- SPAT.- PART-OF HAS- PROPER- SPATIAL -PART IS- PROPER- SPAT.- PART-OF HAS- SPATIAL -PART IS- SPATIAL -PART- OF HAS- OVERLAPPING -REGION HAS- CONNECTING- REGION HAS-SPATIAL- POINT- REFERENCE

47. www.landc.be 47/58Werner Ceusters CTO www.landc.be Example: joint anatomy joint HAS-HOLE joint space joint capsule IS-OUTER-LAYER-OF joint meniscus –IS-INCOMPLETE-FILLER-OF joint space –IS-TOPO-INSIDE joint capsule –IS-NON-TANGENTIAL-MATERIAL-PART-OF joint joint –IS-CONNECTOR-OF bone X –IS-CONNECTOR-OF bone Y synovia –IS-INCOMPLETE-FILLER-OF joint space synovial membrane IS-BONAFIDE- BOUNDARY-OF joint space

48. www.landc.be 48/58Werner Ceusters CTO www.landc.be Linking external ontologies MESH-2001 : “Seizures” MESH-2001 : “Convulsions” Snomed-RT : “Convulsion” Snomed-RT : “Seizure” L&C : ConvulsionL&C : Seizure L&C : Health crisis L&C : Epileptic convulsion IS-A IS-narrower-than ISA Has-CCC

49. www.landc.be 49/58Werner Ceusters CTO www.landc.be Linguistic and domain ontologies Having a healthcare phenomenon Generalised Possession Healthcare phenomenon Human IS-A Has- possessor Has- possessed Patient Is-possessor-of Cancer patient IS-A Has-Healthcare- phenomenon Malignant neoplasm IS-A 1 1 1 2 2 3 3 lung carcinoma IS-A Mr. Kovács has a pulmonary carcinoma

50. www.landc.be 50/58Werner Ceusters CTO www.landc.be Halliday’s systemic functional grammar A “complete” theory for NLU –constructivistic basis: “language construes human experience” –English: It is raining –Chinese: The sky drops water hence: natural languages are instances of generic schemes –macro-structure of documents derive a “structural formula” –micro-structure of documents lexical cohesion in-conjunction analysis

51. www.landc.be 51/58Werner Ceusters CTO www.landc.be Mr Kovács male human human Dom-ent “Kovács” name Mr Kovács Is-assigned- name-of Concept Instance Text From text to meaning

52. www.landc.be 52/58Werner Ceusters CTO www.landc.be “Mr Kovács” analysed syntactically, and features used to drive mapping. The Orth slot of a word gives its surface string. The append( ) operator joins together its arguments as a single string. human-name Sem: human HUMAN- NAME-TYPE4 human-surname HUMAN- NAME-TYPE titled-human Title: title Title -2 TITLED- HUMAN-TYPE female-titled Title: female-title Sem: female-human male-titled Title: male-title Sem: male-human genderless-titled untitled-human HUMAN- NAME-TYPE3 prenamed-provided Prename: human-firstname Prename -1 Sem.Assigned_Name = append{Prenam.Orth, Orth} prename-not-provided Sem.Assigned_Name = Orth human-firstname

53. www.landc.be 53/58Werner Ceusters CTO www.landc.be Conclusions “Understanding a message” comes down to identifying what parts of that message correspond to reality, and what parts are expressions of what doesn’t exist. If a machine has to understand, it must be based on algorithms that use a realist ontology that takes the world, language(s) and the relationship amongst them, properly into account. The medical informatics community (specifically that part dealing with “concept systems”) must become aware that most current approaches confuse “what is real” with “what is thought to be real”.

54. www.landc.be 54/58Werner Ceusters CTO www.landc.be Analysis of “History of CVA”

55. www.landc.be 55/58Werner Ceusters CTO www.landc.be Concepts-terms-criteria-definitions

56. www.landc.be 56/58Werner Ceusters CTO www.landc.be Ontology alignment

57. www.landc.be 57/58Werner Ceusters CTO www.landc.be

58. www.landc.be 58/58Werner Ceusters CTO www.landc.be