1. A Real-World Knowledge Engineering Application: The NeuroScholar Project Gully APC Burns K. M. Research Group University of Southern California

2. Structure of the presentation 1. Ideas & Concepts 2. Design 3. Implementation 4. Demonstration

3. I. Ideas & Concepts In which we are reminded of what most people think knowledge is, how it is currently used (and misused) and how we might improve matters.

4. Main Entry: knowl·edge Pronunciation: 'nä-lij Function: noun Etymology: Middle English knowlege, from knowlechen to acknowledge, irregular from knowen Date: 14th century 1 obsolete : COGNIZANCE 2 a (1) : the fact or condition of knowing something with familiarity gained through experience or association (2) : acquaintance with or understanding of a science, art, or technique b (1) : the fact or condition of being aware of something (2) : the range of one's information or understanding c : the circumstance or condition of apprehending truth or fact through reasoning : COGNITION d : the fact or condition of having information or of being learned 3 archaic : SEXUAL INTERCOURSE 4 a : the sum of what is known : the body of truth, information, and principles acquired by mankind b archaic : a branch of learningCOGNIZANCEknowingCOGNITIONSEXUAL INTERCOURSEknown What does the word ‘Knowledge’ mean? [from http://www.m-w.com/]

5. The published literature Image taken from U.S. Geological Survey Energy Resource Surveys Program … is the end-product of research and as such forms the basis for human understanding of the subject … is very valuable. … is structured. … is interpretable.

6. The published literature Image taken from U.S. Geological Survey Energy Resource Surveys Program … is large and unwieldy. … has varying reliability. … is inconsistent. … is based on natural language. … is difficult to automate. … is terse … is qualitative … is 2-D

7. The published literature Image taken from U.S. Geological Survey Energy Resource Surveys Program … is a valid target for attack with informatics-based methods. This permits … (a) Increased clarification through formalization (b) large-scale data-handling capability (c) analysis of existing data to examine organization

8. A semantic continuum [Mike Uschold, Boeing Corp] Shared human consensus Text descriptions Semantics hardwired; used at runtime Semantics processed and used at runtime ImplicitInformal (explicit) Formal (for humans) Formal (for machines) Further to the right means: Less ambiguity More likely to have correct functionality Better inter-operation (hopefully) Less hardwiring More robust to change More difficult The current status of ‘theory’ in Neuroscience How we would like neuroscientists to think Where we would like to work

9. What’s wrong with this picture? …from a neuroscientist’s point of view… From Swanson (1998), “Brain Maps, Structure of the Rat Brain”, 2 nd edition, Elsevier, Amsterdam. Number of structures = 500 x 2 Number of Cell Groups per structure = 10 Number of Possible Connections between cell groups = 10,000 x 10,000 = 10 8 Estimated Number of Connections between cell groups = 250,000

10. … it’s even worse than that … Neuroscience is extremely multidisciplinary Spatial Scales of Measurement: 10 1 – 10 -9 m Temporal Scales of Measurement: 70 yrs (2.21 x 10 9 s) to 10 -3 s (not even including evolutionary time!) Study occurs in a heterogeneous theoretical framework involving: Anatomy, Physiology, Psychology, Ethology, Biochemistry (Molecular Biology, Genetics, Bioinformatics), Biophysics, Behavioural Ecology, Biology … to name a few… All of these subjects are specialized, hard to link work between disciplines and across levels

11. … & it’s even worse than that !!! Neuroanatomical nomenclature are the closest thing that neuroscience has for a standardized framework… In any given paper, the same name may be used for different structures, or different names may be used different structures. e.g., ‘Globus Pallidus, pars medialis (GPm)’ also called the ‘Entopeduncular Nucleus’ by others. See the index of Swanson (1998), “Brain Maps, Structure of the Rat Brain”, 2 nd edition, Elsevier, Amsterdam list of synonyms according to one source.

12. We restrict the problem space to a specific soluble strategy 1. Describe a given phenomenon (e.g., the stress response). 2. Identify which populations of neurons are involved in the phenomenon (i.e., any neurons that turn on, turn off, change their firing, affect the phenomenon if messed with, etc.). 3. Represent how these populations of neurons are interconnected. 4. Represent the dynamic processes of there neurons that underlie the phenomenon.

13. A Construct: ‘A Knowledge Model’ = A personalized representation of an individual’s knowledge. e.g., A review article is an example of a non- computational knowledge model

14. Another Construct: ‘Knowledge Landscape’ = A map of Knowledge Models (where each KM is timestamped) e.g., An list of the best reviews of a given subject over time is an example of a non-computational knowledge landscape

15. II. Design In which all of these high-falutin’ ideas are put into a logical design and it becomes clear that the design criteria of the NeuroScholar project distinguish it from pure research in computer science

16. Some design requirements In order of importance 1. Powerful & enabling to neuroscientists in their everyday work 2. Easy to use! (i.e., free, multi-platform, one-click installation) 3. Knowledge acquisition / data collation is the rate limiting step 4. Open-source for future development as an academic project.

17. Knowledge Landscapes NeuroScholar Screenshot- (dummy data)

18. Knowledge Landscapes ‘Knowledge Landscape’ ‘Knowledge Model’ ‘Fragments’ ‘Entities’ ‘Properties’‘Relations’ ‘Annotations’ ‘Data Collection’ NeuroScholar Screenshot- (dummy data)

19. ‘Fragments’ ‘Entities’ ‘Properties’‘Relations’ ‘Annotations’ ‘Data Collection’ A set of data fragments e.g. a publication: Allen GV & DF Cechetto. (1993) J Comp Neurol 330:421-438. Knowledge Models & examples

20. ‘Entities’ ‘Properties’‘Relations’ ‘Annotations’ ‘Data Collection’ ‘Fragments’ individual pieces of the literature e.g. descriptions of experimental results. “… Moderate to light terminal labeling was present in the parvocellular portions of the paraventricular nucleus, anterior-hypothalamic nucleus, anterior portion of the lateral hypothalamic area (Figs. 2D, 3B), and in the central nucleus of the amygdala (Fig, 2D)….” From Allen & Cechetto (1993) Knowledge Models & examples

21. ‘Fragments’ ‘Relations’ ‘Annotations’ ‘Data Collection’ Abstract data structures that capture the meaning of a set of fragments within the framework of the NeuroScholar system ‘Entities’ ‘Properties’ injectionSitelabeling experimentalMethod e.g. neuronPopulation object knowledge type = description domain type = tract-tracing experiment brainVolumes Knowledge Models & examples

22. ‘Fragments’ ‘Entities’ ‘Properties’‘Relations’ ‘Annotations’ ‘Data Collection’ Rules that link two objects together. ‘Relations’ LHA ZI Knowledge Models & examples

23. ‘Fragments’ ‘Entities’ ‘Properties’‘Relations’ ‘Annotations’ ‘Data Collection’ ‘Summaries’ Sets of objects and relations, explicitly selected and prioritized within system Knowledge Models & examples neuronPopulation2

24. ‘Fragments’ ‘Objects’ ‘Properties’‘Relations’ ‘Annotations’ ‘Data Collection’ Human-interpretable text to make contents of knowledge base understandable ‘Annotations’ Knowledge Models & examples

25. Distributed Online Sources of Information ‘Fragments’ Local Implementation

26. Distributed Online Sources of Information ‘Fragments’ Local Implementation Users’ Spaces & Models Centralized Published Knowledge Repository

27. Distributed Online Sources of Information Users’ Spaces & Models ‘Fragments’ ‘Pending Review’

28. Distributed Online Sources of Information Users’ Spaces & Models ‘Fragments’ P2P sharing Knowledge Model Comparison

29. Given two users A & B, with Knowledge Models K A & K B being shared under the P2P model. We want A to be able to run a program that automatically compares K B to K A so that the discrepancies and contradictions between the two models can be understood and reconciled.

30. What’s wrong with this picture? …from an computer scientist’s point of view… Where is the formal logic? It’s o.k. if we only export knowledge models to a formal logic-based representation rather that base our entire approach on it. Knowledge Acquisition is the rate-limiting step!

31. Knowledge Representation Knowledge representation is a multidisciplinary subject that applies theories and techniques from three other fields: 1. Logic provides the formal structure and rules of inference. 2. Ontology defines the kinds of things that exist in the application domain. 3. Computation supports the applications that distinguish knowledge representation from pure philosophy… Sowa (2000), Knowledge Representation: Logical, Philosophical, and Computational Foundations, Brooks Cole Publishing Co., Pacific Grove, CA.

32. Knowledge Representation … Without logic, a knowledge representation is vague, with no criteria for determining whether statements are redundant or contradictory. Without ontology, the terms and symbols are ill- defined, confused, and confusing. And without computable models, the logic and ontology cannot be implemented in computer programs. Knowledge representation is the application of logic and ontology to the task of constructing computable models for some domain. Sowa (2000), Knowledge Representation: Logical, Philosophical, and Computational Foundations, Brooks Cole Publishing Co., Pacific Grove, CA.

33. III. Implementation In which the design issues become concerned with more pressing concerns like: ‘how are we actually going to build this thing?’

34. Some implementation choices u Built under UML-based software engineering paradigm u The View-Primitive-Data-Model framework (‘VPDMf’) u Object Oriented Design u Unified Modeling Language (UML) u PerlOO u Java u Relational Databases u MySQL u Informix u Exporting Ontologies (via the VPDMf) u XML, RDF, Flogic u Exporting Logic u Embedded within typed Relation objects within the OO knowledge model. u Use simple method overloading in Java to run Knowledge Model Comparison

35. VPDMf System Builder VPDMf specs (Data Model file & VPDMf XML files) UML-based documentation DBMS User Interface Component Final Working System Forward Engineering Reverse Engineering

36. Implementation Plan Main Database PluginsVPDMf Client App Local Database ServerClient Review Database VPDMf Admin App Plugins

37. Implementation Plan Main Database PluginsVPDMf Client App Local Database Local Apps ServerClient Review Database VPDMf System Builder VPDMf Admin App Plugins

38. Implementation Plan Main Database PluginsVPDMf Client App Local Database ServerClient Review Database VPDMf Admin App Plugins Demonstration

39. Large scale organization of NeuroScholar’s schema Data management of publication data General knowledge management structures Annotations, Justifications, Judgements Experimental data, General histological data Neuroanatomical tract tracing data Final output of the system: the knowledge model Components of the knowledge model specific to neuronal data General data constructs used throughout the system

40. e.g., Views from ‘bibliography’

41. ViewDefinition Article ViewDefinition Fragment ViewLink

43. Basic Functionality: The ViewStateMachine & Forms

44. Additional Functionality: Specialized Form Controls & Plugins 1. The Article Robot Form Control Uses PubMed to retrieve citation information easily 2. The Fragmenter Plugin Allows delineation of fragments on pdf files 3. The AtlasMapper Plugin Allows delineation of regions on brain maps

50. IV. Demonstration In which the truth is finally revealed

51. Acknowledgements This work is funded by the National Library of Medicine (RO1-LM07061-01) Thanks to Arshad Khan Shahram Ghandehanderazdeh Cyrus Shahabi Mark O’Neill Larry Swanson Alan Watts Mihail Bota Wei Cheng Chen Shyam Kapadia Shanshan Song Ning Zhang Yi-Shin Chen