Bruce Porter (University of Texas) Peter Clark (Boeing) and Colleagues Building KB’s by Assembling Components: An early evaluation of the approach

Claims A component is a set of axioms that: describe a consensus view of a common concept can be frequently reused with little modification … … to combine with other components … … to represent domain knowledge well.

Claims A component is a set of axioms that: describe a consensus view of a common concept can be frequently reused with little modification … … to combine with other components … … to represent domain knowledge well. versus an idiosyncratic view

Claims A component is a set of axioms that: describe a consensus view of a common concept can be frequently reused with little modification … … to combine with other components … … to represent domain knowledge well. versus infrequently reused without significant modification

Claims A component is a set of axioms that: describe a consensus view of a common concept can be frequently reused with little modification … … to combine with other components … … to represent domain knowledge well. versus difficult or impossible to combine (“round holes and square pegs)

Claims A component is a set of axioms that: describe a consensus view of a common concept can be frequently reused with little modification … … to combine with other components … … to represent domain knowledge well. versus representations are incomplete or inconsistent We’ll evaluate each of these claims in turn …

The Context for our Evaluation: The Component Approach in Practice We are building a component library while assembling a microbiology KB for the TKCP For each microbiology topic, we will: –Identify the core concepts used in textbook accounts –Build components for these concepts –Use these components to represent the topic in the KB

Do the components capture a consensus view? Internal evaluation: –Have multiple KE’s (team members) independently encode each component. –Measure the agreement among the representations External evaluation: –Translate components from KM into English, yielding “dictionary definitions” –Have n subjects review the definitions then revise them if they deem appropriate. –Have an independent subject extract the “consensus view” from the n definitions. –Measure the agreement between the consensus view and the original ones.

Can the components be frequently reused with little modification? Count the amount of reuse of each component. Count the number of modifications made for each instance of reuse, and weight it by its severity.

Do the components combine together well? For each microbiology topic: –Write a high-level design for its representation which shows: The components that comprise it Their instantiations and inter-relationships –Implement the design –Measure and study the times when implementation details force changes to the design

Do the components represent domain knowledge well? For each topic represented in the KB: –Measure how much of its representation was provided by components; the balance was coded specifically for this topic. –The TKCP will measure the quality of the overall representation, in terms of consistency, completeness, support for inference, and so on. –Analyze the KB’s successes and failures at the TKCP to assign responsibility to the components (versus other aspects of the KB system).

Summary We have identified the “strong claims” of the component approach We plan to evaluate the claims early – while building a microbiology KB The data we collect will be invaluable to study questions we’ve only begun to formulate