1. A Knowledge-Rich Approach to Understanding Text about Aircraft Systems Peter Clark Lisbeth Duncan Heather Holmback Tom Jenkins John Thompson Boeing Engineering and Information Technology

2. Overview Situation: Using a hand-built, airplane knowledge-base (KB) at Boeing (e.g., for concept-based search) Goal: Extend this KB by processing text: from airplane training manuals directly from an aerospace engineer Target: airplane parts, connections, and behaviors

3. Challenge Interpretation of text requires background knowledge –author trying to convey a model to the reader –only some parts of this are explicitly stated in text –rest is assumed to be already known to the reader “The hydraulic system supplies power to the rudder.” Hydraulic System Rudder Supply Power Powered-System Script ConsumeTransmit Conduit Airplane sourcesink behavior

4. Approach Use an aerospace knowledge base (KB): –Taken from an earlier project and extended –Contains general models about what might be said Contains general knowledge of aircraft and systems Doesn’t know details of specific aircraft Text interpretation = specializing and refining this background knowledge

5. Constraints on the Scope 1.Working with restricted sublanguage of English –Simple, declarative sentences –Regular, “boring” style “The hydraulic system supplies power to the rudder.” “A ripple damper smooths the pump pressure output.” “The EDP is on the left side of the engine main gearbox.” “The gearbox turns the EDP when the engine turns.” 2.Constrained domain (aircraft systems) –Reduces vocabulary, background KB 3.Domain of mechanical artifacts –Simple statements about structure and behavior

6. KB 777-200-Airplane Approach User specifies topic (eg. “777-200-Airplane”) Tail Powered System Rudder Vertical Stabilizer Power Source Supplying FuselageWing Flap parts sourcesink script systems parts purpose connects agentrecipient Initial (general) representation of airplane built. “The hydraulic system supplies power to the rudder.” Text is provided.

7. 777-200-Airplane “The hydraulic system supplies power to the rudder.” Tail Powered System Rudder Vertical Stabilizer Power Source Supplying FuselageWing Flap KB parts sourcesink script systems parts purpose connects agentrecipient Approach Then: 1.Text  NLP structure Rudder Hydraulic System Supplying Power recipient object agent Stage 1

8. 777-200-Airplane “The hydraulic system supplies power to the rudder.” Tail Powered System Rudder Vertical Stabilizer Power Source Supplying FuselageWing Flap Rudder Hydraulic System Supplying Power KB ? recipient object agent parts sourcesink script systems parts purpose connects agentrecipient Approach Then: 1.Text  NLP structure 2.Match NLP structure with airplane repn. Stage 1Stage 2

9. 777-200-Airplane “The hydraulic system supplies power to the rudder.” Tail Powered System Rudder Vertical Stabilizer Supplying FuselageWing Flap Rudder Hydraulic System Supplying Power KB Hydraulic System Power Approach Then: 1.Text  NLP structure 2.Match NLP structure with airplane repn. Unify the matching structures. recipient object agent parts sourcesink script systems parts purpose connects Stage 1Stage 2

10. 777-200-Airplane Tail Powered System Vertical Stabilizer FuselageWing Flap KB parts sourcesink script systems parts purpose connects Stage 2 The Knowledge Base Rudder Power Source Supplying agentrecipient “The hydraulic system supplies power to the rudder.” Rudder Hydraulic System Supplying Power recipient object agent Stage 1

11. The Knowledge Base ~500 concepts, ~1500 axioms Mainly airplane parts, behaviors, and connections Axioms converted to prototypes (graph structures) Axiom: “Every airplane has a fuselage, and a tail connected to the fuselage.” (every Airplane has (parts ((a Fuselage) (a Tail with (connected-to (the Fuselage parts of Self)))) Airplane FuselageTail parts connected-to Prototype (sketch)

12. Inference: Building the Initial Airplane Representation 777-200-Airplane Tail Airplane FuselageTail parts Vertical Stabilizer Horizontal Stabilizer parts Flight TakeoffCruiseLand Flight behavior Horizontal Stabilizer parts Vertical Stabilizer subevents TakeoffCruiseLand FuselageTail parts connected-to behavior Flight Axioms in KB:Airplane representation:

13. KB 777-200-Airplane Tail Powered System Rudder Vertical Stabilizer Power Source Supplying FuselageWing Flap parts sourcesink script systems parts purpose connects agentrecipient “The hydraulic system supplies power to the rudder.” Stage 1: Text to NLP Structure

14. 777-200-Airplane “The hydraulic system supplies power to the rudder.” Tail Powered System Vertical Stabilizer FuselageWing Flap Rudder Hydraulic System Supplying Power KB recipient object agent parts sourcesink script systems parts purpose connects Stage 1 Stage 1: Text to NLP Structure Rudder Power Source Supplying agentrecipient

15. Stage 1: Text to NLP Structure Superficial linguistic variations normalized Add initial semantic interpretation Commits to: overall parse/syntactic configuration, word senses May underspecify: some semantic relationships (e.g., modifiers) Rudder Hydraulic System Supplying Power recipient object agent Abstract & rewrite “The hydraulic system supplies power to the rudder.” Parse & semantic analysis Supply_1 [sing,pres,3S] To_5 Hydraulic System_1 [sing,N] The_1 [det] delim recipient range object causer delim Power_1 [sing,mass] Rudder_1 [sing,N] The_1 [det]

16. 777-200-Airplane “The hydraulic system supplies power to the rudder.” Tail Powered System Vertical Stabilizer FuselageWing Flap Rudder Hydraulic System Supplying Power KB recipient object agent parts sourcesink script systems parts purpose connects Stage 1 Rudder Power Source Supplying agentrecipient

17. 777-200-Airplane Tail Powered System Vertical Stabilizer FuselageWing Flap parts sourcesink script systems parts purpose connects Rudder Power Source Supplying agentrecipient “The hydraulic system supplies power to the rudder.” Rudder Hydraulic System Supplying Power recipient object agent Stage 1 ? Stage 2 KB Stage 2: Match NLP Structure with Airplane Representation

18. Stage 2: Match with Airplane Representation Goal: Integrate text into airplane representation Approach: find matching (subsuming) structure(s) Rudder Hydraulic System Supplying Power recipient object agent Rudder Power Source Supplying agentrecipient …… … Text KB (Topic airplane representation)

19. Stage 2: Match with Airplane Representation Goal: Integrate text into airplane representation Approach: find matching (subsuming) structure(s) If match found, unify the two structures together Rudder Hydraulic System Supplying Power recipient object agent Rudder Power Source Supplying agentrecipient …… … Text KB (Topic airplane representation)

20. Stage 2: Match with Airplane Representation Goal: Integrate text into airplane representation Approach: find matching (subsuming) structure(s) If match found, unify the two structures together Rudder Hydraulic System Supplying Power recipient object agent Rudder Power Source Supplying agentrecipient …… … Rudder Hydraulic System Supplying agentrecipient … Power object ……

21. KB expects: RudderPower Source SupplyingPower “The hydraulic system supplies power to the rudder.” But text may deviate with: RudderHydraulic System Providing Power “The hydraulic system provides power to the rudder.” 1. Synonyms Rudder Hydraulic System Powering “The hydraulic system powers the rudder.” 2. Contractions Rudder Hydraulic System Be Supplier Power “The hydraulic system is the supplier of power for the rudder.” 3. Roles 4. … Matching Problem 1:Linguistic Variation

22. Use simple transformation rules to modify NLP structure No match  transform and re-search for a match RudderHydraulic System SupplyingPower RudderHydraulic System Providing Power synonym Rudder Hydraulic System Be Supplier Power role Rudder Hydraulic System Powering contraction/ expansion Matching Problem 1: Linguistic Variation

23. Matching Problem 2: Expression of Implied Facts Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable PressMovePull Rudder Control System Script objectagent behavior subevents agent object agent object Explicit facts:

24. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable Press MovePull Rudder Control System Script objectagent behavior subevents agent object agent object “The pilot presses the pedal.” Explicit facts: Matching Problem 2: Expression of Implied Facts

25. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable PressMove Pull Rudder Control System Script objectagent behavior subevents agent object agent object “The pilot presses the pedal.” “The pedal pulls a cable.” Explicit facts: Matching Problem 2: Expression of Implied Facts

26. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable Press Move Pull Rudder Control System Script objectagent behavior subevents agent object agent object “The pilot presses the pedal.” “The pedal pulls a cable.” “The cable moves the rudder.” Explicit facts: Matching Problem 2: Expression of Implied Facts

27. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable PressMovePull Rudder Control System Script objectagent behavior subevents agent object agent object “The pilot presses the pedal.” “The pedal pulls a cable.” “The cable moves the rudder.” Explicit facts: Inference Rule: If A does X, and X causes Y then add A does Y. Matching Problem 2: Expression of Implied Facts

28. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable PressMovePull Rudder Control System Script objectagent behavior subevents agent object agent object “The pilot presses the pedal.” “The pedal pulls a cable.” “The cable moves the rudder.” Implied facts: Move object agent instr- ument. “The pilot moves the rudder using the pedal.” Matching Problem 2: Expression of Implied Facts

29. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Rudder Pedal Pilot Cable PressMovePull Rudder Control System Script objectagent behavior subevents agent object agent object “The pilot presses the pedal.” “The pedal pulls a cable.” “The cable moves the rudder.” Implied facts: Move object agent instr- ument. “The pilot moves the rudder using the pedal.” “The pedal moves the rudder using the cable.” Move object instrument agent Matching Problem 2: Expression of Implied Facts

30. Problem: –Not all airplane facts are explicit in the KB structures –If user refers to an implied fact, matcher will not find it Approach: –Match text against an extended version of the KB –Extensions generated on demand Airplane representation: Explicit facts Implied facts

31. Status and Assumptions While some components are mature, only complete throughput for a small number of sentences Relies on strong expectations from the KB –Everything that might be said is encompassed by KB Assumes stage 1 output is correct Assumes a single model of the airplane is described

32. Challenges… Inaccurate knowledge –“The pump supplies a source of power to the rudder.” Simplifications/multiple models –“There are three hydraulic systems in the 777-200.” –“The flight control surfaces steer the airplane.” Fluctuating (“fuzzy”) concept boundaries –“hydraulic system” includes controlled devices (eg rudder)? Ambiguities not locally resolvable –“The EDP is attached to the airplane’s engine.”

33. Summary Goal: extend an airplane KB from text Use of background knowledge can help: –constrains possible interpretations –provide the surrounding context Presented a simple model of how this can be done: –text interpretation = iterative refinement of a representation –use of graph matching and unification