1. The REVERE project: experiments with the application of probabilistic NLP to systems engineering Paul Rayson 1, Luke Emmet 2, Roger Garside 1 and Pete Sawyer 1 1 Lancaster University and 2 Adelard

2. Abstract zDespite natural language's well-documented shortcomings as a medium for precise technical description, its use in software- intensive systems engineering remains inescapable. This poses many problems for engineers who must derive problem understanding and synthesise precise solution descriptions from free text. This is true both for the largely unstructured textual descriptions from which system requirements are derived, and for more formal documents, such as standards, which impose requirements on system development processes. This paper describes experiments that we have carried out in the REVERE project to investigate the use of probabilistic natural language processing techniques to provide systems engineering support.

3. Natural language products of the systems engineering process Systems Engineering Process Requirements Specification Test plans

4. Natural language inputs to the systems engineering process Systems Engineering Process User interview transcripts Standards documents User manuals

5. The REVERE project

6. Robust NLP Large free text corpora NLP toolbox Training data REVERE

7. Syntactic and Semantic analysis zPOS analysis using CLAWS yHybrid tagger using HMM yError rate of 1.5% zSemantic analysis yGeneral sense field of words and idioms yInvestigating applicability to technical domain zWmatrix retrieval tool yFrequency profiling and KWIC

8. Air Traffic Control zEthnographic studies at ATC centre yVerbatim transcripts of observations and interviews with controllers yUnstructured reports y103 pages

9. Role Analysis

10. Corpus analysis Log-likelihoodSemanticWord sense (and examples from the text) tag 3366S7.1power, organising (‘controller’, ‘chief’) 2578M5flying (‘plane’, ‘flight’, ‘airport’) 988O2general objects (‘strip’, ‘holder’, ‘rack’) 643O3electrical equipment (‘radar’, ‘blip’) 535Y1science and technology (‘PH’) 449W3geographical terms (‘Pole Hill’, ‘Dish Sea’) 432Q1.2paper documents and writing (‘writing’, ‘written’, ‘notes’) 372N3.7measurement (‘length’, ‘height’, ‘distance’, ‘levels’, ‘1000ft’) 318L1life and living things (‘live’) 310A10indicating actions (‘pointing’, ‘indicating’, ‘display’) 306X4.2mental objects (‘systems’, ‘approach’, ‘mode’, ‘tactical’, ‘procedure’) 290A4.1kinds, groups (‘sector’, ‘sectors’)

11. Corpus Analysis (2)

12. A Standards Document zNational standard for procurement of safety-critical military systems y21,000 words yStrongly structured and highly stylised

13. POS analysis

14. Role Analysis

15. Conclusions zRapid analyses of complex and voluminous free text zSupport tools for system engineers zShallow analysis to help focus attention zTwo experiments: ATC & Standards Doc