1. CassBeth Inc. www.cassbeth.com1 Specification Analysis Tool (SAT) Overview www.cassbeth.com/sat October 2005

2. CassBeth Inc. www.cassbeth.com2 Importance of Good Reqs Make Sure You Design and Implement What You Thought You Needed –Reqs Driven Development Show That You Actually Implemented What You Needed –This May Include Certifications ROI Justifications Go Back 25+ Years

3. CassBeth Inc. www.cassbeth.com3 Problem Statement How Can you Create a Clean Useful Specification? Buzz Words are: Clear, Complete, Consistent, Testable, etc

4. CassBeth Inc. www.cassbeth.com4 Different Views on Specs A NASA View –Automated Reqs Measurement (ARM) tool An FAA View –Prepared by the Plain English Network (PEN) (www.plainlanguage.gov) –Adapted for FAA, February 2000 (http://www.faa.gov/language/) –Edited for AUA Plain Language Month, April 2003 Walt's View

5. CassBeth Inc. www.cassbeth.com5 Observations No one approach is better or worse –Optimized for each organization An approach is appropriate if it works –Only organization can make determination Appears to be need for User defined analysis rules –Can allow each approach to be optimized

6. CassBeth Inc. www.cassbeth.com6 Levels of Consciousness surface key reqs that terrify all No Politics - Truth did you get all leaves Completeness eliminate similar, duplicates, conflicts Consistency source, sink, nouns, action verbs Clear Complete Testable Imperatives shall, must, will, should... Consistent Imperative Use

7. CassBeth Inc. www.cassbeth.com7 Alternatives Do Nothing Use Manual Techniques Create Your Own Tool Use Other Specification Analysis Tools –QuARS, ARM (NASA), TIGER Use SRDB (DOORS) –Filtering and View Mechanisms Use SAT with SRDB Parent Child Reports

8. CassBeth Inc. www.cassbeth.com8 What Does SAT Do? Performs automated specification analysis –Very fast but Humans make final decisions Looks for bad requirement text –Uses rules spread throughout your organization Looks for complex spec problems –Missing capabilities & key requirements Makes recommendation for each problem –Shows newly minted staff spec creation Gathers Metrics

9. CassBeth Inc. www.cassbeth.com9 Where Does SAT Fit Analysis Identify Key Reqs Prelim Spec Doc Expand Key Reqs DOORS Import Clean Spec Doc Reqs Clean Up DOORS Parent Child Reports Reqs Clean Up Final Spec Doc

10. CassBeth Inc. www.cassbeth.com10 SAT Services Predefined Services –Requirement Text Assessment (RTA) –Generic Structure Analysis (GSA) –Domain Structure Analysis (DSA) –Generic Capabilities Analysis (GCM) –Domain Capabilities Analysis (DCA) –Key Requirements Analysis (KRA) User Defined Services Metrics

11. CassBeth Inc. www.cassbeth.com11 SAT Operations SAT LeadStake Holders Updated Specs Reports Official Reports 1. 2.

12. CassBeth Inc. www.cassbeth.com12 SAT Portal - Demo

13. CassBeth Inc. www.cassbeth.com13 SAT Portal - Demo

14. CassBeth Inc. www.cassbeth.com14 How Does SAT Work User Uploads File –From Local Computer or Network SAT Examines Extension –If.doc SAT Attempts to Parse Into Objects Looks for PUI –If None Matches PUI MASK, creates PUI Converts Each Line Into Object –DOORS Excel Export –MsWord Saved as.txt

15. CassBeth Inc. www.cassbeth.com15 How Does SAT Work Subjects Each Object To Each SAT Rule –Based on Display Filter Setting –Tabulates Metrics For Each Rule Subjects Each Object to Duplicate Check –If Enabled Outputs On-The-Fly Object Results Outputs Metrics at End of Last Object User Modifies Display Filters and Rules

16. CassBeth Inc. www.cassbeth.com16 SAT Control Panel

17. CassBeth Inc. www.cassbeth.com17 SAT Control Panel

18. CassBeth Inc. www.cassbeth.com18 RTA Rule Templates SAT Rules –Untestable, Unsure, Vague 1, Vague 2, Vague 3, Vague 4, Compound Req, Internal Reference, Not Standalone, Fragment, Declarative, Duplicates CMSEI Rules –vague, Subjective, Choice or Option, Implicit Expressions, Weakness, Under Specification, Multiplicity, Duplicates INCOSE Rules –Standard Constructs, Words to Avoid - Vague, Completeness, Duplicates NASA Rules –Completeness, Continuances, Directives, Options, Weak Phrases CMSEI, INCOSE, NASA as interpreted by SAT staff

19. CassBeth Inc. www.cassbeth.com19 SAT Report

20. CassBeth Inc. www.cassbeth.com20 SAT Report

21. CassBeth Inc. www.cassbeth.com21 CMSEI Report

22. CassBeth Inc. www.cassbeth.com22 CMSEI Report

23. CassBeth Inc. www.cassbeth.com23 CMSEI Report

24. CassBeth Inc. www.cassbeth.com24 INCOSE Report

25. CassBeth Inc. www.cassbeth.com25 INCOSE Report

26. CassBeth Inc. www.cassbeth.com26 INCOSE Report

27. CassBeth Inc. www.cassbeth.com27 NASA Report

28. CassBeth Inc. www.cassbeth.com28 NASA Report

29. CassBeth Inc. www.cassbeth.com29 NASA Report

30. CassBeth Inc. www.cassbeth.com30 SAT Report All Services

31. CassBeth Inc. www.cassbeth.com31 SAT Report All Services

32. CassBeth Inc. www.cassbeth.com32 Find Duplicates

33. CassBeth Inc. www.cassbeth.com33 Find Duplicates

34. CassBeth Inc. www.cassbeth.com34 Generic Structure Analysis

35. CassBeth Inc. www.cassbeth.com35 Domain Structure Analysis

36. CassBeth Inc. www.cassbeth.com36 Generic Capabilities Analysis

37. CassBeth Inc. www.cassbeth.com37 Generic Capabilities Analysis

38. CassBeth Inc. www.cassbeth.com38 Domain Capabilities Analysis

39. CassBeth Inc. www.cassbeth.com39 Key Reqs Analysis

40. CassBeth Inc. www.cassbeth.com40 Key Reqs Analysis

41. CassBeth Inc. www.cassbeth.com41 Key Reqs Analysis

42. CassBeth Inc. www.cassbeth.com42 Other Domains

43. CassBeth Inc. www.cassbeth.com43 Other Domains

44. CassBeth Inc. www.cassbeth.com44 Other Domains

45. CassBeth Inc. www.cassbeth.com45 Other Domains

46. CassBeth Inc. www.cassbeth.com46 Other Domains

47. CassBeth Inc. www.cassbeth.com47 SAT Big Picture 3. TEMPLATES 1. RULES 2. SERVICES 5. DOMAINS COMMUNICATIONS DEFENCE ATC 6. PRECISE LANGUAGE INDUSTRIES ENGINEERING GOV REGULATION LEGAL 4. APPLICATION.

48. CassBeth Inc. www.cassbeth.com48 SAT Features Its Simple and Fast –Automates Tedious Tasks With Greatest Benefit Dynamic Display Filters –Change with User Defined Rules Dynamic Metrics –Change with User Defined Rules Report Format Contains Everything –Offending Object Text –Analysis Findings and Recommendations –Metrics

49. CassBeth Inc. www.cassbeth.com49 SAT Features Ubiquitous –Uses Internet Technologies Highly Scalable –Uses Internet Technologies User Defined Rules –Allows Build Up of Templates Its New –You Can Influence Its Engine, Services, Rules, etc

50. CassBeth Inc. www.cassbeth.com50 Why is SAT Different User defined rules with template examples –Allows users to grow their own Structured in terms of services that span simple to highly abstract consciousness –User decides what level makes sense for them Realizes that simple services based on keyword searches remove significant distractions / noise from problem set –So hard stuff can begin to be addressed

51. CassBeth Inc. www.cassbeth.com51 What I Would Like To See Happen Community Buys SAT Community Creates and Publishes Rules Community Suggests Engine Attributes Community Suggests Engine Features Other Domains Adopt SAT –Legal (e.g. contracts, laws, constitutions)

52. CassBeth Inc. www.cassbeth.com52 Where Can I Get A Web Server Window Professional Versions –2000 and XP professional Apache –www.apache.org IndigoPerl –www.indigostar.com Your Computer Network –Ask Your Computer Support Staff SAT May Come Bundled with Web Server –Depends on Early Marketing Findings

53. CassBeth Inc. www.cassbeth.com53 Company Overview CassBeth established in August 1997 Early Goals Were to Study Internet Technologies CassBeth Incorporated in 2000 Large Traffic at E-Commerce Sites SAT is a CassBeth Product Direct Result of Cassbeth web Mining Solutions

54. CassBeth Inc. www.cassbeth.com54 Links / References www.cassbeth.com/sat satc.gsfc.nasa.gov/metrics www.seecforum.unisa.edu/SEECTools.html#tiger www.indigostar.com www.apache.org www.plainlanguage.gov www.faa.gov/language/ www.pupress.princeton.edu/titles/7929.html www.archive.org www.incose.org

55. CassBeth Inc. www.cassbeth.com55 References Bob Ferguson, INCOSE Presentation - Automated Natural Language Analysis of Requirements, Carnegie Mellon University, Software Engineering Institute, June 21 2005. Professor Harry G. Frankfurt, On Bullshit, Princeton University, ISBN: 0-691- 12294-6 F. Fabbrini, M. Fusani, S. Gnesi, G. Lami, An Automatic Quality Evaluation for Natural Language Requirements, Istuto diElaborazione dellInformazione del C.N.R. - Pisa Italy Pradip Kar, Michelle Bailey, Characteristics of Good Requirements, 1996 INCOSE Symposium William M. Wilson, Linda H. Rosenberg, Lawrence E. Hyatt, Automated Quality Analysis of Language Requirement Specifications, SATC NASA Joseph E. Kasser, Xuan-Linh Tran, Simon P. Matisons, Prototype Education Tools for Systems and Software (PETS) Engineering, 14th Annual AAEE Conference, September 2003.

56. CassBeth Inc. www.cassbeth.com56 Back Up Slides Specification Analysis Tool (SAT) Overview

57. CassBeth Inc. www.cassbeth.com57 Where Do Reqs Come From Clean Slate System Analysis Previous Systems Similar Systems Architecture Algorithms Automation HMI Timing & Sizing TEMPEST Rel Maint (RMA) Train Supt (ILS) Specs = Reqs A-Level Segment Subsystem Hardware Software Interface Mechanical Test Unit Integration Subsystem System Performance Functional Ops Suitability Interoperability Key Site

58. CassBeth Inc. www.cassbeth.com58 Where Do Reqs Come From Incremental System Analysis Reuse Specs New Analysis Specs = Reqs Levels As Needed Test Tests As Needed

59. CassBeth Inc. www.cassbeth.com59 A NASA View Imperatives are words and phrases that command something must be provided Imperatives are in descending order of strength as a forceful statement of a requirement –Shall, Must, Must Not, Is Required To, Are Applicable, Responsible For, Will, Should The NASA requirements documents judged most explicit have majority of imperative counts associated with the upper list items Automated Reqs Measurement (ARM) tool

60. CassBeth Inc. www.cassbeth.com60 A NASA View Shall –dictates provision of a functional capability Must or must not –establishes performance reqs or constraints Is required to –spec statements written in passive voice Are applicable –includes by reference, standards or other documentation as additions to specified reqs Automated Reqs Measurement (ARM) tool

61. CassBeth Inc. www.cassbeth.com61 A NASA View Responsible for –systems with predefined architectures –e.g. "The XYS function of the ABC subsystem is responsible for responding to PDQ inputs." Will –cite operational or development environment things provided to capability being specified –e.g. "The building's electrical system will power the XYZ system Automated Reqs Measurement (ARM) tool

62. CassBeth Inc. www.cassbeth.com62 A NASA View –In a few instances "shall" and "will" used interchangeably containing both reqs and descriptions of operational environment –system boundaries not always sharply defined Should –not frequently used as imperative –when used, statement is always very weak –e.g. "Within reason, data files should have same time span to facilitate ease of use & data comparison Automated Reqs Measurement (ARM) tool

63. CassBeth Inc. www.cassbeth.com63 An FAA View "Shall" is one of those officious and obsolete words that has encumbered regulations and other documents for many years. The message that "shall" sends to the reader is, "this is boring deadly material. "Shall" is imprecise. It can indicate either an obligation or a prediction. Dropping "shall" is a major step in making your regulation more reader-friendly. Prepared by the Plain English Network (PEN) (www.plainlanguage.gov) Adapted for FAA, February 2000 (http://www.faa.gov/language/)

64. CassBeth Inc. www.cassbeth.com64 An FAA View Don't be intimidated by the argument that using "must" will lead to a lawsuit. Many agencies already use the word "must" to convey obligations with no adverse legal effects. You can avoid "shall" by substituting "must" to indicate an obligation or "will" to indicate that an action will occur in the future. Be careful to consider which meaning you intend to communicate to your readers. Prepared by the Plain English Network (PEN) (www.plainlanguage.gov) Adapted for FAA, February 2000 (http://www.faa.gov/language/)

65. CassBeth Inc. www.cassbeth.com65 Walts View Keep it Simple –Remove all special cases that need to be remembered –No time for fine distinctions of English syntax Use shall for ALL reqs –Use SRDB attributes like future Dont try to spec other systems –Spec your interface to other systems Minimize descriptive text to minimize noise –Use analysis and design documents