CassBeth Inc. Specification Analysis Tool (SAT) Overview October 2005

CassBeth Inc. Importance of Good Reqs Make Sure You Design and Implement What You Thought You Needed –Reqs Driven Development Show That You Actually Implemented What You Needed –This May Include Certifications ROI Justifications Go Back 25+ Years

CassBeth Inc. Problem Statement How Can you Create a Clean Useful Specification? Buzz Words are: Clear, Complete, Consistent, Testable, etc

CassBeth Inc. Different Views on Specs A NASA View –Automated Reqs Measurement (ARM) tool An FAA View –Prepared by the Plain English Network (PEN) ( –Adapted for FAA, February 2000 ( –Edited for AUA Plain Language Month, April 2003 Walt's View

CassBeth Inc. Observations No one approach is better or worse –Optimized for each organization An approach is appropriate if it works –Only organization can make determination Appears to be need for User defined analysis rules –Can allow each approach to be optimized

CassBeth Inc. Levels of Consciousness surface key reqs that terrify all No Politics - Truth did you get all leaves Completeness eliminate similar, duplicates, conflicts Consistency source, sink, nouns, action verbs Clear Complete Testable Imperatives shall, must, will, should... Consistent Imperative Use

CassBeth Inc. Alternatives Do Nothing Use Manual Techniques Create Your Own Tool Use Other Specification Analysis Tools –QuARS, ARM (NASA), TIGER Use SRDB (DOORS) –Filtering and View Mechanisms Use SAT with SRDB Parent Child Reports

CassBeth Inc. What Does SAT Do? Performs automated specification analysis –Very fast but Humans make final decisions Looks for bad requirement text –Uses rules spread throughout your organization Looks for complex spec problems –Missing capabilities & key requirements Makes recommendation for each problem –Shows newly minted staff spec creation Gathers Metrics

CassBeth Inc. Where Does SAT Fit Analysis Identify Key Reqs Prelim Spec Doc Expand Key Reqs DOORS Import Clean Spec Doc Reqs Clean Up DOORS Parent Child Reports Reqs Clean Up Final Spec Doc

CassBeth Inc. SAT Services Predefined Services –Requirement Text Assessment (RTA) –Generic Structure Analysis (GSA) –Domain Structure Analysis (DSA) –Generic Capabilities Analysis (GCM) –Domain Capabilities Analysis (DCA) –Key Requirements Analysis (KRA) User Defined Services Metrics

CassBeth Inc. SAT Operations SAT LeadStake Holders Updated Specs Reports Official Reports 1. 2.

CassBeth Inc. SAT Portal - Demo

CassBeth Inc. SAT Portal - Demo

CassBeth Inc. How Does SAT Work User Uploads File –From Local Computer or Network SAT Examines Extension –If.doc SAT Attempts to Parse Into Objects Looks for PUI –If None Matches PUI MASK, creates PUI Converts Each Line Into Object –DOORS Excel Export –MsWord Saved as.txt

CassBeth Inc. How Does SAT Work Subjects Each Object To Each SAT Rule –Based on Display Filter Setting –Tabulates Metrics For Each Rule Subjects Each Object to Duplicate Check –If Enabled Outputs On-The-Fly Object Results Outputs Metrics at End of Last Object User Modifies Display Filters and Rules

CassBeth Inc. SAT Control Panel

CassBeth Inc. SAT Control Panel

CassBeth Inc. RTA Rule Templates SAT Rules –Untestable, Unsure, Vague 1, Vague 2, Vague 3, Vague 4, Compound Req, Internal Reference, Not Standalone, Fragment, Declarative, Duplicates CMSEI Rules –vague, Subjective, Choice or Option, Implicit Expressions, Weakness, Under Specification, Multiplicity, Duplicates INCOSE Rules –Standard Constructs, Words to Avoid - Vague, Completeness, Duplicates NASA Rules –Completeness, Continuances, Directives, Options, Weak Phrases CMSEI, INCOSE, NASA as interpreted by SAT staff

CassBeth Inc. SAT Report

CassBeth Inc. SAT Report

CassBeth Inc. CMSEI Report

CassBeth Inc. CMSEI Report

CassBeth Inc. CMSEI Report

CassBeth Inc. INCOSE Report

CassBeth Inc. INCOSE Report

CassBeth Inc. INCOSE Report

CassBeth Inc. NASA Report

CassBeth Inc. NASA Report

CassBeth Inc. NASA Report

CassBeth Inc. SAT Report All Services

CassBeth Inc. SAT Report All Services

CassBeth Inc. Find Duplicates

CassBeth Inc. Find Duplicates

CassBeth Inc. Generic Structure Analysis

CassBeth Inc. Domain Structure Analysis

CassBeth Inc. Generic Capabilities Analysis

CassBeth Inc. Generic Capabilities Analysis

CassBeth Inc. Domain Capabilities Analysis

CassBeth Inc. Key Reqs Analysis

CassBeth Inc. Key Reqs Analysis

CassBeth Inc. Key Reqs Analysis

CassBeth Inc. Other Domains

CassBeth Inc. Other Domains

CassBeth Inc. Other Domains

CassBeth Inc. Other Domains

CassBeth Inc. Other Domains

CassBeth Inc. SAT Big Picture 3. TEMPLATES 1. RULES 2. SERVICES 5. DOMAINS COMMUNICATIONS DEFENCE ATC 6. PRECISE LANGUAGE INDUSTRIES ENGINEERING GOV REGULATION LEGAL 4. APPLICATION.

CassBeth Inc. SAT Features Its Simple and Fast –Automates Tedious Tasks With Greatest Benefit Dynamic Display Filters –Change with User Defined Rules Dynamic Metrics –Change with User Defined Rules Report Format Contains Everything –Offending Object Text –Analysis Findings and Recommendations –Metrics

CassBeth Inc. SAT Features Ubiquitous –Uses Internet Technologies Highly Scalable –Uses Internet Technologies User Defined Rules –Allows Build Up of Templates Its New –You Can Influence Its Engine, Services, Rules, etc

CassBeth Inc. Why is SAT Different User defined rules with template examples –Allows users to grow their own Structured in terms of services that span simple to highly abstract consciousness –User decides what level makes sense for them Realizes that simple services based on keyword searches remove significant distractions / noise from problem set –So hard stuff can begin to be addressed

CassBeth Inc. What I Would Like To See Happen Community Buys SAT Community Creates and Publishes Rules Community Suggests Engine Attributes Community Suggests Engine Features Other Domains Adopt SAT –Legal (e.g. contracts, laws, constitutions)

CassBeth Inc. Where Can I Get A Web Server Window Professional Versions –2000 and XP professional Apache – IndigoPerl – Your Computer Network –Ask Your Computer Support Staff SAT May Come Bundled with Web Server –Depends on Early Marketing Findings

CassBeth Inc. Company Overview CassBeth established in August 1997 Early Goals Were to Study Internet Technologies CassBeth Incorporated in 2000 Large Traffic at E-Commerce Sites SAT is a CassBeth Product Direct Result of Cassbeth web Mining Solutions

CassBeth Inc. Links / References satc.gsfc.nasa.gov/metrics

CassBeth Inc. References Bob Ferguson, INCOSE Presentation - Automated Natural Language Analysis of Requirements, Carnegie Mellon University, Software Engineering Institute, June Professor Harry G. Frankfurt, On Bullshit, Princeton University, ISBN: F. Fabbrini, M. Fusani, S. Gnesi, G. Lami, An Automatic Quality Evaluation for Natural Language Requirements, Istuto diElaborazione dellInformazione del C.N.R. - Pisa Italy Pradip Kar, Michelle Bailey, Characteristics of Good Requirements, 1996 INCOSE Symposium William M. Wilson, Linda H. Rosenberg, Lawrence E. Hyatt, Automated Quality Analysis of Language Requirement Specifications, SATC NASA Joseph E. Kasser, Xuan-Linh Tran, Simon P. Matisons, Prototype Education Tools for Systems and Software (PETS) Engineering, 14th Annual AAEE Conference, September 2003.

CassBeth Inc. Back Up Slides Specification Analysis Tool (SAT) Overview

CassBeth Inc. Where Do Reqs Come From Clean Slate System Analysis Previous Systems Similar Systems Architecture Algorithms Automation HMI Timing & Sizing TEMPEST Rel Maint (RMA) Train Supt (ILS) Specs = Reqs A-Level Segment Subsystem Hardware Software Interface Mechanical Test Unit Integration Subsystem System Performance Functional Ops Suitability Interoperability Key Site

CassBeth Inc. Where Do Reqs Come From Incremental System Analysis Reuse Specs New Analysis Specs = Reqs Levels As Needed Test Tests As Needed

CassBeth Inc. A NASA View Imperatives are words and phrases that command something must be provided Imperatives are in descending order of strength as a forceful statement of a requirement –Shall, Must, Must Not, Is Required To, Are Applicable, Responsible For, Will, Should The NASA requirements documents judged most explicit have majority of imperative counts associated with the upper list items Automated Reqs Measurement (ARM) tool

CassBeth Inc. A NASA View Shall –dictates provision of a functional capability Must or must not –establishes performance reqs or constraints Is required to –spec statements written in passive voice Are applicable –includes by reference, standards or other documentation as additions to specified reqs Automated Reqs Measurement (ARM) tool

CassBeth Inc. A NASA View Responsible for –systems with predefined architectures –e.g. "The XYS function of the ABC subsystem is responsible for responding to PDQ inputs." Will –cite operational or development environment things provided to capability being specified –e.g. "The building's electrical system will power the XYZ system Automated Reqs Measurement (ARM) tool

CassBeth Inc. A NASA View –In a few instances "shall" and "will" used interchangeably containing both reqs and descriptions of operational environment –system boundaries not always sharply defined Should –not frequently used as imperative –when used, statement is always very weak –e.g. "Within reason, data files should have same time span to facilitate ease of use & data comparison Automated Reqs Measurement (ARM) tool

CassBeth Inc. An FAA View "Shall" is one of those officious and obsolete words that has encumbered regulations and other documents for many years. The message that "shall" sends to the reader is, "this is boring deadly material. "Shall" is imprecise. It can indicate either an obligation or a prediction. Dropping "shall" is a major step in making your regulation more reader-friendly. Prepared by the Plain English Network (PEN) ( Adapted for FAA, February 2000 (

CassBeth Inc. An FAA View Don't be intimidated by the argument that using "must" will lead to a lawsuit. Many agencies already use the word "must" to convey obligations with no adverse legal effects. You can avoid "shall" by substituting "must" to indicate an obligation or "will" to indicate that an action will occur in the future. Be careful to consider which meaning you intend to communicate to your readers. Prepared by the Plain English Network (PEN) ( Adapted for FAA, February 2000 (

CassBeth Inc. Walts View Keep it Simple –Remove all special cases that need to be remembered –No time for fine distinctions of English syntax Use shall for ALL reqs –Use SRDB attributes like future Dont try to spec other systems –Spec your interface to other systems Minimize descriptive text to minimize noise –Use analysis and design documents