© 2014 Systems and Proposal Engineering Company. All Rights Reserved Using Natural Language Parsing (NLP) for Automated Requirements Quality Analysis Chris Ritter, Daniel Hettema, Steven H. Dam, Ph.D., ESEP, SPEC Innovations April 2014

© 2014 Systems and Proposal Engineering Company. All Rights Reserved 1 Requirement Levels The Requirements Hierarchy Number of Requirements, Level of Detail Increases User Needs Conceptual Requirements System Requirements Application/Component Specifications A capability or feature identified by a User as being needed to perform his mission A high-level requirement generated during the concept development phase. Contained in ORD, CONOPS A requirement that describes in technical language the desired capabilities of a system. Requirement that is at the level of detail needed for actually designing a new capability. Contained in System Requirements Specification (SRD)

© 2014 Systems and Proposal Engineering Company. All Rights Reserved 2 Characteristics of Good Requirements Each individual requirement should be: –Correct: Describes the users true intent and is legally possible –Complete: Express a whole idea –Clear: Unambiguous and not confusing –Consistent: Not in conflict with other requirements –Verifiable: Provable (within realistic cost and schedule) that the system meets the requirement –Traceable: Uniquely identified, and able to be tracked to predecessor and successor lifecycle items/objects –Feasible: Able to be implemented with existing technology, and within cost and schedule –Modular: Can be changed without excessive impact on other requirements –Design: Does not impose a specific solution on design; says what, Independent not how

© 2014 Systems and Proposal Engineering Company. All Rights Reserved 3 Avoid these Pitfalls DONT use ambiguous language DONT use bullet lists; use numbered lists instead DONT use jargon DONT use language that provides an escape clause –Ex: The user shall be able to access the Internet as often as is practicable DONT write long, rambling sentences DONT put two requirements in one sentence; e.g., The system shall … and … DONT use vague terms -- Ex: user-friendly DONT include suggestions or possibilities – Ex: may, should, ought DONT include wishful thinking – Ex: The system shall be 100% reliable

© 2014 Systems and Proposal Engineering Company. All Rights Reserved How Do We Ensure Quality Requirements? Follow the rules for what makes a good requirement Enforce the rule requires significant training and discipline Most tools help manage the requirements, but do not help you track the quality of the requirements 4

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Solution 1: Identify Requirements on Import Innoslates one button Import Analyzer identifies requirements vs. statements (context for a requirements) by using a key word search (will, shall, should, or requirement) Requirements entities have quality attributes associated with them The user can transform a statement to a requirement and vice versa 5

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Requirement Entities Each quality factor has a Yes/No value to determine if it meets the requirement Uncertain what attributes means? –Hover over name and definitions appear 6

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Solution 2: Innoslates Quality Check 7 Run quality checker to automatically analyze the quality attributes of the requirements

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Quality Check Uses NLP Technology Natural Language Processing: –a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages Innoslate uses this technology to break down sentences into nouns, verbs and adjectives to identify when conjunctions are used (clear), specific types of hardware/software specified (design), and other parameters that affect the requirements quality 8 Wikipedia

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Solution 3: Roll-up Quality Score Change values from No to Yes to adjust score upward 9

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Solution 4: Add Comments Authors and reviewers can add comments and describe changes in more detail 10

© 2014 Systems and Proposal Engineering Company. All Rights Reserved Summary We need to do more than just manage requirements Innoslate brings new technology (NLP) into the automation of requirements analysis However, it does not substitute for good engineering judgment – it is meant as a way to cue the analyst to potential problems 11