Nonbehavioral Specifications 0810

Non-behavioral Characteristics Portability Portability Reliability Reliability Efficiency Efficiency Human Engineering Human Engineering Testability Testability Understandability Understandability Modifiability Modifiability

Portability Degree to which software running on one host computer environment can be converted to run on another. Degree to which software running on one host computer environment can be converted to run on another. Not necessary for all applications (embedded systems, single-use systems). Not necessary for all applications (embedded systems, single-use systems). Some applications, it is essential. Some applications, it is essential.

Problems with specification May be impossible to quantify: “The maximum time to port to host system X shall be …” May be impossible to quantify: “The maximum time to port to host system X shall be …”  We don’t know what the next generation will be;  Does the maximum time affect the design of the system?

Approaches to Specifying Portability Source language Source language  Java: JVM ported to lots of platforms  Ada: DoD certifies – no extentions, no subsets  Ideally, language is a design issue, but if its effect on portability is critical, it’s a requirement  Language selection tends to be political in organizations Host operating systems Host operating systems  Say which ones up front, if you know Compiler selection Compiler selection  Ansi Standard compilers

Reliability This is difficult in software: This is difficult in software: “The system shall be % reliable.” “The system shall be % reliable.”  What does this mean?  It could mean that the phone system may lose a call now and again, but the entire system must not fail for more than 5 minutes a year.  In a patient monitoring system, it may mean that if it does go down, it alerts staff. It must not make a mistake in monitoring more than one patient in 100,000.

Traditional reliability (hardware) MTTF MTTF  MTTF of system is well defined in terms of MTTF of components.  Redundant components improve reliability because failure of components is independent. Hardware degrades in its environment. Hardware degrades in its environment. Bathtub curve for electronics Bathtub curve for electronics time failures Burn-in

MTTF and Software Software doesn’t degrade over time. Software doesn’t degrade over time.  Suppose you run a program for 10 years without failure, then it suddenly fails. Why?  Software was changed.  Software was used a new way.

Bugs Failure: Software does not do what is required (specified). Behavior is different from that needed. Failure: Software does not do what is required (specified). Behavior is different from that needed. Fault: A cause of a failure. Fault: A cause of a failure.  Not all faults result in failure.  All failures result from faults.  A state in which there is a fault without a failure is a hazard state. Error: A design or implementation flaw. Error: A design or implementation flaw.

Testing The purpose of testing is not to demonstrate correct execution of the program. The purpose of testing is not to demonstrate correct execution of the program. The purpose of testing is to discover faults. The purpose of testing is to discover faults.

Problems specifying reliability in terms of bugs. Assume quality is designed in from the start: Assume quality is designed in from the start:  The software testing shall require no more than two months.  Software testing shall discover no more than 10 bugs. The software shall have no more than one bug per thousand lines of code. The software shall have no more than one bug per thousand lines of code.  We want zero bugs, so this must be better than 5 per  How do we know how many there are? Wait until software is retired, then count them.

Fault Seeding Before testing, insert some bugs Before testing, insert some bugs Track how many of these are found in testing. Track how many of these are found in testing. Total bugs in system = Total bugs in system = (#seeded * total_detected)/seeded_detected

Example Secretly seed 10 bugs. Secretly seed 10 bugs. Test team discovers 120 bugs, 6 of which are seeds. Test team discovers 120 bugs, 6 of which are seeds. Bugs = 10 * 120 / 6 = 200 bugs Bugs = 10 * 120 / 6 = 200 bugs # bugs remaining = 200 – 120 = 80, 4 of which are “known”. # bugs remaining = 200 – 120 = 80, 4 of which are “known”.

Notes Not all bugs are equal Not all bugs are equal  Equally difficult to find  Equally difficult to repair Seeding is harder than it looks. Seeding is harder than it looks. Intuitive Intuitive Measure of testing effectiveness Measure of testing effectiveness

Reliability Levels of criticality Levels of criticality  Cause mild inconvenience  Cause minor financial loss  Cause major embarrassment  Cause major financial loss  Injure people  Kill a few people  Kill many people  Destroy humankind

Example Reliability Requirement No more than five bugs per 10K lines of executable code may be detected during integration and system testing. No more than ten bugs per 10K lines of executable code may remain in the system after delivery, as calculated by the Monte Carlo seeding technique defined in Appendix III. The system must be 100% operational 99.9% of the calendar time during its first year of operation.

Efficiency The utilization of scarce resources. The utilization of scarce resources.  Memory  CPU  Disk  Communication Easy to specify if limits are given Easy to specify if limits are given  Example: ATC system: The system shall trace the movements of up to fifty aircraft.

Need to say how it will degrade: What if there are 51 aircraft? Possibilities: What if there are 51 aircraft? Possibilities:  Software fails.  Track first 50, ignore 51 st.  Software notifies 51 st pilot to leave area.

Human Engineering: Levels of Specification The system shall have an easy-to-use human interface. The system shall have an easy-to-use human interface. The system shall be menu driven. The system shall be menu driven. The system shall be menu-driven. Appendix A shows sample menus. The system shall be menu-driven. Appendix A shows sample menus. The system shall be menu-driven. Appendix A shows all menus to be used. The system shall be menu-driven. Appendix A shows all menus to be used.

Human Engineering: Error Messages Unless there is a sound understanding of the types of error messages the system can generate, there is insufficient knowledge of the system’s expected normal behavior. Unless there is a sound understanding of the types of error messages the system can generate, there is insufficient knowledge of the system’s expected normal behavior. It is a good idea to have an appendix that specifies the text of all error messages. It is a good idea to have an appendix that specifies the text of all error messages.

Testability, Modifiability, Understandability Very difficult to quantify. Very difficult to quantify. These are important contributors to cost (maintenance). These are important contributors to cost (maintenance). One suggestion is to specify conformity to a set of programming standards One suggestion is to specify conformity to a set of programming standards

Programming standards specify Naming conventions Naming conventions Invocation conventions Invocation conventions  Calls, interrupts, synchronization  Message formats Component headers Component headers  Format and content In-line documentation style In-line documentation style Use of global constructs and variables Use of global constructs and variables Use of named constructs Use of named constructs Modularity standards Modularity standards

Where to put these? Be sure you really have these requirements Be sure you really have these requirements Sections 5 and 6 in SRS Sections 5 and 6 in SRS

5.NON-BEHAVIORAL REQUIREMENTS 5.1.PERFORMANCE REQUIREMENTS 5.2.SECURITY 5.3.QUALITATIVE REQUIREMENTS Availability Maintainability Portability Design and Implementation Constraints

6. Other Requirements 6.1.DATABASE 6.2.OPERATIONS 6.3.SITE ADAPTATION