VERIFICATION AND VALIDATION TECHNIQUES

The goals of verification and validation activities are to assess and improve the quality of the work products generated during development and modification of software 2 types of verification 1. life-cycle verification is the process fo determining the degree to which the work products of a given phase of the development cycle fulfill the specifications established during prior phases.

2. formal verification is a rigorous mathematical demonstration that source code conforms to its specifications. Validation is the process of evaluating software at the end of the software development process to determine compliance with the requirements Boehm Verification: are we building the product right? Validation: are we building the right product?

Requirements errors are caused by incorrect statement of user needs, by failure to completely specify functional and performance requirements, by inconsistencies among the requirements, and by infeasible requirements. Design errors are introduced by failure to translate the requirements into correct and complete solution structures, by inconsistencies with design specifications. Implementation errors are the errors made in translating design specifications into source code.

Techniques for assessing and improving the quality of source code include systematic quality assurance procedures, Walkthroughs and inspections, Static analysis, symbolic execution, Unit testing and systematic integration testing.

8.1 Quality Assurance Quality assurance is a planned and systematic pattern of all actions necessary to provide adequate confidence that the item or product conforms to established technical requirements. The purpose of the software quality assurance group is to provide assurance that the procedures, tools, and techniques used during product development and modification are adequate to provide the desired level of confidence in the work products.

Software quality assurance personnel are distinct from the software development group. Preparation of a software quality assurance plan for each software project is a primary responsibility of the software quality assurance groups Topics in a software quality assurance plan include 1. purpose and scope of the plan 2. documents reference in the plan 3. organizational structure, tasks to be performed, and specific responsibilities as they relate to product quality.

4. documents to be prepared and checks to be made for adequacy of the documentation. 5. standards, practices, and convention to be used. 6. reviews and audits to be conducted. 7. a configuration management plan that identifies software product items, controls and implements changes, and records and reports changed status. 8. practices and procedures to be followed in reporting, tracking, and resolving software problems.

9.Specific tools and techniques to be used to support quality assurance activities. 10. methods and facilities to be used maintain and store controlled versions of identified software 11. methods and facilities to be used to protect computer program physical media. 12. provisions for ensuring the quality of vendor provided and subcontractor developed software. 13.mehtods and facilities to be used in collecting, maintaining, and retaining quality assurance records.

Other duties performed by quality assurance personnel include: 1. development of standard policies, practices and procedures. 2. development of testing tools and other quality assurance aids 3. performance of the quality assurance functions described in the software quality assurance plan for each project 4. performance and documentation of final product acceptance tests for each software product.

A software quality assurance group perform the following functions 1. during analysis and design, a software verification plan and an acceptance test plan are prepared. The verification plan describes the methods to be used in verifying that the requirements are satisfied by the design documents and That the source code is consistent with the requirements specifications and design documentation. The source code test plan is an important component of the software verification plan.

The acceptance test plan be developed by quality assurance personnel. 2. during product evolution, in-process audits are conducted to verify consistency and completeness of the work products. Items to be audited for consistency include interface specifications for hardware, software and people. 3.

8.2 walkthroughs and inspections Walkthroughs and inspections can be used to systematically examine work products thought the software life cycle. In a walkthrough session, these items are examined principles of operation, user’s manuals, and maintenance procedures

In a walkthrough session, the material being examined is presented by a reviewee and evaluated by a team of reviewers. The reviewee “walk through” the work product, and reviewers raise question on issues of concern. Inspection differ from walkthroughs It consists of a team of trained inspectors, working from checklists of items to be examined.

8.2.1 walkthroughs A walkthrough team usually consists of a reviewee and 3 to 5 reviewers. Members of a walkthrough team include Project leader, other members of the project team, a representative from the quality assurance group, a technical writer, and other technical personnel. Customers and users should be included.

Walkthrough sessions held in open and no defensive atmosphere. Several guidelines for walkthroughs 1. everyone’s work should be reviewed on a scheduled basis Project leader’s technical work should be reviewed. 2. emphasis should be placed on detecting errors. One member of the review team should be designated recording secretary for the session.

3.Major issues should be addressed. One reviewer should be designated as moderator in order to maintain a +ve atmosphere. 4. walkthrough sessions should be limited to 2 hours. Provides incentive for active participation by the reviewers.

Successful use of walkthroughs is strongly dependent on establishing a +ve, nonthreatening atmosphere for the walkthrough sessions. The time spent in walkthroughs is repaid by numerous benefits: Errors are caught at the earliest possible time. When they are easiest and least expensive to fix.

Project team communication is improved Project personnel learn new techniques from one another and New personnel quickly learn project details Motivation is improved. Work products come to be viewed as public documents Team members derive greater job satisfaction from their work.

Inspections Inspections, like walkthroughs, use throughout the software life cycle to assess and improve the quality of the various work products. Design and code inspections were first described by Fagan. In Fagan's experiment, 3 separate inspections were performed. One following design, but prior to implementation

One following implementation, but prior to unit testing and One following unit testing. According to Fagan, an inspection team consists of 4 persons, who play the roles of moderator, designer, implementer, and tester. Items to be inspected in a design inspection include Completeness of the design with respect to the user, and Functional requirements, Internal completeness and consistency in definition and Usage of terminology and Correctness of the interfaces between modules.

Items to be inspected in a code inspection include Subprogram interfaces, Decision logic, Data referencing, Computational expressions, I/O statements, comments, data flow and memory usage.

Actual items examined under the category of subprogram interfaces include 1. are the numbers of actual parameters and formal parameters in agreement? 2. do the type attributes of actual and formal parameters match? 3. do the dimensional units of actual and formal parameters match? 4. are the numbers, attributes, and ordering of arguments to built in functions correct? 5. are constants passed as modifiable arguments? 6. are global variable definitions and usage consistent among modules?

In fagan’s initial experiment, the inspection team conducted two 2 hour sessions per day and expended 25 hours per person. Of the errors found during development, 67 % were found before any unit testing was performed. During the first 7 months of operation, 38 % fewer errors were found. In a subsequent experiment, fagan found that 82 % of all errors discovered during development of a software product Were found during design and code inspections.

The savings in programmer resources on the project was 25% of estimated cost because most errors were found before unit testing. Subsequent experiments have reported 70% and greater error removal during design and code inspections. Inspections and walkthroughs are valuable for real time systems. number of advantages for inspection teams Inspectors have roles to play: the moderator Directs the effort an error checklist is utilized. Detailed error feedback is provided to individual programmers.

8.3 static analysis Static analysis is a technique for assessing the structural characteristics of source code, design specifications or any notational representation that conforms to well defined syntactic rules. In static analysis, the structure of the code is examined, but the code in not executed. It is useful for discovering questionable coding practices and departures from coding standards, in addition to detecting structural errors such as uninitialized variables and mismatches between actual and formal parameters.

Static analysis is often used to denote examination of program structure by an automated tool. A static analyzer construct a symbol table and a graph of control flow for each subprogram, as well as a call graph for the entire program. The symbol table contains information about variable. Its type attributes, the statement where declared, statements where set to a new value, and statements where used to provide values.

Fig 8.1 illustrates a code segment and the corresponding control flow graph. The nodes correspond to basic blocks of source code The arcs represent possible transfers of control between blocks. A basic block of source code has the property that if the first statement in the block is executed, every statement in the block will be executed.

In a call graph, the nodes represent program units The arcs represent potential invocation of one program unit by another. Given the control flow graph and a symbol table contains, for each variable in a subprogram, the statement numbers where the variables are declared, set, and used, A static analyzer determine data flow information such as uninitialized variables, Variables that are declared but never used, and Variables that are set but not subsequently used.

See fig 8.1 Variable x is uninitialized on path ABC and is set but not used on path ABD Static analyzers produce lists of errors, questionable coding practices, and Departures from coding standards. For instance, a variable that is uninitialized on all paths is a structural error. A variable that is set but not used on any subsequent control path, or a variable that is declared but never used, is not an error. It may be symptomatic of an error.

Table 8.1 contains a list of typical items that are provided by static analyzers. These are both practical and theoretical limitations to static analysis. A major practical limitation involves dynamic evaluation of memory references at run time. Dynamic test cases used to obtain information that is difficult or impossible to obtain by static analysis techniques. Major theoretical limitations are imposed on static analysis by decidability theory.

By algorithmic manner we mean that it is impossible to write a computer program to perform this task for all possible programs. The term “arbitrary” is used to mean that there are programs and data for which