1. Assistance - Savita Kini November 15, 1997 2 Software Quality Assurance - Outline ä What is Software Quality assurance(SQA)? ä Quality Concepts. ä Software Quality Assurance Activities. ä Software Reviews and their importance ä Statistical SQA. ä Software Reliability ä ISO 9000 approach to SQA

2. Assistance - Savita Kini November 15, 1997 3 What is SQA? ä Software Quality Assurance is an umbrella activity that is applied throughout the software process...

3. Assistance - Savita Kini November 15, 1997 4 It encompasses.. ä A quality management approach ä Effective software engineering technology ä Formal technical reviews that are applied throughout the software process ä A multitiered testing strategy ä Control of software documentation and changes to it ä A procedure to assure compliance with software development standards ä Measurement and reporting techniques

4. Assistance - Savita Kini November 15, 1997 5 Quality ??? ä Quality refers to any measurable characteristics such as correctness, maintainability, portability, testability, usability, reliability, efficiency, integrity, reusability and interoperability.

5. Assistance - Savita Kini November 15, 1997 6 Quality Concepts ä Quality of Design refers to the characteristics that designer’s specify for an item. ä Quality Control is the series of inspections, reviews and tests used throughout the development cycle to ensure that each work product meets the requirements placed upon it. ä Quality of Conformance is the degree to which the design specifications are followed during manufacturing.

6. Assistance - Savita Kini November 15, 1997 7 (cont'd)... ä Quality policy refers to the basic aims and objectives of an organization regarding quality as stipulated by the management. ä Quality assurance consists of the auditing and reporting functions of management. ä Cost of Quality includes all costs incurred in the pursuit of quality or in performing quality related activities such as appraisal costs, failure costs and external failure costs.

7. Assistance - Savita Kini November 15, 1997 8 (cont'd)... ä Quality planning is the process of assessing the requirements of the procedure and of the product and the context in which these must be observed. ä Quality testing is assessment of the extent to which a test object meets given requirements ä Quality assurance plan is the central aid for planning and checking the quality assurance. ä Quality assurance system is the organizational structure, responsibilities, procedures, processes and resources for implementing quality management.

8. Assistance - Savita Kini November 15, 1997 9 Relative cost of correcting an error

9. Assistance - Savita Kini November 15, 1997 10 Defn. of Software Quality Assurance ä Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software.

10. Assistance - Savita Kini November 15, 1997 11 SQA Group Plan ä Evaluations to be performed ä Audits and reviews to be performed ä Standards that are applicable to the project ä Procedures for error reporting and tracking ä Documents to be produced by the SQA group ä Amount of feedback provided to software project team

11. Assistance - Savita Kini November 15, 1997 12 SQA Group Activities ä Participates in the development of the projects software process description ä Reviews software engineering activities to verify compliance with the defined software process. ä Audits designated software work products to verify compliance with those defined as part of the software process.

12. Assistance - Savita Kini November 15, 1997 13 (cont'd)... ä Ensures that deviations in software work and work products are documented and handled according to a document procedure. ä Records any non-compliance and reports to senior management.

13. Assistance - Savita Kini November 15, 1997 14 Software Reviews ä ‘Filter’ for the software engineering process ä ‘Purify’ the software work products that occur as a result of analysis, design, and coding. ä Achieve technical work of more uniform, greater and more predictable quality. ä Detect errors and problems at the earliest possible time.

14. Assistance - Savita Kini November 15, 1997 15 Formal Technical Reviews ä To uncover errors in function, logic, or implementation for any representation of the software ä To verify that software meets its requirements ä To ensure that software representation meets predefined standards ä To achieve software development in a uniform manner ä To make projects more manageable

15. Assistance - Savita Kini November 15, 1997 Defect Amplification Model

16. Assistance - Savita Kini November 15, 1997 Defect Amplification with Reviews

17. Assistance - Savita Kini November 15, 1997 Cost Comparison of Error Repair

18. Assistance - Savita Kini November 15, 1997 16 Review Guidelines.. ä Review the product, not producer ä Set an agenda and maintain it ä Limit the debate ä Enunciate problem areas, not to solve every problem noted ä Take written notes ä Allocate resources and time schedule for FTR’s ä Limit the number of participants and insist upon advance preparation ä Develop a checklist for each work product to be reviewed ä Training for all reviewer’s ä Reviewing earlier reviews

19. Assistance - Savita Kini November 15, 1997 Additional Structures ä Requirements Control Board ä All requirement changes must be formally reviewed and approved ä Software Control Board ä All design changes must be formally reviewed and approved ä Interface Control Board

20. Assistance - Savita Kini November 15, 1997 17 Statistical Quality Assurance ä Implies information about software defects is collected and categorized ä An attempt is made to trace each defect to its underlying cause ä Isolate the vital few causes of the major source of all errors ä Then move to correct the problems that have caused the defects

21. Assistance - Savita Kini November 15, 1997 Categories of Errors ä Incomplete or erroneous specification (IES) ä Misinterpretation of customer comm (MCC) ä Intentional deviation from specification (IDS) ä Violation of programming standards (VPS) ä Error in data representation (EDR) ä Inconsistent module interface (IMI) ä Error in design logic (EDL)

22. Assistance - Savita Kini November 15, 1997 Categories of Errors (cont'd) ä Incomplete or erroneous testing (IET) ä Inaccurate or incomplete documentation (IID) ä Error in programming lang. Translation (PLT) ä Ambiguous or inconsistent human-computer interface (HCI) ä Miscellaneous (MIS) ä Most often IES, MCC and EDR are the vital few causes for majority of errors.

23. Assistance - Savita Kini November 15, 1997 18 Definitions ä E i = the total number of errors uncovered during the i th step in the software engineering process ä S i = the number of serious errors ä M i = the number of moderate errors ä T i = the number of minor errors ä PS = size of the product (LOC, design statements, pages of documentation)

24. Assistance - Savita Kini November 15, 1997 19 error index ä Phase index for each step and then error index is calculated PI i = w s (S i /E i )+w m (M i /E i )+w t (T i /E i ) ä Formula:

25. Assistance - Savita Kini November 15, 1997 20 Software Reliability ä Defined as the probability of failure free operation of a computer program in a specified environment for a specified time. ä It can measured, directed and estimated ä A measure of software reliability is mean time between failures where ä MTBF = MTTF + MTTR ä MTTF = mean time to failure ä MTTR = mean time to repair

26. Assistance - Savita Kini November 15, 1997 21 Software Availability ä Availability =MTTF/(MTTF + MTTR) * 100% ä Software availability is the probability that a program is operating according to requirements at a given point in time

27. Assistance - Savita Kini November 15, 1997 Software Safety ä Processes that help reduce the probability that critical failures will occur due to SW ä Hazard analyses ä Identify hazards that could call failure ä Develop fault tree ä Identify all possible causes of the hazard ä Formally review the remedy for each ä Redundancy ä Require a written software safety plan ä Require independent verification & validation

28. Assistance - Savita Kini November 15, 1997 Example Fault Tree -- Thermal Loss of heat Power failure Computer failure Incorrectinput SW failed to throw switch... Computer failure SW failed to throw switch... Logic reversed

29. Assistance - Savita Kini November 15, 1997 Software Safety ä Redundancy ä Replicated at the hardware level ä Similar vs.. dis-similar redundancy ä Verification ä Assuring that the software specifications are met ä Validation ä Assuring that the product functions as desired ä Independence

30. Assistance - Savita Kini November 15, 1997 22 Overview of SQA Plan ä Purpose of Plan ä References ä Management ä Documentation ä Standards, Practices and Conventions ä Reviews and Audits ä Test ä Problem Reporting and Corrective action ä Tools, Techniques and Methodologies ä Code Control ä Media Control ä Supplier control ä Records Collection, Maintenance and Retention ä Training ä Risk Management

31. Assistance - Savita Kini November 15, 1997 23 ISO 9000 Quality Standards ä ISO 9000 describes quality assurance elements in generic terms that can be applied to any business. ä It treats an enterprise as a network of interconnected processes. ä To be ISO-complaint processes should adhere to the standards described. ä Elements include organizational structure, procedures, processes and resources. ä Ensures quality planning, quality control, quality assurance and quality improvement.

32. Assistance - Savita Kini November 15, 1997 24 ISO 9001 ä An international standard which provides broad guidance to software developers on how to Implement, maintain and improve a quality software system capable of ensuring high quality software ä Consists of 20 requirements... ä Differs from country to country..

33. Assistance - Savita Kini November 15, 1997 25 ISO 9001 (cont'd)..requirements ä Management responsibility ä Quality system ä Contract review ä Design Control ä Document and data control ä Purchasing ä Control of customer supplied product ä Product identification and traceability ä Process control ä Inspection and testing ä Control of inspection, measuring and test equipment

34. Assistance - Savita Kini November 15, 1997 26 ISO 9001 (cont'd).. ä Inspection and test status ä Control of non- confirming product ä Corrective and preventive action ä Handling, storage, packaging, preservation and delivery ä Control of quality records ä Internal quality audits ä Training ä Servicing ä Statistical techniques

35. Assistance - Savita Kini November 15, 1997 27 Summary- ä SQA must be applied at each step ä SQA might be complex ä Software reviews are important SQA activities ä Statistical SQA helps improve product quality and software process ä Software Safety is essential for critical systems ä ISO 9001 standardizes the SQA activities