1 Definition Introduction and key stages Steps and standards used Plan Team Characteristics Implementation documentation Reviews & Audit Software Quality Assurance

2 Definition Software quality assurance is a planned and systematic pattern of actions that are required to ensure high quality in software. Software Quality Assurance is an umbrella activity that is applied at each step in the process of building the software. SQA is a collection of activities during s/w development that focus on increasing the quality of the s/w being produced.

3 SQA Includes Analysis,design,coding and testing methods and tools Formal technical reviews during s/w development Testing methodologies s/w documentation s/w measurement and reporting mechanisms

4 Three key stages in SQA i)Design review and document inspection ii)Intermediate quality audit iii)Product and system inspection

5 Steps - SQA

6 Standards used Planning Analysis Review Configuration management s/w testing specifications

7 Quality Assurance Plan The plan for quality assurance activities should be in writing Some elements that should be considered by the plan are: defect tracking, unit testing, source-code tracking, technical reviews, integration testing and system testing.

8 Quality Assurance Plan Defect tracing – keeps track of each defect found, its source, when it was detected, when it was resolved, how it was resolved, etc Unit testing – each individual module is tested Source code tracing – step through source code line by line Technical reviews – completed work is reviewed by peers Integration testing -- exercise new code in combination with code that already has been integrated System testing – execution of the software for the purpose of finding defects.

9 SQA Plan – 1 Management section –describes the place of SQA in the structure of the organization Documentation section –describes each work product produced as part of the software process Standards, practices, and conventions section –lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work

10 SQA Plan - 2 Reviews and audits section –provides an overview of the approach used in the reviews and audits to be conducted during the project Test section –references the test plan and procedure document and defines test record keeping requirements

11 SQA Plan - 3 Problem reporting and corrective action section –defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities Other –tools, SQA methods, change control, record keeping, training, and risk management

12 SQA – Team Members i)Design and analysis team ii)System team iii)Application support team iv)Testing team

13 RoleResponsibilities Quality CoordinatorResponsible for ensuring all quality activities are planned and carried out accordingly Responsible for ensure all team member are properly trained and equipped for their given roles and responsibilities Ensures SQA activities align with available resources Module Leadercoordinates activities related to a specific system module Software Quality LeaderResponsible for leading SQA activities(i.e. coordinating reviews and walkthroughs) Quality ReviewerReviews and identifies defects in project artifacts Provides feedback for improved quality in software artifacts SQA Team MemberResponsible for providing support during SQA activities by carrying out assigned tasks as they relate to quality of the system

14 Team(Traditional Organization View)

15 Functional Managers, Empowered Teams

16 Self-Organizing Product Teams

17 Characteristics Functionality Performance Constraints Technical innovations Technological and management risk

18 Implementation The SQA team’s responsibility in this phase is to ensure the implementation of the proposed functions of the intended application. Along with the implementation, the process of developing the application will also be monitored by the application. There are a couple of design methods that could be used such as the language or framework and the SQA team should aid the development team in selecting the proper language and/or framework.

19 Minimum documents required to ensure that the implementation of software satisfy requirements, may be listed as follows. i) Software Requirement Specification (SRS) ii) Software Design Description iii) Software Verification and validation plan iv) Software verification and validation report v) User Documentation vi) Software Configuration management plan vii) Other Documentations

20 Documentation The SQA team’s responsibility in this phase To ensure the implementation of the proposed functions of the intended application. Along with the implementation, the process of developing the application will also be monitored by the application.

21

22

23 Reviews and Audits

24 Reasons for Audit conducted A specific project milestone has been reached and an audit is initiated as planned or as required by the auditing organization's charter. External parties or customers request an audit of a specific item, at a specific date, or at a project milestone. This could be part of a contract agreement. An internal organization has requested the audit, establishing a clear and specific need.

25 Software Reviews Purpose is to find defects (errors) before they are passed on to another software engineering activity or released to the customer. Software engineers (and others) conduct formal technical reviews (FTR) for software engineers. Using formal technical reviews (walkthroughs or inspections) is an effective means for improving software quality.

26 Following reviews shall be conducted as a minimum. Software requirement review: This review will guarantee the technical feasibility, adequacy, and completeness of the requirements. Preliminary design review: To evaluate the technical adequacy of the preliminary design of the s/w as depicted in the s/w architectural design description. Critical design review: To determine the acceptability of the detailed s/w design as depicted in the software design description

27. Software verification and validation plan review: Tries to access the correctness of the transition to the next phase. Also meet the users requirements. Functional audits: To be performed prior to the software delivery to end- users. Physical audits: To verify that the software and its documentations are internally consistent and are ready for delivery.

Reviews Reviews are the principle means of validating both process and product quality Basic procedure is to have a group of people examine a process artifact to find potential problems Common software review types include –defect inspection and removal (product) –progress reviews (product and process) –quality reviews (product and standards)

29 Review Roles Presenter (designer/producer). Coordinator (not person who hires/fires). Recorder –records events of meeting –builds paper trail Reviewers –maintenance oracle –standards bearer –user representative –others

Quality Reviews Group of knowledgeable people examines a software component and its documentation Code, design models, specifications, test plans, standards, etc. can be subjected to review Once an artifact has been reviewed and ‘signed off’ by the reviewers, management has given its approval to proceed to the next stage of development

Quality Review Process Select a review team Arrange a time and place for the review Distribute documents to review Conduct the review Complete the review forms Decide whether to approve artifacts or have them reworked and review them again

Review Purposes Quality function –part of the general quality management process Project management function –provide information to project managers Training and communication function –product knowledge is shared among development team members

Quality Review Results Purpose is the discovery of system defects and inconsistencies Review comments need to be classified as –no action (no changes to artifact are required) –refer for repair (author needs to correct faults) –reconsider overall design (problem identified impacts other design components) Requirement and specification problems may require involvement of client to resolve

34 Formal Technical Reviews - 1 Involves 3 to 5 people (including reviewers) Advance preparation (no more than 2 hours per person) required Duration of review meeting should be less than 2 hours Focus of review is on a discrete work product Review leader organizes the review meeting at the producer's request.

35 Formal Technical Reviews - 2 Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer) Producer of the work product walks the reviewers through the product Recorder writes down any significant issues raised during the review Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not.

36 Why do peer reviews? To improve quality. Catches 80% of all errors if done properly. Catches both coding errors and design errors. Enforce the spirit of any organization standards. Training and insurance.

37 Audit

38 The generic steps involved in an audit are: Initiation Scope Frequency Preparation Review of documentation The programme Working documents Execution Opening meeting Examination and evaluation Collecting evidence Observations Close the meeting with the auditee

39 Report Preparation Content Distribution Completion Report Submission Retention

40 Software Quality Audit Roles and Processes A software quality audit involves: The client, person or organization which requests the audit, The auditor or team who performs the audit, and The auditiee whose work is being examined.

41 Client The client (i.e., person or organization) is responsible for authorizing the audit and for defining the scope and identifying the requirements of the audit

42 Auditor or Audit Team determining the team size, briefing team members on the audit scope and areas to be audited, providing background about the organization being audited, assigning the workload of who will audit what areas, determining the audit schedule, notifying and briefing the audited organization on the scope of the audit and materials that need to be provided, ensuring that the audit team is prepared to conduct the audit, ensuring that the audit plan or procedures are performed, and issuing reports in accordance with the audit plan or procedures.

43 The auditee is responsible for: participating in the audit, providing all relevant materials and resources to the audit team, understanding the concerns of the auditors and verifying their factual accuracy, providing a response to the audit report, and correcting or resolving deficiencies cited by the audit team.

44 SQA Group Activities - 1 Prepare SQA plan for the project. Participate in the development of the project's software process description. Review software engineering activities to verify compliance with the defined software process.

45 SQA Group Activities - 2 Audit designated software work products to verify compliance with those defined as part of the software process. Ensure that any deviations in software or work products are documented and handled according to a documented procedure. Record any evidence of noncompliance and reports them to management.

46 Formal SQA Approaches 1.Proof of correctness. 2.Statistical quality assurance. 3.Cleanroom process combines items 1 & 2.

47 Reliability Metrics - part 1 Probability of Failure on Demand (POFOD) –POFOD = –For one in every 1000 requests the service fails per time unit Rate of Fault Occurrence (ROCOF) –ROCOF = 0.02 –Two failures for each 100 operational time units of operation

48 Reliability Metrics - part 2 Mean Time to Failure (MTTF) –average time between observed failures – ( MTBF) Availability = MTBF / (MTBF+MTTR) –MTBF = Mean Time Between Failure –MTTR = Mean Time to Repair Reliability = MTBF / (1+MTBF)

49 Time Units Raw Execution Time –non-stop system Calendar Time –If the system has regular usage patterns Number of Transactions –demand type transaction systems

50 Validation Perspectives Reliability validation –Does measured system reliability meet its specification? –Is system reliability good enough to satisfy users? Safety validation –Does system operate so that accidents do not occur? –Are accident consequences minimized? Security validation –Is system secure against external attack?