CSSE Software Engineering Process and Practice Lecture 5 Q UALITY A SSURANCE

Software Engineering Lecture 7: Quality Assurance 2 Semester 1, 2006 T OPICS C OVERED In this lecture we will be exploring the following two key ideas in quality: Quality and Quality Models - how do we deconstruct the idea of quality so that we can measure, monitor and control ‘quality’: This is the ‘Product View of Quality’ (Pfleeger pp 8.); Assurance - how confident are we that the final program or system meets our requirements and how do we guarantee that the final program ‘will’ meet our requirements;

Software Engineering Lecture 7: Quality Assurance 3 Semester 1, 2006 Q UALITY FOR E NGINEERS Ideally, for engineering we would like to be able to monitor, measure, evaluate and control quality! … But what is quality? Quality is considered simply too vague a concept to ‘engineer’. For engineering purposes we need to measure, monitor and control quality and so more quantifiable notions of quality are required.

Software Engineering Lecture 7: Quality Assurance 4 Semester 1, 2006 Q UALITY M ODELS Typically, a quality model is used to give a more precise and (where possible) specify the measurable properties of software quality. Quality models are based on the following idea: “Even if quality cannot be measured directly we can find a set of measurable properties of a system that imply quality”.

Software Engineering Lecture 7: Quality Assurance 5 Semester 1, 2006 E XERCISE Write down 5 aspects or properties that you believe show ‘High Quality Teaching’.

Software Engineering Lecture 7: Quality Assurance 6 Semester 1, 2006 K EY Q UESTIONS B EHIND Q UALITY Start to think about what the characteristics of a quality system might be. For example, should it: 1. Satisfy all explicit requirements, that is, should it be correct, complete and consistent with respect to explicit requirements? 2. Adhere to internal (organisational or project) and external standards imposed on the project, for example IEC61508 (safety), Rainbow standards (US Security), IEEE standards, internal company standards and if so, then to what extent? 3. Conform to our implicit notions of quality such as requirements for performance, reliability, usability, extendibility, safety and security? Note: Some factors may require more attention than others because of the problem domain, customer requirements or business necessity.

Software Engineering Lecture 7: Quality Assurance 7 Semester 1, 2006 T HE ISO9126 Q UALITY M ODEL CharacteristicsSub-Characteristics EfficiencyTime Behaviour Resource Utilization MaintainabilityAnalyseability Changeability Testability Stability PortabilityReplaceability Adaptability Installability Co-existence CharacteristicsSub-Characteristics FunctionalitySuitability Accuracy Interoperability Security ReliabilityFault Tolerance Maturity Recoverability UsabilityUnderstandability Learnability Operability Attractiveness

Software Engineering Lecture 7: Quality Assurance 8 Semester 1, 2006 O THER S OFTWARE Q UALITY A TTRIBUTES

Software Engineering Lecture 7: Quality Assurance 9 Semester 1, 2006 Q UALITY A SSURANCE Can you guarantee that your program has ‘quality’? It is difficult, if not impossible, to guarantee absolutely that we have meet the needs for which it was built (why?). The aim is to provide a high level of assurance - a high degree of confidence - that the resulting system will meet the needs for which it was built. Providing ASSURANCE that a system does meet the needs for which it was built means ensuring that the system meets a range of requirements coming from, for example: The Users and other Stakeholders; or The Operational Environment. An important factor that distinguishes Engineering from ad hoc development is the ability to exert control over the level of assurance achieved in a project – this is where process becomes important.

Software Engineering Lecture 7: Quality Assurance 10 Semester 1, 2006 Q UALITY A SSURANCE In practice, exerting this kind of control over quality and quality attributes (like those in ISO9126) requires that we build quality into the system starting at requirements stages. All the evidence that we have points to a situation where we cannot simply fix up our software post hoc and add in quality!

Software Engineering Lecture 7: Quality Assurance 11 Semester 1, 2006 A SSUMPTIONS A BOUT Q UALITY The Big Assumption About Quality The quality of the process determines the quality of the product. The reason for the assumption is that even with standards such as ISO9126 many of the attributes of software are hard to measure. However, there is a very complex and poorly understood relationship between software processes and product quality.

Software Engineering Lecture 7: Quality Assurance 12 Semester 1, 2006 Decide on what factors will deliver quality for your project and expend effort and resources on those factors (but don’t neglect the others). Decide on standards to follow and tailor the standards for your project for example, IEEE standard or IEC standards. Decide upon process metrics: for example, acceptable defect rates from inspections of code or inspections of designs. Decide on product metrics: For example the target reliability or performance estimates for you system. Don’t use inappropriate practices simply because standards have been established. P RACTICAL Q UALITY

Software Engineering Lecture 7: Quality Assurance 13 Semester 1, 2006 Standards are often a key part of the management of quality in a project – but not the only factor; Product standards define characteristics that all components should exhibit e.g. a common programming style. Process standards define how the software process should be enacted. S TANDARDS

Software Engineering Lecture 7: Quality Assurance 14 Semester 1, 2006 A UDITS, R EVIEWS, I NSPECTIONS AND W ALKTHROUGHS Technical reviews are aimed at uncovering problems in an artefact or seeking ways to improve the artefact. Audits aim to determine if a particular product or process conforms to standards.

Software Engineering Lecture 7: Quality Assurance 15 Semester 1, 2006 T YPES OF T ECHNICAL R EVIEW Technical Reviews - Management Method The aim is to evaluate a requirements, design, processes, code or other artifacts to provide evidence that it: Conforms to specifications; Is being created according to plans and standards; Changes are properly implemented; As a minimum, the review team explores the artifact to evaluate whether or not it is suitable for its purpose. Inspections – Management Method The aim is to detect and identify defects in requirements, designs, processes, code or other artifacts. Verify that the artifact meets its specification; Verify that the artifact meets the relevant standards; Identify where the artifact deviates from standards Collect defect data for measurement. Walkthroughs – Developer Method The aim is to evaluate requirements, design, processes, code or other artifacts to: Find defects, omissions and contradictions; Consider alternatives and to suggest alternatives; Exchange technical information;

Software Engineering Lecture 7: Quality Assurance 16 Semester 1, 2006 Consists of a Review Leader, Recorder and two or more Team Members. Project management is typically responsible for implementing the recommendations of the review. Input – A statement of Objectives, a Specification for the artefact, and the Artefact itself. Output – 1. Aims, objectives, specifications and artefact itself 2. A list of deficiencies in the artefact, a list of issues in the process and management of the artefact; 3. Action items, ownership for resolving the deficiencies and issues; and 4. Recommendations for disposing of the outstanding issues. T ECHNICAL R EVIEWS

Software Engineering Lecture 7: Quality Assurance 17 Semester 1, 2006 I NSPECTIONS Typically, a Moderator, a Reader, a Recorder, the Author(s) of the artefact, and two to three Inspectors. Inputs 1. A statement of Objectives, a Specification for the artefact, and the Artefact itself; 2. Inspection Checklist; 3. Standards and guidelines relevant to the engineering of the artefact; 4. Inspection and reporting forms. Outputs 1. A defect list containing a description of the defect, its location and the classification of the defect (E.g. Minor, Major, Critical). 2. A summary report of the inspection summarised by defect class or defect severity; 3. An Inspection Report.

Software Engineering Lecture 7: Quality Assurance 18 Semester 1, 2006 W ALKTHROUGHS Consists of a Moderator, Recorder, Author, and two or more Team Members. Inputs 1. A statement of Objectives, a Specification for the artefact, and the Artefact itself; 2. Standards and guidelines relevant to the engineering of the artefact; Outputs 1. A list of the objectives for this walk-through; 2. A list of the defects, omissions, contradictions and suggestions for improvement; 3. Recommendations for improving the artefact and the processes surrounding that artefact.