Presentation is loading. Please wait.

Presentation is loading. Please wait.

Software Quality Assurance

Similar presentations


Presentation on theme: "Software Quality Assurance"— Presentation transcript:

1 Software Quality Assurance
Industry statistics for 1994 (WW. Gibb): For every 6 systems that work, 2 are cancelled It is worse for large systems: 3 out of every 4 large systems were reduced in scope or did not work at all (e.g. Denver airport) Still bad today for low maturity software development Good quality software doesn’t just happen, however good the team producing the software and however good the development techniques they use. Good software development needs procedures and management. That is what QA is all about Why does it need procedures and management? * Because people can’t hold everything they need to know in their head at once * Because many techniques are not ‘turn the handle’ - they are creative - and hence there is opportunity for human error

2 Why are procedures and management necessary for quality?
Software development is complex: much to remember Hence standards, procedures and guidelines which embody best practice These also help training of new staff When the pressure’s on… something has to give... Hence defined points at which quality of work so far is assured Hence being explicit with the customer about cost-time-scope-quality trade-off: ‘if you want software of this quality...’ Some defects may go unnoticed by the author Hence reviews… which wouldn’t happen without management! Helps assure the customer that the software is of sufficient quality

3 This lecture What is quality? What is quality assurance?
Quality factors What is quality assurance? Quality manuals and plans Standards for quality manuals: the ISO 9000 family Group Project Standards Reviews Kinds of review: particularly the Formal Technical Review How to do a formal review

4 What is quality? ISO definition (ISO 8402:1994) Or…
“The totality of features and characteristics of a product, process or service that bears on its ability to satisfy stated and implied needs.” Or… Software quality is meeting the requirements of the user With the caveats that… The user requirements may not be precisely stated Some user requirements may not be stated at all, because of ‘shared assumptions’ between user and developer The developer may also have requirements which may not be part of the user requirements (e.g. testability, maintainability)

5 Quality factors (ISO 9126:1992)
Functionality Satisfying stated or implied needs (regarding the software’s functions) Reliability Maintaining performance under stated conditions for stated period of time Usability The effort needed for use, and the individual assessment of such use by a stated or implied set of users Efficiency The relationship between performance and resources required Maintainability The effort needed to make specified modifications Portability The ability of software to be transferred from one environment to another Suitability: Right feature set: suitable functions for tasks to be performed Accuracy: Same as correctness: Provision of right/agreed results or effects Interoperability: interaction with particular systems Security: Ability to prevent unauthorised access Maturity: Frequency of failure Fault tolerance: Ability to maintain some level of performance in spite of failure Recoverability: Ability to recover original level of performance after a failure (e.g. time to recover) Operability: Effort required to operate the software once it’s learnt

6 What is Quality Assurance?
Quality Assurance is the overall management of quality. Within a company, it includes: Specifying a quality policy: General quality objectives and commitments Maintaining a quality manual, which contains details regarding: What constitutes good quality: Standards, procedures, guidelines Quality controls: methods of assuring that quality is good Producing quality plans for each project The parts of the quality manual which apply to this project Assigning specific people to quality assurance, often including a project-independent quality assurance group Planning (schedule/budget), performing, evaluating quality assurance activities This definition of quality control, something that assures quality rather than the general activity of ensuring quality, is consistent with ISO 8402.

7 Recognised standards of Quality Assurance
Increases customer’s confidence that a company produces quality products General Quality Standards e.g. ISO 9000 Quality Manual From Sommerville. Project 1 Quality Plan Project 2 Quality Plan Project n Quality Plan

8 The ISO 9000 family ISO 9000: Guidelines for selection and use of the other documents in the family. ISO 9001: Model for QA in design, development, production, installation and servicing. ISO 9002: Model for QA in production, installation and servicing. ISO 9003: Model for QA in final inspection and test. ISO 9004: Guidelines on quality management and quality system elements. Also ISO : Application of ISO 9001 to the development, supply and maintenance of software. Direct correspondence between clauses of ISO 9001 and ISO

9 Some example sections from ISO 9000-3
Management Responsibility. Quality System. Internal quality system audits. Corrective action. Contract review. Purchaser’s requirements spec. Development planning. Quality planning. Design and implementation. Testing & Validation. Acceptance. Maintenance. Configuration Management. Document Control. Measurement. Rules, practices, conventions. Training. Purchasing. ISO 9001/ are short (10-15 pages): open to interpretation

10 ISO 9001 versus ISO 9000-3: ‘quality planning’ section
… Quality planning shall be consistent with all other requirements of a supplier’s quality system and shall be documented in a format to suit the supplier's method of operation. The supplier shall give consideration to... ISO Quality planning should address… quality requirements, expressed in measurable terms, where appropriate; the life cycle model to be used for software development defined criteria for starting and ending each project phase identification of types of reviews, tests and other verification and validation activities to be carried out identification of configuration management procs to be carried out

11 ISO 9000: A standard for quality manuals
Companies can, and do, get certification that: Their quality manual meets the ISO 9000 standards They run their projects with adherence to their quality manual A company is ISO certified by an independent body The company first receives training in ISO 9000 etc. The quality manual is sent to the body, which assesses it and reports back An assessment team visits the company to ensure that it follows its manual The body then awards the company ISO 9000 status unconditionally, conditionally, or not at all After certification, an assessment team visits unannounced 2-4 times per year - and conducts a full reassessment every 3 years From Darrel Ince, ISO 9001 and Software Quality Assurance, (National Library)

12 Software Engineering Institute’s Capability-Maturity Model (SEI CMM)
Level 5 Optimizing Organisational commitment to continuous process improvement; defect prevention Level 4 Managed Management by measurement: quantitative targets for each project; productivity and quality measures; measurement database Level 3 Defined Stable process model including entry/exit criteria; reviews, quality tracking, full process training program Level 2 Repeatable Stable project planning, tracking of cost/schedule/ functionality, problem identification, project standards; can ‘repeat the success of previous projects’ From Paulk, Mark C. et al. “Capability Maturity Model, Version 1.1”, IEEE Software, July 1993, Level 1 Initial Unstable, ad-hoc processes; project success depends on staff skills/experience (‘heroics’)

13 Comments on the CMM CMM versus ISO 9000 Disadvantages of the CMM
CMM much more prescriptive (500 pages versus pages) Companies rated ISO 9000 are probably partly level 2, partly level 3; but they may only be at level 1 (because ISO open to interpretation) CMM is improvement-oriented, ISO 9000 is ‘minimum quality’ Disadvantages of the CMM Nothing about technology or risk analysis Originally to help DoD select contractors: large, slow-moving systems Key areas a bit arbitrary: 80% level % level 3 = level 1! Too bureaucratic for small companies? At least it collects together some best practices CMM versus ISO 9000 is from a paper by Mark Paulk, IEEE Software January 1995. Disadvantages from Sommerville. ‘Too bureaucratic’ has a question mark because it seems that plenty of small companies have been assessed with respect to the CMM (although they may actually be small suborganisations within large organisations).

14 A project’s quality plan
Should be defined at an early stage of the project Can use quality factors to determine which parts of the quality manual are particularly appropriate to this project Is portability important? How long will this product live? Define activities to be performed. Answer questions for each activity: What is checked and against what? Who performs the check and who approves the checks? Who takes the final decision if there is controversy? How are results recorded? How are corrective actions controlled?

15 Quality controls: The general process
1 Define things which are considered ‘good quality’. 2 Define the procedure for checking quality. 3 Carry out the procedure on a specified item. 4 Record the result. 5 Take and record any corrective action Regarding both the item and the quality system Feedback (step 5) is important - if there are any deficiencies in the QA system, then they should be improved.

16 Group Project Standards
A cut down version of what you might see in industry They cover the following areas: QA.01 Quality plan - see previous slide on ISO-9001 Quality planning QA.02 Project management - how we will run the project QA.03 Documentation - what they will cover/look like QA.05 Design Spec standards QA.06. Test Spec standards QA.07 Review standards - will cover now... QA.08 Configuration management QA.09/10 Coding standards for Java/C# QA.11 Final report / maintenance documentation

17 An important quality control: the review
Bugs are caught more efficiently by review than by testing A review is… A meeting at which some aspect(s) of the project are inspected and assessed by a group of people All reviews have an element of formality, and their outcome is minuted Formal Technical Review To identify defects in a workproduct (design document, code listing, …) Walkthrough To identify defects in a workproduct, but with the emphasis on learning and evaluating alternatives - less formal

18 Formal Technical Review: process
Author submits document or code to the review convenor Briefly examined before accepting it for review Review preparation Review team formed Criteria formed by which the document/code will be examined All necessary documents sent to reviewers, who read them The review meeting (2 hours maximum) Issues (possible defects) are detected, but not corrected Follow-up: document changed in response to issues highlighted in review meeting, and resubmitted if necessary

19 Documents needed for a review
The item under review! The document(s) from which the item is derived For test spec: requirements, user interface design, design? For code: detailed design (or for us, list of classes/attributes/methods) The relevant quality plan documents A review checklist: questions to answer Summarises what to look for when reviewing Derive from the quality plan documents and (to an extent) from the documents from which the item is derived Try to fit on one sheet of paper, to concentrate on major issues

20 Example review checklist: design
Do the reviewers understand the design? Will the design meet the requirements? Does the design satisfy the relevant standards? Can the design be simplified? Will the design lead to a maintainable system? Could the design be modified so as to offer more opportunity for reusing existing software? The more specific the quality plan, the easier to create checklists SE.QA.07 has a more specific design checklist for group project.

21 Example review checklist: code
Are all variables initialised before their values are used? Have all constants been named? Is each loop certain to terminate? Are upper and lower bounds of arrays sensible? Have pointers been correctly reassigned after modification? Does code conform to QA code standards for the language?

22 The review meeting The item being reviewed is ‘walked through’ from start to end Issues are highlighted as they occur Includes issues found before, as well as during, the meeting The author evaluates each issue, and… Accepts as a defect, or explains why not a defect, or answers a question which a reviewer has Minutes are taken by the scribe (QA Manager for us!) ‘Issue’ is not a euphemism for ‘problem’, it means ‘possible problem’ (Note that the item is ‘walked through’ even though this isn’t a walkthrough!)

23 Conclusions QA is the management of doing things in such a way to ensure that end results meet the users requirements. Steps must be appropriate & economically realistic. Aim is not highest possible quality, but is a compromise between Time Cost Quality Scope


Download ppt "Software Quality Assurance"

Similar presentations


Ads by Google