Product Quality, Testing, Reviews and Standards CO3808 and CO2403 Product Quality, Testing, Reviews and Standards

Software Product Quality Ensuring that the required level of quality is achieved in a software product by defining appropriate quality standards and procedures ensuring that these are followed developing a ‘quality culture’ This involves: Quality assurance -procedures and standards Quality planning - for a particular project Quality control - ensuring that procedures and standards are followed This is a separate activity from project management to ensure independence and objectivity

What is Quality and Can it be Measured? Quality is difficult to pin down, it is based heavily upon expectation and perception There are some key words that can be associated with Quality Software

Product Testing Determining whether a product meets its requirements – is it fit for purpose? Involves developing a testing strategy A testing strategy will require some, or all of the following processes Verification & Validation (ensuring that the data entering the system is accurate and reliable) Defect testing (finding and fixing bugs) Integration testing (linking objects, components and modules) Test Reports (confirmations that a particular product meets a particular specification) Testing workbenches (tools that can assist with the testing process)

What should a test strategy do? Consider the overall system and determine the key functional and non-functional requirements These must be met if a system is to be considered a success, so a testing strategy should be designed to ensure (and prove) that they can be verified and validated A testing strategy should be designed ‘around’ the system in question and is intended to identify what is to be tested and the methods by which that testing can take place A testing strategy is not simply about defect testing, it should consider wider project issues such as: Assessing and Evaluating Risks Issues of criticality, performance and reliability Legal issues Security considerations Concurrency issues, system demand, scalability, portability, backups and recovery, disaster management

Product Standards Standards are the key to effective quality management They may be international, national, organisational or specific project standards Product standards define characteristics that all components should exhibit Process standards define how the software process should be enacted Example Product Standards Design Review Form Document Naming Standards Interface Design Standards Object Variable and Procedure Naming Procedure header format Java or C++ style standards Project plan format Change Request Form

Standards Why are standards important They encapsulate best practice in an attempt to avoid the repetition of past mistakes They provide a framework for quality assurance process – products cab be checked for standard compliance Provide continuity in the workplace - new staff can understand the organisation by understanding the standards applied Problems with Standards Not seen as relevant and up-to-date by experienced software engineers (unless they developed the standards!) Often seen to involve too much bureaucratic form filling Can be very specific to an organisation and are therefore unsupported by software tools so tedious manual work is involved to maintain standards

The quality review process The principal method of validating the quality of a process or of a product Group examination of part or all of a process or system and its documentation to find potential problems There are several different types of review with different objectives Code Inspections, for defect identification and removal at module level (Product focussed) Progress Reviews, to assess how the overall project is progressing (product and process focus) Quality Reviews, to examine if the product is matching its requirements and also to assess whether quality procedures are being followed (at all levels of a project)

Continued… A group of people carefully examine part or all of a software system and its associated documentation before they are “signed off” as being complete Code, designs, specifications, test plans, standards, etc. can all be reviewed The objective is the discovery of system defects and/or inconsistencies Comments made during the review should be classified: No action. No change to the software or documentation is required. Refer for repair. Designer or programmer should correct an identified fault. Reconsider overall design. The problem identified in the review impacts other parts of the design. Some overall judgement must be made about the most cost-effective way of solving the problem. Requirements and specification problems may have to be referred back to the client

Summary To determine if a project is a success, something has to be measured against an agreed benchmark The quality of a product can be assessed against defined quality attributes Product Quality can be determined by how well a product meets its specification (functional and non-functional) Standards provide a framework against which to assess individual components of a system Quality reviews provide the opportunity to assess quality against the standards does the product meet the quality attributes does the product fulfil the user requirements have standards been adhered to (do they need changing?)