OHTO -01 SOFTWARE ENGINEERING SOFTWARE QUALITY Today we talk about software process quality and certification

OHTO -01 SOFTWARE QUALITY - QUALITY COMPONENTS Objective quality component: properties that can be measured or approximated objectively Subjective quality component: customer satisfaction (”What does the product feel like?”) Other: features which can not be (even subjectively) evaluated at the time. This is related with future events which can not be predicted - unexpected circumstances, changes, etc.

OHTO -01 THE QUALITY SYSTEM Quality control - controlling the way things are done Quality assurance - making sure quality is achieved Quality policy Quality planning Quality improvement These terms come from the standard ISO 8402

OHTO -01 THE QUALITY SYSTEM (cont’d) The quality system of a company is simply the way the company works and it covers all areas of activity. Therefore, a quality system always exists. It may be documented or not. However, in the long run in a big company it makes a major difference, if the management takes quality seriously and knowingly emphasizes it in all activities.

OHTO -01 QUALITY CONTROL Controlling the software development process Standards for the development process, e.g. - well-defined phases - checklists - reviews: what and when - organisational standards Visibility and bookkeeping of the development process Standards for software code and documentation, e.g. - naming and style - document skeletons and formats - different kinds of review and feedback forms

OHTO -01 QUALITY ASSURANCE Improving software quality by monitoring the products (software) and process Ensuring full compliance with the standards for products and process Ensuring that any inadequacies in the product and process (and standards) are brought to management’s attention.

OHTO -01 QUALITY ASSURANCE - PREREQUISITES Top management commitment The quality assurance organisation must be independent from the development organisation. The quality assurance organisation must be properly staffed. The quality assurance organisation must co- operate properly with the development organisation. Quality must be seen as a common goal.

OHTO -01 SOFTWARE METRICS Measurements which relate to a software system, process, or related documentation Examples: - size of a product in lines of code - number of reported faults - time required to produce a system component Control metrics measure the process Predictor metrics are measurements of a product attribute which can be used to predict an associated product quality.

OHTO -01 PREASSUMPTIONS FOR THE USE OF PREDICTOR METRICS We can accurately measure some property of the software. A relationship exists between what we can measure and what we would like to know about the product’s behavioural attributes. This relationship is understood, has been validated, and can be expressed in terms of a formula or a model. (This last assumption is often ignored.)

OHTO -01 PRODUCT QUALITY METRICS Cohesion and coupling (see the reverse engineering lectures) Cyclomatic complexity (has to do with loops and conditional statements - it does not take the complexity of data definitions into account) - Does this measure the complexity of the problem or the complexity of the solution? Customer satisfaction (survey the customers) Usually the preconditions are not fully met.

OHTO -01 PROCESS QUALITY METRICS Actual vs. estimated time and cost Number of faults detected Phase of the project when faults were found Productivity (e.g. lines of code / man-month) Quality expenses (e.g. repairs required after delivery)

OHTO -01 QUALITY REVIEW TYPES Design or program inspections/audits. These are commonly driven by a checklist of possible errors. Management reviews. These are intended to provide information for the management about the overall progress of the software project. Quality reviews. The work of an individual or a team is reviewed by a panel made up of project members and technical management.

OHTO -01 A GENERAL SCHEME FOR REVIEWS Select the review team. Arrange the place and time. Distributed the documents. Hold the review. Note the actions and complete the review forms.

OHTO -01 POSSIBLE ACTIONS FOR FINDINGS IN REVIEWS No action. Some kind of an anomaly was found, but it was not cost-effective to fix it. Refer for repair. The team or individual in charge of the product has to correct the fault. Reconsider the overall design. It may be more reasonable to fix other components of the system to solve the problem which was found.

OHTO -01 INSPECTIONS / AUDIT SESSIONS The ”Fagan method” - named after an IBM employee who pioneered the technique All types of defects are noted - not just logic or function errors. Inspections can be carried out by collagues at all levels except the very top. Inspections are carried out using a predefined set of steps. Inspection meetings do not last for more than two hours. The inspection is lead by a moderator who has a specific trainging in the technique.

OHTO -01 INSPECTIONS / AUDIT SESSIONS (continued) The other participants have defined roles. For example, one person will act as a recorder and note all defects found and another will act as a reader who takes the other participants through the document under inspection. Checklists are used to assist the fault-finding process. Material should be inspected at an optimal rate of about 100 lines per hour. Statistics are maintained so that the effectiveness of the inspection process can be monitored.

OHTO -01 A CHECKLIST FOR REVIEW EVALUATIONS Is material complete (and does it meet the standards)? Was material distributed on time? Are applicable standards referenced and available? Were attendees prepared to contributed? Did evaluation start on time? Were number of defects identified? Was review conducted per standard protocols? Were entrance criteria met? Were exit criteria met?

OHTO -01 A CHECKLIST FOR REVIEW EVALUATIONS (cont’d) Are all issues scheduled for resolution? Are interface issues coordinated? Is disposition of all defects complete? Were HCI, testing, maintenance, and tools considered? Were quality attributes reported? Has trace of defects been initiated?

OHTO -01 SOFTWARE QUALITY CIRCLES A Japanese practice. A quality circle is a group of four to ten volunteers working in the same area who meet for, say, an hour a week to identify, analyse and solve their work-related problems. One of the group is a group leader and one (maybe from outside) is a facilitator giving advice. Training is required, and so is full support from the management.

OHTO -01 PROBLEM SOLVING BY SOFTWARE QUALITY CIRCLES Identify a list or problems and choose one. Clarify the problem. Identify and evaluate the causes. Identify and evaluate the solutions. Decide on the solution. Develop an implementation plan. Present the plan to the management. Implement the plan. Monitor the plan. Consider wider apllicability of solution. Restart from choosing another problem.

OHTO -01 A RISK ASSESMENT CHECKLIST Have risk areas been identified? Has risk analysis been adequately reviewed? Has it met internal or external standards? Was review of analysis results thorough? Does risk analysis documentation meet standards? Are there any open risk issues? If yes, then what? Is a risk containment plan in place? Have alternatives been defined in the event that failure occurs? Is the reporting frequency appropriate?

OHTO -01 ISO 9000 Series of standards for general use in different fields of industry. ISO 9001 is for companies with product development and production. Separate certification organisations give certification for companies which fulfill the requirements of the standards.

OHTO -01 INGREDIENTS OF ISO 9001 Management responsibility Quality system Contract review Design control Document control Purchasing Purchaser supplied control Product identification adn traceability Process control Inspection and testing

OHTO -01 INGREDIENTS OF ISO 9001 (continued) Inspection, measuring and test equipment Inspection and test status Control of nonconforming product Corrective action Handling, storage, packaging and delivery Quality records Internal quality audits Training Servicing Statistical techniques

OHTO -01 AN OVERVIEW OF ISO 9001 REQUIREMENTS The management must define and document the policy concerning quality and must ensure that this policy is communicated to all levels of the organisation. All quality control procedures must be documented. All contracts to supply goods or services should contain mutually agreed requirements that the developer is capable of delivering. There should be procedures to approve design and other documentation.

OHTO -01 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) Where components of the system to be supplied to the client are obtained from third parties there must be procedures to ensure, check, and maintain the quality of these components. Individual products should be identifiable as should their components. The process by which the final product is created should be planned and monitored. Inspection and testing should take place during the development phase, at its completion and before delivery. Tests and inspections should also be carried out on components obtained from third parties.

OHTO -01 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) The equipment used in the production process itself should be properly controlled with respect to quality. The testing status of all components and systems should be clearly recorded all times. Care must be taken to ensure that items which are known to be defective are not inadvertently used. When a defect is detected, measures must be undertaken to remove the defective part and to ensure that the defect does not occur again. Satisfactory procedures must be in place to deal with correct handling, storage, packaging, and delivery of the product.

OHTO -01 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) Sufficient records must be maintaned to demonstrate that the quality system is working satisfactorily. The software quality management system should be audited on a regular basis. Servicing and supprt activiies must be subject to the quality management system. The developer must establish appropriate statistical techniques to verify the acceptability of the final product.

OHTO -01 CERTIFICATION (for standards like ISO 9001) The company familiarises itself with the standards. The company applies for the certification. The certification organization inspects the company (based on documents which it requires from the company). An evaluation meeting is held with representatives from both the company and the certification organisation. If necessary, some corrective actions are performed. A certification is given to an accepted company. Re-evaluations will be performed.

OHTO -01 GENERAL CONCLUSIONS The development of a quality system requires changes and changes create resistance. Be realistic. Quality depends a lot on attitudes. Internal and management motivation is essential. Small steps can be the best way forward.