1. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 1 SOFTWARE ENGINEERING SOFTWARE QUALITY Today we talk about software process quality and certification

2. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 2 SOFTWARE QUALITY - QUALITY COMPONENTS Objective quality component: properties that can be measured or approximated objectively Subjective quality component: customer satisfaction (”What does the product feel like?”) Other: features which can not be (even subjectively) evaluated at the time. This is related with future events which can not be predicted - unexpected circumstances, changes, etc.

3. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 3 THE QUALITY SYSTEM Quality control - controlling the way things are done Quality assurance - making sure quality is achieved Quality policy Quality planning Quality improvement These terms come from the standard ISO 8402

4. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 4 THE QUALITY SYSTEM (cont’d) The quality system of a company is simply the way the company works and it covers all areas of activity. Therefore, a quality system always exists. It may be documented or not. However, in the long run in a big company it makes a major difference, if the management takes quality seriously and knowingly emphasizes it in all activities.

5. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 5 QUALITY CONTROL Controlling the software development process Standards for the development process, e.g. - well-defined phases - checklists - reviews: what and when - organisational standards Visibility and bookkeeping of the development process Standards for software code and documentation, e.g. - naming and style - document skeletons and formats - different kinds of review and feedback forms

6. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 6 QUALITY ASSURANCE Improving software quality by monitoring the products (software) and process Ensuring full compliance with the standards for products and process Ensuring that any inadequacies in the product and process (and standards) are brought to management’s attention.

7. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 7 QUALITY ASSURANCE - PREREQUISITES Top management commitment The quality assurance organisation must be independent from the development organisation. The quality assurance organisation must be properly staffed. The quality assurance organisation must co- operate properly with the development organisation. Quality must be seen as a common goal.

8. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 8 PROCESS QUALITY METRICS Actual vs. estimated time and cost Number of faults detected Phase of the project when faults were found Productivity (e.g. lines of code / man-month) Quality expenses (e.g. repairs required after delivery)

9. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 9 QUALITY REVIEW TYPES Design or program inspections/audits. These are commonly driven by a checklist of possible errors. Management reviews. These are intended to provide information for the management about the overall progress of the software project. Quality reviews. The work of an individual or a team is reviewed by a panel made up of project members and technical management.

10. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 10 A GENERAL SCHEME FOR REVIEWS Select the review team. Arrange the place and time. Distributed the documents. Hold the review. Note the actions and complete the review forms.

11. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 11 POSSIBLE ACTIONS FOR FINDINGS IN REVIEWS No action. Some kind of an anomaly was found, but it was not cost-effective to fix it. Refer for repair. The team or individual in charge of the product has to correct the fault. Reconsider the overall design. It may be more reasonable to fix other components of the system to solve the problem which was found.

12. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 12 INSPECTIONS / AUDIT SESSIONS The ”Fagan method” - named after an IBM employee who pioneered the technique All types of defects are noted - not just logic or function errors. Inspections can be carried out by collagues at all levels except the very top. Inspections are carried out using a predefined set of steps. Inspection meetings do not last for more than two hours. The inspection is lead by a moderator who has a specific trainging in the technique.

13. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 13 INSPECTIONS / AUDIT SESSIONS (continued) The other participants have defined roles. For example, one person will act as a recorder and note all defects found and another will act as a reader who takes the other participants through the document under inspection. Checklists are used to assist the fault-finding process. Material should be inspected at an optimal rate of about 100 lines per hour. Statistics are maintained so that the effectiveness of the inspection process can be monitored.

14. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 14 A CHECKLIST FOR REVIEW EVALUATIONS Is material complete (and does it meet the standards)? Was material distributed on time? Are applicable standards referenced and available? Were attendees prepared to contributed? Did evaluation start on time? Were number of defects identified? Was review conducted per standard protocols? Were entrance criteria met? Were exit criteria met?

15. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 15 A CHECKLIST FOR REVIEW EVALUATIONS (cont’d) Are all issues scheduled for resolution? Are interface issues coordinated? Is disposition of all defects complete? Were HCI, testing, maintenance, and tools considered? Were quality attributes reported? Has trace of defects been initiated?

16. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 16 SOFTWARE QUALITY CIRCLES A Japanese practice. A quality circle is a group of four to ten volunteers working in the same area who meet for, say, an hour a week to identify, analyse and solve their work-related problems. One of the group is a group leader and one (maybe from outside) is a facilitator giving advice. Training is required, and so is full support from the management.

17. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 17 PROBLEM SOLVING BY SOFTWARE QUALITY CIRCLES Identify a list or problems and choose one. Clarify the problem. Identify and evaluate the causes. Identify and evaluate the solutions. Decide on the solution. Develop an implementation plan. Present the plan to the management. Implement the plan. Monitor the plan. Consider wider apllicability of solution. Restart from choosing another problem.

18. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 18 ISO 9000 Series of standards for general use in different fields of industry. ISO 9001 is for companies with product development and production. Separate certification organisations give certification for companies which fulfill the requirements of the standards.

19. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 19 INGREDIENTS OF ISO 9001 Management responsibility Quality system Contract review Design control Document control Purchasing Purchaser supplied control Product identification adn traceability Process control Inspection and testing

20. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 20 INGREDIENTS OF ISO 9001 (continued) Inspection, measuring and test equipment Inspection and test status Control of nonconforming product Corrective action Handling, storage, packaging and delivery Quality records Internal quality audits Training Servicing Statistical techniques

21. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 21 AN OVERVIEW OF ISO 9001 REQUIREMENTS The management must define and document the policy concerning quality and must ensure that this policy is communicated to all levels of the organisation. All quality control procedures must be documented. All contracts to supply goods or services should contain mutually agreed requirements that the developer is capable of delivering. There should be procedures to approve design and other documentation.

22. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 22 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) Where components of the system to be supplied to the client are obtained from third parties there must be procedures to ensure, check, and maintain the quality of these components. Individual products should be identifiable as should their components. The process by which the final product is created should be planned and monitored. Inspection and testing should take place during the development phase, at its completion and before delivery. Tests and inspections should also be carried out on components obtained from third parties.

23. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 23 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) The equipment used in the production process itself should be properly controlled with respect to quality. The testing status of all components and systems should be clearly recorded all times. Care must be taken to ensure that items which are known to be defective are not inadvertently used. When a defect is detected, measures must be undertaken to remove the defective part and to ensure that the defect does not occur again. Satisfactory procedures must be in place to deal with correct handling, storage, packaging, and delivery of the product.

24. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 24 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) Sufficient records must be maintaned to demonstrate that the quality system is working satisfactorily. The software quality management system should be audited on a regular basis. Servicing and supprt activiies must be subject to the quality management system. The developer must establish appropriate statistical techniques to verify the acceptability of the final product.

25. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 25 CERTIFICATION (for standards like ISO 9001) The company familiarises itself with the standards. The company applies for the certification. The certification organization inspects the company (based on documents which it requires from the company). An evaluation meeting is held with representatives from both the company and the certification organisation. If necessary, some corrective actions are performed. A certification is given to an accepted company. Re-evaluations will be performed.

26. 26.3.2003Software Engineering 2003 Jyrki Nummenmaa 26 GENERAL CONCLUSIONS The development of a quality system requires changes and changes create resistance. Be realistic. Quality depends a lot on attitudes. Internal and management motivation is essential. Small steps can be the best way forward.