1. Software Quality Management Copyright, 1999 © Jerzy R. Nawrocki Jerzy.Nawrocki@put.poznan.plwww.cs.put.poznan.pl/jnawrocki/mse/psp/ Personal Software Process Lecture 12

2. J. Nawrocki, PSP, Lecture 12 From the previous lecture 1 Introduction 2 Management plan 3 Quality plan 4 Purchasing plan 5 Development plan 6 Test plan 7 Installation and delivery 8 Product management 9 Maintenance 10 The project plan

3. J. Nawrocki, PSP, Lecture 12 From the previous lecture A good plan should be: complete (forms)complete (forms) accessible (a proper place & format)accessible (a proper place & format) clear (readable)clear (readable) specific (what, when, by whom, at what costs)specific (what, when, by whom, at what costs) precise (unit of time)precise (unit of time) accurate (random errors - no problem)accurate (random errors - no problem)

4. J. Nawrocki, PSP, Lecture 12 Introduction What is software quality ? Is quality an important issue ? How to improve quality ?

5. J. Nawrocki, PSP, Lecture 12 Software quality American Heritage Dictionary: a characteristic or attribute of something Słownik języka polskiego: właściwość, rodzaj, gatunek, wartość

6. J. Nawrocki, PSP, Lecture 12 Software quality Philip Crosby83: conformance to requirements

7. J. Nawrocki, PSP, Lecture 12 Software quality Quality of design Quality of conformance

8. J. Nawrocki, PSP, Lecture 12 Software quality Quality control product-oriented a part of the manufacturing process passive inspections, reviews, tests

9. J. Nawrocki, PSP, Lecture 12 Software quality Quality assurance process-oriented a part of the management process active audits and reports

10. J. Nawrocki, PSP, Lecture 12 Software quality Quality management What level of quality do we need ? How quality is important for our business ? How much is it going to cost ?

11. J. Nawrocki, PSP, Lecture 12 Software quality Product quality Process qualityProduct quality Process quality The user of the PSP is the programmerThe user of the PSP is the programmer The main requirement: to produce a quality softwareThe main requirement: to produce a quality software Other requirements: efficiency, ease of learning,..Other requirements: efficiency, ease of learning,.. I like that process..

12. J. Nawrocki, PSP, Lecture 12 Finding and fixing defects The cost elements: Determining that there is a problemDetermining that there is a problem Isolating the source of the problemIsolating the source of the problem Determining exactly what is wrongDetermining exactly what is wrong Fixing the requirements as neededFixing the requirements as needed Fixing the design as neededFixing the design as needed Fixing the implementation as neededFixing the implementation as needed

13. J. Nawrocki, PSP, Lecture 12 Finding and fixing defects The cost elements (contd.): Inspecting the fixInspecting the fix Testing the fix to ensure it fixes the identified problemTesting the fix to ensure it fixes the identified problem Testing the fix to ensure it doesnt case other problemsTesting the fix to ensure it doesnt case other problems Changing the documentation as need to reflect the fixChanging the documentation as need to reflect the fix

14. J. Nawrocki, PSP, Lecture 12 Some fix time data The relative times to identify the defects (TRW): during requirements: 1 during design: 3 - 6 during coding: 10 in development test: 15 - 40 in acceptance test: 30 -70 during operation: 40 - 1000

15. J. Nawrocki, PSP, Lecture 12 Some fix time data The relative time to identify defects (IBM): during design reviews: 1during design reviews: 1 during code inspections: 20during code inspections: 20 during machine test: 82during machine test: 82

16. J. Nawrocki, PSP, Lecture 12 Some fix cost data Relative costs to identify software defects (IBM): during design: 1.5during design: 1.5 prior to coding: 1prior to coding: 1 during coding: 1.5during coding: 1.5 prior to test: 10prior to test: 10 during test: 60during test: 60 in field use: 100in field use: 100

17. J. Nawrocki, PSP, Lecture 12 Some fix cost data JPL, average cost per defect: inspections: $90 - $120inspections: $90 - $120 tests: $10 000tests: $10 000Conclusions: Remove most of the defects before testing

18. J. Nawrocki, PSP, Lecture 12 A case study - problem 5-person team5-person team Estimated size: 50 000 LOC in C++Estimated size: 50 000 LOC in C++ Requirements + prototyping: 10 monthsRequirements + prototyping: 10 months High level design: 3 monthsHigh level design: 3 months Estimated completion time: 21 monthsEstimated completion time: 21 months Integration and system tests: to start in 5 monthsIntegration and system tests: to start in 5 months Testing should take 3 monthsTesting should take 3 months Inspections would cost another 3 monthsInspections would cost another 3 months Can we afford to inspect specification before starting implementation ?Can we afford to inspect specification before starting implementation ?

19. J. Nawrocki, PSP, Lecture 12 A case study - problem How did we get the 3-month testing estimate ? Brooks72: large software projects are about half done when integration starts. More realistic schedule: 2 x 18 = 36 months 2 x 18 = 36 months

20. J. Nawrocki, PSP, Lecture 12 A case study - testing An experienced software engineer injects ~ 100 defects per KLOCAn experienced software engineer injects ~ 100 defects per KLOC Half these defects are found by the compilerHalf these defects are found by the compiler 50 KLOC x 50 defects / KLOC = 2 500 defects50 KLOC x 50 defects / KLOC = 2 500 defects Hours to find a defect (industry data): 3 - 20Hours to find a defect (industry data): 3 - 20 2 500 defects x 8 hours/defect = 20 000 hours2 500 defects x 8 hours/defect = 20 000 hours 360 days x 11 hours/day = ~ 4 000 hours360 days x 11 hours/day = ~ 4 000 hours To test the system 5 programmers would have to work 360 days, 11 hours per dayTo test the system 5 programmers would have to work 360 days, 11 hours per day

21. J. Nawrocki, PSP, Lecture 12 A case study - inspections The yield of the review or inspection: ~ 70%The yield of the review or inspection: ~ 70% Average cost of inspections: 0.5 hours/defectAverage cost of inspections: 0.5 hours/defect 70% of 2 500 defects = 1 750 defects70% of 2 500 defects = 1 750 defects Inspection time: 1 750 defects x 0.5 hours/defect = 875 hoursInspection time: 1 750 defects x 0.5 hours/defect = 875 hours Testing time: 750 defects x 8 hours/defect = 6 000 hoursTesting time: 750 defects x 8 hours/defect = 6 000 hours Total time: 6 875 hours ~ 1/3 x 20 000 hoursTotal time: 6 875 hours ~ 1/3 x 20 000 hours Reviews and inspections would save us 8 months out of 36 monthsReviews and inspections would save us 8 months out of 36 months

22. J. Nawrocki, PSP, Lecture 12 Cost-of-quality components Failure costs (diagnosing a failure, repairs)Failure costs (diagnosing a failure, repairs) Appraisal costs (evaluating the product to determine its quality)Appraisal costs (evaluating the product to determine its quality) Prevention costs (identifying the causes of defects; preventing them in the future)Prevention costs (identifying the causes of defects; preventing them in the future)

23. J. Nawrocki, PSP, Lecture 12 PSP COQ components Failure costs: the total time spent in compile and testFailure costs: the total time spent in compile and test Appraisal costs: the time spent in design and code reviews + any inspection timesAppraisal costs: the time spent in design and code reviews + any inspection times

24. J. Nawrocki, PSP, Lecture 12 PSP COQ measures Failure COQ = 100 * (compile time + test time) / total development timeFailure COQ = 100 * (compile time + test time) / total development time Appraisal COQ = 100 * (design review time + code review time) / total development timeAppraisal COQ = 100 * (design review time + code review time) / total development time Total COQ = Appraisal COQ + Failure COQTotal COQ = Appraisal COQ + Failure COQ Appraisal as a % of Total Quality Costs = 100 * Appraisal COQ / Total COQAppraisal as a % of Total Quality Costs = 100 * Appraisal COQ / Total COQ A/FR ratio = Appraisal to failure cost ratio = Appraisal COQ / Failure COQA/FR ratio = Appraisal to failure cost ratio = Appraisal COQ / Failure COQ

25. J. Nawrocki, PSP, Lecture 12 Developing a Quality Strategy Decide how to measure your processDecide how to measure your process Determine the quality methods that are most effective for youDetermine the quality methods that are most effective for you Periodically re-evaluate your strategy and set new goalsPeriodically re-evaluate your strategy and set new goals

26. J. Nawrocki, PSP, Lecture 12 Measuring the process Yield (step n) = 100 * r / (r + e) r - defects removed in step n e - defects escaping from step n Yield (overall) = 100 * R / I R - defects removed before compile I - defects injected before compile entry

27. J. Nawrocki, PSP, Lecture 12 Basic defect causes Education: dont understand how to do somethingEducation: dont understand how to do something Communication: not properly informed about somethingCommunication: not properly informed about something Oversight: omitted doing somethingOversight: omitted doing something Transcription: knew what to do but made a mistakeTranscription: knew what to do but made a mistake Process: the process somehow misdirected the actionsProcess: the process somehow misdirected the actions

28. J. Nawrocki, PSP, Lecture 12 Defect prevention Select a specific defect typeSelect a specific defect type What are the causes of the defect ?What are the causes of the defect ? Why it has been made ? (education, communication, oversight,..)Why it has been made ? (education, communication, oversight,..) How to prevent the problem in the future ?How to prevent the problem in the future ? Repeat the above steps a few timesRepeat the above steps a few times What are the trends or patterns in the data that might suggest larger problems ?What are the trends or patterns in the data that might suggest larger problems ? What are the actions that had worked in the past ?What are the actions that had worked in the past ? Test your defect prevention ideasTest your defect prevention ideas

29. J. Nawrocki, PSP, Lecture 12 Defect prevention Exemplary defect types: most frequentmost frequent most difficult to find and fixmost difficult to find and fix easiest to find and fixeasiest to find and fix......

30. J. Nawrocki, PSP, Lecture 12 Quality assurance techniques Inspections and reviewsInspections and reviews Software configuration managementSoftware configuration management

31. J. Nawrocki, PSP, Lecture 12 Summary Software quality is not only a technical issue but also an economical one. Cost-of-quality measures allow us to observe our process and to identify the prospective improvements

32. J. Nawrocki, PSP, Lecture 12 Further readings W.S. Humphrey, A discipline for software engineering, Addison- Wesley, Reading MA, 1995.

33. J. Nawrocki, PSP, Lecture 12 Quality assessment 1. What is your general impression ? (1 - 6) 2. Was it too slow or too fast ? 3. Did you learn something important to you ? 4. What to improve and how ?