1. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 1 SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering Module 11 Analyzing the Net Cost of a Process

2. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 2 Analyzing the Net Cost of a Process 1) Identify and list all steps or tasks of the process – Document each step -- purpose, description, procedures, etc. – Include everything you do. This works best if you watch and record actual behavior of practitioners.

3. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 3 Analyzing the Net Cost of a Process 2) For each task, do a basic value- added analysis: – Divide all tasks into value-added and non-value-added – A task is “value-added” if it meets the three strict criteria: Changes the product The change is wanted by the customer It is done right the first time (is not rework)

4. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 4 Scrutinize Value-Added Tasks Inputs – What is their source? – From whom or what? (supplier). – Is there an alternative? – Does the supplier understand my dependence on them? What value does it add, if any? – Why do this? For whom? – What is the impact if I do not do this? – Do they care that I do this?

5. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 5 Scrutinize Value-Added Tasks (continued) Outputs – What do I produce? – To whom does it go? (i.e., who is the customer of this task?) – Do they really need it? You may find that at least some portion of these tasks is really non-value-added

6. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 6 For the Moment, we Ignore Whether the Non-Value-Added Tasks are Essential We can take that into consideration later, when deciding which tasks to focus on eliminating or streamlining But it is a good idea to scrutinize the “essential” tasks to make sure they are really essential

7. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 7 Scrutinize Value-Added- Essential Tasks Why must we do it? – What happens if we don’t do the task? Does anyone really care? Would there be a genuine problem? – What does the task accomplish? – What would happen if we did it a different way? Can it be simplified or streamlined? Is all of it essential?

8. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 8 Sample Result of Analysis Non-EssentialEssential Value-AddedNon-Value-Added (costs $, no value to customer) 1) Customer Wants 2) Changes Product 3) Done Right the First Time Requirements analysis Design Coding Documentation Integration Manufacturing Packaging Shipping Estimating Training Planning Customer- required acceptance test Configuration Control Inspections Debugging Service calls Warranty costs Fedex costs for patches Loss of customer goodwill etc. Approval by 7 people! Delays for test systems Data conversion between design tool and coding tool Wait for subcontracted hardware

9. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 9 Net Cost of a Process (continued) 3) Divide into five categories: A) Costs of performing the process value-added B) Prevention costs non-value-added, cost of quality (conformance) C) Appraisal or evaluation costs non-value-added, cost of quality (conformance) D) Failure response costs non-value-added, cost of quality (non conformance) E) Everything else Non-value-added, non cost of quality

10. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 10 Net Cost of a Process Categorizing Tasks & Subtasks Cost of Non Conformance FailureAppraisalPrevention Cost of Conformance Non Cost of Quality Cost of Quality (all non-value-added) Design Development Fabrication Documentation Assembly Process Creation Upgrade Shipping Training Planning Simulation Modeling Consulting Qualifying Certifying Inspection Testing Audits Monitoring Measure- ment Verification Analysis Rework Service Modification Expediting Recall Correction Retest Error Analysis Non- Value- Added Value Added Over- head Errors Ineff- icien- cies

11. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 11 Some Costs Fit Multiple Categories Example: – The design process is mostly value-added but about 20% of the cost is for a design inspection which is, technically, non-value-added – If you break the process into more detailed sub-processes, you can usually assess these situations more accurately

12. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 12 Net Cost of a Process (continued) 4) Determine costs – Determine the cost in labor or other units for each task in each category – Sum for all tasks. % of time spent Task ___ % aaa 12.5 bbb 8.0 … Time Card Labor Hours Spent Time Card Labor Hours Spent

13. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 13 Example - Cost of a Process Non- value- added Cost of Conformance Cost of Non Conformance FailureAppraisalPrevention Non Cost of Quality Cost of Quality TOTAL 600 100 450 95 300 450 400 275 200 150 3170 100% Program Design Design Reviews Code Code Reviews Unit Test Integration Test System Test Configuration Mgt Problem Solving Rework Retest 540 400 940 30% 60 120 4% 40 35 300 450 400 275 1500 45% 200 150 500 15% value- added 60 50 110 3.5%

14. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 14 At This Point There are Two Possibilities Analysis Results Do we Have Useful Data on Costs? Yes – Proceed to Further Analysis (step 7) No – Improve Measurements (step 5)

15. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 15 What if You Don’t Know Enough for Useful Data? Often at this point you realize how much you do not know – Cost of some tasks – Value of some tasks – Quality of some tasks – Productivity or cycle time of some tasks Which leads us to step 5 - defining a measurement program to help with this analysis

16. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 16 Net Cost of a Process (continued) 5) For each task, determine: – Customer requirements (input & output) – Supplier requirements (input & output) – Measures that determine if requirements are met – Measures that determine why or why not – Other measures needed to assess value or cost

17. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 17 Net Cost of a Process (continued) 6) For each task: – Analyze the Measurement Requirements Is it measured now? Can it be measured? Should it be measured? – Determine the level of effort or cost to measure

18. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 18 Use the Measures to Determine... What tasks cost the most What tasks are worth improving What tasks should be eliminated

19. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 19 Net Cost of a Process (continued) 7) Look for opportunities to improve – Tasks that are not value-added but have high cost – Tasks that have high cost of non- conformance This is where we waste so much time!

20. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 20 Comments As the process matures, the trend is to have more and more prevention costs and fewer appraisal & failure costs Your process maturity may have a lot to do with which techniques work and which do not!

21. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 21 Rework One of the first things to measure is rework – Easy to explain to upper management – Contributes negatively to cost, quality & cycle time

22. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 22 Rework Loops Process Step Undetected Defect Several Steps Process Step Defect Detected

23. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 23 Doing it Over Again Process Step Undetected Defect Several Steps Process Step Defect Detected Rework costs money and time

24. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 24 Comments (continued) By tracking these costs and the impact of process changes, we can determine the cost and the benefit of changes Thus we can optimize the process We can optimize for: – Quality – Cost, or – Cycle Time Process engineering can help improve all three!

25. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 25 Example of Tracking Project Gamma Project Beta Project Alpha

26. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 26 More Comments If we track current costs – We open our eyes to the cost of non conformance If we track the effect of changes – We learn what works to improve our process

27. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 27 Cost of Quality vs. Value-Added The cost of quality chart helps us identify which costs contribute to cost of quality and which costs contribute to cost of failure to have quality Note that both charts put value-added tasks on the left But cost of quality analysis helps us decide which non-value-added costs to focus on and where to add non-value- added tasks (cost of conformance) to reduce net costs

28. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 28 Measuring “Cost of Quality” Money is the language of management You need to tell managers how much quality costs You need to show them that improvements are possible in terms of net cost (In some organizations, other things matter, such as overhead rates, people levels, prestige, etc.)

29. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 29 Measure Actual Costs and Other Data Record actual data in a historical database – Labor – Material – Cost of quality vs. non cost of quality – Cost of conformance vs. cost of non conformance Project future costs on the basis of historical precedent

30. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 30 Analyze Actual Costs and Other Data Compare alternative approaches – Process changes – Process differences – Other differences

31. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 31 Humphrey’s Model What is really happening What you think should happen What you think is happening Measurement can help you understand what is really happening

32. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 32 Beware... Use process costs to measure processes, not people Or people will change performance to make the numbers look good Or you will become overly dependent on having only the best people Success only because you have the best people does not constitute genuine organizational capability improvement

33. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 33 Process Changes Have Many Benefits Consistency Long term impact Not overly dependent on individuals BUT This does not mean that good people are not valuable A good process enables – Best people to maximize their contributions – The organization to function smoothly when personnel changes occur

34. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 34 Summary - I Understanding value is the starting point for effective and efficient processes Value-added analysis can help identify the best places to focus improvement efforts Software quality analysis is different from hardware quality analysis because the processes are different. Software is more dependent on development cost than production cost. Quality tasks can be divided into cost of conformance and cost of non-conformance

35. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 35 Summary - II Cost of conformance can be divided into cost of prevention and cost of detection Invest in cost of conformance to reduce cost of non-conformance Invest in prevention to reduce overall costs Tracking of costs can demonstrate the true costs of non-conformance, especially of rework Cost of quality and value-added are similar concepts but not identical

36. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 36 References Crosby, Philip B. Quality is Free, New York, McGraw-Hill, 1979. Deming, W. Edwards, Out of the Crisis, MIT Press, 1986, ISBN: 0911379010 Juran, J. M., Juran on Leadership for Quality: An Executive Handbook, The Free Press, 1989.

37. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2005, Dennis J. Frailey, All Rights Reserved CSE8314M11 version 5.09Slide 37 END OF MODULE 11