CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 1 SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering Module 15 Six Sigma and Zero Defects - Overview

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 2 Contents Zero Defects Overview Six Sigma Overview Summary

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 3 Zero Defects and Six Sigma General Concepts

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 4 Zero Defects and Six Sigma Each of these is a measure of quality And also a set of principles and concepts that can be used to establish a program for quality improvement

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 5 Goals A measure of quality that can be applied to anything you produce and that will result in unmatched quality across the board Dimensionless Fosters & motivates quality improvement Provides insight into the quality improvement process Correlates to our innate sense of quality

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 6 Zero Defects See Schulmeyer in reference list

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 7 Average Defects per Product Zero Defects is a Stretch Goal Stretch Goal No defects in delivered products Expectation Number of defects will diminish » (asymptotically approach zero) time

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 8 Experience with Zero Defects Many quit about here

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 9 Another Experience with Zero Defects

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 10 How Do You Know How Well You Are Doing Relative to What Is Possible? The “Zero Defects” measure is not directly related to “degree of goodness” or “degree of quality” or “what is possible” It only measures defects on an absolute scale

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 11 What is needed for some products is not needed for others -- you need to know what the customer requires How Do You Correlate the Defect Rate to What is Possible or to Relative Cost?

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 12 How Many Products Must Be Defect Free? 99% – would mean 1 typo per 100 words of course notes which is fairly good – but ,000 wrong drug prescriptions per year - very bad 99.9% – 1 typo per page - good – 500 surgical errors per week 99.99% – 2000 mail delivery errors per hour

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 13 What about Variance? Average product vs worst case product – Which one matters more? – Is it better to have an average product with.3 defects or to have a worst case product with 2 defects? In its simplest form, “zero defects” does not tell us the answer to this kind of question

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 14 Other Problems with Zero Defect Programs How do you gain insight into the nature of the problems? – The measure says nothing about the causes of the defects or how to cure them How do you justify continuous improvements in defect removal? – There is no good way to know if you can justify the cost

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 15 Six Sigma

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 16 Premise of Six Sigma Programs You use a process to produce something The process can vary as well as the product Average number of defects is not an acceptable measure... – You need to understand the worst case and why it happens – You need to control process variance as well as defects

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 17 Product Quality Depends on: 1) The design of the product 2) The materials used to construct it 3) The process used to produce it 1) Design 2) Materials 3) Production Process (Outputs) (Inputs) Products

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 18 Applying to Software For software, the product is essentially a design! So the three factors become: – Inputs: 1) The architecture of the software 2) The requirements of the software – Process: 3) The software development process

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 19 Software Quality Depends on: 1) The architecture of the software 2) The requirements used to construct it 3) The process used to develop it 3) Development Process 1) Architecture 2) Requirements (Outputs) (Inputs) Software

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 20 Basic Units of Six Sigma Programs Opportunity – Any step of the process, type of material, or design element that can cause a defect Defect – Anything wrong with the product Defects per million opportunities (DPMO) – This is the basic measure of quality

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 21 DPMO Defects are problems with the product Opportunities are steps in the process, materials, or design elements where you can make a mistake DPMO DPMO = Number of Defects * 10 6 Number of Opportunities

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 22 To Improve Quality Decrease the number of defects or Increase the number of opportunities? A common mistake when first learning about six sigma concepts is to assume that increasing opportunities is a good thing. In fact, it is a bad thing.

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 23 Count opportunities in such a way as to inflate the number, so that DPMO goes down. Make the process more complex so as to increase the number of opportunities Correct the process to reduce defects Simplify the process so that opportunities go down and total defects go down as well Uses and Abuses of DPMO

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 24 Software Defects & Opportunities A defect is a failure to meet requirements An opportunity is a process step, architecture element, or requirement where a defect could originate In a complex product like software, it is very hard to determine all of the opportunities

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 25 Characteristics of an Opportunity Independent from other opportunities Constant total number for the product/process if there is no change in the process Consistent measuring method is more important than exact definition The real objective is to measure & improve, not to count opportunities perfectly.

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 26 Software Opportunities One shortcut measure of opportunities is lines of code (if you measure size in lines of code) Another might be steps of the process, if your process description is very detailed down to the individual work step Do NOT get overly concerned about the definition of an opportunity or defect

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 27 Goals This goal is derived from a specific analysis of processes and has specific rationale behind it – Details in later slides and modules Other goals (more or less aggressive) can be established, based on understanding the rationale 6 Sigma goal: DPMO < 3.4

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 28 Techniques to Achieve 6 Sigma 1) Design the product for producability 2) Improve the quality of the materials 3) Design the process to produce quality products 1) Design 2) Materials 3) Production Process (Outputs) (Inputs) Products

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 29 Techniques to Achieve 6 Sigma Quality for Software 1) Architect the software for ease of development and maintenance 2) Improve the quality of the requirements 3) Design the development process to produce quality software 1) Architecture 2) Requirements 3) Development Process (Outputs) (Inputs) Software

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 30 Process Analysis for Manufactured Products Each product or raw material has certain measurable characteristics Example: a bolt could be measured in terms of: – Length – Diameter – Spacing of threads – Diameter of head – Weight – Strength of material under specific conditions

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 31 For Each Measurable Property You Establish Specifications Example: “Diameter should be 0.4 inches, plus or minus.002 inches” Notice that there are two components: – A target value specification – A range of values that are acceptable specification limits

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 32 Responsibility of Designer Specify tolerances Design products with reasonable tolerances relative to the manufacturing capability Create designs that can be produced within the stated tolerances

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 33 Responsibility of Software Architect Specify bounds of acceptable software behavior Architect software with reasonable tolerances relative to the design and programming capabilities of the staff Create software architectures that can be produced and maintained

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 34 SEI Claim As you increase maturity level, the cost and schedule decrease and the variance goes down

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 35 Problems with Software and Sigma Software generally has low product volume compared with manufactured products – But what if we measure units, tests, objects, screens, functions, etc? Software development process has very high variance – Does it need to? – Is that necessarily bad?

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 36 Problems with Software (continued) We don’t have the statistical data to back up applying these techniques – But maybe that does not matter if the ideas for improvement still apply NIH (Not Invented Here)

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 37 Summary Six Sigma and Zero Defects programs involve – Measures of quality, such as – Goals that minimize quality problems – Methods of quality improvement Six Sigma focuses on variability as well as overall quality The next module will cover six sigma principles and applications.

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 38 References Harry, Mikel J. and J. Ronald Lawson, Six Sigma Producibility Analysis and Process Characterization, Motorola University Press, Addison-Wesley, ISBN Schulmeyer, G. Gordon. Zero Defect Software. McGraw Hill, ISBN Schulmeyer, G. Gordon and James McManus. Handbook of Software Quality Assurance, Second Edition (especially chapter 17). Van Nostrand Reinhold, New York, ISBN

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M15 version 5.09Slide 39 END OF MODULE 15