1. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 1 SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering Module 01 Overview of Software Quality Engineering

2. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 2 Why is There So Much Ineffective Product Development? Organizations focus on cost or schedule... … instead of looking at the big picture

3. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 3 The “Zero-sum Game” Trap Pick Any Two Quality Productivity Cycle Time

4. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 4 The Secret to Effective Product Development Make the Process Efficient – Eliminate waste – Eliminate mistakes – This makes things faster, less costly, and higher in quality Avoid the mistake of seeing the problem as a zero sum game, such as: “to cut cost or save time you must reduce quality”; “to improve quality you must make the product more expensive.” Avoid the mistake of seeing the problem as a zero sum game, such as: “to cut cost or save time you must reduce quality”; “to improve quality you must make the product more expensive.”

5. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 5 Effective Quality Engineering is Fundamental to Productivity and Cycle Time Improvement Effective Product Development Quality Productivity Cycle Time

6. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 6 Any Banner Will Do Total Quality Management Total Cycle Time Productivity Enhancement Six Sigma

7. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 7 Defining Quality

8. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 8 Concepts of Quality Webster defines quality as: 1) “that which makes something what it is" 2) “the degree of excellence” But is this what we mean for software? 1) “our software is what it is - that makes it a quality product" 2) “the more perfect the software the higher the quality”

9. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 9 “That Which Makes it What it Is” e.g. Purity of tone is a quality of music – But perhaps not in certain musical styles – What defines the quality of “hard rock” music? Is quality in the ear of the beholder? Is there a universally accepted characteristic of musical quality? Is quality in the ear of the beholder? Is there a universally accepted characteristic of musical quality?

10. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 10 “Degree of Excellence” Which has higher quality: a Ferrari or a Toyota Corolla ??? Which has more prestige? Which costs less and leaves money for other expenses? Which is more reliable? Which weighs more?

11. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 11 Concepts of Quality for Products “Quality is conformance to requirements” Crosby “Quality is fitness for intended use” Juran “Quality is value to someone” Weinberg

12. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 12 “Quality is Conformance to Requirements” If testable requirements can be established, then it is possible to decide whether the product meets the criteria Thus you can avoid disputes and have workable contractual relationships HOWEVER...

13. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 13 Issues with “Conformance to Requirements” - I Who establishes the requirements? – Sponsor - The one who pays for the product – End User - The one who will use the product – Sales or Marketing - The one who will sell the product – Engineering - The ones who will design and build it

14. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 14 Issues with “Conformance to Requirements” - II Are the requirements right? – consistent – complete – correct Who determines whether the requirements are right? What if you discover a problem later on?

15. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 15 Issues with “Conformance to Requirements” - III What about implicit vs. explicit requirements? – E.g. coffee should be hot and flavorful – Implicit requirement: not poisonous Furthermore, requirements change during the development process – Who makes and who controls the change? – Who pays for the consequences of change?

16. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 16 “Quality is Fitness for Intended Use” This definition is based on a fundamental concept of law - that a product should be fit for the use that it is intended for This definition accommodates the fact that we may not be able to fully define the requirements HOWEVER...

17. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 17 Issues with “Fitness for Intended Use” - I Who defines fitness? – Consider a TV set -- which fitness characteristics are not understood by Typical User Engineer Sales Personnel – Consider a software program -- which fitness characteristics are not understood by the typical software developer?

18. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 18 Issues with “Fitness for Intended Use” - II Different users have different definitions of fitness – Ease of use for novices vs control of fine details for experts – vs ease of maintenance for support staff Uses change as users grow in experience – Too many “ease of use” and “automatic” features may frustrate an expert

19. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 19 Issues with “Fitness for Intended Use” - III The “pleasant surprise” concept – User gets more than he or she expected – “They really knew what they were doing” There is always a balance between the engineer knowing better than the customer and the customer knowing better than the engineer

20. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 20 “Quality is Value to Someone” This definition incorporates the idea that quality is relative And it places increased emphasis on understanding what quality means to the intended user of the software HOWEVER...

21. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 21 Issues with “Value to Someone” Whose opinion counts? – May need to weigh different opinions – May need to separate explicit from implicit views Logic vs Emotion – “Glitz” v. “Substance” What is it Worth? – Space Shuttle -- 0 defects – Video Game -- good user interface

22. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 22 Definitions of Software Quality IEEE: The degree to which the software possesses a desired combination of attributes Crosby: The degree to which a customer perceives that software meets composite expectations Note that both definitions imply multiple expectations

23. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 23 Software Quality Characteristics

24. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 24 Summary of Quality Definition Issues Define quality – You must define it to know if you have it – … and to engineer it into your product Quality has multiple elements – It reflects a multitude of expectations Quality is relative – Quality is in the eye of the customer Quality encompasses fitness, value, and other attributes

25. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 25 Question How does your organization define quality?

26. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 26 Quality Engineering

27. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 27 The Evolution of Quality Quality Engineering Quality Control Quality Assurance 1916todayfuture1950’s

28. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 28 Quality Control Preventing unacceptable products from being released to the customer Emphasis is on finding defects and fixing them after the fact. “A regulatory process through which we measure actual quality performance, compare with standards, and act on differences.” Juran “A regulatory process through which we measure actual quality performance, compare with standards, and act on differences.” Juran

29. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 29 Quality Control Goal: Keep Quality at an Acceptable Level by Rejecting Unacceptable Products Requirements DevelopmentQC Inspection Pass Fail Standards of Quality

30. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 30 Headrest Story - Part I: Independence Why go to college? I’ll get a job at an automobile assembly plant! My Brother

31. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 31 Headrest Story - Part II: Employment I found a quality control job on the assembly line... finding defective headrests.

32. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 32 Rejects My Brother Headrest Story - Part III: Excitement The highlight of my day!!! They switched from red to blue!

33. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 33 Headrest Story - Part IV: Quality Control QC Manager Production Manager Production rate is too low! You’re too picky! These are substandard! Pay more attention to the criteria!

34. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 34 Headrest Story - Part V: “Discussion” You’re a %#*@ *#&$% You! Discussion (as used in automobile assembly lines): Verbal communication characterized by extensive use of profanity and threats of bodily harm.

35. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 35 Headrest Story - Part VI: The Following Fall My Brother

36. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 36 Problems with Quality Control Does not reduce the number of defects Does not improve the process Does not result in better products Does not motivate improvement Results in adversarial relationships

37. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 37 Quality Assurance Assuring Product Quality: “Building Quality In” Providing evidence that the quality function is being performed adequately Juran Quality assessment and measurement Fisher/Baker

38. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 38 Quality Assurance “A planned and systematic pattern of all actions necessary to provide adequate confidence that the product conforms to established technical requirements” IEEE (George Tice) “A planned and systematic pattern of all actions necessary to provide adequate confidence that the product conforms to established technical requirements” IEEE (George Tice)

39. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 39 Software Quality Assurance Don Riefer These methods and procedures include: – Planning, measuring and monitoring of all work performed by software engineers, software testers, etc. A system of methods and procedures used to assure that the software product meets its requirements.

40. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 40 Quality Assurance Looks at the Entire Process Requirements DevelopmentQC Inspection Pass Fail Standards of Quality Process and Design Standards

41. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 41 Quality Assurance is More Effective than Quality Control...... because the emphasis moves to the development process You attempt to fix problems before and during the development process You improve the process and therefore reduce the number of defects in a lasting manner

42. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 42 But... Quality improvement is still separate from other process improvement and software development activities Adversarial relationships are still there – quality assurance vs. software developers – validating and testing vs. design and coding Motivation to improve is inconsistent It costs more to have people monitoring people

43. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 43 Quality Engineering Similar to quality assurance, but the responsibility shifts to everyone on the team Quality is built into the development process – Requirements, Design, Coding, Testing, etc. This is a very professional and responsible approach to software development

44. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 44 The Philosophical Change in QE Problems result in process changes, not punishment of people Finding errors is good -- it keeps them from leaking through to the customer Everyone appreciates that a competitive process is the way to remain a competitor Measurements are used so that decisions are based on fact (in addition to intuition)

45. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 45 Quality Engineering Requires a Cultural Change Pride in quality in addition to pride in product features or performance Professionalism rather than fear of criticism Overcoming the fear of metrics Seeing software development as much more than programming and design “We” rather than “They”

46. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 46 Quality Engineering Approach Build quality into the product as part of the development process – Measure quality – Understand quality – Improve quality Engineer the whole process for improvements in quality, productivity and cycle time (“Process Engineering”) A defined process is a must !!

47. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 47 Elements of Quality Engineering Understand process and its role Define value and quality - and focus on adding both of these to the product Manage process performance through programs such as six-sigma or zero defects or statistical process control Analyze the cost of quality Define and manage software reliability

48. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 48 Benefits of the Quality Engineering Approach Less adversarial Motivation and information to improve Flexibility to change the process in response to a problem – you understand the problem and its cause – you understand the consequences of a change in the process Knowledge is the foundation of successful quality engineering

49. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 49 Summary Product development is not a “zero sum game” Quality must be defined in terms of things that matter to customers Quality engineering focuses on the whole process and involves the whole project team

50. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 50 References Crosby, Philip B. Quality is Free, New York, McGraw-Hill, 1979. Deming, W. Edwards, Out of the Crisis, MIT Press, 1986, ISBN: 0911379010 Juran, J. M., Juran on Leadership for Quality: An Executive Handbook, The Free Press, 1989. Juran, J. M. and Frank M. Gryna, Quality Planning and Analysis, McGraw-Hill. 1980.

51. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 51 References (continued) Schwartz, James B., 1994, The Hunters and the Hunted, Productivity Press, ISBN 1-56327-043-9 Weinberg, Gerald M., 1992, Quality Software Management, Volume 1, Systems Thinking, Dorset House, New York, ISBN 0-932633-22-6.

52. CSE 8314 - SW Measurement and Quality Engineering Copyright © 1995-2003, Dennis J. Frailey, All Rights Reserved CSE8314M01 Version 3.09Slide 52 END OF MODULE 01