1. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 SMU CSE 8314 Software Measurement and Quality Engineering Module 04 Software Process Maturity

2. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 2 Models of Process Maturity  There are many models of process maturity  The original concept was developed in the 1930’s  One variant is the four quadrant scale of competence and consciousness  Philip Crosby applied the concept to develop his “Five Patterns of Management Attitude”

3. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 3 Software Process Maturity  Watts Humphrey was inspired by Crosby’s model to produce the “Five Levels of Organizational Process Maturity”  Gerald Weinberg identified “Six Patterns of Software Quality Culture”

4. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 4 Four Quadrant Model of Organizational or People Competence 1) Unconscious Incompetence You are not competent You are not aware that you are not competent 2) Conscious Incompetence You are not competent But you are aware of it 3) Conscious Competence You are competent You are aware of it as you execute 4) Unconscious Competence You are competent You are not aware of it because you have internalized it ConsciousConscious UnconsciousUnconscious IncompetenceCompetence

5. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 5 Four Quadrant Model of Organizational or People Competence 1) Unconscious Incompetence You are not competent You are not aware that you are not competent 2) Conscious Incompetence You are not competent But you are aware of it 3) Conscious Competence You are competent You are aware of it as you execute 4) Unconscious Competence You are competent You are not aware of it because you have internalized it ConsciousConscious UnconsciousUnconscious IncompetenceCompetence

6. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 6 Four Quadrant Model of Organizational or People Competence 1) Unconscious Incompetence You are not competent You are not aware that you are not competent 2) Conscious Incompetence You are not competent But you are aware of it 3) Conscious Competence You are competent You are aware of it as you execute 4) Unconscious Competence You are competent You are not aware of it because you have internalized it ConsciousConscious UnconsciousUnconscious IncompetenceCompetence

7. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 7 Four Quadrant Model of Organizational or People Competence 1) Unconscious Incompetence You are not competent You are not aware that you are not competent 2) Conscious Incompetence You are not competent But you are aware of it 3) Conscious Competence You are competent You are aware of it as you execute 4) Unconscious Competence You are competent You are not aware of it because you have internalized it ConsciousConscious UnconsciousUnconscious IncompetenceCompetence

8. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 8 Four Quadrant Model of Organizational or People Competence 1) Unconscious Incompetence You are not competent You are not aware that you are not competent 2) Conscious Incompetence You are not competent But you are aware of it 3) Conscious Competence You are competent You are aware of it as you execute 4) Unconscious Competence You are competent You are not aware of it because you have internalized it ConsciousConscious UnconsciousUnconscious IncompetenceCompetence

9. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 9 Four Quadrant Model of Organizational or People Competence 1) Unconscious Incompetence You are not competent You are not aware that you are not competent 2) Conscious Incompetence You are not competent But you are aware of it 3) Conscious Competence You are competent You are aware of it as you execute 4) Unconscious Competence You are competent You are not aware of it because you have internalized it ConsciousConscious UnconsciousUnconscious IncompetenceCompetence

10. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 10 Philip Crosby’s Five Patterns of Management Attitude  Uncertainty - Do not understand the task  Awakening - Understand the problems, not the solutions  Enlightenment - Understand HOW to solve known problems  Wisdom - Understand WHY the solutions work  Certainty - Can solve unexpected problems

11. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 11  Initial - Depend on heroes; Ad-hoc and chaotic; people are herded  Repeatable - Can duplicate past successes with typical people; people are managed  Defined - Processes are documented and tailored; stick with processes in a crisis  Managed - Processes are understood based on measurement; institutionalized & internalized  Optimizing - Processes are continuously improved Humphrey’s Five Levels of Software Process Maturity

12. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 12 Weinberg’s Six Patterns of Software Quality Culture - I  Oblivious - Don’t know they are following a process, and don’t really care either. Don’t see problems.  Variable - Do what feels right. Rely on intuition.  Routine - Realize that management is needed but don’t spend enough money or time to do it professionally. Work hard and follow procedures / processes that have worked in the past.

13. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 13 Weinberg’s Six Patterns of Software Quality Culture - II  Steering - Management understands what they are doing - both how to manage and how to develop software. They follow processes in routine times and in crises.  Anticipating - Organization measures & analyzes. Management by fact. The organization knows what is coming & is prepared for it.  Congruent - Focus is on quality management. Continuous improvement based on knowledge and facts.

14. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 14 Rough Comparison of Levels 0 1 2 3 4 5 Crosby’s Five Patterns of Management Humphrey's Five Levels of SW Process Maturity Traditional Model of Competence Weinberg’s Six Patterns of SW Quality Culture Oblivious Unconscious Incompetence Conscious Incompetence Conscious Competence Unconscious Competence Uncertainty Awakening Enlightenment Wisdom Certainty Initial Repeatable Defined Managed Optimizing Variable Routine Steering Anticipating Congruent LevelLevel

15. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 15 When are you Successful at Each Level? 0 - When it isn’t very important what the quality is. 1 - When a new and exciting idea is the key thing “Creation Myths” of new companies; must have heroes 2 - Under normal, non-stressful situations Good people, strong leaders, good tools, stable environment & market

16. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 16 When are you Successful at Each Level? (continued) 3 - Most of the time, including under stress conditions, except when totally new situations come up This is the time to automate -- when you know WHAT to automate 4 - Virtually always But beware of unconscious competence sliding into unconscious incompetence as the environment changes 5 - Always (in theory)

17. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 17 What Level Are You At If You Never Have Cost Overruns?  Could mean a high level  Could mean a low level –Perhaps you have a consistent, low-risk environment where it is easy to succeed at a low maturity level –Perhaps you are in serious trouble and don’t know it  Best buggy whip factory in the world

18. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 18 Average vs. Worst Case Performance  If you average 10 weeks per project, what is the risk of promising delivery in 10 weeks? –50% risk of taking more than 10 weeks –Yet how many people make cost, schedule and quality estimates based on their average performance?  You need to know more than the average to make a good estimate

19. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 19 You Need to Know More than the Average  The best and worst cases  How much you vary from the average  Seek predictability when you can find it

20. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 20 Variability can tell you a Lot High Variability = Lack of Predictability = Low maturity Low Variability = High Predictability = High Maturity months required % of total projects % of total projects

21. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 21 Improving Process Maturity and Quality

22. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 22 Improving Process Maturity and Quality  Our goal is a more predictable, mature process so we can confidently estimate our costs, schedules & quality.  The question is how to do this.  The answer is to use the techniques of quality engineering

23. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 23 Start with Culture and Behavior  In most organizations, improvement begins with CHANGING THE BEHAVIOR – of the people & – of the organization

24. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 24 The Challenge  Basic requirement: –Change patterns and behaviors  Basic problem: –Those changing the culture are part of it –They are ignorant of much of the culture “Pigs don’t know pigs stink” (Weinberg)  But outsiders are ignorant of much of what is good –And they are likely to be mistrusted

25. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 25 The Approach  Look at the situation from an outsider’s perspective  Educate the outsider  Educate the insiders  Blend the best of both -- outsider perspective and insider knowledge  This depends on two key factors: –Education –Willingness to be educated

26. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 26 Outside View vs. Inside View (Level 0 - Oblivious)

27. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 27 Outside View vs. Inside View (Level 1 - Variable)

28. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 28 Outside View vs. Inside View (Level 2 - Routine)

29. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 29 Outside View vs. Inside View (Level 3 - Steering)

30. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 30 Outside View vs. Inside View (Level 4 - Anticipating)

31. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 31 Outside View vs. Inside View (Level 5 - Congruent)

32. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 32 So How do we Accomplish Change?  Even internal change agents tend to be distrusted by the organization  Change management skills are important –But beyond the scope of this course  We will focus on the technical side of the change process

33. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 33 The Role of Models in Change  We begin with understanding the role of models in achieving change –Crosby’s, Humphrey’s, Weinberg’s, models help us to understand maturity –Other models can help understand more about our processes, organizations, cultures and products  Models support communication –Common framework for discussion –Basis for extrapolating behavior

34. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 34 Precision Required of Models Depends on Level of Maturity Level 1 or 2: –No stable base, so no need for precision. –Need basic models of processes and organization. Level 3: –Can begin simulation models because of stability

35. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 35 Higher Maturity Levels Call for More Precision in Models Levels 4 and 5: –Can begin precise, formal models due to measurements and data

36. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 36 Models Are Powerful... Which Makes Them Dangerous  At each level of maturity, there are potential uses and abuses of models –E.g., using a model to override common sense –Or using the model to predict things the model is not capable of predicting

37. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 37 Frailey’s Rule of Models If you do not understand how the model works, beware of using the model to make important decisions Corollary: study models, learn how they work, then use them

38. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 38 The Benefits of Models  They show up differences in concepts  They facilitate team building and joint approaches  They help new staff catch on quickly  They provide a record to compare plans with actuals –Basis for improvement –Basis for learning  They can be a tool for creativity

39. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 39 Use Models to...  Define where you are –And help determine if this is OK  Define where you want to be  Define the steps to get there  Perform “what if” exercises  Document/record your decisions & plans  Train newcomers  Communicate

40. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 40 SEI Capability Maturity Model (CMM) and Integrated Maturity Model (CMMI)  Derived from Humphrey’s 5-level model  Each level has 2 to 7 Key Process Areas (KPAs)  KPAs are process skills that must be mastered in order to achieve a given level

41. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 41 KPAs at SEI CMM Level 2  Software Requirements Management  Software Configuration Management  Software Quality Assurance  Software Subcontractor Management  Product Tracking and Oversight  Project Planning We will address this in more detail later

42. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 42 Self-Assessments - I  Used to assist in determining “where you are”  A team of about 5 people gathers information about the organization –Survey –Follow-up questions –All organizational levels are covered from senior management to practitioners

43. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 43 Self-Assessments - II  Information is evaluated against the level 2 goals and key process areas. –Only if these are positive does the team look at higher levels.  Findings are not limited to KPAs  Findings are reviewed before presentation to assure organizational buy-in.  Follow-up action plan and its execution are essential for success of this method.

44. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 44 But Is it Always Good to Move Higher in Maturity?  It depends on the organization –Its business –Its goals –Its resources –Its focus In the next module, we will discuss Weinberg’s method of analyzing an organization to determine the most appropriate level of maturity. We will also discuss why organizations resist improvements.

45. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 45 Summary  There are several models of process maturity  Improving maturity can reduce variability and increase consistency and confidence  Use models to guide your improvement process  But don’t use models you don’t understand

46. Copyright 1995-2007, Dennis J. Frailey CSE8314 - Software Measurement and Quality Engineering CSE8314 M04 - Version 7.09 46 END OF MODULE 04