2. Topic 14Summer 2003 1 ICS 52: Introduction to Software Engineering Lecture Notes for Summer Quarter, 2003 Michele Rousseau Topic 14

3. Summer 2003 2 Process Improvement l Capability Maturity Model l ISO 9000 l PSP

4. Topic 14Summer 2003 3 l Understanding existing processes l Introducing process changes to achieve organisational objectives l usually focused on: quality improvement cost reduction schedule acceleration l Most work so far has focused on defect reduction to improve Quality l Testing can’t do it all Process improvement

5. Topic 14Summer 2003 4 l Process analysis Model and analyse (quantitatively if possible) existing processes l Improvement identification Identify quality, cost or schedule bottlenecks l Process change introduction Modify the process to remove identified bottlenecks l Process change training Train staff involved in new process proposals l Change tuning Evolve and improve process improvements Process improvement stages

6. Topic 14Summer 2003 5 The process improvement process

7. Topic 14Summer 2003 6 l Process quality  product quality These are closely related l A good process is usually required to produce a good product l For manufactured goods, process is the principal quality determinant l For design-based activity, other factors are also involved especially the capabilities of the designers Process and product quality

8. Topic 14Summer 2003 7 Principal product quality factors

9. Topic 14Summer 2003 8 What is CMM? l Capability Maturity Model l Developed by the software community in 1986 with leadership from the SEI. l Has become a de facto standard for assessing and improving processes related to software development l Has evolved into a process maturity framework l Provides guidance for measuring software process maturity l Helps establish process improvement programs

10. Topic 14Summer 2003 9 What is the Software CMM? l “A common-sense application of process management and quality improvement concepts to the software development and maintenance” l A model for organizational improvement

11. Topic 14Summer 2003 10 Software Capability Maturity Model. Maturity Levels Process Capability Indicate Key Process Areas Goals Contain Achieve Common Features Implementation Organized by Address Key Practices Activities Contain Describe CMU/SEI-93-TR-24 p. 29

12. Topic 14Summer 2003 11 What makes up the CMM? l The CMM is organized into five maturity levels: l Initial l Repeatable l Defined l Manageable l Optimizing l Except for Level 1, each maturity level decomposes into several key process areas that indicate the areas an organization should focus on to improve its software process.

13. Topic 14Summer 2003 12 Each Maturity Level l 1. Initial : »ad hoc process. Success depends on individual effort. l 2. Repeatable : »Basic management processes: cost, schedule and functionality l 3. Defined : »Activities are documented, standardized and integrated into an organization-wide software process. l 4. Managed : »Detailed measures are collected: software and product quality. l 5. Optimizing : »Continuous process improvement: quantitative feedback from the process and from testing new ideas and technologies.

14. Topic 14Summer 2003 13 Five Levels of Software Process Maturity Initial Optimizing Managed Defined Repeatable Unpredictable and poorly controlled Focus on process improvement Process measured and controlled Process characterized, fairly well understood Can repeat previously mastered tasks Disciplined process Continually improving process Predictable process Standard, consistent process Level 1 Level 2 Level 3 Level 4 Level 5

15. Topic 14Summer 2003 14 Key Process Areas KPAs associated with each maturity level describe functions that must be present to satisfy good practice at a particular level. (Except Level 1) Each KPA is described by: Goals – Overall objectives Commitments – Requirements that must be met to achieve the goals. Abilities – Things that must be in place to enable the organization to meet the commitment. Activities – Specific tasks required to achieve the KPA function Methods for monitoring implementation Methods for verifying implementation Each KPA is defined by a set of key practices that contribute to satisfying its goals. (ie policies, procedures, and activities)

16. Topic 14Summer 2003 15 Level 2 KPAs - Purpose * Paraphrased

17. Topic 14Summer 2003 16 Level 2 KPAs - Purpose (Cont.) * Paraphrased Changes in commitments are agreed to by affected groups All activities are planned and tracked

18. Topic 14Summer 2003 17 Level 3 KPAs - Purpose * Paraphrased

19. Topic 14Summer 2003 18 Level 3 KPAs - Purpose (Cont.) * Paraphrased

20. Topic 14Summer 2003 19 Level 4 KPAs - Purpose * Paraphrased

21. Topic 14Summer 2003 20 Level 5 KPAs - Purpose * Paraphrased

22. Topic 14Summer 2003 21 Interesting CMM Facts  The number of companies using CMM to assess their software management practices more than doubles every five years (since 1987).  Software Quality Assurance is the biggest obstacle for organizations trying to move from level 1 to level 2.  Organization Process Definition is one of the biggest obstacles for organization trying to move from level 2 to level 3.

23. Topic 14Summer 2003 22 … more interesting facts  On average, it takes an organization: »25 months to move from level 1 to 2 »22 months to move from level 2 to 3 »36.5 months to move from level 3 to 4  About a third of companies engaged in CMM are located overseas (primarily India), and are 3 times more likely to reach CMM level 4 or 5 than US organizations.  Only about 23% of organizations surveyed eventually move from level 2 to level 3 or higher.

24. Topic 14Summer 2003 23 What improved maturity provides Source: Master Systems, Inc. Based on data from 1300 applications, average 200,000 LOC

25. Topic 14Summer 2003 24 Characteristics of Immaturity l Software process improvised during the course of a project. l Even if process is specified, it is not rigorously followed or enforced. l Reactionary, focus on solving immediate crises. l Hard deadlines often mean a compromise in functionality and/or quality. l No objective basis for judging product quality or for solving process problems. l Quality is difficult if not impossible to predict.

26. Topic 14Summer 2003 25 Characteristics of Maturity l Able to manage software development and maintenance organization/project wide. l There is a prescribed, mandated, and enforced process. l Process is consistent with the way that work actually gets done. l Process is updated and improved as necessary. l Roles and responsibilities within the process are clear. l Quality is measured and monitored, and an objective basis for judgment exists. l The necessary infrastructure for supporting the process exists. l Workers see the value in the process.

27. Topic 14Summer 2003 26 Greater Maturity Can Bear Fruit l SE division of Hughes aircraft spent @$500K over a three year period for assessment and improvement programs. By the end of the three year period, assessed at CMM Level 3. Estimated savings of @$2M annually as a result (less overtime, less rework, greater productivity, etc.) l Equipment Division of Raytheon rise to CMM Level 3, at an estimated cost of @$580K resulted in 2-fold increase in productivity along with savings of @$15.8M in rework costs. l Motorola GED (CMM Level 4) documented significant reduction in cycle time reduction in defect rates increase in productivity

28. Topic 14Summer 2003 27 ISO 9000 - Background l International set of standards for quality management (ISO 9000:2000, ISO 9001:2000, ISO 9004:2000, etc.) l ISO is name adopted by the International Organization for Standardization (not an acronym) – comes from isos “equal” l ISO 9000 is the most popular quality standard in the world l Over 13,000 standards issued since 1946 l Made up of representative bodies from over 140 countries l It applies to almost all types of organizations regardless of their function or product.

29. Topic 14Summer 2003 28 From “The Dilbert Principle”

30. Topic 14Summer 2003 29 ISO 9000 – What is it? l ISO 9000 is primarily concerned with "quality management". l ISO 9001:2000 specifies requirements for a quality management system for any organization that needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements and aims to enhance customer satisfaction, in all business sectors l It involves the development of a quality system that meets the quality requirements of the ISO standards.

31. Topic 14Summer 2003 30 Contents of Software Quality Plan from ISO9000 l Management responsibility l Quality system l Contract review l Design control l Quality control l Purchasing l Customer supplied info l Configuration management l Process control l Inspection and testing l Inspection and testing equipment l Control of non-conforming product l Corrective action l Handling, storage, packing and delivery l Quality records l Internal quality audits l Training l Software maintenance l Statistical techniques l Control of the development environment

32. Topic 14Summer 2003 31 ISO 9000 series of standards: l represents an international consensus on good management practices l guidelines on what constitutes an effective quality management system l serves as framework for continuous improvement

33. Topic 14Summer 2003 32 ISO 9000 and quality management

34. Topic 14Summer 2003 33 ISO 9000 - Certification l Quality standards and procedures should be documented in an organisational quality manual l External body may certify that an organisation’s quality manual conforms to ISO 9000 standards (namely ISO 9001) l Customers are, increasingly, demanding that suppliers are ISO 9000 certified

35. Topic 14Summer 2003 34 ISO vs CMM l CMM and the ISO 9000 series of standards share common concerns with quality and process management. l CMM emphasizes continuous improvement l ISO deals with minimum criteria of quality systems l There is a clear correlation between the key processes in the CMM and the quality management processes in ISO 9000 l ISO 9000 has little explicit support for continuous improvement

36. Topic 14Summer 2003 35 ISO vs CMM (2) l The CMM is more detailed and prescriptive and includes a framework for improvement l An ISO 9001-compliant organization would not necessarily satisfy all of the CMM level 2 key process areas (it would satisfy most of the level 2 goals and many level 3 goals. l Organisations rated as level 2 in the CMM are likely to be ISO 9000 compliant

37. Topic 14Summer 2003 36 ISO9000 and CMM compared CMMISO 9001 l Specific to software developmentIntended for most industries l Used in USA, less widely Recognised and accepted in most elsewhere countries l Provides detailed and specific Specifies concepts, principles and definition of what is required safeguards that should be in place for given levels l Assesses on 5 levelsEstablishes one acceptable level l CMM Level 2 - 3  ISO 9000 l Relevant to Stabilises the customer - supplier s/w development processrelationship l No time limit on certificationCertification valid for three years l No ongoing auditAuditors may return for spot checks during the lifetime of the certificate

38. Topic 14Summer 2003 Personal Software Process (PSP) l Developed by SEI in 1994 l A measurement and analyses framework to help you characterize your process l A defined procedure to help you to improve your performance l PSP principles System quality depends on the quality of its worst components Component quality depends on individual developers

39. Topic 14Summer 2003 38 Overview of CMM and PSP l CMM sets out the principal practices for managing the processes in large-scale software development l PSP sets out the principal practices for defining, measuring and analysing an individual’s own processes

40. Topic 14Summer 2003 39 PSP l PSP applies a CMM-like assessment for individual work Measurement & analysis framework to help you characterize your process »Self-assessment and self-monitoring Prescribes a personal process for developing software »defined steps »Forms »Standards Assumes individual scale & complexity »Well-defined individual tasks of short duration

41. Topic 14Summer 2003 PSP Overview l The PSP is introduced in 7 upward compatible steps (4 levels) l Write 1 or 2 small programs at each step Assume that you know the programming language l Gather and analyze data on your work Many standard forms & spreadsheet templates l Use these analyses to improve your work Note patterns in your work

42. Topic 14Summer 2003 PSP Evolution PSP0 Current process Time recording Defect recording Defect type standard PSP1 Size estimating Test report PSP2 Code reviews Design reviews PSP3 Cyclic development PSP2.1 Design templates PSP1.1 Task planning Schedule planning PSP0.1 Coding standard Size measurement Process improvement proposal (PIP) Baseline Personal Process Cyclic Personal Process Personal Quality Management Personal Planning Process

43. Topic 14Summer 2003 PSP Evaluation l Humphrey has used in SE courses Improvements in time-to-compile, quality and productivity l Patchy, but promising use in industry E.g. Nortel (Atlanta) l Still immature l Requires large overhead for data gathering Not clear that you should use permanently or continually

44. Topic 14Summer 2003 43 PSP/TSP/CMM