1. X36SSP CMM + metriky 8. přednáška Ing. Martin Molhanec ČVUT – FEL K13113

2. CMM Capability Maturity Model Model vznikl z potřeby hodnotit pro ministerstvo obrany USA softwarové firmy při výběrových řízeních na státní zakázky v počátku osmdesátých let. Byl vytvořen pracovníky Institutu softwarového inženýrství Carnegie Mellon univerzity v Pitsburgu. Vžil se pod označením CMM (Capability Maturity Model) nebo také pod označením SW-CMM, protože později začaly vznikat jeho modifikace pro různé jiné oblasti (PM-CMM pro projektové řízení a další). Model definuje firemní procesy v softwarových firmách do pěti úrovní.

3. The Capability Maturity Model (CMM) distinguishes between immature and mature software organizations. Immature software organizations are typically reactionary and have little understanding as to how to successfully develop software. Mature software organizations understand the development process, which enables them to judge the quality of the software products and the process that produces them. Mature software organizations have a higher success rate and a lower overall cost of software across the entire life of a software product than do immature software organizations. The CMM has been adopted by hundreds of organizations worldwide who want to improve the way they develop software. Capability Maturity Model provides a framework from which a process for large, complex software efforts can be defined The CMM defines five maturity levels, evolutionary plateaus toward achieving a mature software process, that an organization can attain with respect to the software process. Source: Software Engineering Institute Level 5: OptimizedWork the measures Level 4: ManagedMeasure the work Level 3: DefinedWork the plan Level 2: RepeatablePlan the work Level 1: InitialWorship the hero Maturity

4. Defect Prevention Technology change management Process change management Defect Prevention Technology change management Process change management Quantitative process management Software quality management Quantitative process management Software quality management Requirements management Software project planning Software project tracking and over sight Software subcontract management Software quality assurance Software configuration management Requirements management Software project planning Software project tracking and over sight Software subcontract management Software quality assurance Software configuration management Level 1: Initial Level 2: Repeatable Level 4: Managed Level 5: Optimizing Organization process focus Organization process definition Training program Integrated software management Software product engineering Intergroup coordination Peer reviews Organization process focus Organization process definition Training program Integrated software management Software product engineering Intergroup coordination Peer reviews Level 3: Defined CMM defines issues to be addressed The true value of the CMM is that it defines the issues that must be addressed by a software process to achieve each maturity level. These issues are called key process areas (KPA).

5. The software process for both management and development activities is documented, standardized, and integrated into a standard software process for the entire organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software. Detailed measures, called metrics, of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled Continuous process improvement is enabled by quantitative feedback from the software process and from piloting innovative ideas and technologies. The planning and management of new projects is based on experience with similar projects. Process capability is enhanced at the project level by establishing basic process management techniques Software requirements and deliverables are baselined Processes often differ between projects, reducing opportunities for teamwork and reuse The user community is provided visibility into the project at defined occasions, typically via the review and acceptance of major project deliverables, allowing limited management control Overcommitment si common During a crisis, planned procedures are abandoned and project teams revert to coding and testing Success depends on having an exceptional manager and seasoned and effective software team The software team process is effectively a black box to the user community. Resources go in and software potentially comes out LevelDescription 1. Initial The software process is ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics 2. RepeatableBasic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications 3. Defined A standard process is used, with possible tailoring, on all projects. Management has good insight into technical progress on the project Defined processes allow the user community greater visibility into the project and enable accurate and rapid status updates. 4. ManagedProductivity and quality are measured for important software process activities across all projects. The user community can establish an accurate, quantitative understanding of the software process capability of your organization/team and the project risk before the project begins. 5. OptimizedInnovations that exploit the best software engineering practices are identified and shared throughout the organization The software process is improved by changing common causes of inefficiency Disciplined change is the norm, not the exception. Features The five CMM maturity levels Capability Maturity Model 1. Initial Na této úrovni dominují nahodilé (ad hoc) procesy. Software je vytvářen bez firemních pravidel chaoticky, takže se jeho tvorba často dostává do kritických situací. Dosažený úspěch ve vývoji software je důsledkem šťastné náhody a závisí na individuálních schopnostech a znalostech programátorů. Dohodnuté termíny nejsou většinou dodržovány a ukončení vývoje je dosahováno nesmírným heroickým úsilím na konci projektu. Celkové náklady na projekt jsou sečteny po ukončení vývoje a posuzují se jako důsledek vývoje a nutných výdajů. O kvalitě software se sice hovoří, ale nijak se nezajišťuje. Každý spoléhá na toho druhého, že kvalitu zajistí.

6. The software process for both management and development activities is documented, standardized, and integrated into a standard software process for the entire organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software. Detailed measures, called metrics, of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled Continuous process improvement is enabled by quantitative feedback from the software process and from piloting innovative ideas and technologies. Overcommitment si common During a crisis, planned procedures are abandoned and project teams revert to coding and testing Success depends on having an exceptional manager and seasoned and effective software team The software team process is effectively a black box to the user community. Resources go in and software potentially comes out LevelDescription 1. InitialThe software process is ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics 2. Repeatable Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications 3. Defined A standard process is used, with possible tailoring, on all projects. Management has good insight into technical progress on the project Defined processes allow the user community greater visibility into the project and enable accurate and rapid status updates. 4. ManagedProductivity and quality are measured for important software process activities across all projects. The user community can establish an accurate, quantitative understanding of the software process capability of your organization/team and the project risk before the project begins. 5. OptimizedInnovations that exploit the best software engineering practices are identified and shared throughout the organization The software process is improved by changing common causes of inefficiency Disciplined change is the norm, not the exception. Features The five CMM maturity levels Capability Maturity Model The planning and management of new projects is based on experience with similar projects. Process capability is enhanced at the project level by establishing basic process management techniques Software requirements and deliverables are baselined Processes often differ between projects, reducing opportunities for teamwork and reuse The user community is provided visibility into the project at defined occasions, typically via the review and acceptance of major project deliverables, allowing limited management control 2. Repeatable Opakovaně se dosahuje dobrých výsledků. Firma využívá základních postupů projektového řízení, ale z projektu na projekt se přenášejí jen některé úspěšné prvky řízení. Nicméně intuitivně zaběhané procesy a povědomí, že je potřeba pracovat kvalitně, vytvářejí dost stabilní prostředí pro udržení přijatelné úrovně jakosti softwarových produktů.

7. The software process for both management and development activities is documented, standardized, and integrated into a standard software process for the entire organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software. Detailed measures, called metrics, of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled Continuous process improvement is enabled by quantitative feedback from the software process and from piloting innovative ideas and technologies. The planning and management of new projects is based on experience with similar projects. Process capability is enhanced at the project level by establishing basic process management techniques Software requirements and deliverables are baselined Processes often differ between projects, reducing opportunities for teamwork and reuse The user community is provided visibility into the project at defined occasions, typically via the review and acceptance of major project deliverables, allowing limited management control Overcommitment si common During a crisis, planned procedures are abandoned and project teams revert to coding and testing Success depends on having an exceptional manager and seasoned and effective software team The software team process is effectively a black box to the user community. Resources go in and software potentially comes out LevelDescription 1. InitialThe software process is ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics 2. RepeatableBasic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications 3. Defined A standard process is used, with possible tailoring, on all projects. Management has good insight into technical progress on the project Defined processes allow the user community greater visibility into the project and enable accurate and rapid status updates. 4. ManagedProductivity and quality are measured for important software process activities across all projects. The user community can establish an accurate, quantitative understanding of the software process capability of your organization/team and the project risk before the project begins. 5. OptimizedInnovations that exploit the best software engineering practices are identified and shared throughout the organization The software process is improved by changing common causes of inefficiency Disciplined change is the norm, not the exception. Features The five CMM maturity levels Capability Maturity Model 3. Defined Software je vyvíjen podle předem stanoveného postupu, metodicky, plánovitě, s využitím pokročilého projektového řízení s cílem dosáhnout vypracování požadovaného software v čase, s rozpočtovanými náklady a disponibilními zdroji. Provádí se pravidelné vyhodnocování odchylek od plánu a přijímají se opatření ke krácení termínů jednotlivých činností, aby software byl dodán včas. O kvalitu produktů a služeb se s ohledem na zákazníky explicitně usiluje, proto je kvalita softwaru dodržována na velmi dobré úrovni a má tendenci vykazovat určité zlepšování.

8. The software process for both management and development activities is documented, standardized, and integrated into a standard software process for the entire organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software. Detailed measures, called metrics, of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled Continuous process improvement is enabled by quantitative feedback from the software process and from piloting innovative ideas and technologies. The planning and management of new projects is based on experience with similar projects. Process capability is enhanced at the project level by establishing basic process management techniques Software requirements and deliverables are baselined Processes often differ between projects, reducing opportunities for teamwork and reuse The user community is provided visibility into the project at defined occasions, typically via the review and acceptance of major project deliverables, allowing limited management control Overcommitment si common During a crisis, planned procedures are abandoned and project teams revert to coding and testing Success depends on having an exceptional manager and seasoned and effective software team The software team process is effectively a black box to the user community. Resources go in and software potentially comes out LevelDescription 1. InitialThe software process is ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics 2. RepeatableBasic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications 3. Defined A standard process is used, with possible tailoring, on all projects. Management has good insight into technical progress on the project Defined processes allow the user community greater visibility into the project and enable accurate and rapid status updates. 4. Managed Productivity and quality are measured for important software process activities across all projects. The user community can establish an accurate, quantitative understanding of the software process capability of your organization/team and the project risk before the project begins. 5. OptimizedInnovations that exploit the best software engineering practices are identified and shared throughout the organization The software process is improved by changing common causes of inefficiency Disciplined change is the norm, not the exception. Features The five CMM maturity levels Capability Maturity Model 4. Manage Firma má všechny procesy jasně definovány a stanoven pro ně postup měření, který vyhodnocuje jejich podíl a efektivitu při tvorbě software. Zjištěné charakteristiky procesů jsou postupně upravovány tak, aby se firma přizpůsobila měnícím se podmínkám trhu, aniž by to mělo dopad na jakost vyvíjeného softwaru, jehož kvalitativní parametry jsou firmou cílevědomě stále zvyšovány a dosahují vysoké úrovně.

9. The software process for both management and development activities is documented, standardized, and integrated into a standard software process for the entire organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software. Detailed measures, called metrics, of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled Continuous process improvement is enabled by quantitative feedback from the software process and from piloting innovative ideas and technologies. The planning and management of new projects is based on experience with similar projects. Process capability is enhanced at the project level by establishing basic process management techniques Software requirements and deliverables are baselined Processes often differ between projects, reducing opportunities for teamwork and reuse The user community is provided visibility into the project at defined occasions, typically via the review and acceptance of major project deliverables, allowing limited management control Overcommitment si common During a crisis, planned procedures are abandoned and project teams revert to coding and testing Success depends on having an exceptional manager and seasoned and effective software team The software team process is effectively a black box to the user community. Resources go in and software potentially comes out LevelDescription 1. InitialThe software process is ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics 2. RepeatableBasic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications 3. Defined A standard process is used, with possible tailoring, on all projects. Management has good insight into technical progress on the project Defined processes allow the user community greater visibility into the project and enable accurate and rapid status updates. 4. ManagedProductivity and quality are measured for important software process activities across all projects. The user community can establish an accurate, quantitative understanding of the software process capability of your organization/team and the project risk before the project begins. 5. Optimized Features The five CMM maturity levels Capability Maturity Model Innovations that exploit the best software engineering practices are identified and shared throughout the organization The software process is improved by changing common causes of inefficiency Disciplined change is the norm, not the exception. 5. Optimizing Neustálá zpětná vazba ovlivňuje následné softwarové projekty tak, aby se firemní procesy neustále zlepšovaly a dosahovaly předem definovaných, optimálních parametrů. Firma dosahuje trvale špičkové jakosti, aniž by náklady na kvalitu softwaru měly dopad na hospodaření firmy. Naopak, garantovaná kvalita software usnadňuje jeho prodej.

10. CMM Model se poměrně rychle v průběhu osmdesátých let rozšířil a řada dalších univerzit a firem rozpracovávala a propracovávala jeho dílčí aspekty. Pro model byly definovány jednotlivé kroky, které umožňují postup na vyšší úroveň: z 1. na 2. úroveň - disciplína, zodpovědnost, pořádek, cílevědomost, z 2. na 3. úroveň - standardizace, stálost a provázanost firemních procesů, monitorování procesů, ze 3. na 4. úroveň - kritéria a měření procesů, řízení změn a využívání predikce vývoje procesů, ze 4. na 5. úroveň - neustálé zlepšování procesů, vícekriteriální optimalizace procesů, adaptibilita procesů.

11. CMM Rovněž pro jednotlivé úrovně byly definovány klíčové aspekty úspěchu (key process areas).

12. COCOMO Model The original COCOMO model was first published by Barry Boehm in 1981 at CSE Center for Software Engineering. This is an acronym derived from the first two letters of each word in the longer phrase Constructive Cost Model. The COCOMO cost estimation model is used by thousands of software project managers, and is based on a study of hundreds of software projects. The most fundamental calculation in the COCOMO model is the use of the Effort Equation to estimate the number of man-months required to develop a project. Most of the other COCOMO results, including the estimates for Requirements and Maintenance, are derived from this quantity. The original COCOMO model has been very successful, but it doesn't apply to newer software development practices as well as it does to traditional practices, but there is COCOMO II tuned to modern software life cycles. COCOMO II targets the software projects of the 1990s and 2000s. The COCOMO calculations are in the form of non-linear mathematical functions and are based on estimates of a project's size in Source Lines of Code (SLOC). Then the COCOMO model makes its estimates of required effort (measured in man-months) based on the estimate of the software project's size. From this value, the prediction of following values can be derived: project duration (typically in months) required to complete the software project, and required staffing (typically in FTE). Our approach Since all data were not available in the form required as inputs for the COCOMO method and only a basic information about Function Points and related project sizes in man-months were on hand, the man-months obtained from known Function Points were converted into its comparable COCOMO equivalent - the Effort value. This value was then used in the COCOMO equations to make all subsequent estimates. Our approach Since all data were not available in the form required as inputs for the COCOMO method and only a basic information about Function Points and related project sizes in man-months were on hand, the man-months obtained from known Function Points were converted into its comparable COCOMO equivalent - the Effort value. This value was then used in the COCOMO equations to make all subsequent estimates. Function Points / Feature Points The Function Point methodology was developed in 1979 by Allan Albrecht at IBM and was modified in 90’s as the Feature Point methodology by Jones. This methodology is based on the premise that the size of a software project can be estimated early, during the requirements analysis. If there are counted some requirement characteristics of the project, and then multiplied by weighting and adjustment factors, the sum is the total point count. The method used for estimation COCOMO is a model that allows one to estimate the cost, effort, and schedule when planning a new software development activity. Units counted through either Function Points or Feature Points represent the expected complexity and required effort of the project.

13. Software Project Size 100 FP1000 FP10000 FP Productivity rates in FP per man/month Function Points (Albrecht, 1979) is a metrics used for estimating the size of an application. The basic idea of Function Points is that five items are counted: 1) the inputs to the application 2) the outputs from it 3) inquiries by users 4) the data files that would be updated by the application, and 5) the interface to the application. Once these figures are known, they are multiplied by following coefficients: 4, 5, 4, 10, and 7 (these values can be modified from own experience, if required). The result indicates the estimated size of an application. Function Points worked reasonably for simple transaction processing or management information systems in 80’s, but unfortunately they often proved to be inadequate for estimating complex application such as information systems for high-tech industries. Therefore, there exists the need for Feature Points (Jones, 1995), a superset of classical FP, to correct this problem. Feature Points introduces new measurement, algorithms, giving it a weighting of 3. Additionally, the data file weighting is decreased from 10 to 7. > 100 75 - 100 50 - 75 25 - 50 15 - 25 5 - 15 1 - 5 < 1 (Jones) Source code statements required for one FP Assembler low level: C classical: Fortran, Cobol structured: Pascal, RPG, PL1 hybrid object oriented: Modula-2, C++ non imperative: Lisp, Prolog 4GL languages obj. oriented: VB, Java, C# pure object oriented: Smalltalk, CLOS query languages: SQL, QBE, OQL Programming Language (Jones) 320 150 106 80 71 64 40 32 21 16 Complexity of applications in function points 1.0%0.01%0.0% 3.0%0.1 %0.0% 7.0%1.0%0.0% 15.0%5.0%0.1% 40.0%10.0%1.4% 25.0%50.0%13.5% 10.0%30.0%70.0% 4.0%4.0%15.0%

14. CMM Level NumberRecommended Number of Metrics 1234512345 0 - 11 11 - 22 22 - 30 30 - 36 36 - 50 The need for Metrics in CMM Metrics management is used to potentially improve both the quality of the software that is developed or maintained and also to improve the actual software processes of the organization itself. Metrics management provides the information critical to understanding the quality of the work that the project team has performed, and how effectively that work was performed. Software companies are used to collect metrics because this is the only proven tool to obtain managerial information for software development and quality assurance In the suggested Capability Maturity Model (CMM), there are recommended numbers of metrics to be collected and analyzed. Optimal frequency of collect the recommended metrics is weekly (or earlier, if there is a tool), and optimal frequency of analyze metrics is between weekly and monthly. Why to Collect Metrics?  To support greater predictability and accuracy of schedules and estimates.  To support quality assurance efforts by identifying the techniques that work best and the areas that need more work.  To support productivity and process improvements efforts by measuring how efficient the IT staff is and by indicating where the IT staff need improve.  To improve management control by tracking both projects and people.  To improve the motivation of developers by making them aware of what works and what does not.  To improve communications by describing accurately what is happening, and by identifying trends early on. Metrics management is the process of collecting, summarizing and acting on measurements

15. StagePotential Metrics Number of use case Function/feature points Level of risk Project size Cost/benefit breakpoint Number of reused infrastructure artifacts Number of introduced infrastructure artifacts Requirements instability Procedure/Operation count of a module Number of variables per data structure Size of procedures/operations Number of data types per database Number of data relationships per database Number of interfaces per module Requirements instability Procedures/Operations size Procedures/Operations response Comments per procedure/operation Percentage of commented procedures/operations Global usage Number of candidate items for generalization Percentage of items generalized Effort required to generalize an item Percentage of deliverables reviewed Time to fix defects Defect recurrence Defect type recurrence Define Requirements and Justify Define Management Documents Model Program, Generalize and Test Family of potential SW metrics Recommended metrics are printed in bold face.

16. StagePotential Metrics Time to fix defects Defect recurrence Defect type recurrence Number of defects Defect source count Work effort to fix a defect Percentage of items reworked Enhancements implemented per release Amount of documentation Percentage of customers trained Average training time per user trainer Number of lessons learned Percentage of staff members assessed Average response time Average resolution time Support request volume Support backlog Support request aging Support engineer efficiency Reopened support requests Mean time between failures Software change requests opened and closed Deliver and Test Assess Support Family of potential SW metrics Recommended metrics are printed in bold face.

17. MetricsDescription Calendar time expended Overtime Staff turnover Work effort expended This productivity metric is a measure of the number of people gained and lost over a defined period of time, typically a calendar month. This metric is often collected by position/role and can be used to help define human resources requirements and the growth rate of your information technology (IT) department. This cost metric is a measure of the amount of work expended to complete a task, typically measured in work days or work months. This metric is often collected by task (see above). This cost metric is a measure of the amount of overtime for a project and can be collected as either an amount of of workdays or a percentage of overall effort. This metric is often subdivided by overtime for work directly related to development, such as programming or testing, and by indirect effort, such as training or team coordination. This schedule metric is a measure of a calendar time to complete a task, where a task can be as small as the creation of a deliverable or as large as a project phase or an entire project. Metrics Applicable to All of the Phases and Stages

18. CategoryDescription Cost metrics measure the amount of money invested in a project. Examples of cost metrics include effort expended or investment in training. Cost Productivity metrics measure the effectiveness of the organization`s infrastructure. Examples of productivity metrics include the number of reused infrastructure items and trend analysis of effort expended over given periods of time. Productivity Quality metrics measure the effectiveness of the quality assurance efforts. Examples of quality assurance metrics include the percentage of deliverables reviewed and defect type recurrence. Quality Requirements metrics measure the effectiveness of the requirements definition and validation efforts. Examples of requirements metrics include requirements instability and number of use cases. Requirements Schedule metrics measure the accuracy of your proposed schedule to the actual schedule. Examples of schedule metrics include the calendar time expended to perform a task or project phase. Schedule Size metrics measure, as the name suggests, the size of the development efforts. Examples of size metrics include the number of methods of a class and the function/feature point count. Size Testing metrics measure the effectiveness of the testing efforts. Examples of testing metrics include the defect severity count and the defect source count. Testing Metric categories

19. Závěr Bez metrik nelze provádět kvalitně řízený softwarový proces nemluvě o jeho optimalizaci. CMM nám ukazuje, jak na tom jsme s úrovní řízení softwarového procesu! Vývoj rozsáhlých ICT produktů si žádá vědomé řízení celého projektového procesu!