Presentation is loading. Please wait.

Presentation is loading. Please wait.

Rick Hefner. Marilee J. Wheaton TRW

Similar presentations

Presentation on theme: "Rick Hefner. Marilee J. Wheaton TRW"— Presentation transcript:

1 Using Process Improvement and Knowledge Management for Better Predictive Analysis Capability
Rick Hefner Marilee J. Wheaton TRW One Space Park - R2/ Redondo Beach, CA 90278 This paper discusses TRW's approach for capturing and using project performance data. Metrics data are used for on-going assessments of the progress of the software activities as measured against documented plans and for planning improvements to project processes. Mature organizations typically maintain a repository of metrics data across projects, for use in establishing software process and product thresholds/goals and to facilitate process improvements at the organizational level. The Project Comparative Data Base (PCDB) is TRW’s central repository for historical data, including work descriptions, labor, subcontractor, and other direct cost data. This data is used to calibrate parametric software cost models such as COCOMO, Price-S, and SEER/SEM. The calibrated cost models form a basis for estimating, analyzing and predicting process performance. A corporate estimation staff maintains the PCDB database and cost estimation tools. TRW is using this data to set quantitative quality goals for both software products and processes. Productivity and quality are measured for important software process activities across all projects as part of the organizational measurement program. These measurements establish the quantitative foundation for evaluating the projects' software processes and products. Projects achieve control over their products and processes by narrowing the variation in their process performance to fall within acceptable quantitative boundaries. Meaningful variations in process performance can be distinguished from random variation (noise), particularly within established product lines. The risks involved in moving up the learning curve of a new application domain are known and carefully managed. The software process capability is predictable; the process operates within measurable limits. This level of process capability allows an organization to predict trends in process and product quality within the quantitative bounds of these limits. When these limits are exceeded, action is taken to correct the situation. Software products are of predictably high quality. The description of the PCDB design and usage will be of interest to those establishing corporate metrics repositories, those seeking to improve project performance, and those developing technology and tools to aid these tasks. Presented to Sixteenth International Forum On COCOMO and Software Cost Modeling

2 Agenda Motivation for Improvement
CMMI Level 4 Requirements and Expectations Organizational Metrics Process Database Design and Implementation Lessons Learned Systems Architecture Vision 16th COCOMO Forum

3 Motivation for Improvement to Level 4
Level 3 ensures well-defined, repeatable processes Competition demands better quantitative understanding of the contributors to cost and schedule Process productivity Product quality (which drives rework) Needed better measures, data, and analytic techniques for critical process and product characteristics: Determine whether processes are behaving consistently or have stable trends (i.e., are predictable) Identify improvement in TRW’s standard processes Identify project practices which may be best practices Understand the cost-quality-schedule tradeoffs 16th COCOMO Forum

4 How Does Six Sigma Fit with ISO 9001 and CMMI?
Process Improvement Quality Mgmt. Six Sigma Best-Practices ISO 9000 Business Measures Voice of the Customer Change Management SW CMM ISO 9001 Methods & Tools DMAIC DFSS Process Management CMMI Group and Corporate Initiatives Winter’s spectrum article Newspaper article – recent Cote speech Linda Mills appointment Capability Maturity Model Initiative (CMMI) and ISO 9001 establish the change management and process management framework needed for Six Sigma Six Sigma methods and tools assist in the quantitative analysis needed at CMMI Levels 4/5 16th COCOMO Forum

5 CMMI-SE/SW Staged Representation
Level Focus Process Areas Continuous process improvement Causal Analysis and Resolution Organizational Innovation and Deployment 5 Optimizing 4 Quantitatively Managed Quantitative management Quantitative Project Management Organizational Process Performance Organizational Process Focus Organizational Process Definition Organizational Training Integrated Project Management Risk Management Decision Analysis and Resolution Requirements Development Technical Solution Product Integration Verification Validation Process standardization 3 Defined Requirements Management Project Planning Project Monitoring and Control Supplier Agreement Management Measurement and Analysis Process and Product Quality Assurance Configuration Management 2 Managed Basic project management 1 Performed 16th COCOMO Forum

6 Level 4 Changes in the CMMI
SW-CMM process areas are split by process and product quality Quantitative Process Mgmt Identify and correct special causes of process variation Software Quality Mgmt Develop a quantitative understanding of the quality of the project's software products CMMI process areas are split by organization and project Organizational Process Performance Maintain a quantitative understanding of the performance of the organization’s set of standard processes, and provide the process performance data, baselines, and models to quantitatively manage the organization’s projects. Quantitative Project Management Quantitatively manage the project’s defined process to achieve established quality and process performance objectives. 16th COCOMO Forum

7 Organizational Process Performance (CMMI)
Establish and maintain a quantitative understanding of the performance of the organization’s set of standard processes, and to provide the process performance data, baselines, and models to quantitatively manage the organization’s projects. Required Goals SG 1 Establish Performance Baselines and Models Baselines and models that characterize the expected process performance of the organization's set of standard processes are established and maintained. GG 3 Institutionalize a Defined Process The process is institutionalized as a defined process. Expected Implementation Practices Expected Institutionalization Practices 16th COCOMO Forum

8 Expected Implementation Practices
SP 1.1 Select Processes Select the processes or process elements in the organization's set of standard processes that are to be included in the organization's process performance analyses. SP 1.2 Establish Process Performance Measures Establish and maintain definitions of the measures that are to be included in the organization's process performance analyses. SP 1.3 Establish Quality and Process Performance Objectives Establish and maintain quantitative objectives for quality and process performance for the organization. SP 1.4 Establish Process Performance Baselines Establish and maintain the organization's process performance baselines. SP 1.5 Establish Process Performance Models Establish and maintain the process performance models for the organization's set of standard processes. 16th COCOMO Forum

9 Approach Our Level 3 process database supported cost estimation and process improvement Surveyed management team to establish business drivers Defined measures needed to characterize process performance and quality at the organizational level Defined measures needed to characterize project satisfaction of organizational goals Identified sub-processes amenable to quantitative management Defined project measures needed for quantitative management of those sub-processes Examined improvements in the organization standard process needed to stabilize the process or make measures meaningful Examined improvements in the project’s defined processes 16th COCOMO Forum

10 Example Organizational Metrics Collected and Derived
Collect base measures Size Effort by activity Cost by activity Number of defects (By phase) Derive other measures Defect density (defects/size) Productivity (size/effort) Defect containment (defects saves/escapes from defects by phase) Rework effort SPI/CPI (planned vs. actual effort) 16th COCOMO Forum

11 How Projects Use the Organizational Process Assets
CMM Organizational Standard Process & Organizational Procedures Process Asset  Library (PAL) Organizational Database Organizational Training Office Organizational Policies Senior Management Review Organization Project Project Plans Project Schedules & Budgets Project Results Project Defined Process & Procedures 16th COCOMO Forum

12 Project Comparative Data Base (PCDB)
PCDB contains: Cost data from a certified accounting system Project validated technical characteristics data PCDB relates: Cost data to standardized (WBS) work elements Cost data to the technical characteristics of work performed The Office of Cost Estimation uses the PCDB to: Calibrate the parametric models Derive cost estimating relationships Provide historical data for proposals, project planning/replanning Define risk affordability/analysis Characterize process performance and quality 16th COCOMO Forum

13 What is the PCDB ? Accounting Data Project Inputs
PCDB Standard WBS JN Mapping PCDB Standard WBS Alternate Hours/Cost Mapping Accounting Data Labor hours, dollar costs Non-labor costs Breakdown by PCDB WBS element, by labor category S/W Development Descriptive Data Examples SLOC, other size measures ESLOC (derived) Labor Required to Develop SLOC/ESLOC Software Development Other Project Disciplines Cost Model Parameters Accounting & Descriptive Data Project Management Systems Engineering Hardware Development Software Development Systems Integration & Test Site Activation Integrated Logistics Configuration Management Data/Documentation Management Quality Assurance Development Support Facility Operations and Maintenance Specialty Customer Services Other Activities 16th COCOMO Forum

14 PCDB Support to Cost Modeling and Proposals
Functional Requirements RFP Project and Proposal Team Support Final Estimates Parametric Cost Models First Order Estimates Advanced Pricing System (APS) Model Calibration Project Comparative Data Base (PCDB) Cost Volume Sanity Check Final Estimates Project Technical Description Data & Metrics CERs Certified Accounting Data BOEs Estimation Notebook Guidelines Sanity Check BOE Generation Support Actuals Management Information System (AMIS) Project Data 16th COCOMO Forum

15 PCDB Data Submittal Process
No Project Prepares and Submits Descriptive Data Inputs Proj OK? Yes Place Data in PCDB in the Review State Start Place Data in PCDB in the Input State Retrieve Accounting System Data Project Prepares and Submits Accounting Data Inputs OCE Project Staff Generate Derived Data and Summary Reports Data Review and Validation Update Input Data No Proj & OCE OK? Yes End Place Data in the Reportable State Perform Parametric Validation if Applicable 16th COCOMO Forum

16 Parametric Validation/Calibration
Additional selective validation for software development History Data Points Creation of parametric cost model baseline Input from Project Descriptive Data Forms: Software Development OCE provides guidance and assistance Provided to Project Software Parametric Model (i.e., Costar (COCOMO II), SEER-SEM, Price S, or SLIM) cost estimation validation Iteration with project to achieve validation within 10% - 20% of actual effort Results in calibrated baseline for future estimates 16th COCOMO Forum

17 Lessons Learned - 1 Determining a common set of metrics is tough because there are different needs (and frequency, granularity, accuracy, etc.) Project management decisions Customer insight Senior management oversight Organizational process performance characterization Process improvement Proper support for metrics collection requires changing the culture to “management-by-data” Project managers must use the data to manage their projects Senior managers must use the data to meet organizational business objective Data collectors must be confident that data will not be used against them 16th COCOMO Forum

18 Lessons Learned - 2 Understanding variation in process performance allows more insight into estimation What’s the likely cost of this work? What’s the probability we can perform the work for $____? Project resistance to data collection is primarily due to the time and effort required to collect and report the data Must be integrated and consistent with the process Data collection mechanisms require clear instructions, to ensure the desired information is captured and validated Collection should be a combination of automated and manual methods for cost and accuracy 16th COCOMO Forum

19 Systems Architecture Vision
Current Situation Project Risk Assessment Organizational Metrics Data What if Modeling Predictive Analysis Financial Data Project Risk Assessment What if Modeling Risk Radar Project Mgmt Data Decision Support Layer Organizational Metrics Data Financial Data Group Common Data Repository Proposal Data Project Mgmt Data The End Objective Integrate Metrics with Financial, Project, proposal and other information to support trend and risk analysis Proposal Data 16th COCOMO Forum

20 Conclusions The process database developed at Level 3 is a key asset in achieving Level 4 The many uses of metrics places additional emphasis on innovative database design and usage Characterizing the organization’s process performance requires: Definitizing your business goals Selecting the right metrics Stabilizing the organization and projects’ processes Collecting and analyzing the metrics Management and decisions that are data driven result in better predictive analysis capability 16th COCOMO Forum

Download ppt "Rick Hefner. Marilee J. Wheaton TRW"

Similar presentations

Ads by Google