Completing the Loop: Linking Software Features to Failures 20 July 2004 Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Paul Garnett Keith Lesch Chad Freeman

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Initial Problem Little consistency among IV&V Facility issue repository data Do software features predict software failures and defects?

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Evolution of Problem Statement Linker Project Successfully Set the Stage for Addressing the Problem of Little Consistency Among IV&V Facility Issue Repository Data (Linker is a computer program developed by MSIS that allows users to generate reports of PITS issue data across multiple projects) Other Work has Addressed Linking Features to Failures in Code

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Current Problem Little consistency among IV&V Facility issue repository data PITS data fields inconsistent across projects Reporting and analysis differ across multiple projects Are there consistent measures that can be applied across NASA centers and projects to determine when NASA software assurance efforts on a project have been effective?

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. New Approach Use IV&V Facility as Pilot Project to Define Software Assurance Effectiveness Metrics Standardize defect reporting across IV&V Facility Refine PITS training Define and pilot software assurance effectiveness metrics at IV&V Facility Collaborate with Other Federal Agencies Record Results and Recommend Changes to NASA Software-Related Policies

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Importance/Benefits Positive NASA Project Impact Reports more meaningful Similar software assurance data collected across multiple projects More complete IV&V data per issue analyzable using ODC vs. “Everything is in the description field” More consistent use of PITS Positive NASA Software Assurance Impact Updated policies More accurate measurement of software assurance efforts

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Relevance to NASA Consistent Use of Software Assurance Metrics Across NASA Projects Linker Can Be Used to Support the Software Assurance Metrics Effectiveness Program ODC Can Be Used to Track Issues Across the Full Life-Cycle of NASA Projects PITS Standard Practices with Appropriate PITS Training IV&V Facility To Implement a Master Field Set for Issues Development of Linker translation layer Historical data to be mapped to master field set

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Accomplishments Completed Version 1 of the Linker Tool Imports PITS data across multiple projects ODC data Other data Creates intermediate data repository Reports PITS data across multiple projects Currently installed in the IV&V tools lab for use by NASA personnel Orthogonal Defect Classification Created master field set applying ODC to IV&V Implemented this field set on an IV&V pilot project Modified PITS for Automated Import of Field Set Started ODC Data Collection on Pilot Project

Copyright © 2004, Mountain State Information Systems, Inc. All rights reserved. MOUNTAIN STATE INFORMATION SYSTEMS, INC. Next Steps Contact Other Federal Agencies on Use and Definition of Software Assurance Metrics Establish IV&V Facility Metrics Effectiveness Program as a NASA Software Assurance Pilot Project Implement ODC Field Set on Additional IV&V Pilot Projects Modify Linker to Support the IV&V Facility Metrics Effectiveness Pilot Program