Slide 1V&V 10/2002 Software Quality Assurance Dr. Linda H. Rosenberg Assistant Director For Information Sciences Goddard Space Flight Center, NASA

Slide 2V&V 10/2002 Agenda Introduction Defining Software Quality Assurance Quality Assurance and Software Development IV&V within SQA Summary QA

Slide 3V&V 10/2002 Introduction

Slide 4V&V 10/2002 “Traditional” Development TIME REQUIREMENTS TESTING QUALITYQUALITY

Slide 5V&V 10/2002 Results in

Slide 6V&V 10/2002 Quality Assurance

Slide 7V&V 10/2002 Why SOFTWARE Assurance time functionality Hardware Software time cost Hardware Software

Slide 8V&V 10/2002 Software Quality Assurance IEEE Standard for Information Technology - Software Life Cycle Processes “The Quality assurance process is a process for providing adequate assurance that the software products and processes in the project life cycle conform to their specified requirements and adhere to their established plans. “ IEEE Quality Assurance Plans “Quality Assurance - a planned and systematic pattern of all actions necessary to provide adequate confidence that the time or product conforms to established technical requirements.”

Slide 9V&V 10/2002 Quality Attributes Product Operations Product Transition Product Revision Maintainability - Can I fix it? Flexibility - Can I change it? Testability - Can I test it? Correctness - Does it do what I want? Reliability - Does it do it accurately all the time? Efficiency - Will it run on my machine as well as it can? Integrity - Is it secure? Usability - Can I run it? Portability - Will I be able to use on another machine? Reusability - Will I be able to reuse some of the software? Interoperability - Will I be able to interface it with another machine?

Slide 10V&V 10/2002 SQA Life CYCLE Concept/ Requirements Reviews (SCR. SRR) Requirement trace SW Development Plans Define success criteria Prototyping Metrics Safety Considerations IV&V

Slide 11V&V 10/2002 SQA Across the Life Cycle Design TestDeploymentConcept/ Requirements Devel. & Coding IV&V Risk Management MetricsSafetyReliability

Slide 12V&V 10/2002 Why IV&V at NASA MARS

Slide 13V&V 10/2002 Independent Verification & Validation Independent –Technical: IV&V prioritizes its own efforts –Managerial: Independent reporting route to Program Management –Financial: Budget is allocated by program and controlled at high level such that IV&V effectiveness is not compromised Verification (Are we building the product right?) Validation (Are we building the right product?) Software IV&V is a systems engineering process employing rigorous methodologies for evaluating the correctness and quality of the software product throughout the software life cycle

Slide 14V&V 10/2002 IV&V Approach Req Design Code Test (Verification & Validation) Unit IntegrationAcceptance Req Design Code Testing Unit Test (Verification & Validation) Integration Acceptance Clean Room Approach Traditional Software Development V&V iV&V Req Design Code Test (Verification & Validation) Unit IntegrationAcceptance IV&V Implementation IV&V

Slide 15V&V 10/2002 IV&V Activities Requirements Phase System Reqts Analysis S/W Reqts Analysis Interface Analysis Process Analysis Technical Reviews & Audits Design Phase Design Analysis Interface Analysis Test Program Analysis Supportability Analysis Process Analysis Technical Reviews & Audits Code Phase Code Analysis Test Program Analysis Supportability Analysis Process Analysis Technical Reviews & Audits Test Phase Test Program Analysis Independent Test Supportability Analysis Technical Reviews & Audits Verify Validate

Slide 16V&V 10/2002 Implementing IV&V at NASA

Slide 17V&V 10/2002 IV&V Criteria IV&V is intended to mitigate risk Risk = Probability * Consequence Probability of an undesired event Consequences if that event should occur  IV&V must be based on Risk Probability & Consequence

Slide 18V&V 10/2002 IV&V Probability Risk Factors Factors that impact the difficulty of the development –Software Team Complexity –Contractor Support –Organization Complexity –Schedule Pressure –Process Maturity of Software Provider –Degree of Innovation –Level of Integration –Requirement Maturity –Software Lines of Code

Slide 19V&V 10/2002 IV&V Probability Risk Factors

Slide 20V&V 10/2002 Consequence Factors Potential for loss of life Potential for serious injury Potential for catastrophic mission failure Potential for partial mission failure Potential for loss of equipment Potential for waste of software resource investment- Potential for adverse visibility Potential effect on routine operations GRAVE SUBSTANTIAL MARGINAL INSIGNIFICANT

Slide 21V&V 10/2002 Criteria Determination for IV&V

Slide 22V&V 10/2002 Summary

Slide 23V&V 10/2002 SQA vs. IV&V SQA Risk PROJECT X IV&V  SQA  IV&V

Slide 24V&V 10/2002 IV&V Benefits Better software/system Performance Higher Confidence in Software Reliability Compliance between Specs & Code Criteria for Program Acceptance Better Visibility into Development Better Decision Criteria Second Source Technical Alternative Reduced maintenance cost Reduced Frequency of Operational Change TechnicalManagement

Slide 25V&V 10/2002 Conclusion Applied early in the software development process, IV&V can reduce overall Project cost. NASA policy provides the management process for assuring that the right level of IV&V is applied. IV&V Implementation Criteria provide a quantitative approach for determining the right level based on mission risk IV&V CANNOT replace Quality assurance but must supplement it to be successful IV&V Requires a strong Quality assurance base

Slide 26V&V 10/2002 References IV&V Facility, Fairmont, WV Director – Ned Keeler Deputy Director - Bill Jackson