1. ISA 201 Intermediate Information Systems Acquisition

2. Lesson 19 Software Support; Pre-video
Lesson Point of Contact:
Name: Jason Hamilton
Phone:
Read Ahead: None
Length of Presentation:
Presentation: 2.0 hours
Exercise: 1.0 hours
ELO Identify the important software life cycle planning documents and their major components.
ELO For an existing IT/SIS, recognize critical success factors for software transition.
ELO Describe the keys to successful software sustainment and support.
ELO Compare and contrast the types of software maintenance.
ELO Explain the elements, purpose, outcomes and issues of the DoD software disposal process.
ELO Using the sample exercise, identify critical success factors for software transition.
ELO Identify relevant considerations that influence the acquisition lifecycle of a software-reliant system.
MT Understand the Software Development Plan (SDP), the Integrated Master Schedule (IMS), and the Work Breakdown Schedule (WBS) including their content, purpose, and how they relate to the acquisition development cycle.
MT Understand the impact of the following factors on the acquisition lifecycle of a software-reliant system: the Completeness and Stability of the Requirements, the Development and Documentation of the Software Architecture, Software Reuse including use of COTS and open source software, Software Data Management and Technical Data Rights, the use of Modular Open Systems, Data Protection and Software Assurance, Software Safety, Software Acquisition and Sustainment Costs, Technical Maturity Levels, and Post-Deployment Software Support (PDSS).
Quiz Question(s) and correct answer(s):
Question: Which plan identifies the resources needed to support delivered software and describes the developer’s plans for transitioning delivered software to the support agency?
Correct Answer: Software Transition Plan (STrP)
Question: In terms of overall risk, what software deployment strategy usually has the highest risk?
Correct Answer: Immediate Deployment
Question: According to ISO/IEC 12207:2008 what is the purpose of the IT or software disposal process?
Correct Answer: (all of the above)
Destroys or stores system software elements and related products in a sound manner, in accordance with legislation, agreements, organizational constraints and stakeholder requirements
Where required, it maintains records that may be monitored
To end the existence of system's software entity

3. Development considerations
Lesson Plan Status
No change
Development considerations
Deployment
Transition
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: None at this time
*Policy / Directive / Standard / DTM ID: J-STD-016, NDIA, DAG, Program Manager’s E-Toolkit, USAF Weapon Systems Software Management Guidebook, NAVAIR Logistics Handbook, NavAir Software Logistics Primer
*********************************************************************************************************************************
Key Points: Scenario taken from a Raytheon program management class and modified to reflect government views. Scenarios have proven to be pretty realistic.
This is intended as a short, in lesson exercise. All teams work on the question for 5-8 minutes then do an informal brief-back as part of an instructor led classroom discussion
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work

4. Life Cycle Planning: System Development to Disposal
Describe when software (SW) support activities occur throughout the acquisition life cycle 
Development
Deployment
Transition
Operations & Sustainment
Disposal
Planning for these activities should begin in the development phase.
Planning should include the Software Support Activity (SSA)
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: Describe when software (SW) support activities occur throughout the acquisition life cycle;
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: Topic Introduction slide – Lifecycle planning should be started up front and early.
About 70% of program lifecycle costs occur during Operations and Sustainment. Does our planning reflect this?
Key Questions to Ask and Anticipated Answers:
Q: Why is our sustainment planning often lacking
A: We are focused on getting funding and sometimes we defer planning until we have to. Most PMs last 3 years or less but so what is their motivation to plan for events that will happen long after their term as PM ends?
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work

5. Planning for Support: The Big Picture
Planning for support begins early in the lifecycle and effects design decisions:
Choose architectures that promote ease of change throughout the lifecycle, including tools, languages, and hardware (MOSA)
Address hardware and software as an integrated system and not as isolated entities
Focus on ease of changing technology base, and ease of verification and expansion to support capability growth
Explore cross-cutting solution to minimize proliferation of new architectures, components and software
For Legacy systems:
Migrate toward more affordable/sustainable architectures and support environments
Develop and implement a migration plan
I don't currently call this out in this manner in CP 30, but these essentially align as LP/MT under Describe when software (SW) support activities occur throughout the acquisition life cycle 
SLIDE INFORMATION***************************************************************************************************************************
*Slide Type: Content
*Supporting ELOs ID: ELO Describe the keys to successful software sustainment and support; Describe when software (SW) support activities occur throughout the acquisition life cycle 
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points:
USAF Weapon Systems Software Management Guide
Viable Life-Cycle Computer Systems and Software
It is an acquisition objective to develop and deliver computer systems and software that will provide capable, timely, and affordable functionality throughout all life cycle phases. This is accomplished by ensuring the contractor plans for the following:
a. The ability of the proposed computer system architecture to enable efficient and affordable design change/technology insertion (user capability requirements & technology refresh) over the life phases of the program (system development and demonstration, production and deployment, and operations and support).
b. The ability to efficiently and affordably produce and maintain a specification compliant system over the planned production schedule.
c. The ability to efficiently and affordably meet user requirements (e.g., availability, mission capable rate) for the economic life of the system as affected by product durability and technology obsolescence.
Some approaches that can be applied to support this objective include:
a. Selecting architectures which permit ease of change of underlying tools, languages, and hardware processing platforms.
b. Addressing hardware and software as an integrated system, not isolated entities.
c. Focusing on ease of changing the technology base, ease of verification, and ease of expansion to support future capability growth.
d. Exploring cross-cutting solutions to minimize proliferation of new architectures, components, and software.
For legacy systems:
a. Migrate over time toward more affordable and sustainable architectures and support environments.
b. Develop and implement a plan, including objectives, funding, and schedules, to accomplish this migration over the life cycle.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work

6. Software Sustainment and Support
Design for Support
Support the Design
Understand requirements
Apply the Systems Engineering process
Develop Product Support Strategy
Assess Total Ownership Costs
Plan maintenance and support
Identify SSA -- Deliver support to operational sites
Manage program in the field
Continuously monitor the pulse of the customer
Analyze product availability (Help Desk metrics)
Analyze affordability
Ensure safety
These are fairly generic planning points. I don't currently call these out in this manner in CP 30, but these essentially align as LP/MT under Describe when software (SW) support activities occur throughout the acquisition life cycle 
SLIDE INFORMATION***************************************************************************************************************************
*Slide Type: Content
*Supporting ELOs ID: ELO Describe the keys to successful software sustainment and support; Describe when software (SW) support activities occur throughout the acquisition life cycle 
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: NAVAIR Logistics Handbook, April 2007
Designing for optimal system operational effectiveness requires balance between system effectiveness and system Life Cycle Cost. The emphasis is on:
Designing for support refers to the reliability and maintainability of the prime mission system or equipment to execute mission capability.
Supporting the design includes Human Factors Engineering and the cost-effective responsiveness and relevance of the support infrastructure.
The key to achieving that balance between weapon system effectiveness and cost is to smoothly integrate the Defense Acquisition Management Framework (including its defined Phases and milestones) with Systems Engineering and design maturation processes.
The following table lists activities that facilitate designing for support and activities that facilitate supporting the design.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software Sustainment includes the processes, procedures, people, materiel, and information required to support, maintain, and operate the software aspects of a system.
Software Sustainment Pre-Work

7. Software Development Planning: Infrastructure Overview
Program office organization must allocate functional responsibilities related to software development and management:
Group to oversee the integration and evaluation of all system-level, cost performance, tradeoff analyses (i.e. System Cost or Performance IPT)
Group to develop and coordinate the Program Office Estimate (POE) and Service Cost Position (i.e. System or Software Cost IPT)
Group responsible for developing a cost-effective and responsive acquisition strategy that satisfies all civilian and military services requirements being satisfied by software (i.e. Software Acquisition IPT)
Group that develops and maintains all Test and Evaluation Master Plans (TEMPs) and supports the RFP development process by ensuring requirements are testable (i.e. SW Test IPT)
Group that plans and manages sustainment related decision, including monitoring post-deployment performance and system health (i.e. Program Execution or Sustainment IPT)
SLIDE INFORMATION*******************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify relevant considerations that influence the acquisition lifecycle of a software-reliant system; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID:
*****************************************************************************************************************************
Key Points: Many of the IPTs we may use during development have key responsibilities in planning for deployment, sustainment, and disposal activities
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
SW Cost/Performance IPT
The Cost/Performance IPT is responsible for integrating and evaluating all system-level, cost performance, trade-off analyses. It recommends performance or engineering and design changes to meet Capability Development Document (CDD) and any other software or requirements threshold values for system-level requirements. Office of the Secretary of Defense (OSD) and the Services may be represented in this IPT.
SW Cost IPT
The SW Cost IPT may be a multi-service/agency IPT led by the cognizant SMC Program/Project office, depending on the scope and involvement of other services or agencies. It will be established to develop and coordinate the Program Office Estimate (POE) and Service Cost Position. This IPT works under the Cost IPT, headed by the Air Force Cost Analysis Agency.
SW Acquisition IPT
The SW Acquisition IPT is chaired by the cognizant SMC Program/Project office/officer. This IPT is responsible for developing a cost-effective and responsive acquisition strategy that satisfies all civil and military services requirements being satisfied by software. This IPT will also develop the program Single Acquisition Master Plan (SAMP). Members may include representatives from the user community and other agencies/services, as appropriate to the project.
SW Test IPT
The SW Test IPT will be chaired by cognizant SMC Program/Project office, with support of the Responsible Test Organizations and other Services stakeholders. The scope of this IPT is to develop and maintain all Test and Evaluation Master Plans (TEMPs) and support the RFP development process by ensuring requirements are testable. This IPT is also responsible for addressing the development and application of Modeling and Simulation in testing activities.
Software Sustainment Pre-Work

8. Key Software Development and Sustainment Documents
Software Development Plan
Controls all of the software developer's activities
Provides the government insight into the contractor’s organization responsible for developing the software.
Written and carried out by the SW development contractor
Development contractor uses this to tell the Project Officer how the contractor will do his job.
Major topic areas include: Development and test resources, Error reporting, Configuration Management and Security
Software Maintenance Plan
Also called the Computer Resources Lifecycle Management Plan (CRLCMP)
May be a section within (or appendix to) the Life Cycle Sustainment Plan (LCSP)
Should be prepared by the agency that is acquiring and will be responsible for maintaining and using the software.
The PM should assemble a Computer Resources Working Group (CRWG) to prepare the plan and choose organization responsible for maintenance.
Major topic areas include: Transfer of responsibility, equipment needs, personnel needs, configuration management and funding.
Software Installation Plan (SIP) and Software Transition Plan (STrP)
Detailed later in this video
Changes in red. Also updated the speaker notes.
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify the important software life cycle planning documents and their major components; Describe when software (SW) support activities occur throughout the acquisition life cycle; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID:
*********************************************************************************************************************************
Key Points: Approaches and strategies for SW development, deployment, transition, and disposal activities are documented in a series of plans
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Slide supports ELO #1 – Know the important software life cycle planning documents and their major components.
Software Development Plan
The SDP is the guide that controls all of the software developer's activities and is used to provide the government insight into the contractor’s organization responsible for developing the software. This plan should be tailored specifically for the particular project, but should include proven processes, methods and procedures from past successful software development projects. It is written and carried out by the software development contractor and is the vehicle by which the development contractor tells the Project Officer how the contractor will do his job. Because of the importance of the Software Development Plan, it should be thoroughly reviewed by the Project Officer (or by designated personnel) before acceptance and approval. A draft of the SDP should be specified in the Request For Proposal (RFP) as a compliant document and included as part of the contractor’s proposal.
Software Maintenance Plan
The Software Maintenance Plan (also known as the Computer Resources Lifecycle Management Plan--CRLCMP) should be prepared by the agency that is acquiring and will be responsible for maintaining and using the software. The CRLCMP is no longer required but may be a best practice. Depending on the scope of the project, the Software Maintenance Plan may be within (or an appendix to) the Life Cycle Sustainment Plan (LCSP). The project officer should make certain that a team of DoD personnel from the acquisition, maintenance or logistics, and user organizations is assembled for the plan preparation. This team should also include the SSA, if they have been identified (as early as possible is a best practice). This team, usually called the Computer Resources Working Group (CRWG), should be assembled very early in the development effort, even before the software development contract is awarded, if possible.
The first major decision of the CRWG and a key element of the Software Maintenance Plan is the selection of the organization that will be responsible for the software maintenance after delivery of the system – the SSA. The organization could be either a contractor or a government support organization. The latter is called organic maintenance. Once identified, the SSA should participate in the CRWG and assist with the preparation of the Software Maintenance Plan.
Software Sustainment Pre-Work

9. Core Software Development Documents
Add column for Data Item Description.
The SSA will need all of this to conduct adequate sustainment planning and management.
If intent is to be comprehensive to cover transition/deployment; Add
Software Maintenance Plan (LCSP)
Software Installation Plan
Software Transition Plan
Document
Purpose
DID
Software Requirements Specification (SRS)
What the software must do.
DI-IPSC-81433A
Software Design Description (SDD)
How the software will do it.
DI-IPSC-81435A
Software Test Plan (STP)
How it will be shown that the software fulfills its requirements.
DI-IPSC-81438A
Software Test Description (STD)
The specific inputs, scenario, and acceptance criteria for each test.
DI-IPSC-81439A
Software Test Report (STR)
The results obtained when the test procedures were conducted.
DI-IPSC-81440A
Software Product Specification (SPS)
What was coded, tested, and delivered.
DI-IPSC-81441A
Version Description Document (VDD) / Software Version Description (SVD)
What is included in a particular version of the software that is delivered?
DI-IPSC-81442A
*Slide Type: Content (Content or Exercise)
SLIDE INFORMATION*****************************************************************************************************************
*Supporting ELOs ID: ELO Identify the important software life cycle planning documents and their major components; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID:
*********************************************************************************************************************************
Key Points: See below
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
From: SMC (Space and Missile Center) SW Acq Handbook 2004
All of the documents listed in the table above are required for each SI in every software development activity. The Software Requirements Specification and the Software Design Document, after approval by the Project Officer (PO), will serve as baselines for subsequent software development activities. The Software Test Plan, Software Test Description, and Software Test Report also should require approval by the PO, but they do not become baselines since they do not define mandatory characteristics of the final software product.
Software Requirements Specification:
The Software Requirements Specification (SRS) specifies the requirements for a Computer Software Configuration Item (CSCI) and the methods to be used to ensure that each requirement has been met. Requirements pertaining to the CSCI's external interfaces may be presented in the SRS or in one or more Interface Requirements Specifications (IRSs) (DI-IPSC-81434) referenced from the SRS. The SRS, possibly supplemented by IRSs, is used as the basis for design and qualification testing of a CSCI.
Software Design Description:
The Software Design Description (SDD) describes the design of a Computer Software Configuration Item (CSCI). It describes the CSCI-wide design decisions, the CSCI architectural design, and the detailed design needed to implement the software. The SDD, with its associated Interface Design Descriptions and Database Design Descriptions, is used as the basis for implementing the software. It provides the acquirer visibility into the design and provides information needed for software support.
Software Test Plan:
The Software Test Plan (STP) describes plans for qualification testing of Computer Software Configuration Items (CSCIs) and software systems. It describes the software test environment to be used for the testing, identifies the tests to be performed, and provides schedules for test activities. There is usually a single STP for a project. The STP enables the acquirer to assess the adequacy of planning for CSCI and, if applicable, software system qualification testing.
Software Test Description:
The Software Test Description (STD) describes the test preparations, test cases, and test procedures to be used to perform qualification testing of a Computer Software Configuration Item (CSCI) or a software system or subsystem. The STD enables the acquirer to assess the adequacy of the qualification testing to be performed. This Data Item Description (DID) is used when the developer is tasked to define and record the test preparations, test cases, and test procedures to be used for CSCI qualification testing or for system qualification testing of a software system.
Software Test Report:
The Software Test Report (STR) is a record of the qualification testing performed on a Computer Software Configuration Item (CSCI), a software system or subsystem, or other software-related item. The STR enables the acquirer to assess the testing and its results.
Software Product Specification:
The Software Product Specification (SPS) contains or references the executable software, source files, and software support information, including 'as built' design information and compilation, build, and modification procedures, for a Computer Software Configuration Item (CSCI). The SPS can be used to order the executable software and/or source files for a CSCI and is the primary software support document for the CSCI. Note: Different organizations have different policies for ordering delivery of software. These policies should be determined before applying this DID.
Version Description Document / Software Version Description:
The Software Version Description (SVD) identifies and describes a software version consisting of one or more Computer Software Configuration Items (CSCIs). It is used to release, track, and control software versions. The term ‘version’ may be applied to the initial release of software, to a subsequent release of that software, or to one of multiple forms of the software released at approximately the same time (for example, to different sites). This DID is used when the developer is tasked to identify and record the exact version of software to be delivered to a user, support, or other site.
Other Essential Support Documents
The System Specification is not strictly a software document, but must precede and control the Software Requirements Specification. The Interface Requirements Specification is important for large Software Items (SI). Where there are multiple interfacing SIs, it acts as a linkage point between separate Software Requirements Specifications and Software Design Documents. The Software Users Manual (SUM) is essential for those applications in which a human operator has direct access to and control over software operation. In such applications, the SUM is an expansion of, and equal in importance to, the Software Requirements Specification. There are applications in which the user may not be aware of the software or directly control the software; in these cases, a SUM may have no meaning and may be omitted.
Software Sustainment Pre-Work

10. Deployment Lesson Plan Status Development considerations Transition
No change
Development considerations
Deployment
Transition
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: None at this time
*Policy / Directive / Standard / DTM ID: J-STD-016, NDIA, DAG, Program Manager’s E-Toolkit, USAF Weapon Systems Software Management Guidebook, NAVAIR Logistics Handbook, NavAir Software Logistics Primer
*********************************************************************************************************************************
Key Points: Scenario taken from a Raytheon program management class and modified to reflect government views. Scenarios have proven to be pretty realistic.
This is intended as a short, in lesson exercise. All teams work on the question for 5-8 minutes then do an informal brief-back as part of an instructor led classroom discussion
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work

11. Software Installation Plan (SIP)
A plan for installing software at user sites, including preparations, user training, and conversion from existing systems.
Developed when the developer install software at user sites and installation is complex enough to require a documented plan.
A separate SIP may be unnecessary if the software is embedded in a hardware-software system and a fielding or deployment plan is used.
Should include a system overview, an installation overview and description, site-specific information for software operations staff, schedules, installation and data update procedures, along with site-specific information for software users.
Essentially the “Material Fielding Plan” for software.
Need to consider if this is a developer responsibility or the PMO responsibility, since it will require interaction with the gaining organizations.
Is there a DID for this?
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify the important software life cycle planning documents and their major components; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID: MIL STD 498 / J-STD-16
*********************************************************************************************************************************
Key Points: See below
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software installation planning
The developer shall develop and record plans for performing software installation and training at the user
sites specified in the contract. This planning shall include all applicable items in the Software Installation
Plan (SIP) (see E.2.3).
Software Sustainment Pre-Work

12. Software Deployment—Strategies
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify relevant considerations that influence the acquisition lifecycle of a software-reliant system; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: Different deployment schemes carry different risks. In all cases, IT professionals are called to be agents of change. Two way communication and a planning IPT with input from key departments across the organization are critical. It is very difficult to over-communicate this point.
Key Questions to Ask and Anticipated Answers:
Q: From your observations, what do you think is the best method?
A: There is no one size fits all. We’ll address risks for each method in the next slide, but any of the above methods may be used.
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work
Lesson 03

13. Transition Lesson Plan Status Development considerations Deployment
No change
Development considerations
Deployment
Transition
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: None at this time
*Policy / Directive / Standard / DTM ID: J-STD-016, NDIA, DAG, Program Manager’s E-Toolkit, USAF Weapon Systems Software Management Guidebook, NAVAIR Logistics Handbook, NavAir Software Logistics Primer
*********************************************************************************************************************************
Key Points: Scenario taken from a Raytheon program management class and modified to reflect government views. Scenarios have proven to be pretty realistic.
This is intended as a short, in lesson exercise. All teams work on the question for 5-8 minutes then do an informal brief-back as part of an instructor led classroom discussion
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work

14. Software Transition Plan (STrP)
Documents transition criteria and activities from the development environment to the SSA for sustainment
It should be developed if the software support concept calls for transition of responsibility from the developer to a separate support agency.
Describes the developer’s plans for transitioning deliverable items to the SSA.
Identifies the hardware, software, and other resources needed for life cycle support of deliverable software.
Should include software support resources, recommended procedures, training requirements, and all transition planning activities and schedules
May be used by the acquirer for updating the Software Maintenance Plan/LCSP
DID: DI - IPSC-81429A.
Consider adding a slide somewhere articulating what is transition versus deployment for a common lexicon? Or just on the ‘terminology’ slide?
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify the important software life cycle planning documents and their major components
*Policy / Directive / Standard / DTM ID: MIL STD 498 / J-STD-16
********************************************************************************************************************************
Key Points: See below
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
software transition:
The set of activities that enables responsibility for software development to pass from one organization, usually the organization that performs initial software development, to another, usually the organization that will perform software maintenance.
5.1.5 Software transition planning
The developer shall identify all software development resources that will be needed by the maintenance
organization to fulfil the support strategy specified in the contract. The developer shall develop and record
plans identifying these resources and describing the approach to be followed for transitioning deliverable
items to the maintenance organization. This planning shall include all applicable items in the Software Transition
Plan (STrP) (see E.2.4).
DATA ITEM DESCRIPTION 1. TITLE SOFTWARE TRANSITION PLAN (STrP) 2. IDENTIFICATION NUMBER DI-IPSC DESCRIPTION/PURPOSE 3.1 The Software Transition Plan (STrP) identifies the hardware, software, and other resources needed for life cycle support of deliverable software and describes the developer's plans for transitioning deliverable items to the support agency.
3.2 The STrP is developed if the software support concept calls for transition of responsibility from the developer to a separate support agency. The STrP may also be used by the acquirer for updating the Computer Resources Life Cycle Management Plan
Software Sustainment Pre-Work

15. Software Transition: Success Factors
Critical Success Factors
Identify and acquire any key resources needed by the designated Software Support Activity
Document rationale of supportability decisions (PS BCA)
Match the software support strategy to the system level support strategy
Adhere to development standards
Maintain good configuration management (CM)
Use a Computer Resources IPT (CR-IPT)
Employ a Software Transition Plan (STrP)
Demonstrate the supportability concept
Emphasize Software Supportability Reviews
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO For an existing IT/SIS, recognize critical success factors for software transition
*Policy / Directive / Standard / ********************************************************************************************************************************
Key Points: Transition from fielding to support is often extremely challenging. Responsibilities and ownership can often get blurred and must be planned for appropriately.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Software Sustainment Pre-Work
Lesson 03

16. Backups/Lesson Exercise
Lesson Plan Status
No change
Development considerations
Deployment
Transition
Backups/Lesson Exercise
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: None at this time
*Policy / Directive / Standard / DTM ID: J-STD-016, NDIA, DAG, Program Manager’s E-Toolkit, USAF Weapon Systems Software Management Guidebook, NAVAIR Logistics Handbook, NavAir Software Logistics Primer
*********************************************************************************************************************************
Key Points: Scenario taken from a Raytheon program management class and modified to reflect government views. Scenarios have proven to be pretty realistic.
This is intended as a short, in lesson exercise. All teams work on the question for 5-8 minutes then do an informal brief-back as part of an instructor led classroom discussion
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Lifecycle Planning

17. Exercise 2, Part 1: COTS and Freeware Use Grow Out of Control
Course Title
Exercise 2, Part 1: COTS and Freeware Use Grow Out of Control
In the early stages of the program you decided to limit COTS and freeware use so severely that little would be used, at least in your tactical (mission critical) applications.
However, that didn’t work. Little by little, exceptions were made—usually to save time and money.
By PDR, COTS HW and SW products and “Freeware” SW are being used extensively, throughout the design. Different suppliers often use different releases of the same COTS or Freeware, resulting in integration risks.
Moreover, each individual instance has its own licensing agreement, refresh plan, conditions of use, documentation format, etc.
There is concern that if things continue, transition to production cannot proceed in an orderly fashion due to the great variety of terms and conditions that must be satisfied. There are no centralized records of the licenses or other requirements for the COTS and freeware. Moreover, there are security concerns with all the “Free” SW.
This happened because COTS and freeware were selected by individual suppliers and/or contractor engineers to meet individual needs, with no defined process, and no coordination of efforts or approval at a management level with total program visibility.
Supply chain staff were not familiar with software, so they deferred to individual engineers for decisions involving freeware and COTS software.
Task: You must decide your subsequent steps and plan of action.
OK – no change
SLIDE INFORMATION***************************************************************************************************************************
*Slide Type: Exercise
*Supporting ELOs ID: ELO Describe the keys to successful software sustainment and support
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: Exercise.
Exercise has no “approved school solution” teams must be able to provide a reasonable defense for their answers. Emphasize the potential impacts of COTS on sustainment
COTS (Commercial Off The Shelf) products carry unique issues because they are often proprietary designs and we typically have no control over product configuration or manufacturing processes. Low cost COTS suppliers often operate with thin margins that make them susceptible to business problems. Problems can arise rapidly.
It may be possible to buy the design and manufacturing rights for this product and produce it ourselves.
More risky but possible is to buy the company and bring them out of bankruptcy.
There may be other COTS alternatives but there will be cost and schedule penalties for adapting our system to their product.
We could design our own version of the COTS product and produce it ourselves, but there will be cost and schedule penalties.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Lifecycle Planning
Copyright © 2009 Raytheon Company. All rights reserved.

18. Exercise 2, Part 2: Bankrupt COTS Supplier
The program Supply Chain Manager walks into your office and informs you that the supplier filed bankruptcy last week. This was a surprise to everyone in Supply Chain—no reasons for the bankruptcy were known.
You learn that the bankrupt suppler provided a COTS module that included both hardware and its software. Both are proprietary to the supplier. Since their code was already successfully functioning, your system software, at unanticipated cost, was developed to interface with their proprietary code.
The Buyer can not talk with anyone associated with the suppler as all phones have been disconnected. Another meeting is quickly called that includes your COTS subject matter expert (SME), software IPT lead, Operations Manager, and lead logistics engineer.
The objective of this second meeting is to determine the magnitude of the issue, impact on current and next production lots plus fielded systems.
Task: What is the root cause of this issue? How can the impact of this problem be reduced in the future? Prepare your talking points and preliminary recommendation to deal with this issue. State the projected program impact.
OK – no change
SLIDE INFORMATION***************************************************************************************************************************
*Slide Type: Exercise
*Supporting ELOs ID: ELO Describe the keys to successful software sustainment and support
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: Exercise
Exercise has no “approved school solution” teams must be able to provide a reasonable defense for their answers. Emphasize the potential impacts of COTS on sustainment
COTS (Commercial Off The Shelf) products carry unique issues because they are often proprietary designs and we typically have no control over product configuration or manufacturing processes. Low cost COTS suppliers often operate with thin margins that make them susceptible to business problems. Problems can arise rapidly.
It may be possible to buy the design and manufacturing rights for this product and produce it ourselves.
More risky but possible is to buy the company and bring them out of bankruptcy.
There may be other COTS alternatives but there will be cost and schedule penalties for adapting our system to their product.
We could design our own version of the COTS product and produce it ourselves, but there will be cost and schedule penalties.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Lifecycle Planning

19. Key Terminology Refresher (See handout)
Software sustainment:
Software maintenance:
Corrective maintenance:
Adaptive maintenance: 
Perfective maintenance: 
Preventative maintenance:  
Software Configuration Management (SCM):  
Software Data Rights:  
Software Documentation: 
Software Support Environment:  
Software Testing (Validation and Verification):  
Software Support Activity (SSA): 
Post Deployment Software Support (PDSS) (aka Post Production Software Support (PPSS):
Software reuse:  
Software Refactoring: 
Software Reengineering:  
Capability Maturity Model Integrated (CMMI):
Identify key terminology for software sustainment.   
Add:
Release?
Patch?
SI (formerly CSCI?)
Production Environment?
Transition?
Deployment?
*Slide Type: Content (Content or Exercise)
SLIDE INFORMATION*****************************************************************************************************************
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
*Supporting ELOs ID: Identify key terminology for software sustainment
About 70% of program lifecycle costs occur during Operations and Sustainment. Does our planning reflect this?
Key Points: Topic Introduction slide – Lifecycle planning should be started up front and early.
Q: Why is our sustainment planning often lacking
Key Questions to Ask and Anticipated Answers:
A: We are focused on getting funding and sometimes we defer planning until we have to. Most PMs last 3 years or less but so what is their motivation to plan for events that will happen long after their term as PM ends?
Software sustainment: The processes, procedures, people, material, and information required to support, maintain, and operate the software aspects of a system” (Software Engineering Institute working definition). Software Sustainment addresses other issues not always an integral part of maintenance such as documentation, operations, deployment, security, configuration management, training (users and sustainment personnel), help desk, COTS product management, and technology refresh. Successful software sustainment consists of more than modifying and updating source code. It also depends on the experience of the sustainment organization, the skills of the sustainment team, the adaptability of the customer, and the operational domain of the team. Thus, software maintenance as well as operations should be considered part of software sustainment. The Software Engineering Institute has developed a report under DoD contract on “Sustaining Software Intensive Systems”, found at (IPSE Guidebook, 2011)
Terms \ Definitions \ Acronyms:
Software maintenance: The process of modifying a software system or component after delivery to correct faults, improve performance or other attributes, or adapt to a changed environment” (IEEE Standard Glossary of Software Engineering Terminology). Software maintenance consists of correcting faults, improving performance or other attributes, and adapting software to a changing organization and technical environment. To be complete, there is usually a fourth category of maintenance activities focused on anticipated problems, or preventive maintenance. (IPSE Guidebook, 2011)
Corrective maintenance: Reactive modification (or repairs) of a software product performed after delivery to correct discovered problems. Included in this category is emergency maintenance, which is an unscheduled modification performed to temporarily keep a software product operational pending corrective maintenance.
Adaptive maintenance: Modification of a software product performed after delivery to keep a software product usable in a changed or changing environment. For example, the operating system might be upgraded and some changes to the software may be necessary.
Perfective maintenance: Modification of a software product after delivery to provide enhancements for users, improvement of program documentation, and recoding to improve software performance, maintainability, or other software attributes.
Preventative maintenance: Modification of a software product after delivery to detect and correct latent faults in the software product before they become operational faults. Some organizations may consider preventative maintenance a subset of Perfective maintenance.
Software Configuration Management (SCM): The task of tracking and controlling changes in the software. SCM is a formal engineering discipline that, as part of overall system configuration management, provides the methods and tools to identify and control the software throughout its development and use. From "IEEE Standard for Software Configuration Management Plans" (IEEE Std )
Software Data Rights: The standard license rights in computer software that a licensor grants to the Government are unlimited rights, Government purpose rights, or restricted rights. The standard license in computer software documentation conveys unlimited rights. Those rights are defined in the clause at FAR , Rights in Noncommercial Computer Software and Noncommercial Computer Software Documentation.
Software Documentation: The software development library should contain a controlled collection of software, documentation, other intermediate and final software products, and associated tools and procedures used to facilitate the orderly development and subsequent support of software.
Software Support Environment: A host computer system and other related equipment and procedures located in a facility that provides a total support capability for the software of a target computer system (or a set of functionally and physically related target computer systems). The environment enables the performance of a full range of services including: performance evaluation, system and software generation, development and testing of changes, simulation, emulation, training, software integration, configuration management, and operational distribution for the software.
Software Testing (Validation and Verification): Planning for verification and validation of software begins at onset of development. During verification, the work product (the ready part of the software being developed and accompanying documentation) is reviewed, examined personally by one or more persons in order to discover the defects. Verification and validation processes go hand in hand, but the validation process starts after the verification process ends (after coding of the product ends). During the validation process, test plans, test suites, and test cases are developed, which are used during the various phases of validation.
Software Support Activity (SSA): A Software Support Activity assumes the role of providing post deployment life cycle support for modifications or upgrades made to a system's software following the system's initial fielding. System modifications and upgrades include multi-system changes, block changes, preplanned product improvements, repair of deficiencies reported by the user, and other types of system change packages.
Post Deployment Software Support (PDSS) (aka Post Production Software Support (PPSS): Software support activities that occur after the deployment of the system. “The management of the software development process and the implementation of a process that ensures software supportability are among two of the most difficult challenges facing the Program Manager in management of software-intensive systems. The Program Manager should effectively address the issues of software supportability, the software test environment, and other equipment, material, and documentation, including data rights that are required to provide PDSS for those end users identified in the SDP or in other documents similar to the Computer Resources Life Cycle Management Plan. Successful PDSS planning should assist the Program Manager in controlling software life-cycle costs.” (DAG )
A reusable component may be code, but the bigger benefits of reuse come from a broader and higher-level view of what can be reused. Software specifications, designs, tests cases, data, prototypes, plans, documentation, frameworks, and templates are all candidates for reuse.
Software reuse: When a segment of source code is able to be used again to add new functionalities with slight or no modification. Reuse reduces implementation time.
Software Refactoring: The process of restructuring existing computer code – changing the factoring – without changing its external behavior. Refactoring improves nonfunctional attributes of the software. Advantages include improved code readability and reduced complexity; these can improve source code maintainability and create a more expressive internal architecture or object model to improve extensibility. If done extremely well, code refactoring may also resolve hidden, dormant, or undiscovered computer bugs or vulnerabilities in the system by simplifying the underlying logic and eliminating unnecessary levels of complexity. If done poorly it may fail the requirement that external functionality not be changed, and/or introduce new bugs.
Software Reengineering: the process of examining and altering software to implement a desired modification. Reengineering consists of 2 steps. First, reverse engineering is applied to to understand the current software system and represent it in a new form. Second, forward engineering is applied, implementing and integrating the new desired functionality, giving rise to a new software system (Grubb & Takang, 2003).  
(Is there a better IEEE definition?)
Capability Maturity Model Integrated (CMMI): A management model that emphasizes continuous improvement rather than meeting a standard. There are two types of representations in CMMI® models: staged and continuous.

20. Top Software Engineering Issues
Describe when software (SW) support activities occur throughout the acquisition life cycle;
Software life–cycle planning and management by acquirers and suppliers is ineffective.
Recommendation: Establish a culture of quantitative planning and management, using proven processes with collaborative decision-making across the software life cycle.
Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk.
Recommendation: Improve and expand guidelines for addressing total lifecycle COTS/NDI issues.
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify relevant considerations that influence the acquisition lifecycle of a software-reliant system; Describe when software (SW) support activities occur throughout the acquisition life cycle; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: These are issues from a NDIA report. Bullets 3 and 7 indicate issues with our Lifecycle planning have continued since the mid-2000’s
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
NDIA Top Software Engineering Issues, Sept 2006 – The findings for #3 and #7 were the same in 2010.
Issue 1:
The impact of system requirements upon software is not consistently quantified and managed in development or sustainment.
Recommendation:
Enforce effective software requirements development and management practices, including assessment of change impacts, for both the acquirer and the supplier organizations.
Issue 2:
Fundamental system engineering decisions are made without full participation of software engineering.
Institutionalize the integration and participation of software engineering in all system engineering activities.
Issue 3:
Software life-cycle planning and management by acquirers and suppliers is ineffective.
Establish a culture of quantitative planning and management, using proven processes with collaborative decision-making across the software life cycle.
Issue 4:
The quantity and quality of software engineering expertise is insufficient to meet the demands of the government and the defense industry.
Collaborate on innovative strategies to staff to appropriate levels, and to attract, develop, and retain qualified talent to meet current and future software engineering needs in government and industry.
Issue 5:
Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems.
Recommendation
Study current software verification practices in industry, and develop guidance and training to improve effectiveness in assuring product quality across the life cycle.
Issue 6:
There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments.
Collaborate with industry to develop approaches, standards, and tools addressing system assurance issues throughout the acquisition life cycle and supply chain.
Issue 7:
Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk.
Improve and expand guidelines for addressing total lifecycle COTS/NDI issues.
These issue were identified in 2006 and 2010 reports by the National Defense Industrial Association
Lifecycle Planning

21. Development: Key Planning Aspects
Software Development Planning
Software Test Planning
Software Installation Planning
Software Transition Planning
Software Support Planning
No change
But, research the graphic and understand the tie in to the phases on the left?
Also, perhaps the whole development section belongs in a different lesson or can be part of the CAMTASIA? (currently slides 11 – 21)
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify relevant considerations that influence the acquisition lifecycle of a software-reliant system; Describe when software (SW) support activities occur throughout the acquisition life cycle; Analyze software sustainment planning considerations; ELO Identify the important software life cycle planning documents and their major components.
*Policy / Directive / Standard / DTM ID: *********************************************************************************************************************************
Key Points: Planning for deployment, support and disposal should begin during development – so we are going to begin our discussion with planning for development
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Lifecycle Planning

22. Software Development Key Factors— Questions
No change in slide
Has a software architecture been established? Is it reasonable?
Have communication and interface requirements been established?
Have software requirements been established? Are they prioritized? Are derived requirements understood?
What are reuse and COTS levels?
Are personnel, training, and support impacts understood?
Are interfaces specified in sufficient detail to support coding and implementation?
How will changes be incorporated into the fielded software?
Do results of software tests to date verify performance, reliability, safety, security, and other critical requirements?
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify relevant considerations that influence the acquisition lifecycle of a software-reliant system; Describe when software (SW) support activities occur throughout the acquisition life cycle; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID:
*********************************************************************************************************************************
Key Points: Many of these issues have been discussed throughout the course. If you get to milestone C and haven’t done appropriate planning in these areas the costs will be exponentially higher.
Key Questions to Ask and Anticipated Answers: Each bullet asks a question. Notice the ongoing themes of architecture, interfaces, and COTS and reuse
Terms \ Definitions \ Acronyms:
Lifecycle Planning

23. Language Selection Areas to Address
Language supports the planned application and architecture.
System / software requirements, including performance, interoperability, reliability, safety, security, interface requirements, etc.
Expected software size & complexity, system life-cycle change rate, and sustainment needs.
Reuse of existing systems / software (i.e., programming languages already being used within the system that may be reused or adapted).
Language capabilities of the contractor.
Commercial viability of candidate languages, including current and future availability of compilers, tools, training, etc.
Extent of compatibility with and impact of other related direction (e.g. use of standards such as the Joint Technical Architecture (JTA) and open systems).
Languages defined by standards from organizations such as American National Standards Institute (ANSI) or International Standards Organization (ISO).
Except when refactoring, is this also a developmental (vice sustainment) consideration?
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO For an existing IT/SIS, recognize critical success factors for software transition; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / ********************************************************************************************************************************
Key Points: USAF Weapon Systems Software Management Guidebook
Computer Programming Language Selection
When a programming language decision must be made during the development process, programs should conduct a trade study to determine the best computer programming language, or mix of programming languages, to be used to satisfy system life cycle requirements. This includes the necessary criteria for PDSS that would concern an SSA. Suggested areas to be addressed include:
a. Language supports the planned application and architecture.
b. System / software requirements, including performance, interoperability, reliability, safety, security, architecture, partitioning, advanced functionality, and interface requirements.
c. Expected software size & complexity, system life-cycle change rate, and sustainment needs.
d. Reuse of existing systems / software (i.e., programming languages already being used within the system or within components from other sources that may be reused or adapted).
e. Language capabilities of the contractor.
f. Commercial viability of candidate languages, including current and future availability of compilers, tools, general-purpose development computer system equipment, training, and skilled software engineers.
g. Extent of compatibility with and impact of other related direction (e.g. use of standards such as the Joint Technical Architecture and open systems).
h. Languages defined by standards from organizations such as American National Standards Institute (ANSI) or International Standards Organization (ISO).
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Perl?
Visual Basic?
Ada? C?
C++?
Java?
Lifecycle Planning

24. Assessing Reuse Support Risks
Originally in sustainment, but this is really a developmental consideration I think. Part of the ‘make or buy’ thought process for software???
Reuse concern – documentation is often missing…
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO For an existing IT/SIS, recognize critical success factors for software transition; Describe when software (SW) support activities occur throughout the acquisition life cycle; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / ********************************************************************************************************************************
Key Points: There are often two schools of thought with reuse. One is we should employ software reuse for just about every system. The other is we should never reuse software. In reality, neither side is fully correct. Risks must be weighed and the matrix above is a good start.
The % growth indicates the anticipated increase in lines of code based in the described attributes.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
Lifecycle Planning

25. Software Test Plan (STP)
Produced by the developer
Describes the tasks, schedules, approach, resources, and tools for integrating/testing the software application.
Describes the scope of the testing, the approach to be taken, specific test scripts, resources required, schedules, test databases or files, and documentation requirements.
Should answer the following questions:
What is being tested?
What are pass/fail criteria?
When will each test occur?
What hardware and software environment is required?
What features will/ will not be tested?
What are the responsibilities of individuals and organizations involved in the project?
Followed by Software Test Description(s) and Software Test Report(s) prior to deployment.
Analyze software sustainment planning considerations.
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO Identify the important software life cycle planning documents and their major components; Analyze software sustainment planning considerations.
*Policy / Directive / Standard / DTM ID: MIL STD 498 / J-STD-016
*********************************************************************************************************************************
Key Points: An important part of development is testing. Approaches to testing by the developer are documented in the SW Test Plan
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
J-STD
The developer shall develop and record plans for conducting software item qualification testing. This planning shall include all applicable items in the Software Test Plan (STP)
The Test Plan answers such questions as: What is being tested?
What are pass/fail criteria?
When will each test occur?
What hardware and software environment is required?
What features must be tested?
What features will not be tested?
What are the responsibilities of individuals and organizations involved in the project?
Steps in Test Planning
Software test planning begins before the software design. The steps in test planning include, in this order: Establish acceptance criteria.
Develop an overall plan for integrating software modules.
Test the modules in an operational environment.
Lifecycle Planning

26. Software Support and Test Readiness
Should this be pre-deployment?
SLIDE INFORMATION*****************************************************************************************************************
*Slide Type: Content (Content or Exercise)
*Supporting ELOs ID: ELO For an existing IT/SIS, recognize critical success factors for software transition
*Policy / Directive / Standard / ********************************************************************************************************************************
Key Points: Several of the key areas addressed by DTRR and OTRR are sustainment focused.
Key Questions to Ask and Anticipated Answers:
Terms \ Definitions \ Acronyms:
MFP = Materiel Fielding Plan
“Life cycle management plan” refers to something like a CRLCMP.
Lifecycle Planning
ISAM 05