1. University of Southern California Center for Systems and Software Engineering Issues and Recommendations Emanating from the Management Issues Group of the Integrating Systems and Software Engineering Workshop Art Pyster 31 October 2007

2. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE2 Background A workshop on integrating software and systems engineering processes was held at USC on October 29-30 The workshop was sponsored by Kristen Baldwin, OSD’s Deputy Director of Software Engineering and Systems Assurance A workshop group, led by Art Pyster from Stevens Institute, spent 4 hours focused on management issues. While the group identified numerous issues and produced a rough list of top issues, 4 hours was not enough time to hone the wording of the issues or to produce focused recommendations to OSD. The following refinement of the issues and specific recommendations are based on the workshop discussions, but are not a direct output of the workshop.

3. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE3 Management Group Attendees Dewitt Latimer, USC Art Pyster, Stevens (Chairman) William Bail, Mitre Jim Cain, BAE Systems Stan Settles, USC Brandon Gautney, Dynetics Brian Gallagher, SEI Don Reifer, Reifer Consultants David Seaver, Price System Chris McCauley, EMsolve Stuart Glickman, LM

4. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE4 Primary Management Issues A number of specific management problems were proposed during the workshop. The problems can largely be rolled up into two primary management issues: 1.Key acquisition personnel do not understand how important software engineering and software technology are in making critical decisions throughout the entire acquisition lifecycle. 2.Throughout the entire acquisition lifecycle, critical decisions are made by key acquisition personnel who lack the necessary competencies in software engineering and software technology. The next four slides offer scenarios illustrating these two primary management issues based on problems discussed at the workshop.

5. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE5 Scenario #1: Missing Software Engineers 1.The Acquisition Strategy and Analysis of Alternatives assume a large degree of reuse of an existing software component, or use of a COTS software package, or reuse of a hardware component whose behavior is driven largely by embedded software. 2.Software engineers with the necessary domain knowledge are not actively engaged in developing the acquisition strategy or performing the analysis of alternatives. 3.The assumption about how much reuse is possible or the suitability of the COTS software package turn out to be false or the assumption about how difficult it will be to modify the reused components turns out to be false. 4.Program cost and schedule are negatively impacted and possibly performance as well.

6. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE6 Scenario #2: Technical Naiveté 1.A Service Oriented Architecture is selected as the technical approach for a mission planning system to allow for the greatest flexibility in adding new type of analyses and new sources of information after initial deployment. 2.The PM and Chief Engineer worked on a software-intensive program before, but were never deeply immersed in software engineering and software technologies. They believe they know more about software than they actually do. 3.The PM and Chief Engineer do not appreciate the policy, performance, security, and other “ility” challenges inherent in current SOA technology. 4.Because they lack insight into the technical challenges of SOA, the PM and Chief Engineer do not involve software engineers that have the appropriate SOA skills to help them address those challenges. 5.The PM and Chief Engineer build a schedule and budget that do not reflect the engineering work necessary to achieve the “ility” related KPPs. 6.The program appears to make excellent progress until integration reveals the engineering challenges in realizing the “ility” related KPPs.

7. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE7 Scenario #3: Lost Opportunity 1.A platform program requires extensive multi-year hardware development, but much of the functionality and value of the platform will be realized through software. 2.The PM and Chief Engineer do not appreciate the opportunity to continually demonstrate and refine software and system capability by integrating software with emulated hardware. 3.The PM and Chief Engineer build a schedule that delays software development, depriving them of an opportunity to gain invaluable insight into system performance, human factors, algorithm performance, and hardware/software integration mismatches as well as to build a stronger relationship with the customer through continual visible demonstration of progress.

8. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE8 Scenario #4: Unrealistic Requirements 1.The concept for a communications system identifies very demanding non-functional requirements, such as security and availability. 2.The ability to deliver those non-functional requirements will be driven largely by software. 3.The implementation of those non-functional requirements will not fit within the anticipated cost and schedule, but the software engineers who could help determine the true cost and schedule implications of those requirements are left off the team that develops the RFP. 4.The acquisition proceeds with infeasible cost and schedule, which is recognized later in the development.

9. University of Southern California Center for Systems and Software Engineering October 2007©USC-CSSE9 Recommendations 1.Determine the validity of the two primary management issues. Select 3 MDAP programs for analysis. Independently review the process and personnel involved in key decisions made in the program where understanding of software engineering and software technology should have been critical. Independently determine the software competencies possessed by the staff who made those key decisions. Independently determine whether the quality of those decisions suffered when staff did not possess necessary understanding of software engineering and software technology. 2.Based on what is found in step #1, provide new guidance on: The qualifications of key staff for MDAP programs The processes those key staff use Ways to grow key staff through such means as education at DAU