1. Systems Engineering: It’s the Law!
An Overview of the
Defense Acquisition Management System and Weapon Systems Acquisition Reform Act (WSARA)
Geoff Draper
Harris Government Communications Systems Division

2. Systems Engineering: It’s the Law!
Topics:
Performance issues in Major Defense Acquisition Programs (MDAPs)
Summary of studies and reports
Overview of the Defense Acquisition Management System
DoD policies and guidance
DODI
Weapon Systems Acquisition Reform Act of 2009 (WSARA)
Potential Implications to the Defense Industry
References:
P. L , Weapon Systems Acquisition Reform Act of 2009.
May 22,
Defense Technical Memorandum, DTM
“Implementation of the Weapon Systems Acquisition Reform Act of 2009”. Dec
DoDI , The Defense Acquisition Management System. Dec 2008.
DoD Strategic Management Plan. July 2009.
Defense Acquisition Guidebook (DAG), Chapter 4: Systems Engineering.
Implementation of Weapon Systems Acquisition Reform Act (WSARA) of Mona Lush, OUSD (AT&L). Oct 2009.
DODI and WSARA Impacts on Early System s Engineering.
Sharon Vannucci, OSD (AT&L) / DDRE. NDIA SE Conference, Oct ’09.
Weapon System Acquisition Reform Act (WSARA) 2009.
Mr. Nicholas Torelli, OUSD (AT&L) / DDRE. NDIA SE Div mtg, July 2009.
A Multi-Level Approach to Addressing SE Issues in Defense Programs.
NDIA SE Division, June
27-Mar-17

3. The Problem – Improving Program Performance through SE
Numerous studies and reports document program performance issues and the role of effective systems engineering
GAO T - Actions Needed to Overcome Long-standing Challenges with Weapon Systems Acquisition and Service Contract Management
“DOD’s major weapon systems programs continue to take longer to develop, cost more, and deliver fewer quantities and capabilities than originally planned.”
“costs … of major defense acquisition programs increased 26 percent and development costs increased by 40 percent from first estimates”
“programs … failed to deliver capabilities when promised—often forcing warfighters to spend additional funds on maintaining legacy systems”
“current programs experienced, on average, a 21-month delay in delivering initial capabilities to the warfighter”
NRC/USAF Study - Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Acquisition
“Attention to a few critical systems engineering processes and functions particularly during preparation for Milestones A and B is essential…”
“…critically dependent on having experienced systems engineers with adequate knowledge of the domain…”
“Decisions made prior to Milestone A should be supported by a rigorous systems analysis and systems engineering process involving teams of users, acquirers, and industry representatives.”

4. Addressing SE Issues - Key Studies and Reports
Study / Report
Summary Issues / Findings
Pre-Milestone A and Early-Phase Systems Engineering
[report] (NRC/USAF study)
Inexperienced leadership
External interface complexity
System complexity
Incomplete/unstable reqts at MS B
Reliance on immature technology
Reliance on large amts of new SW
NDIA Top 5 SE Issues (2006)
[report, briefing]
Inconsistent SE practices
Insufficient SE early in life cycle
Lack SE resources (qty, quality)
Requirements not well-defined and managed
Inadequate SE tools/environments
NDIA Top SW Issues (2006)
SE decisions without SW involvement
Ineffective SW life cycle plans
Lack SW resources (qty, quality)
SW verification: costly, ineffective
Requirements impact on SW
SW assurance (predictable, secure)
COTS/NDI impacts
Systemic Root Cause Analysis (SRCA) (2008)
(publication 2009, pending)
Unrealistic acq. strategies and plans
Milestone decision gates, criteria
Staffing shortfalls (quantity, skills, experience)
Systems Engineering of Tactical Air Launched Systems: An Industry Examination (2008) [briefing]
(Air Armament Center, NDIA Gulf Coast Chapter)
Incomplete design-to requirements
Requirements creep, not stable
Unplanned reqts verification effort
Unplanned performance/design tests
Configuration variation issues
Component qualification test
Weapon level integration tests
Weapon level qualification tests
Weapon flight testing
Redesign for producibility/cost
Field reliability, sustainability
House Armed Services Panel on Defense Acq. Reform (final report, March 2010)
Manage performance of acq. system
Lacking good reqts process (JCIDS)
Ad hoc acquisition of services
Develop/incentivize acq. workforce
Ineffective financial mgmt system
Protect the industrial base (all sizes)
Responsiveness of core acq system
Details

5. Systemic Issues of Big “A” Acquisition
Resources (PPBE)
Funding instability
Insufficient resource trade space
Budget not properly phased/magnitude to support planned development
Defense Acquisition
System
(DAS)
Small “a” Acquisition
Requirements
(JCIDS)
Immature technologies
Inadequate systems engineering
Inadequate requirements flow-down/ traceability/ decomposition
Insufficient schedule trade space
Inadequate implementation of Earned Value Management System
Lack of time and assets for testing
To deliver capable, effective, efficient material solutions, Requirements Managers must understand “Big A Acquisition” which is the combination of the Requirements, Planning, Programming, Budgeting, and Execution (PPBE) and acquisition processes. Program success depends on integrating the three components of “Big A Acquisition.”
Chairman Joint Chiefs of Staff Instruction (CJCSI) states that the requirements process, the acquisition process, and the Planning, Programming, Budgeting, and Execution (PPBE) process must work together to deliver capabilities to the warfighters. Our goals are to align the three processes, to make consistent decisions, and to avoid the mistakes of the past.
Welcome. I am Major General William Troy, the Joint Staff J-8 Vice Director for Force Structure, Resources and Assessment. One of our most important missions in J-8 is to be an advocate for the warfighter and assist the Joint Requirements Oversight Council in properly defining and validating future warfighting capabilities.
The course you are taking will help you, a requirements manager, play your role in defining the requirements for future warfighting capabilities. While we are all committed to giving the warfighter the best possible solutions, there is often a gap in both experience and expectation between the warfighters on one hand and the acquisition and resource communities on the other. This may result in misinterpreted requirements and lead to systems which:
do not perform to the level required by the warfighter, cost too much, or are not delivered when needed.
This certification course will help address that challenge. With this training, you will be better able to establish a common understanding between these two communities on how the requirements are developed and how they are turned from performance requirements on paper into acquisition programs. Ultimately it will help you ensure the right capabilities are delivered to the warfighter.
As requirements managers, we need to make sure we get the requirement right, and I think that means four things.
First, we need to ensure the performance requirements defined by the warfighter are feasible given available or developing technologies. We have to be realistic about the amount of technological risk we are assuming.
Second, the requirement must have an affordable solution. If it is too expensive, chances are we’ll never get it in the quantities we need.
Third, the system must be delivered on time. An overly complex or rigid requirement can result in a system that’s late-to-need and may well be useless.
Fourth, and most importantly, we need to make sure the requirement meets the needs of the combatant commanders.
To meet these four imperatives, we must engage key players within the Services, Agencies, combatant commands, Joint Staff and the Office of the Secretary of Defense early in the process. All have important roles, and we need to make sure their voices are heard. In doing this job you must think of yourself as the warfighter’s advocate, and understand the perspectives of the various players to find the right balance between capability investments and operational risk. Decisions to pursue new capabilities without sufficient engagement up front can result in:
pursuing the wrong solution
multiple efforts for the same capability, or
not delivering what the warfighter really needs.
From a warfighter’s perspective, an approach or solution by one Service may not meet the needs of all combatant commanders. The Services often develop new capabilities based on their existing and future doctrine with a service-centric approach. Combatant command requirements, however, are based on their OPLANs and CONPLANs. And remember, the joint warfighter requires capabilities that can be operated across Service boundaries.
As a Requirements manager we need you to help reconcile the natural tensions that exist in these two perspectives. By engaging with the Combatant Commanders and their staffs early you will ensure we understand how they intend to employ these future capabilities and make sure their needs are accurately represented to the services. The requirements manager must help ensure we deliver capabilities with joint operations and versatility designed in from the start.
In the war we are now fighting in Iraq and Afghanistan, we have seen many battles fought in extremely limited battlespace, with every Service directly involved in the fight. We have seen soldiers and Marines fighting side-by-side, calling in Navy and Air Force close air support with Special Operations Forces in critically close quarters, all taking place on a battlefield that includes non-combatants, religious shrines, and friendly indigenous forces. More than ever, the tools of our trade must be precise and must fit together. We need to be able to see each other and talk to each other and reduce the fog of war.
We’ll never get there if we don’t get the requirements right at the outset of the process. As you proceed through this course, I ask you to remember the impact you have when you write, review, interpret, or develop new capabilities for the warfighter.
Lack of JROC-validated requirements document for basic program (ORD, CDD, CPD)
Inadequate requirements for basic program and any increments
Critical dependence on external programs with developmental issues
Lack of inter- and intra-departmental stakeholder coordination and support
Joint Capabilities Integration and Development System (CJCSI-3170)
Program, Planning, Budgeting, and Execution (DoD-7000, FMRs)
Defense Acquisition System (DoDI-5000)
*Systemic Issues of Nunn-McCurdy “Class of 2007” Programs
Synchronize JCIDS, DAS, and PPBE to deliver capabilities to Warfighters.

6. Summary of Key SE Initiatives and Reports
Common program issues:
-Poor program planning -Unrealistic estimates
-Unstable requirements -Immature technology
-Not following SE processes -Insufficient SE expertise
-Ineffective reviews -Poor system reliability
Other sources:
Program Support Reviews (PSRs)
Nunn-McCurdy breaches
Congressional oversight
Defense Science Board
Other studies, reviews, reports
2003
2006
2008
2009
2010
NDIA Top 5
SE Issues [1/03]
DODI [5/03]
Defense Acq Guide (DAG); Chap 4: SE [7/06]
OSD policy, guidance, reviews: SEP; IMS; training; risk mgmt; PSRs
NDIA Top SE Issues [7/06]
NDIA Top SW Issues [9/06]
NRC/USAF Study: Pre-MS A and Early SE [1/08]
DODI updates [12/08]
Systemic Root Cause Analysis (SRCA) [draft]
PL (WSARA) [5/09]
DTM Implementation of WSARA [12/09]
QDR [2/10]
House Armed Services Committee report [3/10]
OSD WSARA report to Congress [3/10]
NDIA Reports
DoD policy and actions
Other (Congress, studies, reviews)
There is no shortage of sources confirming we have issues –
How do government and industry collaborate to act upon solutions?

7. Secretary of Defense Direction
Chief among institutional challenges facing the Department is acquisition.”

8. Weapons Systems Acquisition Reform Act
“The key to successful acquisition programs is
getting things right from the start with sound
systems engineering, cost estimating, and
developmental testing early in the program cycle.
The bill that we are introducing today will require the
Department of Defense to take the steps needed to
put major defense acquisition programs on a sound
footing from the outset. If these changes are
successfully implemented, they should help our
acquisition programs avoid future cost overruns,
schedule delays, and performance problems.”
–Senator Carl Levin, Chairman, Senate Armed Services Committee
“The Weapon System Acquisition Reform Act of
2009 is an important step in efforts to reform the
defense acquisition process. This legislation is
needed to focus acquisition and procurement on
emphasizing systems engineering; more effective
upfront planning and management of technology
risk; and growing the acquisition workforce to meet
program objectives.”
–Senator John McCain, Ranking Member, Senate Armed Services Committee

9. P. L. 111-23: Weapon Systems Acquisition Reform Act (WSARA) – May 2009
Key Elements of Legislation:
Organizational
Establishes Directorates for SE and DT&E as principal advisors
Joint tracking of component and MDAP progress against plans and measurable criteria with annual reporting to Congress
Independent cost estimation and cost analysis (Director, CAPE)
Provide adequate trained staff for SE and development planning
Conduct MDAP performance assessments and root cause analysis
Role of SE across program lifecycle
Developmental planning, lifecycle management, sustainability
Ensure reliability, availability, maintainability (RAM)
Mandates
Measurable performance criteria in SE/DT&E plans
Competitive prototypes for MDAPs; prime make/buy analyses
System PDR before MS B, with formal MDA assessment
Assessment of technical maturity and integration risk of critical technologies during Technology Development (TD)
Technical and cost oversight
Independent estimates, Problem Assessment Root Cause Analysis
Technical analysis of cost/schedule breaches; presumed termination
27-Mar-1727-Mar-1727-Mar-1727-Mar-17

10. 27-Mar-17

11. Key Acquisition Business Process ChangesX
Increased Emphasis on Milestone A
Mandatory for MDAPs with Technology Development Programs
Likely for Most Programs
When PDR is Conducted after Milestone B an MDA Post-PDR Assessment is Required
Preliminary Design Review (PDR) before Milestone B to Enhance Understanding of Derived Requirements and Improve Cost Estimation
Post-Critical Design Review Assessment – A Mandatory Decision Point to Review Progress
MS A
MS B
MS C X
FRP DR
ICD
MDD
Materiel Solution Analysis
PCDRA
Technology Development
PDR
P-PDRA
Engineering & Manufacturing Development
PDR
CDD
Production & Deployment
CPD
Operations & Support
Competitive Prototyping
Re-structured “EMD” Phase
Effective Contracting via Pre-Award Peer Reviews
Materiel Development Decision – Mandatory Process Entry Point
Enhanced Emphasis on:
Technology Maturity
Systems Engineering
Integrated Testing and Test Planning
Manufacturing and Producibility
Logistics and Sustainment Planning
Competitive Prototyping
Configuration Steering Boards Established to Stabilize Requirements

12. 27-Mar-17

13. 27-Mar-17

14. Technology and Manufacturing Readiness Test and DemonstrationB C
IOC
FOC
Materiel
Solution
Analysis
TECHNOLOGY
DEVELOPMENT
Engineering &
Manufacturing
Development
PRODUCTION &
DEPLOYMENT
OPERATIONS &
SUPPORT
Materiel
Development
Decision
FRP
Decision
Review
Post CDR
Assessment
TRLs 1-3
TRL 4
TRL 5
TRL 6
TRL 7
TRL 8
TRL 9
Technology
Readiness
Levels
Defense Acquisition
Guidebook
para
Analytical/
Experimental
Critical
Function/
Characteristic
Proof of
Concept
Component
And/or
Breadboard
Validation
In a
Laboratory
Environment
Component
And/or
Breadboard
Validation
In a
Relevant
Environment
System/
Subsystem
Model or
Prototype
Demonstrated
In a Relevant
Environment
System
Prototype
Demonstrated
In an
Operational
Environment
Actual
System
Completed
Qualified
Through
Test and Demonstration
Actual
System
“Mission
Proven”
Through
Successful
Operations
MRLs 1-3
MRL 4
MRL 5
MRL 6
MRL 7
MRL 8
MRL 9
MRL 10
Manufacturing
Readiness
Levels
Draft MRA Deskbook
May 2008
Manufacturing
Feasibility
Assessed.
Concepts
defined/
developed
Capability to produce
Technology In Lab
Environment.
Manufacturing Risks
Identified
Capability to
Produce
Prototype
Components
Capability to
Produce System/
Subsystem
Prototypes
Capability to Produce
Systems, Subsystems
Or Components in a
Production
Representative
Environment
Pilot Line
Capability
Demonstrated.
Ready for LRIP
Low Rate
Production
Demonstrated.
Capability In
Place for FRP
Full Rate
Production
Demonstrated.
Lean Production
Practices In Place
Manufacturing
Cost Drivers
Identified
Cost Model
Constructed
Detailed Cost
Analysis
Complete
Cost Model Updated
To System Level
Unit Cost Reduction
Efforts Underway
Engineering
Cost Model
Validated
LRIP Cost
Goals Met
Learning Curve
Validated
FRP Unit
Cost Goals
Met
Section 2366b of Title 10, United States Code, requires certification that: the technology in the program has been demonstrated in a relevant environment to enter Milestone B. [TRL 6] 14

15. DoD Strategic Management Plan (7/09) Key acquisition-related excerpts
Goals
Measures
Key Initiatives
Improve acquisition processes and execution to support warfighter reqts
# of months to IOC
Acquire systems through evolutionary acquisition
Increase use of fixed price contracts
% of contracts at MS B that are FP
% of contracts at MS C that are FP
Milestone decision reviews
Increase % of MDAPs initiated with low technical risk
# of MDAPs initiated in GFY
% of MDAPs with technologies demonstrated in a relevant environment (i.e., TRL 6)
Competitive prototyping
PDR before MS B
Independent Technical Readiness Assessments (TRA)
Assess programs as directed in WSARA
% of Problem Assessment Root Cause Analysis (PARCA) offices
% of programs assessed
Root cause analysis
Ensure supportability, maintainability and costs are considered in lifecycle
% programs meeting DoD sustainment metrics
Establish sustainment metrics reporting (availability, reliability, TOC)
Implement recommended next-gen sustainment strategies
Focus R&D to address warfighting requirements
% of completing demonstration programs transitioned per year
Track programs transitioning with CY
Review RDT&E funding for transition
Sponsor technology leadership strategy outreach to industry, academia
# of 2-day dialogues with academia technology leaders
Understand investment drivers and strategies to sustain tech leadership
Right-shape and re-balance the acquisition workforce - goal to grow by 20,000 positions by FY2015
% certification positions filled (annual)
Annual increase in end-strength
# contractor work-yr equiv in-sourced
Grow the acquisition workforce
Achieve 10,000 acquisition positions through in-sourcing

16. What’s Next? Areas of emphasis: Defense workforce
Defense Strategy
Defense objectives, emerging threats
Rebalancing the Force
Counterinsurgency, counterterrorism, WMD, cyber
U.S. force structure: sizing, shaping, evolution
Defense workforce
Supporting troops, deployment
Recruiting, retention, development
Strengthening relationships
U.S. defense posture, interagency, abroad
Reforming How We Do Business
Rapid acquisition, security, how we buy, exports
Strengthening the industrial base
Strategic approach to climate and energy
Defense Risk Management Framework
Operational, force management, institutional, future challenges, strategic, military, political
A U.S. force “prepared to conduct a wide variety of missions under a range of different circumstances.”

17. Summary DODI 5000.2 and WSARA are changing the game Questions?
Early life cycle planning (SEP)
Adherence to effective SE practices
Independent cost estimates
Competitive prototyping
Managed technology risks (TRL)
Increased emphasis on reliability and supportability (RAM)
Congressional reporting and oversight
More fixed priced contracts and evolutionary acquisition likely
Revitalization of DoD SE and acquisition workforce
Questions?
Geoff Draper )

18. Backup
27-Mar-1727-Mar-1727-Mar-1727-Mar-17

20. Addressing SE Issues – Key NDIA Task Groups and Milestones
Government Studies/Reports (GAO, NRC, DCMA, etc.)
Reviews (QDR, etc.)
Program Support Reviews (PSRs)
Congressional oversight (Nunn-McCurdy, McCain-Levin, …)
Working Groups
Conferences
Forums
2003
2006
2007
2008
2009
NDIA Top 5
SE Issues
NDIA Top 5 SE Issues
NDIA Top SW Issues
Defense Software Strategy Summit
SE Effectiveness Survey
Systemic Root Cause Analysis (SRCA) [draft]
Industrial Committee on Program Mgmt (ICPM)
SE of Tactical Air Launched Systems (Gulf Coast Chapter) (Air Armament Center)
Top SE/SW Issues Update (planned)
Industrial Committee on SE (ICSE) (planned)
Common program issues:
-Poor program planning -Unrealistic estimates
-Unstable requirements -Immature technology
-Not following SE processes -Insufficient SE expertise
-Ineffective reviews -Poor system reliability
There is no shortage of sources confirming we have issues –
How do government and industry collaborate to act upon solutions?

21. NRC Study – Pre-MS A and Early-Phase SE
Findings
Recommendations
Attention to a few critical systems engineering processes and functions particularly during preparation for Milestones A and B is essential to ensuring that Air Force acquisition programs deliver products on time and on budget.
Require that Milestones A and B be treated as critical milestones in every acquisition program and that … the “Pre-Milestone A/B Checklist” … be used to judge successful completion.
The creation of a robust systems engineering process is critically dependent on having experienced systems engineers with adequate knowledge of the domain relevant to a contemplated program.
Assess needs for officers and civilians in the systems engineering field and evaluate whether either internal training programs … or external organizations are able to produce the required quality and quantity of systems engineers and systems engineering skills. …
The government, FFRDCs, and industry all have important roles to play throughout the acquisition life cycle.… Source selection for system development and demonstration should not be made until after the work associated with Milestones A and B is complete.
Decisions made prior to Milestone A should be supported by a rigorous systems analysis and systems engineering process involving teams of users, acquirers, and industry representatives.
The Air Force used to have a development planning organization that applied pre-Milestone A systems engineering processes to a number of successful programs, but that organization was allowed to lapse.
A development planning function should be established in the military departments to coordinate the concept development and refinement phase
of all acquisition programs to ensure that the capabilities … as a whole are considered and that unifying strategies such as … interoperability are addressed.
Reference: Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Acquisition.

22. NDIA Top 5 SE Issues NDIA SE Division (July 2006)
Provides an update and status from a previous task group report conducted in 2003.
Issues
Recommendations
1. Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle.
Ensure institutionalization of effective SE practices into program planning and execution
2. Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development.
Integrate engineering planning within the acquisition life cycle to ensure adequate time and effort for SE early in the program life cycle
3. Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs
Emphasize the application of SE practices and resources to the capability definition process to address warfighter needs and translation into executable programs.
4. The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry.
Grow SE expertise through training, career incentives, and broadening “systems thinking” into other disciplines.
5. Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels.
Strengthen and clarify policy and guidance regarding use of collaborative environments, models, simulations, and other automated tools.
Reference: NDIA Top 5 Systems Engineering Issues Report. July NDIA Systems Engineering Division. [report, briefing]

23. NDIA Top Software Issues NDIA SE Division (Sept 2006)
Recommendations
1. The impact of requirements upon software is not consistently quantified and managed in development or sustainment.
Enforce effective software requirements development and management practices, including assessment of change impacts, for both the acquirer and the supplier organizations.
2. Fundamental system engineering decisions are made without full participation of software engineering
Institutionalize the integration and participation of software engineering in all system engineering activities.
3. Software life-cycle planning and management by acquirers and suppliers is ineffective.
Establish a culture of quantitative planning and management, using proven processes with collaborative decision-making across the software life cycle.
4. The quantity and quality of software engineering expertise is insufficient to meet the demands of government and the defense industry.
Collaborate on innovative strategies to staff to appropriate levels, and to attract, develop, and retain qualified talent to meet current and future software engineering needs in government and industry.
5. Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems.
Study current software verification practices in industry, and develop guidance and training to improve effectiveness in assuring product quality across the life cycle.
6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments.
Collaborate with industry to develop approaches, standards, and tools addressing system assurance issues throughout the acquisition life cycle and supply chain.
7. Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk.
Improve and expand guidelines for addressing total lifecycle COTS/NDI issues.
Reference: NDIA Top Software Engineering Issues Report. Sep NDIA Systems Engineering Division. [report, briefing]

24. Systemic Root Cause Analysis (SRCA) - Conducted 2007-2008, publication pending
SRCA workshops and task group
Based on PSR findings (44 programs)
Tagged to core/systemic root causes
95 preliminary recommendations
48 systemic issues
3 Recommendation Areas
Recommendation Area
Problem Statement
Industry Actions*
Implement Achievable Acquisition Strategy and Planning
Acquisition strategies and plans are incomplete, ineffective and unrealistic, resulting in unachievable program expectations
Require the government, prior to RFP release, to provide industry with govt. expectations and common understanding for IMP/IMS/risk management/business rhythms
Enhance Gate Review Process
Lack of timely process and adequately defined and enforceable criteria to assess program maturity at milestones and linkage to technical reviews
Define criteria for trigger conditions
Enhance Staff Capabilities
Staffing shortfalls (numbers, skills, and experience) lead to adverse acquisition consequences specifically in the areas of requirements, planning, execution and expectations
Develop and validate a representative staffing model for DoD based on industry that can be applied to the govt. given the current OSD acquisition guidance
Develop a workload analysis to estimate the numbers and expertise needed in the acquisition workforce
Broaden expertise to enhance cross-functional and domain knowledge and skills
* Reference SRCA briefing for additional details and recommended government actions.
© 2006 by Carnegie Mellon University 24

25. SE of Tactical Air Launched Weapons NDIA Gulf Coast Chapter (2008)
SE Deficiencies
Root Causes
Program Structure and Control
Insufficient Maturity Of Design At Critical Decision Points
•Insufficient Testing And Analysis Planned To Achieve Maturity
•Late Integration Of Production Critical Processes And Controls
•Program Funding Profiles Not Structured For Improved Practices
Requirements And Verification
Lack Of Service Use Profile Leaves Interpretive Requirements
•Insufficient Mapping Of Requirements To Design
•Ineffective Maturation And Verification Planning
Design Best Practices
•Inadequate Design Analysis
- Fault Tree Analysis On All Subsystems During Design
- Single Point Failure Analysis During Design
•Inadequate Maturation Analysis And Testing
- COTS Integration
- Design Margin And Sensitivity Development
- Critical Manufacture And Assembly Process ID / Control
•Use Of “Production Representative” Configuration For Verification
Risk Identification And Management
•Inadequate Relationship To “Knowledge” Of Design
Supply Chain Practices
•Inconsistent Approaches To Design Characterization
Reference: “Systems Engineering of Tactical Air-Launched Weapons: An Industry Examination.”
Air Armament Symposium, [briefing]

26. A Survey of Systems Engineering Effectiveness (2007)
Joint NDIA/SEI survey of 46 programs correlating the effectiveness of SE processes with program performance.
SE processes most strongly correlated with better program performance:
Architecture
Trade Studies
Technical Solution
IPT Capability
Requirements Development and Management
Report:
documents/08.reports/08sr034.html
Projects with better Systems Engineering Capabilities deliver better Project Performance (cost, schedule, functionality)

27. Defense Acquisition System Weighted Expenditures
DoDI Perspective
Program Initiation A B C
IOC
FOC
Materiel Solution Analysis
Engineering and
Manufacturing Development
Technology Development
Production & Deployment
Operations &
Support
Materiel Development Decision
FRP Decision
Review
Post-CDR
Assessment
LRIP/IOT&E
Pre-Systems Acquisition
Systems Acquisition
Sustainment
Warfighter and Sustainment Organization Perspective A B
(Program
Initiation) C
Materiel
Solution
Analysis
The point of this chart s that the system is with the warfighter and supporter a lot longer than it was in development. So, a good reason to make sure systems are reliable and supportable up front.
Technology
Development
Engineering and
Manufacturing
Development
Production &
Deployment
Operations & Support
Materiel
Development
Decision
FR5P
Review
Decision
LRIP/IOT&E
Assessment
Post-CDR
Pre-Systems Acquisition
Systems Acquisition
Sustainment
65-80%
20-35%
30+ YEARS
Nominal Life Cycle Cost Distribution