1. Lead Systems Integrator Approach to Major Systems Procurement The Good The Bad, and The Ugly Presented to Huntsville Regional Chapter INCOSE August 21, 2008 Robert L. Lord, Ph.D.

2. What are LSIs? What do LSIs do? The role of Systems Engineering in LSI contracting Are LSIs necessary? What are the pitfalls to using the LSI approach? Case Study: U.S. Coast Guard Integrated Deepwater Systems (IDS) Program (Deepwater) Legislation regarding Lead Systems Integrator approach to systems procurement References Lead Systems Integrator (LSI)

3. What is a Lead Systems Integrator? (1) A lead systems integrator is a contractor, or team of contractors, hired by the federal government to execute a large, complex, defense-related acquisition program, particularly a so-called system-of-systems (SOS) acquisition program. An SOS program is aimed at acquiring a collection of various platforms (i.e. [sic], ground vehicles, aircraft, and ships) that are to be linked together by computer networking technology so as to create a larger, integrated overall system.

4. What is a Lead Systems Integrator? (1) Section 805 of the FY2006 National Defense Authorization Act defines two types of LSI: 1.Prime contractors who develop major systems* that are expected at the time of contract award to perform a substantial portion of the work on the system and major subsystems; 2.Contractors who perform acquisition functions that are closely associated with inherently governmental functions in the development of a major system*. A third type of LSI combines the two to a degree such that: 3. The prime contractor performs acquisition functions that are closely associated with inherently governmental functions while performing a substantial portion of the work on the system and/or major subsystems. * Major systems are defined in the National Defense Authorization Act as systems for which the total expenditures for research, development, test and evaluation (RDT&E) are estimated to be more than $155 million, or for which the total amount projected for procurement is estimated to be more than $710 million. (1) contractors who develop major systems expected at the time of contract award to perform a substantial portion of the work perform acquisition functions that are closely associated with inherently governmental functions

5. What do Lead Systems Integrators Do? (1) LSIs can have broad responsibility and latitude for executing their programs, and may perform some or all of the following functions: Requirements generation Technology assessment and development Source selection Procurement of systems or components Management of supplier firms Construction or modification work Risk assessment and management Verification and validation Life-cycle cost estimation Administration Which functions create the potential for a conflict of interest to arise for the LSI? Why is understanding the LSI concept important to Systems Engineers ? LSIs perform many functions traditionally performed by the government contracting officer or other government officials.

6. Role of Systems Engineering in LSI Contracting Requirements Metrics Modeling and Simulation Technology Readiness Assessment Verification, Validation, and Accreditation Risk Assessment and Tracking Life Cycle Cost Analysis System of Systems Integration System Life Cycle Support Trade Studies/Make or Buy Decisions

7. Why are LSIs Necessary? (1) Government lacks in-house technical and project- management expertise to execute large, complex acquisitions programs. Fifty percent downsizing of government acquisition workforce between 1994 and 2005 Increase in scope/level of government procurement activity Growing complexity of systems being acquired Stove-pipe effect within and between governmental agencies System integration and optimization require tailoring of subsystems (compromise between agencies) Difficult for government acquisition agencies to keep pace with technical innovation

8. Potential Problems with the LSI Concept (1) Lack of transparency to customer Program costs System optimization Source-selection procedures System technical performance Potential conflicts of interest for LSI Self-Certification Make – Buy decisions Lack of competition after initial LSI award Option awards Re-competition Follow-on programs

9. Examples of LSI Managed Programs (1) National Missile Defense Program – DoD Missile Defense Agency/Boeing U. S. Army Future Combat Systems – U.S. Army/Boeing- SAIC U. S. Coast Guard Deepwater Program – Department of Homeland Security/Integrated Coast Guard Systems U. S. Air Force Transformational Communication System – U.S. Air Force/Booz Allen Secure Borders Initiative (SBInet) - Department of Homeland Security/Boeing

10. U.S. Coast Guards Integrated Deepwater Systems (IDS) Program Awarded to Integrated Coast Guard Systems (ICGS) in June 2002 (2) Largest and most complex acquisition program in Coast Guard history (3) (Pre-9/11 - $17 B over 30 years) (2) (Post–9/11- $24 B over 25 years) (3) Project to replace and modernize the Coast Guards fleet of deepwater-capable ships and aircraft (3) Scope of work includes develop and deliver an improved, integrated system of ships, aircraft, unmanned aerial vehicles, C 4 ISR (Command, Control, Communications, Computer, Intelligence, Surveillance, and Reconnaissance), and supporting logistics. (2) Performance-Based Acquisition (Indefinite Delivery, Indefinite Quantity (ID/IQ - Award Fee) On March 1, 2003, the Coast Guard, originally part of the DOT, became a part of DHS.

12. A critical element of the Deepwater program will be the modernization of C4ISR (command, control, communications, computers, intelligence, surveillance and reconnaissance) systems. The C4ISR System of Systems Enterprise establishes common software, systems, and components across all surface, air and shore assets. The Coast Guards common Command and Control System (CG-C2) provides interoperability, improved situational awareness and new levels of Maritime Domain Awareness (MDA). This provides operational commanders the tactical awareness, planning and decision tools required to prosecute their missions more effectively. C4ISR includes robust communications systems establishing clear and secure voice and data communication across the broad frequency spectrum required by USCG operations, and improved INTEL capabilities (collection, dissemination, fusion and analysis, and INTEL integration with national systems and operational forces is also provided). Most importantly, the C4ISR solution networks each assets passive and active sensor information enhancing the USCG Common Operational Picture (COP). Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance (C4ISR) interoperability, improved situational awareness and new levels of Maritime Domain Awareness (MDA) enhance the USCG Common Operational Picture (COP) common software, systems, and components across all surface, air and shore assets robust communications systems; clear and secure voice and data communication; improved INTEL capabilities tactical awareness, planning and decision tools

13. U. S. Coast Guard Procuring Agency Integrated Coast Guard Systems, LLC (ICGS) Prime Contractor Lockheed Martin (LM) First Tier Subcontractor Northrop Grumman Ship Systems (NGSS) First Tier Subcontractor ICGS, LLC is a business entity jointly owned by Northrop Grumman Ship Systems and Lockheed Martin Corporation. (2) Department of Homeland Security Parent Organization Deepwater Acquisition Structure (2)

14. Maximize Operational Effectiveness Minimize Total Ownership Cost Satisfy the Customer Deepwater Program Goals (2)

15. Search and Rescue Cocaine Seizure Rate Illegal/Undocumented Migrant Interdiction Foreign Fishing Vessel Interdiction Protection of Living Marine Resources National Defense/Military Readiness International Ice Patrol Deepwater Missions, Performance Metrics, and Assessment Results from Fiscal Year 2004 (6) Percentage of lives saved after CG notification in cases with Deepwater participation Percentage of cocaine shipped through transit zone that is seized by Deepwater assets Percentage of total migrant flow interdicted with Deepwater assets Percentage of foreign fishing vessels detected in U.S. Economic Enforcement Zone interdicted with Deepwater assets Percentage of Deepwater living marine resources law enforcement boardings without significant violations Maintain 100% combined readiness of high- endurance cutters and patrol boats to support DoD requirements Maintain 95% accuracy of all limit of all known ice broadcasts Performance Goal Results 93.0 92.6 6.5 Not Determined 32.0 41.9 6.7 4.5 97.0 94.8 100.0 99.3 98.0 97.8 Are these metrics important? Why or why not? Who develops these metrics? Are the Metrics usable in measuring and monitoring program progress?

16. GAO Outlines Deepwater Risks in Testimony to Committee on Transportation and Infrastructure, House of Representatives, 3 May 2001 (Prior to contract award) (9) Area of RiskRisk LevelReasons for Assigning Level of Risk Attaining a stable, sustained funding level High Several years of funding below planned funding levels can have adverse consequences for the acquisition strategy. Budget constraints and other budget priorities threaten USCG ability to achieve large, sustained increases in its budget for capital spending. Controlling costs in the contracts later years Moderate to High The risks center on the potential lack of future competition and reliance on a single contractor to procure the entire system. The level of risk depends on the effectiveness of provisions the USCG designs and includes in the contract to encourage or require subcontract competition and increases its leverage in negotiating future contracts with the prime contractor. Overseeing the Acquisition Moderate... there are many uncertainties about contract management as the USCG increases the size of the administrative effort,.... Using unproven technology Low to Moderate The steps needed to mitigate this risk are relatively few and straightforward. The lack of assessment tools to measure technology maturity poses short-term (low) and long-term (moderate) risk.

17. Key components needed to manage the program and oversee the system integrators performance have not been effectively implemented. Integrated Product Teams (IPT), the Coast Guards primary tool for overseeing the system integrator, have struggled to effectively collaborate and accomplish their mission. The Coast Guards assessment of the system integrators performance in the first year of the contract lacked rigor. Comments by technical specialists responsible for monitoring design and delivery of ships were not included in the evaluation score. Factors that formed the basis of the award fee determination were unsupported by quantifiable metrics. Coast Guard has not yet begun to measure the system integrators performance on the three overarching goals of Deepwater, i.e., Operational Effectiveness, Total Ownership Costs, and Customer Satisfaction. Coast Guard officials had not projected a time frame for when they would be able to hold the system integrator accountable against these goals. Coast Guards original plan for measuring the system integrators performance on an annual basis had slipped. Coast Guard had neither measured the extent of competition nor held the system integrator accountable for taking steps to achieve competition. Deepwater acquisition structure allows two first-tier subcontractors to have sole responsibility for determining whether to hold competitions or to provide assets themselves. Coast Guard had taken a hands off approach to make or buy decisions made by the system integrator. March 2004 GAO Findings (Two years into the program) (2)

18. Key components of management and oversight are not effectively implemented. Put into place a human capital plan to ensure adequate staffing of the Deepwater program. Improve integrated product teams responsible for managing the program by providing better training, approving charters, and improving systems for sharing information between teams. Provide field personnel with guidance and training on transitioning to new Deepwater assets Procedures for ensuring contractor accountability are inadequate. Develop measurable award fee criteria consistent with guidance from the Office of Federal Procurement Policy. Provide for better input from USCG performance monitors. Hold the system integrator accountable for improving effectiveness of the IPTs. Establish a baseline for determining whether the acquisition approach is costing the government more than the traditional asset replacement approach (USCG refused to accept this recommendation.). Establish a time frame for putting steps in place to measure contractors progress toward improving operational effectiveness. Establish criteria to determine when to adjust the project baseline and document the reasons for change. Control of future costs through competition remains at risk due to weak oversight. For subcontracts over $5 million awarded by the contractor to one or both of the two major subcontractors, require notification of the USCG about decision to do work in-house rather than contracting it out. Develop a comprehensive plan for holding the contractor accountable for ensuring adequate competition among suppliers. April 2006 GAO Findings (Four years into the program) (6, 7)

19. 13 added Matagorda was the first 110 to 123 conversion delivered to the CG Artists drawing of the new 140 Fast Response Cutter

20. 123 Patrol Boat (WPB) Conversion Program Since the FRC was not scheduled for delivery until 2018, conversion programs for existing Island Class Patrol Boats were initiated. To combat hull corrosion and add other capabilities to the 110 Patrol Boat, CG initiated the Hull Conversion Program (HSP) and the 123 WPB conversion program. Original Deepwater Program plan called for all 49 110 WPB to be converted to 123 WPB, ultimately to be replaced by 58 140 FRCs delivered between 2018 and 2022. (6) ICGS awarded contract to LM for 123 WPB conversion program. (13) Eight of the 110 patrol boats in worst condition placed in the 123 conversion program. First 123 WPB ( Matagorda ) delivered March 2004. It became operational in Feb 2005 (6) September 2004 Matagorda experienced hull buckling (6) March 2005 CG suspended work on the 123 conversion program. August 2006 whistleblower accuses LM of knowingly delivering unsafe and ineffective patrol boats to the CG. CG IG upheld whistleblowers allegations. (13) November 2006 CG announces suspension of operations of all eight 123 WPB due to structural problems,. (10) April 2007 all eight boats decommissioned and stripped of equipment. CG admits conversion program a failure. (10)

21. Fast Response Cutter (FRC) (12) Contract for design of 140 FRC let to NGSS, New Orleans, LA. July 2004 ICGS decides to use composites on FRC hull design. March 2005 CG submitted plan to modify FRC to meet post 9/11 mission requirements. Due to problems with the conversion program, revised plan called for delivery of first FRC, originally scheduled to begin in 2018, in 2007 or 2008 (FRC-A). (6) May 2005 CG Engineering Logistics Center (ELC) published a white paper outlining concerns over design process, estimated weight increases, hull form, propulsion, et. al. May 2005 System Requirements Review approved moving to Preliminary Design Review. Sep 2005 Preliminary Design Review Considered successful because it met contractual requirements. ICGS authorized to initiate contract for detailed design of FRC with NGSS. After PDR, CG issued a letter expressing concerns with design and inconsistent total ownership cost data that had to be addressed before CDR. Jan 2006, ICGS identifies cavitation problem stemming from hull design. Feb 2006, an Independent Design Review validated ELC concerns. Feb 2006, after spending $26 M, CG temporarily suspends design work on FRC-A due to technical risk.

22. Composites Technology Assessment (12) Oct 2002 agreement between Northrop Grumman Ship Systems (NGSS) and Kockums AB, and its parent company Howaldtswerke Deutsche Werft AG (HDW) covering design, development, construction and sale of Visby-class ships and/or derivative technology (composite, stealth) to U. S. government and friendly international governments through Foreign Military Sales program. May 2004 NGSS and HDW expand agreement allowing for continued sharing of composite shipbuilding technologies. July 2004 ICGS decides to use composites on CG FRC hull design citing lower maintenance and life cycle costs, longer service life, and reduced weight. CG has no prior experience with composite materials used for vessels with FRC operational requirements. Navy does in Osprey Class Coastal Minehunting ships, the world's largest Glass-Reinforced Plastic (GRP) ships, some built by NGSS. January 2005, CG engineering officials raised concern about viability of the FRC design, e.g., hull form, potential speed, propulsion plant studies, and weight. May 2005 CG Engineering Logistics Center (ELC) published a white paper outlining concerns over design process, estimated weight increases, hull form, propulsion, et. al. September 2005 Preliminary Design Review considered successfully passed.

23. Current Patrol Boat/FRC Activities Due to suspension of the FRC development and the urgent need for patrol boats, CG initiated a dual path procurement strategy: Continue work to mitigate technical risk on FRC (FRC-A) Procure an off the shelf patrol boat (FRC-B) to fill the gap until FRC-A boats are ready. (12) March 2007 CG terminated ICGS responsibilities for Deepwater FRC-B off-the-shelf procurement and reassigned it to the CG Acquisition Directorate. (12) April 2007 CG issued an RFP for the FRC-B. Proposals are currently under evaluation. (14)

24. Current Deepwater Program Status In April 2007, the USCG reclaimed the Lead Systems Integrator role from Integrated Coast Guard Systems. (11) The Deepwater Program is now a collection of more than a dozen USCG acquisition programs for replacing and modernizing its fleet of deep-water capable ships and aircraft. (10) At the end of the five-year baseline term, Integrated Coast Guard Systems was awarded a 43-month first additional award term, reflecting good, but not excellent performance. Their contract now runs thru January 2011. USCG says the 43 months is for completion of acquisition efforts already underway. (10) Dec 2006 Department of Justice notified Lockheed, Northrop, and certain other firms involved in the Deepwater program of an investigation and directed them to preserve all documents relating to the program. Investigation centers on communications systems, the conversion of the Coast Guards 110-foot patrol boats, and the National Security Cutter (NSC). Jan 2008, CG sought a $96 million refund from ICGS for conversion program. June 2008, Justice Department requests CG to temporarily stop pursuing contractual remedies against ICGS.

25. Potential Problems with the LSI Concept (1) Revisited Lack of transparency to customer Program costs System optimization Source-selection procedures System technical performance Potential conflicts of interest for LSI Self-Certification Make – Buy decisions Lack of competition after initial LSI award Option awards Re-competition Follow-on programs

26. Legislation (1) H.R. 1815/P.L. 109-163 [109 th ]. FY2006 Defense Authorization Act Section 805 required DOD to submit a report to Congress on the use of LSIs for the acquisition of major systems. H.R. 5122 / Public Law No: 109-364 [109th]: John Warner National Defense Authorization Act for Fiscal Year 2007 Section 807: … no entity performing lead systems integrator functions in the acquisition of a major system by DoD may have any direct financial interest in the development or construction of any individual system or element of any system of systems. Section 115: Requires the Comptroller General to report to Congress on the FCS Lead Systems Integrator H.R. 1639 [110 th ]: provides that no entity performing lead system integrator functions in the acquisition of a major system by the Department of Homeland Security may have any direct financial interest in the development or construction of any individual system or element of any system of systems (Referred to the Subcommittee on Management, Investigations, and Oversight). S. 680 [110th] Accountability in Government Contracting Act of 2007 : Requires federal government to study the use of LSIs (not yet referred to House) S.889 [110 th ] Deepwater Accountability Act : Prohibits use of LSI by DHS on remaining contracts under the Deepwater Program (referred to the Committee on Commerce, Science, and Transportation).

27. Can the Government completely do away with the LSI concept? What role did the CG play in the Deepwater fiasco? What role did the contractor play in the Deepwater fiasco? What can prime contractors do to prevent this type of fiasco? Considering the Coast Guards inability to manage ICGS as the LSI, do you think they are capable of managing the program? What key program requirements do you think will not be met by the restructured program? Discussion Questions

28. (1)Bailey Grasso, Valerie, CRS Report to Congress: Defense Acquisition: Use of Lead System Integrators (LSIs) – Background, Oversight Issues, and Options for Congress, March 26, 2007. (2)U.S. General Accounting Office, Coast Guards Deepwater Program Needs Increased Attention to Management and Contractor Oversight, GAO-04-380, March 2004. (3)ORourke, Ronald, CRS Report to congress: Coast Guard Deepwater Program Needs Increased Attention to Management and Contractor Oversight, September 6, 2006. (4)Hutton, John P. and Stephen L. Caldwell, Coast Guards Deepwater Program Management Initiatives and Key Homeland Security Missions: Testimony before the Subcommittee on Homeland Security, Committee on Appropriations, House of Representatives, U. S. Government Accountability Office, March 5 2008. (5)U.S. General Accounting Office, Progress Being Made on Addressing Deepwater Legacy Asset Condition Issues and Program Management, but Acquisition Challenges Remain, GAO-05-757, July 2005. References

29. (6) U.S. General Accounting Office, Changes to Deepwater Plan Appear Sound, and Program Management has Improved, but Continued Monitoring is Warranted, GAO-06-546, April 2006. (7) U.S. General Accounting Office Testimony, Status of Efforts to Improve Deepwater Program Management and Address Operational Challenges, GAO-07-575T, 8 March 2007. (8) U.S. General Accounting Office Testimony, Deepwater Program Management Initiatives and Key Homeland Security Missions, GAO-08- 531T, 5 March 2008. (9)U.S. General Accounting Office Testimony, Actions Needed to Mitigate Deepwater Project Risks, GAO-01-659T, 3 May 2001. (10)ORourke, Ronald, CRS Report to Congress: Coast Guard Deepwater Acquisition Programs: Background, Oversight Issues, and Options for Congress, June 5, 2008. (11)Lipowicz, Alice, Troubled waters, Washington Technology, Vol. 22 No. 10, June 11, 2007. References

30. (12)Fast Response Cutter A (FRC-A) WPC Patrol Coastal, GlobalSecurity.org. (13)Axe, David, Whistleblower Call Out Lockheed, Military News, Sep 13, 2006. (14)Fein, Geoff, Coast Guard Delays FRC Selection to September, Early October, Defense Daily, 14 July 2008. References