1. Presented to: NDIA PMSC By: Keith Kratzert Date: January 29, 2009 Federal Aviation Administration Improving Program Performance at FAA

2. 2 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 2 NEWS FLASH! On January 22, 2009 in the update of their High-Risk List, the GAO removed the FAA ATC Modernization Program from their High-Risk List “GAO is removing this program from its 2009 High-Risk List because of FAA’s progress in addressing most of the root causes of its past problems and the agency’s commitment to sustaining progress.”

3. 3 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 3 EVM at FAA FAA goal for Major Programs is to manage Major DME programs using EVMS in accordance with OMB Circular A-11 and ANSI/EIA 748 –Manage/measure all resources –Program level, not just contractors –All contractual effort measured, including FFP –Requires changes in behavior by all parties

4. 4 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 4 Recent Accomplishments Established Portfolio/Program Performance Metrics (PPPM) –Now Implementing Agency Wide Closing out first contractor acceptance review First Program Level IBR conducted, second in planning First program surveillance results documented

5. 5 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 5 FAA P 3 M Program Summary Metrics Supporting Metrics

6. 6 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 6 Contractor Acceptance Review Progress Assistance Visit conducted July 9 – 10, 2008 Validation Review conducted September 22 – 26, 2008 Most recent review of CAR closeout activity January 22, 2009 Anticipate formal acceptance in February 2009

7. 7 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 7 Program Level IBR Conducted IBR on the Surveillance and Broadcast System (SBS) October 10, 2008 –One day review –Six CAM interviews –Two outside technical experts Next is the System Wide Information Management (SWIM) Program, February 2009

8. 8 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 8 Surveillance Results The purpose of the Primary Program Performance Surveillance was to verify the reliability and consistency of program performance data Results: Data is Reliable: ASDE-X ASR-11 ATCBI-6 NEXCOM TFM WAAS 627% Some data is questionable: ADS-B ASR-9 Ph II ATOP ASWON ERAM IFPA STARS SWIM TAMR II 940% Questionable Data: ITWS SASO 210%

9. 9 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 9 Current Evaluations of FAA EVM GAO report published GAO-08-756 DOT IG, review of EVM and Security –Draft report out

10. 10 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 10 GAO-08-756, Recommendations Modify acquisition policies governing EVM to: –require the use of a product-oriented standard work breakdown structure –enforce existing EVM training requirements and expand these requirements to include senior executives responsible for investment oversight and program staff responsible for program oversight, and –define acceptable reasons for rebaselining and require programs seeking to rebaseline to (1) perform a root cause analysis to determine why significant cost and schedule variances occurred and (2) establish mitigation plans to address the root cause. Direct the ERAM program office to work with FAA’s Value Management Office to: –determine the root causes for the anomalies found in the contractor’s EVM reports and –develop a corrective action plan to resolve these problems. Direct the Value Management Office to improve its oversight processes by: –including an evaluation of contractors’ performance data as part of its program assessment criteria, when FAA has the authority to do so, and –distinguishing between programs that collect earned value data on fully integrated programs and those that do not in its agency wide progress reports to provide transparency to decision makers.

11. 11 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 11 FAA EVM Summary Continuing to evolve both policy and practice Initiated Program Level Surveillance Initiated FAA Major Contractor Acceptance Addressing GAO recommendations

12. 12 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 12 Backup information

13. 13 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 13 GAO Report GAO-08-756 “FAA has taken important steps to oversee program compliance with EVM policies, but its oversight process lacks sufficient rigor.” Seven Key Components of an Effective EVM Policy Policy componentAssessment of FAA policy Establish clear criteria for which programs are to use EVMFully met Require programs to comply with national standardsFully met Require programs to use a standard structure for defining the work products that enables managers to track cost and schedule by defined deliverables (e.g., hardware or software component) Partially met Require programs to conduct detailed reviews of expected costs, schedules, and deliverables (called an integrated baseline review) Fully met Require and enforce EVM trainingPartially met Define when programs may revise cost and schedule baselines (called rebaselining)Partially met Require system surveillance—routine validation checks to ensure that major acquisitions continue to comply with agency policies and standards Fully met

14. 14 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 14 FAST.FAA.GOV

15. 15 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 15 FAA EVM

16. 16 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 16 Contractor Pre-Acceptance

17. 17 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 17 Contractor Acceptance Process

18. 18 Federal Aviation Administration Improving Program Performance at FAA January 29, 2009 18 FAA Contractor Acceptance An EVM Systems Implementation Plan is prepared by the contractor and reviewed by the FAA. The contractor EVMS is designed and implemented. The Validation Review Plan is prepared by the FAA and provided to the contractor. An EVMS Progress Assistance Visit may be conducted (Optional). The Validation Review is conducted; the Validation Review Report is prepared. Corrective actions are identified, addressed and resolved. The contractor EVMS is accepted and the Letter of Systems Acceptance or Advance Agreement is issued to the contractor. Usually, Agencies require an acceptable surveillance plan and system change management plan from the supplier to obtain an Advance Agreement. EVMS surveillance commences.