1. CDC PM SUMMIT Operational Analysis

2. Federal Concierge LLC All Rights Reserved Janelle B. Hill Lead Consultant of Federal Concierge LLC, www.federalconcierge.comwww.federalconcierge.com Forum Director, Federal CPIC Forum www.cpicforum.orgwww.cpicforum.org 10-years+ experience in Life Cycle Solutions & Professional Services to the Federal Government Sector 3 Masters in CIO studies, Information Assurance, and Information Management, proceeding towards Doctorate work in CPIC / Information Management Certified in the Governance of Enterprise IT (CGEIT) Co-Author of the CPIC Guidebook and the Draft CDC Operational Analysis Guide. 5/23/20152

3. Federal Concierge LLC All Rights Reserved What is an Operational Analysis? Operational Analysis is an evaluation process that formally assesses how well an investment is meeting strategic and business objectives, financial goals, customer needs, system performance goals, and how well it is performing within baseline cost and performance goals. An operational analysis is, by nature, less structured than performance reporting methods applied to developmental projects and should trigger considerations of how the investment's objectives could be better met, how costs could be reduced, and whether the organization should continue performing a particular function. A system for this collection of performance data is the key to effective and efficient operational analysis. 5/23/20153

4. Federal Concierge LLC All Rights Reserved Operational Analysis Operational analysis involves –The collection of information concerning a capital asset’s performance –Comparison of this performance with an established baseline Asset performance measures should include –How well the asset supports customers and stakeholders –How well the asset is managed by the agency The outputs of this process are recommendations to agency resource managers as to the asset’s continued use, modification, improvement, or termination 5/23/20154

5. Federal Concierge LLC All Rights Reserved Where Does Operational Analysis Fit into the EPLC Equation? Must be performed on all Operations & Maintenance Investments, or on the O&M components of an investment in Mixed Life Cycle. Commences somewhere between 6 months to 18 months after your Post – Implementation Review (PIR) 5/23/20155

7. Federal Concierge LLC All Rights Reserved What Requires Me To Perform Operational Analysis ? OMB Capital Programming Guide requires and prescribes Operational Analysis. Also Leverage –Post Implementation Review (PIR) –Program Assessment Rating Tool (PART) –Federal Enterprise Architecture (FEA) Performance Reference Model (PRM) –GAO Cost Assessment Guide –GAO Audits or “High Risk” Information 5/23/20157

8. Federal Concierge LLC All Rights Reserved Effective Monitoring Using Operational Analysis Are you managing your O&M investment to cost & schedule with a WBS? Are you able to predict even a modest partial-point variance or deviation from your cost and schedule? Do you have an OMB approved baseline? Are you regularly monitoring your alternatives to ensure that steady state on this investment is a better management idea than performing the business another way? Is running this investment in O&M the best, most cost-effective or most mission-essential way to leverage the agency-level and department-level inventory of information technology assets? 5/23/20158

9. Federal Concierge LLC All Rights Reserved New CDC Operational Analysis Guide Release scheduled for February 2009 Has been a team effort by the Compass Systems Consulting team and the CDC CPIC Office to gather, review, analyze and select best practices and a best-of-breed approach. Seven agency Operational Analysis methodologies and documents were reviewed for best practices including NOAA, USDA, VA, VHA, DHS, Dept. of Commerce, and HHS. Best-of-breed approach has been blended with CDC processes, the CDC Operational Analysis template and the EPLC framework. The Guide walks the reader through the process for ensuring and documenting compliance as well as ensuring quality, effective reporting of performance to cost and schedule targets. Appendices in the Guide provide helpful examples of performance measures and goals as well as the CDC Operational Analysis template/ form. 5/23/20159

10. Federal Concierge LLC All Rights Reserved Cost & Schedule Performance You should –Periodically be measuring the performance of operational assets against the baseline established during the planning or full acquisition phase (i.e., operational analysis), and –Be properly operating and maintaining the asset to maximize its useful life 5/23/201510

11. Federal Concierge LLC All Rights Reserved Cost & Schedule Performance Operational analysis may identify the need to redesign or modify an asset by –Identifying previously undetected faults in design, construction, or installation/integration –Highlighting whether actual operation and maintenance costs vary significantly from budgeted costs, or –Documenting that the asset is failing to meet program requirements 5/23/201511

12. Federal Concierge LLC All Rights Reserved Who Performs the Operational Analysis? The PPM – Project/ Program Manager is responsible for determining whether or not the system or investment meets expectations –For example Section ID of the OMB Circular 300 describes the strategic performance goals and Section IIIB provides operations and maintenance cost estimates for operating and maintenance –These elements form the baseline for accomplishing an OA 5/23/201512

13. Federal Concierge LLC All Rights Reserved Tools and Techniques There are numerous analytical models and tools that are useful for conducting Operational Analysis (all of which should be a part of your EPLC deliverables and/ or your repository for your project). Examples: –Cost benefit analysis—do benefits outweigh cost? –Trend analysis—investment performance over time –Bench marking—measuring performance against other systems or standards –Lesson Learned—what can be learned from other systems? –Performance against technology strategy—is it inline with CDC’s technology strategy? –Performance against environmental strategy—is it in line with CDC’s environmental management strategy? –Vendor company’s commitment to the product (for off-the-shelf investments)— concern about long term support? –Risk analyses and security assessments—define system vulnerabilities, strengths and weaknesses –Previous OMB 300 reports—provides past technical and cost performance assessments 5/23/201513

14. Federal Concierge LLC All Rights Reserved Estimating O&M Costs Use actual data where it exists Use planned O&M data derived during system design phase Use similar system performance data Consider technology implications (e.g., the use of certain technologies may become more economical in the future Be aware of the new GAO Cost Guide that prescribes how we assess and estimate our costs, including our risk-adjusted costs. 5/23/201514

15. Federal Concierge LLC All Rights Reserved And Your OA Output Includes Recommendations… Provide your conclusions –Investment is ___ is not ___ meeting its performance goals ___ Investment should remain in steady state or mixed life cycle for foreseeable future ___ Investment team should commence planning for DME (Development, Modernization, & Enhancement) in order to ensure continued investment viability ___ Operational Analysis results suggest that investment should be discussed at IT-Investment Review Board ___ Operational Analysis results suggest that IT-Investment Review Board does not need to discuss this investment 5/23/201515

16. Operational Analysis Results The results of your Operational Analysis – will be presented to the IT Governing Body – can be presented with Project reviews –Are essential when conducting the system re- certification –Are essential when conducting the Re-accreditation review –Are an output of the EPLC stage gate review indicated between O&M and disposition. 5/23/2015 Federal Concierge LLC All Rights Reserved 16

17. Federal Concierge LLC All Rights Reserved Thank You For Attending If your office is interested in utilizing support when coordinating and performing your OA efforts, contact: Within CDC: Sandra McGill & Robin Eggleston Compass / Federal Concierge Presenter Contact Information: Janelle B. Hill, Lead Consultant 760-208-0488 mobile – Eastern Standard Time jbb@federalconcierge.com 5/23/201517