1. IT Capital Planning and Management Meeting OMB Exhibit 300 Requirements July 2003

2. Topics All About IT Capital Planning –IT Investment Management –Role of the Exhibit 300 Technical Aspects of the Exhibit 300 –IT Performance Measures –Alternatives and Cost-Benefit Analysis –Earned Value Analysis –Project Risk –Security and Confidentiality –Enterprise Architecture

3. Part 1 – IT Capital Planning Managing and Mitigating IT Risk

4. Basis for Exhibit 300 We “did it” to ourselves –IT projects have poor track records Over budget, over schedule, and don’t meet requirements Don’t find out until very late in the process Clinger-Cohen Act of 1996 –Implemented via OMB Circular A-130 and the Exhibit 300 Reduce risk, increase transparency, and encourage dialog among agency officials and project advocates and managers

5. Legislative Underpinnings Laws and Regulations: Clinger-Cohen Act (ITMRA) Paperwork Reduction Act Government Performance & Results Act (GPRA) OMB Circular A-130 OMB Circular A-11 GAO Guidance Common Themes : Agency plans that emphasis priorities and ROI to American public IT aligned with agency programs and priorities Use IT to improve program and process efficiency & effectiveness IRM operations integrated with organizational management, planning and budgeting processes

6. Clinger-Cohen Act (CCA) Focus on IT results Analyze, track, and evaluate risks and results of all major IT capital investments Cover life of each system Maximize value and assessing & managing IT risks Include explicit criteria for analyzing projected & actual costs, benefits, and risks Means for senior management to obtain timely information regarding progress IT investment process is integrated with processes for budget, financial, and program mgmt decisions

7. OMB Circular A-130 IRM strategic plan, IT capital plan, enterprise architecture, and comprehensive inventory A capital planning and investment selection/control/evaluation process –Investment review board –Management of broad inventory of IT assets (using portfolio approach) –Investment management process Projects evaluated in context of IT portfolio and based on business case, including CBA, ROI, performance measures, and risk clarification A comprehensive information systems security A contemporary IT acquisition approach

8. IT Problems and Challenges …. Long-standing problems persist –Global Risks –Business Practices –Project Practices Risks have increased –Technological landscape is more complex Which vendor’s technology? Microsoft? Sun? Web-enabled? Is it secure enough? Java programming language? Can I find experienced programmers? Let’s understand the nature and types of IT risk!

9. Global Risks (1) IT initiatives not aligned with organizational priorities –Possible Causes: - No “real” agency strategic or annual plan -IT does not have a seat at the table -IT analysts historically have focused on technical aspects of initiatives and do not have a culture that fosters linking initiatives and priorities Inventory of IT Risks

10. Global Risks (2) No information technology strategic plan or enterprise architecture. -Imagine building a house without any architectural plans. -…With only general sketches as to how its supposed to look, or only detailed diagrams for wiring, plumbing, etc. -… With each subcontractor doing whatever they thought best, without consulting with the owner or other contractors. -… With no specialized functions for the rooms (e.g., every room has its own little stove, bed, and bathtub). -… Where a room had to be completely torn down just to remodel it. The same is often true for building information systems in public sector organizations. Inventory of IT Risks

11. Business Practices (1) 1.General preferences to build it in-house rather than find and use an existing solution (parametric design – find one and modify it rather than build it from scratch) 2.Attempting to “fix” or “improve” a business process by developing an automated solution rather than re-design and/or streamline the business process itself. Inventory of IT Risks

12. Business Practices (2) 3.Program and senior management remain distant from information technology initiatives rather than own and endorse them. 4.Approving, reviewing, and managing each IT investment independently, rather than looking at entire inventory of IT investment. Inventory of IT Risks

13. Project Practices (1) 1.Executive level project oversight/control/performance measures 2.Project management risk a. Lacking project management experience/track record b. Lack project management discipline, structuring all aspects of project that consume resources into “service requests” or other format that can be estimated and prioritized c. Lack of configuration management and change control methodology Inventory of IT Risks

14. Project Practices (2) 3.Staffing risk a.Lacking experienced staff b.Significant increase in number of staff c.People problems-morale, turnover, etc. 4.Requirements risk a.Requirements come from managers rather than those that do the work b.Loose, general requirements Inventory of IT Risks

15. Project Practices (3) 5.User involvement risk a.Lack of project advocates b.Insufficient user involvement 6.Technology risk a.Developing with risky tools, methodologies, and vendors Inventory of IT Risks

16. Project Practices (4) 7.Methodological risk a.Problems associated with system development life cycle -Over-planning and under-planning b.Lack of clear technological architecture –Which products and tools? c.Poor/weak application and database analysis and design d.Lack of incremental/modular approach e.Under-estimation or too little testing attention Inventory of IT Risks

17. Project Practices (5) 8. Schedule risk a. Critical path and inter-dependencies not clear 9. Cost sensitivity and risk a. Omitted/hidden costs (conversion, bridges to other systems, and on-going operational costs) b. Risk if specialists are not experienced at estimating cost and schedule for complex IT projects Inventory of IT Risks

18. Project Practices (6) 10. Security and confidentiality 11. Contractor risk a. Over-reliance on contractors b. Coordinating activities of contractors and sub-contractors 12. Operational risk a. Ensuring sufficient size and capacity b. Managing changes Inventory of IT Risks

19. Addressing the Risks The Clinger Cohen Act of 1996 and OMB Circular A-130 outline a variety to planning and management techniques for addressing these risks The objective of these requirements is to improve IT return-on-investment and to better manager IT risk

20. Exhibit 300 Requires Risk Inventory

21. Clinger-Cohen Act (CCA) Focus on IT results Analyze, track, and evaluate risks and results of all major IT capital investments Cover life of each system Maximize value and assessing & managing IT risks Include explicit criteria for analyzing projected & actual costs, benefits, and risks Means for senior management to obtain timely information regarding progress IT investment process is integrated with processes for budget, financial, and program mgmt decisions

22. OMB Circular A-130 IRM strategic plan, IT capital plan, enterprise architecture, and comprehensive inventory A capital planning and investment selection/control/evaluation process –Investment review board –Management of broad inventory of IT assets (using portfolio approach) –Investment management process Projects evaluated in context of IT portfolio and based on business case, including CBA, ROI, performance measures, and risk clarification A comprehensive information systems security A contemporary IT acquisition approach

23. Establishing an Effective IT Investment Review Process Procedures and Staffing

24. Implementing an Effective CPIC A process is needed –Provide information –Review information –Have dialog among decision-makers and proponents of the investment Some areas require special attention –Senior officials will need to be trained –They will require criteria and need to know how to ask tough questions –Advocates for the IT initiative will have to take the Exhibit 300 seriously –Decision-making and funding need to be linked to process

25. IT Investment Management Process Control What are you doing to ensure that the projects will deliver the benefits projected? Evaluate Based on your evaluation, did the systems deliver what you expected? Select How do you know you have selected the best projects? Process Information

26. IT Investment Decision-Making Investment Review Boards review proposals EVALUATION of projects implemented or canceled On-going IT projects Feedback SELECTION and adjustment of of IT portfolio New IT proposals Analysis and ranking of all IT projects Pre-review of proposals before going to IRB CIO Office reviews project progress Places accountability at the top of the Agency! Major purpose is to improve communication!

27. Contents of An Exhibit 300 BENEFITS COST Strategy RISK

28. A-130 “Map” PLANNINGMANAGEMENT SelectionControlEvaluation IT Investment Review Board (IT-IRB) 300B/300 Business Case Cost-Benefit Analysis/ROI Risk Assessment Project Implementation Plan Project Schedule/Budget Enterprise Architecture IRM Strategic Plan IT Capital Plan

29. IT Investment Approach Emphasizes return on investment and risk management Facilitates communication among technical and program managers  Requires business case  Project approval is not automatic  Risks are identified and discussed Project ProposalsIRB Reviews  Program managers decide  Includes pre-review and decision-making criteria  Program managers monitor progress and take corrective action as needed A Basis for Informed Decision-Making

30. Getting Started Get executive management understanding and support of the investment approach & process Develop process suitable for your agency Communicate that process is important internally - - not just to comply with law. It is about enhancing communication about IT investments and placing accountability at appropriate levels! Start gradually and build processes steadily. Cultural change will be difficult and require persistence!

31. Critical Success Factors Executive management support Training and education about business case, cost/benefit analysis, and performance measurement Development and use of Enterprise Architecture Willingness to stop projects with poor business case Commit to monitoring progress and making adjustments Dedicated information management resources

32. Part 2 - Preparing the Exhibit 300 Business Case, CBA, ROI, EVA, and other Contents

33. A-11 Section 300 Since capital budgeting is part of the budget process, Exhibit 300 guidance is contained in the A-11 instead of A-130 –CPIC is a decision-making process for ensuring that IT investments integrate strategic planning, budgeting, procurement, and IT management in support of agency missions and business needs

34. Exhibit 300 Exhibit 300 helps: –OMB understand capital programming –Ensure that IT capital spending supports agency mission and provides ROI equal to or better than alternative uses of funding –Identify poorly performing projects (behind schedule, over budget, or lacking in capability) –Identify capital assets that are no longer fulfilling ongoing mission requirements –Ensure that strong business cases are made for IT investments

35. Reporting Requirements (OMB) New Projects –Projects in initial concept or planning phase will have less detailed and defined specificity –Must be reported as full acquisitions Ongoing Projects –Report all sections of Parts I and II –If cost, schedule, or performance variances exceed 10%, provide a complete analysis of reasons for variances and corrective action Infrastructure Projects –Require different level of information in business case, since infrastructure supports major applications –Business case must still make sense

36. Exhibit 300 Parts I & II: Capital Asset Plan & Business Case I.Summary of project information Summary of spending for project stages I.A.Project description I.B.Justification (All Assets) I.C.Performance goals and measures (All Assets) I.D.Program management (All Assets) I.E.Alternatives analysis (All Assets) I.F.Risk inventory and assessment (All Assets) I.G.Acquisition strategy I.H.Project and funding plan II.AEnterprise Architecture II.B.Security and Privacy II.CGovernment Paperwork Elimination Act (GPEA)

37. Exhibit 300 Part II: Additional Business Case Criteria II.AEnterprise Architecture II.B.Security and Privacy II.CGovernment Paperwork Elimination Act (GPEA)

38. Justification: Link to Strategic Goals

39. Performance Goals & Strategy Use a “movement over time” approach –Start with IT activities –At some point, indicate IT outputs –Show “closely related” outcomes Deal with high-level outcomes at high level, above the measurement table –See next page Completing Exhibit 300

40. Types of Measures Internal –Resource performance (budget units/inputs) –Activity performance (milestones, cycle time, etc.) –Output performance (quantity and quality of goods and services) Customer Satisfaction Outcomes –Immediate and Intermediate outcomes What happens when the customer used outputs Measure value to customer / benefit derived from outputs –Societal implications

41. Measurement Elements Performance measure: characteristic to be measured that makes sense relative to the logic model (e.g., cost, cycle time, quality, qty) Performance indicator: unit of measure (dollars, hours, errors per 1000, number of reports per month) Baseline: initial/starting measurement Performance targets: starting from baseline measure, targets represent performance goals Tolerances: acceptable ranges of variance from performance targets

42. Focusing on Outcomes is Inherently Unnatural Outcomes: style, comfort, speed, durability, health Running Shoe Production Process Buyers Factory raw cut and shape assemble quality warehouse/ material raw material shoes tests shipping/accting Running shoes Customers

43. Think of Performance As Series of Dominos Activities Outputs Immediate Outcomes Intermediate Outcomes Impact

44. Diagram Relationships Using A Performance Logic Model People tend to make quantum leaps from outputs to outcomes because it is easy and quick Reality is that an organization must develop logic that shows how its outputs lead to immediate outcomes, and the possible secondary outcomes that may also occur Analyst must also indicate external factors that could inhibit achievement of those secondary outcomes The results of this analysis is a performance logic model that shows the linkages between outputs and various outcomes

45. S T A R T H E R E Measures At Each Level ResourcesPeople, Facilities, Contracts$, fte ActivityInstalling LAN, Implementingtime, cost application systems OutputsServers, workstations,#, quality, data bases, programsrqmts ImmediateProcessing time reducedhrs, lost OutcomesBetter control of information RelatedMore responsive to customers,satisfaction OutcomesIncreased demandcycle time Phases Examples Measures

46. Reflecting IT Outcomes

49. Alternatives and Risk Alternative Analysis & Risk Management –1. What were the results of the life-cycle cost analysis performed for this investment? –2. Describe alternatives considered and underlying assumptions for each. –3. What were the results of the benefits/cost analysis or ROI analysis for each alternative? –4. Explain the savings and cost recovery schedule –5. Describe your risk assessment and mitigation plan for this project Completing Exhibit 300

50. Earned Value Analysis (EVA) Standard methodology for measuring project performance –Uniform unit of measure for total project progress for any sub-element of the project –Consistent method for analysis of project progress and performance –Basis for cost performance analysis of a project As project progresses, actual performance if factored into adjusting cost/schedule for remainder of project –Enables project team to adjust expectations, if necessary

51. Provide Milestones & Cost

52. Provide Planned and Earned Value

53. Provide EVA Summary Info

54. OMB Graciously Provided Definitions

55. OMB Requires Assess of EVA Info C.If cost and/or schedule variance are a negative 10 percent or more, explain the reason(s) for the variance(s) D.Provide performance variance. Explain whether, based on work accomplished to date, you still expect to achieve your performance goals. If not, explain the reasons for the variance. E.Discuss the contractor, government, and at least the two EAC index formulas, current estimates at completion. Explain the differences and the IPTs selected EAC for budgeting purposes. F.Discuss the corrective actions that will be taken to correct the variances, the risk associated with the actions, and how close the planned actions will bring the project to the original baseline. Define proposed baselines changes, if necessary. G.Has the Agency Head concurred in the need to continue the program at the new baseline?

56. OMB Scoring for EVA Performance Based Management System (PB) 5 Agency uses an Earned Value Management System (EVMS) that meets ANSI/EIA Standard 748 and project is earning the value as planned for costs, schedule, and performance goals. 4 Agency uses the required EVMS, is within the variance levels for two of the three criteria, and needs work on the third issue. 3 Agency uses required EVMS but the process within the agency is very new and not fully implemented or there are weaknesses for this individual project's EVMS information. 2 Agency seems to re-baseline rather than report variances. 1 There is no evidence of PB.

57. CBA Role In Business Case Trying to make govt officials make prudent “business decisions” with taxpayer dollars Business case is important step Cost-Benefit Analysis is important element of an effective business case –It provides evaluation of costs and benefits of a proposed initiative –Together, with risk analysis, it enables decision-makers to decide upon alternative investment opportunities

58. Understanding CBA What is a Cost-Benefit Analysis? Calculating Costs Calculating Benefits Special Considerations

59. CBA Basics Why do we do a CBA? –Required (Circulars A-11, A-76, A-94, and A-130) –Technique for formal economic analysis of government programs and investments When do we do a Cost-Benefit Analysis? –Update throughout IT investment life cycle Who reviews CBAs and makes decisions? –IT Investment Review Board –Office of Management and Budget Who prepares the Cost-Benefit Analysis? –Various options: IT team, budget analysts, CIO staff, etc.

60. CBA Challenges Including All Costs –Obvious costs will emphasize data bases, programs, reports, etc. –Less obvious costs Transition and bridge software, additional hardware, systems software, and data communications, etc. Accurately Estimating Costs –Mythical Man Month (Yourdon) Data proves that 50% of all project time is spent testing, so estimates need to reflect sufficient time in this category

61. More CBA Challenges Including Indirect Benefits –Some indirect benefits are not quantifiable (at least not easily quantifiable) –IT performance measures should take these into account in cases where they cannot be quantified and explicitly included in the cost-benefit analysis

62. Enterprise Architecture II-E Enterprise Architecture –1. Does this investment support your agency’s current architecture or is it part of a modernization initiative? –2. Explain how the project conforms to: Agency IT architecture Agency technology infrastructure Federal Enterprise Architecture Completing Exhibit 300

63. Exhibit 300 Requires Risk Inventory

64. Estimating Costs Comprehensive Identification of All Relevant Project Costs

65. ESTIMATING COSTS Cost estimate accuracy depends on careful identification of all costs –All relevant costs Government personnel Contractor personnel Other contractor costs Hardware, software, and data communications Inter-agency services Supplies and other related costs –Costs over time (throughout the investment life cycle) Determine expected useful life of investment Identify various life cycle phases –Concept, development, implementation, and operation

66. Cost Estimation Steps 1.Identify costs (use a matrix) a.Over time - Life cycle phases b.Specific costs for each phase -labor, hardware, software, etc. 2.Convert life cycle phases into specific dates 3.Convert all costs (especially labor) to dollars a.Determine hourly labor rates for government personnel and contractors b.Convert labor hours into dollars 4.Calculate total project cost over entire life cycle

67. Estimate Life Cycle Costs Planning (Mostly personnel costs) –Pre- and Post-IRB approval –Labor costs for identifying requirements, developing business case, preparing project plan, attending meetings, etc. Design (Mostly personnel costs) –Labor costs for high-level design, specifications, data base analysis, capacity analysis, etc. Development (Personnel and hardware/software costs) Operations (Personnel and hardware/software costs) –Implementation, training, help desk, day-to-day operations, etc. Maintenance (Mostly personnel costs) Identify CostsSTEP 1

68. Cost Estimation Table Plan Design Develop Operate Maintain Govmt personnel Contract personnel Other contractor $ Hardware, software, & data comm. Inter-agency svcs Supplies & other related costs Identify CostsSTEP 1

69. Earned Value Analysis Placing EVA Within the Appropriate Context

70. EVA Is A “Tool” Preparing EVA information during the conceptual phase involves providing a forecast –Decision makers are most concerned that the estimates are accurate EVA is used to monitor progress as project activities occur –It provides useful information about project progress, especially when a project is falling behind schedule and exceeding its budget

71. How EVA Benefits CPIC Select Phase –Results in better cost and schedule estimates Control Phase –Results in earlier identification of schedule and cost overruns –Allows for rapid correction of problems Evaluate Phase –Provides useful data to identify and resolve gaps in the CPIC process

72. Traditional Approach: Milestone Oriented Calendar Dates Milestones 5 4 3 2 1 Status Date Planned Actual X Based on this presentation, the project appears to be in excellent shape.

73. Disconnect: Cost & Schedule Cost displays and scheduled displays typically are not integrated –Cost is discussed when finances are reviewed –Schedule/milestones are discussed in other project status venues –Schedule slippage is often “under stated” in terms of severity or impact

74. Project Status “Softness” IT project managers are typically optimists and natural “trouble shooters” Given contemporary project progress reporting tools, they can portray a rosy picture, thinking that they can fix any problems along the way –Alerting project oversight groups and individuals early in the process is perceived as not in the best interests of the project

75. Solution: EVA As will be seen, EVA is a more systematic and integrated method for measuring progress –It integrates cost and schedule, using the phrase “value” –It enables small variances earlier in the project to be forecast to the end of the project The magnitude of a small slippage can by projected onto a larger scale The result is very useful for identifying small problems that have potential to become large problems

76. Earned Value Objectives Properly plan the project Monitor the costs and schedule Forecast project completion date and costs

77. Earned Value Definitions Earned Value –The amount of “value” that is claimed, or earned, when a segment of work is accomplished Earned Value Analysis –A consistent methodology for measuring and analyzing: Project schedule performance Project cost performance Project progress –A set of metrics for making meaningful comparisons between planned and completed work

78. A Form of Modularity As will be seen, EVA is similar, in many respects to modular design and development –Lowering cost and improving quality by developing and assembling components (coupling and cohesion) –Examples include electronics industry and computer hardware industry It is the modular design and management of tasks and work steps

79. EVMS and EVPM Earned Value Management System (EVMS) –An integrated system that uses earned value to measure progress objectively Earned Value Project Management (EVPM) –A project management technique that uses earned value to determine if a project is achieving its cost and schedule goals

80. Is EVA A New Concept? It is an operations research tool that originated in the early 1900s –Program Evaluation and Review Techniques (PERT and PERT Charts) –Critical Path Method (CPM) It was a key element of a project control system used by DOD in the 1960s –Cost/Schedule Control Systems Criteria (C/SCSC)

81. EV Validity – Findings 1.700 DOD projects show it scientifically forecasts completion dates and costs 2.Allows for integrated performance measurement 3.Can be used to: Predict final range of costs Project final cost and date Monitor remaining work 4.Allows managers to use management by exception (MBE) monitoring approach

82. Part 4 Earned Value Approach Estimation: From EVA Perspective, An Accurate Baseline Is Essential

83. EVA Baseline EVA depends on sound estimation EVA baseline is important because: –Throughout project lifecycle, current information will be compared back to baseline –Variances will be calculated against baseline –Going back to OMB to “re-baseline” is not a good thing

84. Accurate Estimates Are Critical! Out-of-the-sky estimates are common –Technology people want to build, not do paper- intensive work associated with planning and estimated – do quick estimates to satisfy management demands –Planning and estimating requires different skills and temperament than building Consequences of inaccurate estimates can be substantial –Under-estimate effort and cost –Deliver products that don’t meet expectations

85. Developing An Accurate EVA Baseline Accurate estimates require considerable pre-work before accurate Exhibit 300 estimates (cost & schedule) are available 1.Requirements (analysis) 2.General design (high level design) –Sufficient information to assess number and complexity of deliverables 3.Work design –What work needs to be done to develop, implement, and support deliverables 4.Work schedule (labor hours and dates) 5.Cost estimate/budget Steps parallel system development lifecycle approach (SDLC)

86. Develop Deliverables Inventory ArtifactComplexity Level Inter- Dependencies Estimated Labor Hrs Special Rqmts Program A Program B Relational DB Tables Script A Script B Web Artifacts Total

87. Define the Work Work Breakdown Structure (WBS) –Deliverable-oriented grouping of project elements that organizes and defines the total project scope –Work not in the WBS is outside of project scope –Each descending level represents increasingly detailed description of project elements Accuracy of WBS is key aspect of effective EVA –Accurately identify needed artifacts to meet requirements –Accurately estimate complexity and labor needed to develop artifacts –Focus on indirect as well as direct activities Activities that do not contribute directly to artifact development, but are still needed to implement and support the users

88. Sample WBS Project Activity 1Activity 2Activity 3 Task 1-1Task 1-2 Subtask 1-2.01Subtask 1-2.02 Task 3-1Task 3-2 Level1 Level 2 Level 3 Level 4

89. Sample WBS Project Design DBDevelop ProgramsDevelop Website Design & Normalize Generate DDL Create Data Structures Populate Data Structures DesignConstruct Web Pages Level1 Level 2 Level 3 Level 4

90. Work Packages/WBS Dictionary Work Packages –Lowest level WBS elements –Have accompanying narrative –Scoped by organizational unit responsible for accomplishing work WBS Dictionary –Repository for work package narratives –Serves as SOW technical component For solicitation or internal development

91. Develop Work Package Budgets For each work package –In addition to narrative, develop budget Estimate type and quantity of resources for each work package –Personnel, hardware, equipment, software, supplies, space, & facilities Convert resource estimates to dollars Since project managers compute labor hours, but not actual dollars, you will need a “dollar converter”

92. BCWS – Planned Value Budgeted Cost of Work Scheduled (BCWS) –Value varies over time –BCWS is total estimated cost of all completed and partially-completed work packages on date that earned value is conducted –Once baseline is approved, BCWS can be charted (see next page)

93. BCWS Can Be Calculated From Work Package Baseline $40K $30K $20K $10K Project Milestones BCWS

94. Calculating BCWS (Example) Work Package Est. Budget (Baseline) Est. Start/Stop Date Est. % Complete Earned Value WP 1$10K6/1-9/30100%$10K WP 2$10K8/1-12/31100%$10K WP 3$10K11/1-2/2850%$5K WP 4$10K1/15-3/31-- BCWS As Of 12/31$25K

95. Exhibit 300 (Section 300-23) B.Provide the following project summary information: As of: (date) B.1.Show the budgeted (planned) cost of work scheduled (BCWS): $ B.2.Show budgeted (planned) cost of work performed (BCWP): $ B.3.Show the actual cost of work performed (ACWP): $ BCWS

96. No Earned Value Before Project Starts Project Budget $ Project Milestones Project Start (BCWS = $0) Project planning (Preparing the Exhibit 300)

97. Catching Our Breath Thus far, not much is new –Same old planning and estimating These steps, however, are particularly critical to an earned value project –Once baseline is in place, actual performance against this baseline will be measured continuously for project duration

98. Calculating Earned Value At specified time (90 days, 6 months, etc.), consult with project team: 1.Discuss work package baseline (BCWS) Original completion date for each WP Original budget for each WP 2.Determine status of each work package (BCWP) 3.Determine dollar cost of each work package (ACWP)

99. Three EVA Variables Budgeted Cost of Work Scheduled (BCWS) –How much should have been completed by now? Budgeted Cost of Work Performed (BCWP) –How much has actually be completed by now? Actual Cost of Work Performed (ACWP) –What is actual cost of work completed by now?

100. More About the EVA Variables BCWS –This is where we thought we would be at this point in time BCWP –This variable is used to calculate schedule slippage ACWP –This variable is used to calculate cost slippage

101. Example – EVA At Midpoint? Work Package 1 Work Package 2 Work Package 3 Work Package 4 $10K 100%50%-- $12K$7K-- BCWS? _____ BCWP? _____ ACWP? _____

102. Mid-Point EVA $40K $30K $20K $10K Project Milestones BCWS ACWP BCWP Schedule Variance = -$5K Cost Variance = -$4K

103. Calculating BCWP (Example) Work Package Est. Budget (Base) Est. Start/Stop Date Act. Start/Stop Date Act. % Comp Earned Value WP 1$10K6/1-9/30 100%$10K WP 2$10K8/1-12/318/1-11/30100%$10K WP 3$10K11/1-2/28 Not started -- WP 4$10K1/15-3/31 Not started -- BCWP As Of 12/31$20K Note that BCWS for 12/31 was $25K and BCWP is only $20K. BCWP < BCWS = schedule slippage

104. Calculating ACWP (Example) Work Package Est. Budget (Base) Est. Start/Stop Date Act. Start/Stop Date Est. Cost Act. Cost WP 1$10K6/1-9/30 $10K$25K WP 2$10K8/1-12/318/1-11/30$10K$5K WP 3$10K11/1-2/28 Not started -- WP 4$10K1/15-3/31 Not started -- ACWP As Of 12/31$20K$30K Note that BCWP for 12/31 was $20K and ACWP for 12/31 is $30K. BCWP < ACWP = cost overrun

105. Comparing the Variables When we compare BCWS to BCWP, what should be the “ratio”? –If BCWS = $10,000 and BCWP = $15,000, is this good or bad? When we compare BCWP to ACWP, what should be the “ratio”? –If BCWP = $15,000 and ACWP = $12,000, is this good or bad?

106. “Value” Performance Measurement Methods Measurement Categories –Percent complete Easiest, but subjective –Milestones with weighted values Good for short duration work packages –Fixed formula Useful for purchasing material (25% on order and 75% when task is completed) –Combination % complete and milestones –Other techniques suitable for private sector Equivalent completed units, earned standards, and apportioned relationships

107. Part 5 Earned Value Approach Analyzing Variance and Monitoring Project Progress

108. Once Project Work Begins, EVA Work Begins Collect performance information for each work package Analyze variances: –Compare planned (baseline) values to actual values Compute cost variance Compute schedule variance Assess progress Decide if corrective action is necessary

109. Compute Variances After determining BCWS, BCWP,and ACWP, use “tool” to compute variances –Cost variance –Schedule variance Analyze variances –Assess progress –Decide if corrective action is necessary

110. Variances and Performance Indexes Are Now Complete Project Summary (Cumulative) Value Cost Variance = (BCWP – ACWP) = Cost Variance % = (CV/BCWP) X 100% = Cost Performance Index (CPI) = (BCWP/ACWP) = Schedule Variance = (BCWP – BCWS) = Schedule Variance % = (SV/BCWS) X 100% = Schedule Performance Index (SPI) = (BCWP/BCWS) = Two Independent Estimates at Completion (EAC) = Variance at Completion (VAC) = (BAC – EAC) for both EACs Variance at Completion % (VAC/BAC) X 100% for both EACs Expected Funds to Completion (ETC) = Expected Completion Date =

111. Cost Variance Cost Variance (CV) –CV = BCWP minus ACWP –A negative CV means the project is “over budget” Cost Variance Percentage –(CV divided by BCWP) times 100% Cost Performance Index (CPI) –For every planned dollar of work that is being accomplished, it is costing this much –OMB wants to know how much work is being done for a dollar of cost –CPI = BCWP / ACWP –A value of less than one means the project is over budget

112. Schedule Variance Schedule Variance (SV) –SV = BCWP minus BCWS –A negative SV means the project is behind schedule Schedule Variance Percentage –(SV divided by BCWS) times 100% Schedule Performance Index (SPI) –For every planned dollar of work that is being accomplished, this is how much of the planned work is being achieved –OMB wants to know how much achievement is being attained for a dollar of cost –SPI = BCWP / BCWS –A value of less than one means the project is behind schedule

113. Use Variances and Indexes to Assess Current Status The top six lines of the Project Summary provide an “as is” assessment Project Management Responsibility –Determine if the project is over budget and/or behind schedule –Identify which work packages have the most significant issues –Identify and address causal factors It is a good idea to do this well in advance of when information must be reported to OMB

114. Use EVA Information for Decision Making Behind Schedule –How critical is the schedule? –What is the impact of overtime on project cost? –Can some tasks be done concurrently? –Are the dependencies accurate? –Can process improvements increase speed? –Can tasks be re-organized to shorten critical path? Over Cost –Can activities and tasks be rescheduled? –Are less costly resources available? –Should some tasks be outsourced? –Are there tasks which do not add value that can be deleted?

115. Part 6 Earned Value Approach Forecasting Project Completion Date and Costs

116. Forecast Project Completion Costs Forecasting Tools –Cost Performance Index How much is it costing to do a planned dollars worth of work? Also referred to as the “cost efficiency factor” Represents relationship between actual costs and budgeted costs of work performed at time of measurement –Schedule Performance Index How much of the actual work is being achieved compared to the planned work? Also referred to as “planned schedule efficiency factor” Represents relationship between value of initial planned schedule and value of work performed at time of measurement

117. Research Findings Once a contract/project is 20% complete, the Cost Performance Index (CPI) does not vary from its current value by more than 10% One a contract is at least 10% complete, the overrun at completion will not be less than the overrun incurred to date The percent overrun at completion will be greater than the percent overrun incurred to date The combination of the cumulative CPI and cumulative SPI provides the ability to statistically forecast the “most likely” or “high end” final cost results of a project Bottom line: Once you get off track, research shows that you are unlikely to recoup as the project progresses

118. Estimating Cost At Completion EAC 1 : Low-end estimate –EAC 1 = ACWP + (BAC – BCWP) / CPI cum EAC 2: High-end estimate –EAC 2 = ACWP + (BAC – BCWP) / (CPI cum X SPI cum ) Elements needed to compute estimate at completion –ACWP: Actual Cost of Work Performed –BAC: Budget At Completion –BCWP: Budgeted Cost of Work Planned –CPI: Cost Performance Index –SPI: Schedule Performance Index

119. Estimating Completion Variance and Funds to Completion Variance At Completion (VAC) –VAC 1 = BAC – EAC 1 –VAC 2 = BAC – EAC 2 –OMB wants to know the variance between the baseline (original estimate) and the budget at completion Variance At Completion Percentage (VAC%) –VAC 1 % = (VAC 1 / BAC) X 100% –VAC 2 % = (VAC 2 / BAC) X 100% Estimate To Completion (ETC) –ETC 1 = EAC 1 - ACWP –ETC 2 = EAC 2 - ACWP –OMB wants to know the funding needed to complete the project

120. Forecasts Are Now Complete Project Summary (Cumulative) Value Cost Variance = (BCWP – ACWP) = Cost Variance % = (CV/BCWP) X 100% = Cost Performance Index (CPI) = (BCWP/ACWP) = Schedule Variance = (BCWP – BCWS) = Schedule Variance % = (SV/BCWS) X 100% = Schedule Performance Index (SPI) = (BCWP/BCWS) = Two Independent Estimates at Completion (EAC) = Variance at Completion (VAC) = (BAC – EAC) for both EACs Variance at Completion % (VAC/BAC) X 100% for both EACs Expected Funds to Completion (ETC) = Expected Completion Date =

121. Use Forecasts to Assess Magnitude of Variance The bottom five/six lines of the Project Summary provide perspective regarding the impact of to-date cost overrun and schedule slippage Project Management Responsibility –Determine how significant the end-of-project variance is likely to be –Identify causes and actions to remedy variance –Communicate to IT-IRB and others in order to moderate expectations Prepare to address the issues when the information is provided to OMB

122. Part 7 Preparing the Exhibit 300 Getting a Good Score

123. Provide Requested OMB Information B.4 Provide a cost curve graph plotting BCWS, BCWP, and ACWP on a monthly basis from inception of this phase through the latest report. In addition, plot the ACWP curve to the estimated cost at completion (EAC) value, and provide the following EVMS variance analysis

124. Responding to OMB Questions About EVA Results C.If cost and/or schedule variance are a negative 10 percent or more, explain the reason(s) for the variance(s) D.Provide performance variance. Explain whether, based on work accomplished to date, you still expect to achieve your performance goals. If not, explain the reasons for the variance. E.Discuss the contractor, government, and at least the two EAC index formulas, current estimates at completion. Explain the differences and the IPTs selected EAC for budgeting purposes.

125. Responding to OMB Questions About EVA Results F.Discuss the corrective actions that will be taken to correct the variances, the risk associated with the actions, and how close the planned actions will bring the project to the original baseline. Define proposed baselines changes, if necessary. G.Has the Agency Head concurred in the need to continue the program at the new baseline?

126. Enterprise Architecture Basics Status Report

127. Minimal Visibility for Several Years CIO Council Publications –Federal Enterprise Architecture Framework (September 1999) –Architecture Alignment and Assessment Guide (October 2000) –Practical Guide to Federal Enterprise Architecture (February 2001)

128. Substantial Current Activity OMB and GAO have become more active OMB (www.feapmo.gov) –Federal Enterprise Architecture The Federal Enterprise Architecture is a business-based framework for cross-agency, government-wide improvement. It provides OMB and the Federal agencies with a new way of describing, analyzing, and improving the federal government and its ability to serve the citizen. –Business Reference Model 1.0 (July 2002) GAO Reports –EA Use Can Be Improved –OMB Leadership Critical to Making Needed Enterprise Architecture & E-Govt Progress

129. Key Facts Architecture –Essential to the enterprise –Must be enterprise-wide Enterprise Architecture –Must model existing and planned processes, applications, and systems –Must indicate reusable components and points of reuse –Must provide information sharing in seamless manner across the enterprise

130. Reality Many organizations are incapable of sharing key business information across the enterprise because they suffer from stovepipe information systems –Legacy technology –No interoperability

131. Achieving the Architecture Three ways for information systems to provide an enterprise solution: –Extend existing applications –Add new technology solutions –Connect existing applications and new technology solutions Success means that the EA supports: –Implementing new applications –Implementing reusable enterprise components –Extending existing, often legacy, applications

132. What is the EA? Explicit description and documentation of the current and desired relationships among business and management processes and information technology –Current architecture –Target (future) architecture –Framework for investment review process Select IT investments that “close the gap” Define principles, goals, and directions for interoperability, open systems, public access, compliance with GPEA, end user satisfaction, and IT security OMB Circular A-130

133. EA Technical Reference Model and Standards Profile Technical Reference Model (TRM) –Identifies and describes the information services (such as database, communications, intranet, etc.) used throughout the agency Standards Profile –Defines the set of IT standards that support TRM services –Must include a Security Standards Profile that is specific to the security services specified in the EA

134. Enterprise Architecture The Enterprise Architecture (EA) –Document and submit initial EA to the OMB –Submit significant changes to EA What is the EA? How do agencies create and maintain the EA? What are the Technical Reference model (TRM) and Standards Profile? How will Agencies ensure security in information systems? OMB Circular A-130 A-130 (Nov 30, 2000) Pages 15 – 17

135. Enterprise Architecture Elements Complements IT strategic plan –Mission context –As-is analysis –Target architecture –Transition strategies Enterprise architecture is technology blue print for the organization. IRM Strategic Plan is the document that describes strategy, organizational implications, and budget forecast for achieving desired future state.

136. Creating and Maintaining the EA Create an EA Framework –Document linkages among mission needs, information content, and IT capabilities –Use framework to guide strategic and operational IT planning Use EA Framework to create EA –Business processes –Information flow and relationships –Applications –Data description * relationships –Technology infrastructure OMB Circular A-130

137. TRM & Standards Profile Technical Reference Model (TRM) –Identifies and describes information services used throughout agency Database Communications Intranet Etc. Standards Profile –IT standards that support TRM services -Include Security Standards Profile OMB Circular A-130

138. Information Systems Security Part of TRM/Standards Profile –Security Standards Profile Identification, authentication, and non- repudiation Audit trail creation and analysis Access controls Cryptography management Virus protection Fraud prevention Detection and mitigation Intrusion prevention and detection OMB Circular A-130

139. Federal Enterprise Architecture FEA Project Management Office OMB Circular A-130

140. WHAT IS THE FEDERAL ENTERPRISE ARCHITECTURE(FEA)? To facilitate efforts to transform the Federal Government to one that is citizen-centered, results-oriented, and market-based, OMB is developing the Federal Enterprise Architecture (FEA), a business-based framework for Government-wide improvement. –The FEA is being constructed through a collection of interrelated "reference models" designed to facilitate cross-agency analysis and identification of duplicative investments, gaps, and opportunities for collaboration within and across Federal Agencies

142. Business Reference Model "The Business Reference Model is a function-driven framework for describing the business operations of the Federal Government independent of the agencies that perform them." The BRM is the first layer of the Federal Enterprise Architecture and is the main viewpoint for the analysis of data, service components, and technology It provides an organized, hierarchical construct for describing the day-to-day business operations of the Federal government –While many models exist for describing organizations - org charts, location maps, etc. - this model presents the business using a functionally driven approach. –The Lines of Business and Sub-functions that comprise the BRM represent a departure from previous models of the Federal government that use antiquated, stove piped, agency-oriented frameworks.

143. Performance Reference Model PRM is framework for performance measurement that provides common application measures throughout the federal government. It allows agencies to better manage the business of Government at a federal strategic level while providing a means for gauging progress towards the target FEA –PRM establishes common set of general performance outputs and measures that agencies use to achieve much broader program and business goals and objectives –Model articulates linkage between internal business components and achievement of business and customer-centric outcomes –PRM aligns and leverages existing federal guidance and application/ architecture recommendations. –It facilitates resource allocation decisions based on comparative determinations of which programs/organizations are more efficient and effective

144. PRM Has A Relationship to the PART Assessment The PRM will be designed to integrate with and complement OMB's development of the Program Assessment Rating Tool (PART) and common measures initiative By defining outcome and output measures for lines of business and sub-functions, the PRM will provide the tools necessary to measure cross-agency initiatives at the federal enterprise level. Additional guidance on both the PRM and PART will be provided as these two models undergo continued development.

145. Data and Information Reference Model (DRM) Describes, at an aggregate level, the data and information that support program and business line operations Model aids in describing types of interaction and exchanges that occur between the Federal Government and its various customers, constituencies, and business partners.

146. Service Component Reference Model (SRM) Identifies and classifies horizontal and vertical IT service components that support Federal agencies Aids in recommending service capabilities to support the reuse of business components and services across the Federal Government

147. Technical Reference Model A hierarchical foundation to describe how technology is supporting the delivery of the service component The TRM outlines the technology elements that collectively support the adoption and implementation of component-based architectures

148. Agency Enterprise Architectures Similar But Different

149. Zachman Framework Many agencies are using the “Zachman Framework” –A two-dimensional structure for describing the information architecture of an enterprise –1st dimension: the roles involved in information systems design (planner, owner, designer, builder) –2nd dimension: What, How, Where, etc

152. EA Success Depends on It’s Simplicity It is crucial that everyone understand the EA While Zachman has significant benefit for an organization, it poses significant “practicality” and “ease of use” challenges Program managers may: –“Glaze over” when they see it –Lose appreciation for EA benefits –Ignore the EA Recommendation: –Keep the Zachman Framework behind close doors –Develop an easy-to-understand and easy-to-use EA in order to engage key players

153. EA Layers  Business Processes  Information Flows and Relationships  Applications  Data Descriptions and Relationships  Technology Infrastructure

154. Time-Phased View of EA Every Agency’s enterprise architecture has three time-phased views. –Baseline Architecture - Present IT environment (i.e., "as is" model) –Target Architecture - Future IT environment (i.e, "to be" model) –Transition Architecture - Plan for getting from "as is" to the "to be" model (i.e., migration plan)

156. Develop An EA Model That Everyone Can Understand LayersBaseline (“As Is”) Architecture Target Architecture Transition Architecture (Strategy) Business Processes Major programmatic activities today How programs will operate in future Specific strategies (from agency strategic plan) Information Needs Major program information needs Future program information needs How we will get from now to then ApplicationsCurrent automated support Future automated support How we will get from now to then InfrastructureCurrent infrastructure Future infrastructure How we will get from now to then

157. EA Example CURRENT ARCHITECTURE - 2002 Program Area ->RegulationEnforcementAdministration Business Processes and Data requirements -Legislation -Current Regs -Complaints -Case information -Financial Mgmt -HR -Acquisitions ApplicationsMS-Excel Spreadsheet tracking system Mainframe-based complaint and case tracking system (COBOL) -Mainframe FM system -Mainframe HR [No acq. System] InfrastructureAmdahl Mainframe, COBOL, WAN, Novell LANs, Lotus Notes Mail, Corel Office Suite and MS Office Suite. Some small MS-Access DBs.

158. EA Example TARGET ARCHITECTURE - 2006 Program Area ->RegulationEnforcementAdministration Business Processes and Data requirements -Legislation -Current Regs -Complaints -Case information -Financial Mgmt -HR -Acquisitions ApplicationsN-tier legislation and regulation tracking system Complaint and case tracking Modernization Integrated Oracle FM, HR, and Procurement Application (COTS) InfrastructureCentral server center, MS Visual Basic and C++, WAN, MS-NT Server, MS-Exchange, and MS Office Suite.

159. Transition Strategy for the Example EA Transition Strategy Step 1 Upgrade network infrastructure (2002-2003) Replace Novell with NT Step 2 Upgrade infrastructure (2003) – Replace mainframe with integrated server structure Step 3 Replace legacy admin applications (2003) Step 4 Modernize enforcement applications (2004) Step 5 Upgrade regulation applications (2005)

160. Don’t Forget the TRM and SP Technical Reference Model (TRM) –Identifies and describes information services (e.g., database, communications, Intranet, etc.) used throughout the Agency Standards Profile (SP) –Defines the set of IT standards supporting the services articulated in the TRM –Agencies are expected to adopt standards necessary to support the entire EA. These adopted standards must be enforced consistently throughout the Agency. Security Standards Profile (SSP) –Identifies the security standards specific to the security services specified in the EAF and covers such services as identification, authentication, and non-repudiation.

161. Use Time-Phased Approach for the TRM BASELINE TRM - 2002 Product TypeCentralDecentralized DatabaseIBM-IMS MS-Access dBase IV Data Communications IBM-SNA Novell Netware IBM-SNA Novell Netware Programming Tools COBOLMS-Visual Basic COBOL Office AutomationCorel OfficeMS Office

162. Future TRM TARGET TRM - 2006 Product TypeCentralDecentralized DatabaseOracle MS SQL Server MS-Access Data Communications TCP/IP Microsoft 2000 Server MS 2000 Client Programming Tools C++ Java MS-Visual Basic Office AutomationMS Office

163. Standards Profile Standards CategoryBaseline StandardsTarget Standards DatabaseN/AODBC Compliant Data Communications IBM SNA Novell TCP/IP Ver 4.2 TCP/IP Ver 6.0 OtherHTMLHTML/XML

164. The EA and the CPIC Once Agency management approves an EA, the Agency CIO staff within the CPIC function must ensure it is followed. This is done via the CPIC procedures developed by the Agency. Regardless of how good an EA is when first developed, it must be continually updated

165. EA-Based Decisions Proposed investments and changes to existing legacy systems, which undergo architecture alignment and assessment, result in one of four outcomes. 1.The investment is sufficiently aligned to the architecture such that it can be recommended to proceed. 2.The new investment is rejected because of poor alignment or other CPIC criteria failure. 3.The investment is determined to be valid even though not aligned to the EA, and the EA is updated to reflect missing alignment, functions, data objects, and the target application. 4.Agency management decides (for documented reasons) to pursue the investment via a waiver irrespective of EA misalignment. Justifications for these investments require strong rationale, which are reviewed by external oversight.

166. IT Security Key Area of Attention

167. Information Systems Security Part of TRM/Standards Profile –Security Standards Profile Identification, authentication, and non- repudiation Audit trail creation and analysis Access controls Cryptography management Virus protection Fraud prevention Detection and mitigation Intrusion prevention and detection OMB Circular A-130

168. IT Security Pressures Agency Computer Security Report Card –Government-Wide Grade = F GISRA and Computer Security Act –NIST responsible for promulgating IT security guidance and assistance –NIST published extensive documentation Federal IS Management Act of 2002 – Nation is facing growing and real threats –Increase requirements for GISRA compliance Annual information security report Clarifies use of term “national security system” to be sure that more systems are covered by GISRA Establishment of Federal information security incident center

169. Six Common IT Security Weaknesses Lack of agency senior management attention to IT security Non-existent IT security performance measures Poor security education and awareness Failure to fully fund and integrate security into CPIC Failure to ensure that contractor services are adequately secure Lack of detecting, reporting, and sharing information on vulnerabilities

170. Security Contradictions Make information… –Easy to access by authorized personnel but hard to access by others –Simple to modify properly and hard to modify/destroy improperly –Contain all the details we need but none that invade privacy –Open to the public but protected from Internet attacks

171. Challenges Potentially conflicting goals –The free flow of information between the government and the public is essential to a democratic society (7c) –The individual's right to privacy must be protected (7g) Costs –Security must be commensurate with the value of the resource protected (8a1g) Entropy –Security requires an on-going process to achieve the desired result Dual-edged sword of technology –Efficiency vs. vulnerability

172. Can We Succeed? Yes, but it will take –Ongoing management attention –Creation of Information Security Steering Committee –Ongoing worker attention –Some technical expertise –Resources –A definition of success that doesn’t include total security An enterprise architecture can help…

173. What Do We Need to Succeed? A-130 Perspective Security plan (per CSA 1987) including a risk assessment –Risk assessment need not be formal/quantitative –Differentiate between general support systems and major applications Assigned security responsibility –One person responsible for each system Review security plans –New, changed, or every three years (Re-)Authorization for processing –IAW security plan and risk assessment –New, changed, or every three years *

174. How Can an Enterprise Architecture Help? Clearly identify interfaces between systems for… –Firewall protection –Other access rules –Incident response (pull the plug) Identifies fail-over components for contingency planning Identifies replicated components for public access Define network partitioning that meets security needs –Physical AND logical System Architecture

175. How Can the EA Help? Identify security components for authorization, authentication, enforcement –User directory Identify application components with security features –Document management Identify audit and validation points in data flows Define flows to public access points Define a security architecture –Data & user categories –Event logging & analysis Operational Architecture

176. Challenges to the Security Architecture Huge legacy investments –Older firewalls do not provide monitoring –Older applications implemented custom security mechanisms Monolithic COTS applications –Take it or leave it Missing specific guidance and examples –Requirements are very general Missing resources

177. Questions? K eep in Touch: Tom Kessler 479 Jumpers Hole Road, Suite 406 Severna Park, Maryland 21146 Phone: 410-431-5005 Fax: 410-431-5111 E-mail: Tom@DenaliAssociates.comTom@DenaliAssociates.com Web site: www.DenaliAssociates.comwww.DenaliAssociates.com