1. Improving ERP Cost Estimating
Presented by:
Mr. Wilson Rosa
Air Force Cost Analysis Agency
Mr. David Cashin
Naval Center for Cost Analysis
Mr. Noel Bishop
Office of the Deputy Assistant Secretary of the Army
for Cost & Economics

2. Topic
Speaker
Introduction
Mr. Wilson Rosa
Estimating Challenges
Way Ahead
Mr. David Cashin
Lessons Learned
Mr. Noel Bishop

3. Objective
Share challenges and lessons learned from five recent milestone reviews
Describe a plan to improve ERP cost estimating

4. Introduction
Procuring ERP is becoming increasingly popular within DoD

5. 5 Programs Deployed (2000-2005) Total Investment Cost ~$1.0B (TY)
# of Programs
By The Way, All slide are in TY budget $
As of today there has been five pure ERP’s deployed.
Added together these programs add up to approximately one billion dollars
These programs were built to budget constraints
Therefore our data and experience is limited
Total Investment Cost ~$1.0B (TY)

6. 17 Programs in Development (Present)
# of Programs
As you can see there is growing number of programs who employ ERP.
These programs are full scale ERP programs with growing scope and higher costs
This is why we need to prepare now, to estimate programs with improved methods
Total Investment Cost >$6.0B (TY)

7. How ERP differs from Traditional Software Intensive Programs?
Factor
Traditional Software Programs
ERP
Technology
Custom-Built
Configured Commercial Off the Shelf (COTS) software
Interoperability
Standalone IT systems
Interconnected IT systems
Design Concept
System Analysis and Design
Business Process Re-engineering, Product Configuration
Primary Cost Driver
1 Parameter
(Lines of Code)
Over 30 Parameters
(e.g. scripts, reports, interfaces)
DoD Experience
+35 Years,
Over 7,000 AIS Deployed
4 Years,
5 ERP Deployed
17 ERP In-development
Why improved, because ERP is a process development not a product development.
Let me explain…

8. Main Challenges with ERP Estimating
Understanding Impact of Cost Drivers
Software development represents only a small fraction of total cost
No empirical research on specific ERP cost drivers
Limited historical data to support parametric tools
Because of these differences
We have four main challenges
1. Brain ticklers - projets

9. Challenge # 1: Understanding Impact of Cost Drivers
More than 30 cost parameters
Still learning which are major cost drivers
Currently, analysts can quantitatively estimate only 6 out of 30
Subject matter experts are unable to measure size or impact of
24 out of 30
Put 30 parameters in back up…
The six parameters are similar to current estimating practices for example SLOC, help desk needs, H/W
Since ERPs are based on process and not products, current recognized experts are at a lost to measure things such as
Business Logic
Blueprinting actives such as interviewing experts
Retraining of personnel

10. Challenge # 2: Software Development is not the primary cost driver
It is not just the parameters that differ greatly.
The big difference is that an ERP includes a significant amount of non-system design task
For example = Change Management: Changing the culture of your community to fit new processes
Blue Printing: Understanding the current environment though mapping business logic and transactions
The sheer head count of users can drive training, software licenses, and hardware needs and in some cases dwarf software development altogether
Traditional
Software Program
ERP

11. Challenge # 3: No empirical research on specific ERP cost drivers
No journal articles or dissertations address cost drivers
0 out of 543 ERP Articles
0 out of 56 ERP Dissertations
Few “white papers” found on cost drivers but none provide supporting data
Parametric tools are just now under development
Did I mention that there is little to no knowledge of ERP.
You can read the statistics.
There is only 5-6 years of ERP history in the commercial world and much less in Government circles
Another reason for the derth of research may be because ERPs are process driven and seamingly immeasurable

12. Challenge # 4: Limited historical data to support parametric tools
Five previous ERP programs
Poor EVM reporting from all five
Cost presented at total level only
None reporting into DCARC
Inconsistent definitions/parameters
Enterprise Software Initiative website provides good pricing metrics but not based on actual data
You saw earlier that we have 17 programs in development but only 5 fielded programs. How are we to assess these 17 programs?
Can we reach back to 1980s technology for analogy ?
Can we rely on the contractors estimates ?
Can we focus on SLOC for all CERs ?
Say yes to any of these and

13. Consequence: Significant Cost Growth
Program
Cost Growth* 1
>5% (MS-A compared to MS-B) 2
50% (MS-B compared to IOC) 3
100% (MS-B compared to Preliminary Design Review) 4
350% (MS-A compared to Critical Design Review) 5
400% (MS-B compared to IOC)
Here are the consequences.
Our inability to get our hands around the cost of these programs is unacceptable.
We are not sitting on our hands.
Dave Cashin is going to talk about the road ahead for cost agencies
*Preliminary data from 5 of 17 on-going efforts

14. Road Ahead

15. Identify Cost Drivers and Metrics
Establish methodology for utilizing size metrics
Establish a single data set supporting these metrics
Create standard definitions and map between various industry terms
The cost agencies are already working together on AIS databases.
When we saw ERPs coming down the road we looked to collect data here as well
We need to come up with definitions to map to all industry terms because they are all different but the same

16. Data Collection Efforts
Ensure cost data reporting requirements are included in ERP contracts
Improved CSDR and EVM reporting tailored for ERP programs
Create single database across DoD:
Air Force
Navy
Army and Marine Corps
Army, Air Force, and Navy Cost Centers are working together to build a database for AIS programs starting with ERP’s

17. Standardize Cost Element Structures
Survey industry to develop a single cost element structure
Map between industry work structures and DoD CES structures
Need to work with PA&E to amend policies/handbooks to address ERP specific guidelines
include some sort of standard skills needed set for different activities

18. Develop new ERP parametric tools
Establish a foothold in new data sets
Standardize language and translation guides to bridge the gap
Develop analogies, metrics, and parametric tools
Build rigorous approach to Risk analysis

19. Note: ERP definitions not standardized
Lessons Learned
Program Office must clearly define target processes
Blueprinting and Realization schedule
RICE Objects by some level of complexity
Business Process Master Lists by some level of complexity
Legacy system integrations
Requirements, (e.g., SW licenses, hardware and satellite time)
Training and Fielding plans
Note: ERP definitions not standardized

20. Lessons Learned Calibrate existing models with valid data
Verify CARD metrics with other official documents
Operational Requirements Document
Interface Control Document
GAP Analysis
Process Flow Diagrams
Product Assessment
C4ISR Plan
Business Process Master List
Technical and Functional Requirements

21. Lessons Learned
Add cost growth factor to estimate based on DoD history—fact, not risk
Program Office needs ERP Subject Matter Experts on hand
Ensure Change Management, Data Cleansing and Normalization, and Data Warehousing are adequately addressed

22. Questions? 30 metric list - definitions Expectation of completion date
- ESI contracting toolset for crosscheck information and task lists
- Survey results late summer
- Dictionary by Fall
- Data collection ongoing into winter
- Some Service Data
- Little federal data
- Commercial Data

23. Acronyms AIS Automated Information System BSM
Business System Modernization
CES
Cost Element Structure
CARD
Cost Analysis Requirements Description
COTS
Commercial Off The Shelf
CSDR
Cost and Software Data Reporting
C4ISR
Command, Control, Communication, Computer, Intelligence, Surveillance, and Reconnaissance
DoD
Department of Defense
DCARC
Defense Cost and Resource Center
ECSS
Expeditionary Combat Support System
ERP
Enterprise Resource Planning
EVM
Earn Value Management
IOC
Initial Operational Capability

24. Acronyms IT Information Technology MS-A Milestone A Decision Review
MS-B
Milestone B Decision Review
OSD
Office of the Secretary of Defense
PA&E
Program Analysis & Evaluation
PMO
Program Management Office
RICE
Report, Interface, Conversion, Extension SW
Software