1. By: David Hoang Martin Hoffman
Measuring the Economic Impacts of an Inadequate Infrastructure for Software Testing
By:
David Hoang
Martin Hoffman

2. Improved infrastructure can…
Remove more bugs
Detect bugs earlier
Locate bugs faster 2

3. The longer it takes the more is costs…3

9. What does this mean for the User?
Fewer bugs in software operations stage
Lower maintenance costs
Change in avoidance activities 9

10. Custom vs. Commercial Software Products
Custom Products:
no formal release
more resources needed for implementation (developer)
post-purchase service (developer)
3rd party integrators
To quantify the cost associated with an inadequate infrastructure for software testing we look at the costs of custom and commercial products. This is to prevent double counting.
Release is different from deliverable

11. Estimating Software Developer Costs
Data collected:
Company’s total pre-release testing and post-release service costs.
Surveyed developers.
Additional resources required due to an Inadequate infrastructure for software testing:
Labor costs
Software costs
Hardware costs
After-sales service cost
Delay costs
Reputation costs
Labor costs – additional employee and contract labor for pre-purchase testing and error correction, installation, and post-purchase repair
- accounts for the majority of the impact
Software – additional or redundant testing software purchases
Hardware – equipment, computers needed in testing process
After-sales – non-testing and debugging activities: servicing additional service calls and distribution of patches
Delay costs – lost profits and delayed adoption du to delays and high bug counts in early versions
Reputation – lost sales or market share

12. Estimating Software User Costs
Software purchase decision
Evaluation of products
Lack of information on the quality of products
Unable to compare across products based on price and quality
uncertainty of bugs and interoperability increases
Installation and Integration
Higher for custom products
Business operations
System failures
Installation and integration of custom products require more frequent integration with legacy systems.
Bug or problems in either the new or legacy system can delay the system, require additional patches or drivers
Potential for additional cost is very high here.
Can involve both the user and developer.
Most commercial SW products are typically less then custom sw.
Business operations
cost of loss of production, loss of data, customer information, lost sales, production delays, lost reputation, market share

13. Software User Cost Categories
labor costs
failure costs
performance cost
redundant systems
delayed profits
sales forfeited
Labor – additional employees, contract labor. Testing installation, repair of software, testing sw before purchaing
Failure – cost catastrophic failure of system/sw
Performance – inpact on user, cost of work-arounds, loss of productivity when sw does not perform as expected
Redundant – need for additional hw or sw to maintain support or back-up data in case of sw failture
Common to maintain parallel system for 1+ year as a security measure.
Delayed – delays in production and transactions
Sales – lost transactions, discounted value.

14. Estimating Software Bug Costs
Cost of bugs still remaining with improved sw testing infrastructure.
Estimate the total cost of bugs to the user
Determine linearly related costs and nonlinearly related costs
User inputs:
total costs associated with bugs
general change in total costs in a 75% reduction in bugs
Cost of product failure or installation is linear with the number of bugs
Redundant system costs is not linear.
Difference in cost reduction is industry dependant.
Redundant systems costs – HW costs do not decrease linearly. Need for backup systems is not significantly reduced.

15. Estimating Software Bug Costs

16. Industry-specific User Costs
Industries encounter different types of costs
Two industries were selected:
Financial services – FEDI/clearinghouse software users
Transportation equipment manufacturing
Quality Categories:
Functionality
Reliability
Usability
Efficiency
Maintainability
Portability
FEDI - Financial Electronic Data Interchange

18. Questions?