Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep.

Similar presentations


Presentation on theme: "Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep."— Presentation transcript:

1 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Quantitative Methods Make A Difference

2 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Overview Quantitative methods (probabilistic analysis, operations research, etc.) are widely used in other industries, but mostly lacking in IT investment analysis Over the past 7 years, we have been focusing specifically on the application of more advanced quantitative methods to IT This presentation will review the key findings from the application of quantitative methods to over 30 projects

3 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Quantitative Methods Include: Computing uncertainties and risks Computing the economic value of information Measurement methods for many items usually considered intangible Optimizing solutions when there are huge combinations of options for: –Roll-out priorities of systems –Selection of a portfolio of functions –Any other problem where different alternatives about different aspects of the investment generate lots of possibilities

4 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value AppliedInformationEconomics Economics Decision/Game Theory Empirical Decision Theory Statistics Information Theory SoftwareMetrics InformationEngineering Modern Portfolio Theory Operations Research Method: AIE Applied Information Economics is the practical application of scientific and mathematical methods to quantify the value of IT- enabled business investments

5 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Some HDR Clients Booz-Allen & Hamilton Accenture w/ the State of North Carolina Giga Information Group American Express The Discovery Channel U.S. Federal Government: –Department of Treasury –Bureau of The Census –Department of Veterans Affairs –General Services Administration –Housing and Urban Development The Axa Group – 6 major companies The Banking Industry Technology Secretariat Blue Cross Blue Shield of Illinois

6 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value What Do the Critics Say? Quantifying the risk and comparing its risk/return with other investments sets AIE apart from other methodologies. It can substantially assist in financially justifying a project -- especially projects that promise significant intangible benefits. The Gartner Group AIE represents a rigorous, quantitative approach to improving IT investment decision making…..this investment will return multiples by enabling much better decision making. Giga recommends that IT executives learn more about AIE and begin to adopt its tools and methodologies, especially for large IT projects. Giga Information Group AIE-like methods must become the standard way to make (IT) investment decisions. Forrester Research, Inc.

7 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Five Key Revelations 1.Quantifying risk radically changes IT investment priorities 2.Current measurement priorities are upside- down when compared to priorities based on economic value of information 3.Technology regret is a significant and overlooked factor in the the value of IT investments 4.The true cost of scope creep is much higher than most would think 5.The value of quantitative analysis would make it the best investment in most IT portfolios

8 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Finding 1: Risk Analysis When IT computes risk in the same way that an actuary would, many IT investments will look completely different We define risk a The probability of a quantified loss Risk analysis of IT investments involves a probabilistic analysis of all uncertain variables

9 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Normal Distribution Uniform Distribution Lognormal Distribution Hybrid Threshold confidence 15%85% Ideal Values: Point Real-world Meas. Real-world Measurements vs. Ideal Values

10 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value When asked to provide a subjective 90% confidence interval, most managers provide a range that only has about a 40%-50% chance of being right When asked to provide a subjective 90% confidence interval, most managers provide a range that only has about a 40%-50% chance of being right Actual 90% Confidence Interval Calibrated Estimates Measuring your own uncertainty about a quantity is a general skill that can be taught with a measurable improvement Studies show that most managers are statistically overconfident when assessing their own uncertainty Training can calibrate people so that when they provide a 90% confidence interval, it still has a 90% chance of being right (even though it is subjective) Perceived 90% Confidence Interval

11 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Distribution-based ROI Administrative Cost Reduction Total Project Cost % Improvement in Customer Retention 5%10%15 % 10%20%30 % $2 million$4 million$6 million ROI -50%50%100 % 0% Inputs

12 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Analyzing the Distribution 25%50%75%100%125%-25%0% Risk of Negative ROI Expected ROI ROI = 0% Probability of Positive ROI Return on Investment (ROI) The cancellation hump

13 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Return Risk 10% 20% 30% 40% 10%20%30%40%50%60% Probability of less than a risk-free return A proposed IT investment with a 15% risk and 54% return X Plotting the Risk and Return

14 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Example of Risk Effects 50% 40% 30% 20% 10% 0% 50%100%150%200% Expected IRR over 5 years Chance of a negative IRR These are real IT investments of $2M-$3M plotted against a clients investment boundary The 27% ROI investment is actually preferred to the 83% ROI investment Region of Unacceptable Investments Region of Acceptable Investments

15 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value 10% 100% 1000% 20%40%60%80%100% 20% 30% 50% 200% 300% 500% Size of the Project Relative to the Entire IT Portfolio (i.e. 50% = project makes up half the work in the entire portfolio) Required Minimum Return (IRR over 5 years) Most Risk Averse Approximate Median Most Risk Tolerant Range of Typical Hurdle Rates Risk Increases Required ROIs Adjusting for risk causes some previously-acceptable projects to be rejected Also, some low return but low risk projects would now be acceptable More projects with intangible benefits are now economically justified The net result: A completely reshuffled deck of IT project approvals

16 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Using Risk Analysis to Improve Decisions If the Risk is significant (it usually is), consider doing the following: Reduce the size and functionality of the proposed system - focus on fewer high-return features Wait until specific uncertainties in the environment subside - e.g. major mergers, reengineering, etc. Wait to tackle big projects until proper skills are developed and methods are in place Consider purchased packages that arent a perfect fit but close enough - they may look more attractive now Invest more on a proper economic analysis of the largest IT investments - this should reduce uncertainty about critical quantities Include deferred benefits in any estimate of scope creep costs

17 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Finding 2: Measurements Information has a value that can be calculated with a formula known for 50 years Computing the value of additional information on uncertain variables causes some surprising changes in what gets measures

18 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value The Decision Theory Formula: What it means: Information reduces uncertainty Reduced uncertainty improves decisions Improved decisions satisfy business objectives (by definition) The Economic Value of Information

19 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value The IT Measurement Inversion Typical Attention Economic Relevance Receives Most Attention Least Relevant to Approval Decisions Receives Least Attention Most Relevant to Approval Decisions Costs –Initial Development Costs –Ongoing Maintenance/Training Costs Benefits –A specific benefit (productivity, sales, etc.) –Utilization (when usage starts and how quickly usage grows) Chance of Cancellation See my article The IT Measurement Inversion in CIO Magazine (its also on my website at under the articles link)

20 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Finding 3: Technology Regret Most business cases treat IT investments implicitly as a now or never choice Technology regret is an economic quantity associated with committing to a technology after which, for whatever reason, becomes regrettable The equivalent of buyers remorse Technology regret becomes and important consideration in any environment where changing technology is a constant The issue becomes balancing technology regret (which tends to defer IT investments) vs. the opportunity loss of deferring the benefits of making the investment now

21 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Year Relative Computing Power Per $ (1980=1) % Annual Growth Rate Some Areas of Growth: Processors & Memory (Moores Law) Storage Communications (Paynes Law) Internet Users Sensory devices Competition makes capitalizing on new technology more critical to survival Changing Technology

22 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Changing With Technology A Critical Technology Measure Time How often should you change with technology? Avoiding technology regret is often a major driver in IT decisions. Upgrade 1 Upgrade 2

23 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Real Option Theory Single Period Option Value (Value of Waiting one period) Invest in this cycle, high priority Net Value of the Investment 0 + Invest this cycle, low priority, may be deferred if resources are strained - Re-evaluate in the next decision cycle Reject the investment 1.The option value tells us when an investment, even if it looks positive now, should be deferred until the opportunity is better 2.In the case of IT, waiting for the possibility of better technology around the corner should be considered

24 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value The Effects of Tech Regret Very long duration IT projects that commit to a proprietary solution tend to look much less favorable Short turnaround projects based on non-proprietary standards tend to look better Large scale commitments to the fastest improving technologies (like data storage, bandwidth) tend not to be favorable

25 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Finding 4: Scope Creep The cost of adding one additional function to an software development project is rarely addressed properly If anything, the only cost of new features considered is development cost Long term maintenance, increased risk of cancellation plus deferred benefits is much more significant

26 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value True Scope Creep Costs 24%: Initial development costs 24%: Future maintenance costs (computed over 5 years) 1%: Incremental contribution to probability of total project cancellation 51%: Deferred benefits of the other functions delayed by the proposed new function 24% 1% 51%

27 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Finding 5: Value of Quant. Organizations have successfully adopted more advanced quantitative methods for evaluating IT investments The cost of analysis routinely comes in below 1% and has always been under 2% of the investment size - including initial training This is still less than non-IT investments of similar size and risk It is also sometimes less time-consuming than the previous non-quantitative analysis techniques used by the firm (One of the reasons this analysis is efficient is we conduct a Value of Information Analysis - we only measure what is economically justified) Using the standard VIA calculation for the value of AIE analysis, AIE itself was the best investment of all the IT investments we analyzed - very conservative measures of payoffs put $20 to every $1 spent on AIE

28 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value Overview of RRA Analysis Intangibles Customer Satisfaction Strategic Alignment Technology Risk Information Quality etc. Measurables Errors in Decision X Change to Strategic Measure M Productivity in Activity Y Chance of cancellation, etc. 5%10%15% 10%20%30% $2 mill$4 mill $6 mill Measurements $ $$$ $$ Value of Info. Calculate Risk/Return Position "expected" ROI 50%100%150%200%250%-50%0% Probability of a negative ROI Probability of a positive ROI Organization's investment limit Acceptable region of investment Return Risk Classification

29 Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep Finding 2 What to Measure Finding 3 Tech. Regret Finding 5 AIE Value In Conclusion… Quantitative methods like AIE cause major IT decisions to be very different – and better Advanced methods can and have been learned and adopted by IT organizations More quantitative analysis can be the highest return investment in your IT portfolio


Download ppt "Hubbard Decision Research The Applied Information Economics Company Intro to Quant. Methods Finding 1 The effect of Risk Analysis Finding 4 Scope Creep."

Similar presentations


Ads by Google