Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright HDR 2007 1 How To Measure Anything: Finding the Value of Intangibles in Business.

Similar presentations

Presentation on theme: "Copyright HDR 2007 1 How To Measure Anything: Finding the Value of Intangibles in Business."— Presentation transcript:

1 Copyright HDR How To Measure Anything: Finding the Value of Intangibles in Business

2 Copyright HDR How to Measure Anything It started 12 years ago… I conducted over 60 major risk/return analysis projects so far that included a variety of impossible measurements I found such a high need for measuring difficult things that I decided I had to write a book The book was released in July 2007 with the publisher John Wiley & Sons This is a sneak preview of many of the methods in the book

3 Copyright HDR How To Measure Anything I love this book. Douglas Hubbard helps us create a path to know the answer to almost any question, in business, in science or in life. Peter Tippett, Ph.D., M.D. Chief Technology Officer at CyberTrust and inventor of the first antivirus software Doug Hubbard has provided an easy-to-read, demystifying explanation of how managers can inform themselves to make less risky, more profitable business decisions. Peter Schay, EVP and COO of The Advisory Council As a reader you soon realize that actually everything can be measured while learning how to measure only what matters. This book cuts through conventional clichés and business rhetoric and it offers practical steps to using measurements as a tool for better decision making. Ray Gilbert, EVP Lucent This book is remarkable in it's range of measurement applications and it's clarity of style. A must read for every professional who has ever exclaimed Sure, that concept is important but can we measure it? Dr. Jack Stenner, CEO and co-founder of MetaMetrics, Inc.

4 CFO Measurement Problem 1.The Risk Paradox: The largest and riskiest decisions often get the least quantitative risk analysis 2.The Measurement Inversion: According to an economic valuation of the benefits of a measurement, most measurement priorities are the opposite of the optimal solution. 3.Better alternatives than: –Traditional business cases that dont quantify uncertainty, risks, and intangibles –Scores that quantify nothing Copyright HDR

5 Style vs. Substance If you are adding and multiplying subjective scores on a scale of 1- 5 for things like risk, alignment, etc. chances are your method doesnt improve on your intuition Also dont be fooled by the terms structured or formal (Astrology is both structured and formal, it just doesnt work) The Following Charts Mean Nothing: Copyright HDR Strategy Efficiency EffectivenessCustomer Value Alignment Relationship Process Innovation

6 6 Assessing Assessment Methods Proven should mean more than some previous users feel good about it (the testimonial proof) Only empirical evidence that forecasts and decisions are actually improved can separate real benefits from a placebo effect Effective methods for evaluating IT investments should have a lot in common with well-known methods in other fields (actuarial science, portfolio optimization, etc.) Copyright HDR 2007

7 7 My Three Measurement Heroes Eratosthenes – measured the Earths circumference to within 1% accuracy Enrico Fermi – the physicist who used Fermi Questions to break down any uncertain quantity (and was the first to estimate the yield of the first atom bomb) Emily Rosa – the 11 yr old who was published in JAMA (youngest author ever) for her experiment that debunked therapeutic touch

8 Copyright HDR Three Illusions of Intangibles (The approach) The perceived impossibility of measurement is an illusion caused by not understanding : –the Concept of measurement –the Object of measurement –the Methods of measurement See my Everything is Measurable article in CIO Magazine (go to articles link on

9 Copyright HDR An Approach Model what you know now Compute the value of additional information Where economically justified, conduct observations that reduce uncertainty Update the model and optimize the decision

10 Copyright HDR Uncertainty, Risk & Measurement Measuring Uncertainty, Risk and the Value of Information are closely related concepts, important measurements themselves, and precursors to most other measurements The Measurement Theory definition of measurement: A measurement is an observation that results in information (reduction of uncertainty) about a quantity. We model uncertainty statistically – with Monte Carlo simulations

11 A Few of My Examples Risk of IT The value of better information The value of better security Forecasting the demand for space tourism Forecasting fuel for Marines in the battlefield Measuring the effectiveness of combat training to reduce roadside bomb/IED casualties The Risk of obsolescence The value of a human life The value of saving an endangered species The value of public health The value of IQ points lost by children exposed to Methyl- Mercury Copyright HDR

12 12 Calibrated Estimates Decades of studies show that most managers are statistically overconfident when assessing their own uncertainty Studies also show that measuring your own uncertainty about a quantity is a general skill that can be taught with a measurable improvement Training can calibrate people so that of all the times they say they are 90% confident, they will be right 90% of the time Copyright HDR 2007

13 13 Giga Analysts Giga Clients Statistical Error Ideal Confidence 30% 40% 50% 60% 70% 80% 90% 100% 50%60%80%90%100% % Assessed Chance Of Being Correct Percent Correct 99 # of Responses 16 IT Industry Analysts and 16 CIOs, the analysts were calibrated In January 1997, they were asked To Predict 20 IT Industry events Example: Steve Jobs will be CEO of Apple again, by Aug 8, True or False? 1997 Calibration Experiment Source: Hubbard Decision Research Copyright HDR 2007

14 The Value of Information Copyright HDR What it means: 1.Information reduces uncertainty 2.Reduced uncertainty improves decisions 3.Improved decisions have observable consequences with measurable value I use macro in Excel for this formula. In the book, I made a simple table that can estimate it with some simple multiplication

15 Copyright HDR Next Step: Observations Now that we know what to measure, we can think of observations that would reduce uncertainty The value of the information limits what methods we should use, but we have a variety of methods available Take the Nike Method: Just Do It – dont let imagined difficulties get in the way of starting observations

16 Copyright HDR Some Useful Suggestions It has been done before You have more data than you think You need less data than you think It is more economical than you think

17 The Math-less Statistics Table Measurement is based on observation and most observations are just samples Reducing your uncertainty with random samples is not made intuitive in most statistics texts This table makes computing a 90% confidence interval easy Copyright HDR

18 Copyright HDR Measuring to the Threshold Measurements have value usually because there is some point where the quantity makes a difference Its often much harder to ask How much is X than Is X enough Samples Below Threshold 20% 30% 40% 50% 0.1% 1% 10% Number Sampled Chance the Median is Below the Threshold % 5% 0.2% 0.5% 0

19 Copyright HDR Statistics Goes to War Several clever sampling methods exist that can measure more with less data than you might think Examples: estimating the population of fish in the ocean, estimating the number of tanks created by the Germans in WWII, extremely small samples, etc.

20 Copyright HDR Reducing Inconsistency The Lens Model is another method used to improve on expert intuition The chart shows the reduction in error from this method on intuitive estimates In every case, this method equaled or bettered the judgment of experts 0%10%20%30%40% Cancer patient life-expectancy Life-insurance salesrep performance Graduate students grades Changes in stock prices Mental illness using personality tests Student ratings of teaching effectiveness IQ scores using Rorschach tests Psychology course grades Business failures using financial ratios Reduction in Errors Battlefield Fuel Forecasts IT Portfolio Priorities My Studies Source: Hubbard Decision Research

21 Copyright HDR The Simplest Method Bayesian methods in statistics use new information to update prior knowledge Bayesian methods can be even more elaborate that other statistical methods BUT… It turns out that calibrated people are already mostly instinctively Bayesian

22 Copyright HDR Comparison of Methods Calibrated Estimator Bayesian Typical Un-calibrated Estimator Non- Bayesian Statistics Ignores Prior Knowledge; Emphasizes new data Ignores New data; Emphasizes Prior Knowledge Stubborn Gullible Under-confident (Stated uncertainty is higher than rational) Overconfident (Stated uncertainty is lower than rational) Cautious Vacillating, Indecisive Calibrated Skeptic

23 Copyright HDR Risk/ROI w/ Monte Carlo Administrative Cost Reduction Total Project Cost Customer Retention Increase 5%10%15% 10%20%30% $2 million$4 million$6 million ROI -50%50%100%0% A Monte Carlo simulation generates thousands of random scenarios using the defined probabilities and ranges The result is a range ROI not a point ROI

24 Quantifying Risk Aversion Copyright HDR Acceptable Risk/Return Boundary Investment Region The simplest element of the Nobel Prize-winning method Modern Portfolio Theory is documenting how much risk an investor accepts for a given return Investment

25 Define Decision Model Calibrate Estimators Conduct Value of Information Analysis (VIA) Measure according to VIA results and update model Populate Model with Calibrated Estimates & Measurements Analyze Remaining Risk Optimize Decision Approach Summary 25Copyright HDR 2007

26 26 Final Tips Learn how to think about uncertainty, risk and information value in a quantitative way Assume its been measured before You have more data than you think and you need less data than you think Methods that reduce your uncertainty are more economical than many managers assume Dont let exception anxiety cause you to avoid any observations at all Just do it

27 27 Questions? Doug Hubbard Hubbard Decision Research Copyright HDR 2007

Download ppt "Copyright HDR 2007 1 How To Measure Anything: Finding the Value of Intangibles in Business."

Similar presentations

Ads by Google