1. Conceptualizing multi- criteria decisions: what’s wrong with weighted averages? Presentation to be given at EURO XXI 2006, Reykjavik, Iceland, July 2006 Michael Wood Portsmouth University Business School, England michael.wood@port.ac.uk

2. Abstract There are a number of conceptual schemes for modeling multi-criteria decisions, but the dominant one in practice is based on the idea of a weighted average of ratings on the various criteria. This talk looks at the ubiquity of this idea, and at some of its difficulties—perhaps the main one being that it encourages decision makers to think that their problem is solved. An alternative conceptual framework—Pros And Cons On Common Scales—is then suggested as a way of addressing these difficulties.

3. I will … Give examples of weighted average schemes Give examples of weighted average schemes –including: choosing a mode of transport Look at their difficulties Look at their difficulties –Defining weights and values – more complex than it seems to many people –Thinking the problem is solved Introduce Pros and Cons on Common Scales (PACOCS) Introduce Pros and Cons on Common Scales (PACOCS) –Look at Pros and Cons of Pacocs Consider why multicriteria DA should not be possible Consider why multicriteria DA should not be possible

4. Motivation for this talk Teaching mcda—impossible to get students to question methods or assumptions Teaching mcda—impossible to get students to question methods or assumptions Feeling that there are problems in principle with mcda Feeling that there are problems in principle with mcda General feeling that concepts and methods tend to get unnecessarily complicated. Other examples: SPC and statistics in general, beer delivery routes … and arguably much of academia … General feeling that concepts and methods tend to get unnecessarily complicated. Other examples: SPC and statistics in general, beer delivery routes … and arguably much of academia …

5. Examples of weighted average value functions Choosing a car, laptop … Choosing a car, laptop … Best country for tackling environmental problems (New Zealand), best company to work for (W L Gore), best university (Portsmouth) Best country for tackling environmental problems (New Zealand), best company to work for (W L Gore), best university (Portsmouth) Choosing a mode of transport to get to work … Choosing a mode of transport to get to work … Often difficult to find basis for functions. Authors have considerable power?

8. Difficulties with SMARTS678 (Edwards and Barron, 1994) Scores need standardising Scores need standardising Weights need defining carefully – e.g. swing weights Weights need defining carefully – e.g. swing weights Aggregate value scale difficult to interpret Aggregate value scale difficult to interpret and … Problem appears to be solved! Problem appears to be solved! Answer may always be the same – but different day, different situation Answer may always be the same – but different day, different situation Answer may be “wrong” Answer may be “wrong”

9. So … Get rid of the problematic bits …

10. Pros And Cons On one or more Common Scales of value The procedure is to choose one of the objects of evaluation as the base, and then measure differences on each criterion between this base and the other objects using one or more common scales of value. Then these differences—pros if positive and cons if negative—can be added up to find the aggregate difference for each value scale. If there is just one value scale this aggregate can then be used as a basis for comparing the overall value of each object. (Similar to Tversky’s 1969 additive difference model.)

12. PACOCS No weights or value scores to confuse the issue No weights or value scores to confuse the issue Just scores on a common scale Just scores on a common scale Scores elicited directly for specific situation, or via a general conversion factor (e.g. -£0.117 per minute for duration) Scores elicited directly for specific situation, or via a general conversion factor (e.g. -£0.117 per minute for duration) Input and output easy to interpret Input and output easy to interpret Common scale could be based on one of the other criteria, but difficult to visualise Common scale could be based on one of the other criteria, but difficult to visualise For models not based on linear conversions results may depend on choice of base (consequence of Tversky’s intransitivity theorem) For models not based on linear conversions results may depend on choice of base (consequence of Tversky’s intransitivity theorem)

13. But it may not be possible to reduce everything to one scale! Exactly. In which case we shouldn’t be doing it!

15. Advantages of several scales … Admit things can’t be reduced to one dimension Admit things can’t be reduced to one dimension Make different trade-offs in different circumstances Make different trade-offs in different circumstances A deterministic numerical model with a single scale likely to lead to the same decision being made repeatedly A deterministic numerical model with a single scale likely to lead to the same decision being made repeatedly –Not optimum? –Evolutionary advantages of diversity—perhaps the distinguishing feature of human intelligence is its difficulty in reaching decisions? (Monkeys base decisions on a simple weighting system?)

16. PACOCS vs SMARTS (based loosely on Payne et al, 1993)

17. Why multicriteria DA should not be possible … Future preferences are unpredictable, particularly as they are likely to be modified by future experience Future preferences are unpredictable, particularly as they are likely to be modified by future experience Pretending that the problem can be solved may lock us into behaviour with little to recommend it Pretending that the problem can be solved may lock us into behaviour with little to recommend it This may be obscured by sophisticated methods, but … This may be obscured by sophisticated methods, but … PACOCS is such a crude approach that when it is very silly this should be obvious