1. Comparative Knowledge Discovery with Partial Order and Composite Indicator Partial Order Ranking of Objects with Weights for Indicators and Its Representability by a Composite Indicator.

2. Objective To develop data based methodology for multicriterion prioritization using partial order To determine if a partial order ranking can be represented by a composite indicator. To determine equivalence classes of composite indicators To seek reconciliation between stake holder designed composite indicator and partial order ranking based on evidence supplied by data

3. Problem Rank a set of n objects based on some intrinsic quality Intrinsic quality is not directly measurable Measurements on m surrogate indicators are used Data matrix: n by m matrix with columns with one column for one indicator

4. Approach Modification of classical partial order (POSET) ranking –Use stochastic ordering based on cumulative rank frequency (CRF) distribution of objects instead of averaging ranks assigned by linear extensions –Use indicator weights in forming CRF’s based on data –Indicator weights are based on proximity of linear extensions to ranks assigned by individual indicator columns to objects

5. Why Weighting Expert opinions and stakeholders’ interests may need to be incorporated. Makes sense to take into consideration weights based on data to prioritize. This allows us an opportunity for reconciliation between stakeholder based and evidence based weighted rankings.

6. Data based weights Traditional partial order based prioritization does not use any weighting. Instead, a novel modification that involves cumulative rank frequencies allows us to use weights. Weights can originate from an external, stakeholder source or can be data based. We begin with data based weights for POSET ranking.

7. Data Based Weighted POSET Ranking (DBWPR) Iterative method. Initially, uses uniform weights to form cumulative rank frequency distribution (CRF) matrix based on object ranks by each indicator column. The CRF matrix is used in place of the original data matrix The CRF method due to Patil and Taillie (2004) is used to compute POSET ranking Also data-based proximity weights for different indicators based on correlation between ranks of objects assigned by linear extensions and ranks of objects in indicator columns are computed End of first iteration

8. DBWPR continued Subsequent iterations New CRF matrix is computed based on object ranks by each indicator column. The CRF matrix is used in place of the original data matrix The CRF method due to Patil and Taillie (2004) is used to compute POSET ranking Also data-based proximity weights for different indicators based on correlation between ranks of objects assigned by linear extensions and ranks of objects in indicator columns are computed If data based weights match data based weights of the previous accept the most recent ranking as DBWPR and terminate iteration else do another iteration.

10. An Example 10 objects, 3 indicators Data matrix Rank Matrix IDq1q2q3 0 1.56.52 1 945 2 93 3 58 4 4 5 5 411 6 1.539.5 7 895 8 1099.5 9 427 IDq1q2q3 0 9310 1 278 2 619 3 644 4 738 5 71412 6 9112 7 318 8 112 9 7136

11. Rank Frequency Matrix with equal weights for three indicators ID 12345678910 0 11000 1 0000 1 000 1100 0 1 0 2 0010010010 3 00 0011010 0 4000111000 0 5200100000 0 6101000010 0 7000010011 0 80000000021 9010100100 0

12. Cumulative Rank Frequency Matrix (CRFM) ID 12345678910 0 12222 3 3333 1 000 1222 2 3 3 2 0011122233 3 00 0012233 3 4000123333 3 5222333333 3 6112222233 3 7000011123 3 80000000023 9011222333 3

13. Example (Continued)

14. Equivalent Weights Given a composite indicator with a given vector of weights, it is of interest to find another equivalent composite indicator, that is, another vector of weights that will yield a composite indicator inducing the same ranking that the given indicator does. Aside from academic curiosity, knowing all such classes of equivalent weight vectors can be used to bring about reconciliation between competing indicators.

15. Determining Regions of Equivalent Weights For each object there is a vector of m indicator values in the m dimensional Euclidean space – a data vector. Given a vector of weights = (w 1, w 2, …, w n ), corresponding composite indicator values for different objects are projections of data vectors on the vector of weights. Determination of regions of equivalent weights amounts to examining, for each pair of incomparable objects, the position of the weight vector relative to the two respective data vectors such that they have equal projections on the weight vector. Intersection of the weight vector having equal projections of two incomparable data vectors with the weight space w 1 + w 2 … +w n = 1 divides the weight space into two. All such intersections arising from all pairs of incomparable data vectors partitions the weight space into equivalence classes of weight vectors.

16. An illustration with two indicator space The vector w’ has equal projections of data vectors x and y. Vector w’ intersects the line w1 + w2 = 1 at a certain point. If we look at such intersections of different weight vectors with respect to all pairs of incomparable objects we get a partition of the line w1+w2 = 1. Any two weight vectors with their intersection with w1+w2 = 1 in the interior of an interval of the partition, will define equivalent composite indicators.

17. Example 1. 9 by 2 Data Matrix Objectq1q2 a93 b27 c61 d64 e74 f96 g31 h11 i85

18. Example 2. 7 by 3 Data Matrix Objectq1q2q3 a937 b278 c619 d644 e735 f318 g749

19. Representability Having obtained POSET ranks, is there an index, say, with a vector w of indicator weights that produces an identical ranking of objects? If so the POSET ranking is representable by a composite index Not always a POSET ranking is representable If it exists we denote it by w # How do we find a w # if one exists? If one exists are there more, leading to equivalent weights in the weight space with common rankings. We reduce the problem of finding w # to a linear programming problem.

20. Search for an equivalent composite index Let X = (x ij ) be the n by m data matrix with rows arranged in increasing order by the POSET ranking of objects. n = number of objects m = number of indicators Consider an (n-1) by m matrix D = (d ij ), d ij = x ij - x i+1j i = 1, 2, …., n-1 Let w’ = (w 1, w 2, …, w m ) Then an index with weights w exists if and only if there is a solution to maximization or minimization of some suitable linear function of w’s subject to D’w >= 0 together with some additional constraints. To enforce ties some constraints involve strict equalities.

21. Representability Region If given a ranking is representable then its representability region is the set of all weight vectors each one of which defines a composite indicator that represents the given ranking. Given a ranking its representability region can be obtained by solving the associated linear programming problem with different objective functions. For a data matrix with two or three indicators the representability region can be obtained with simple graphing tools.

22. Example 1. 9 x 2 Data MatrixRepresentability Region Objectq1q2DBWPR a933 b276 c617 d645 e744 f961 g318 h119 i852

23. Example 2 7 x 3 Data MatrixRepresentability Region Obj.Idq1q2q3DWBPR a9372 b2785 c6193 d6446.5 e7354 f318 g7491

24. Absence of Representability Representability of DBWP ranking is desirable. If DBWP ranking is representable then there is possibly a representability region in which case there is some scope for reconciliation with another index if it becomes necessary. Hence if DBWP ranking is not representable we would like to look for approximate representability. DBWP ranking is approximate representable if there is a composite indicator whose ranking has statistically significant correlation coefficient with DBWP ranking.

25. Search for Approximate Representable Composite Indicator Relaxing constraint/s in the linear programming problem solving. –By examining output of the linear program solver used one may be able to eliminate some constraints from the linear programming set up and be able to obtain a composite index which is highly correlated with DBWP ranks. Search the weight space for a weight vector with maximum correlation with DBWP ranks. Not enough is known about possibility of success of either of the above two methods nut we have an example for which both methods worked.

26. Example Approximate Representability 11 x 3 Data Matrix Obj Idq1q2q3DBWPR a9310 2 b278 9 c619 6.5 d5107 8 e644 6.5 f738 5 g71412 1 h9112 3 j318 10 k112 11 l7136 4

27. Microsoft Excel Solver Output Row Number Zw1w2w3Constraint Right Side Solution1.19E-111.22E-12 11.43E-11111 2-2112-7.9E-120 30-888.88E-160 42-2-41.64E-110 5010-29.78E-120 6121.31E-110 70-352.44E-120 81-6-38.82E-130 9333.81E-110 1060-4.5E-120 112063.11E-110 121001.19E-110 130101.22E-120 140011.22E-120 151111.43E-111

28. Approximate Representability

29. Reconciliation DBWP Ranking based on evidence supported by data Stakeholders ranking, if based on subjective basis, may need to be reconciled with DBWP ranking Possibilies –Latter may be in representability or approximate representability region – no reconciliation needed –Latter may be in an equivalency region adjoining representability/approximate representability (RAR) region. Still a difficult situation to handle –Latter may be in a distant equivalency region Harder situation (continued)

30. Reconciliation Harder situation Suggested solution –Locate equivalency region of stakeholder index –Consider a path from the center of the stakeholder region to the center of RAR region along the path toward the RAR region through intervening regions –At each border region compute correlation between ranking due to stakeholder index at current location and DBWP ranking. –If correlation is significant try to persuade stakeholder to accept the modified composite indicator –If not acceptable it is time to review selection of surrogate indicators, their measurements, data collection

31. Reconciliation Example Positionw1w2w3Corr.Coeff P0.1340.4750.3910.6307 Q0.1990.4220.3790.7748* R0.2800.360 0.8829** S0.3690.2790.3620.9550