UNCLASSIFIED Value Focused Metrics Professor D.F. Davis Presentation to 28 ISMOR 1 September 2011

UNCLASSIFIED 2 Metrics modeling: Using the metrics framework as the base for analysis. All assessment/metrics/evaluations normally consist of a set of measurements that are aggregated to some overall meaning. This aggregation technique is what I’m referring to as a metrics model. Definitions

UNCLASSIFIED 3 Most Frequent Top down decomposition aggregated by Multi-Attribute Utilities Most Important Next Important And so on….

UNCLASSIFIED 4 If and only if the decompositions are collectively exhaustive and mutually exclusive – cover every thing, doesn’t leave anything out, and are independent in definition. Most common error is that they are nowhere near independent, and are often highly dependent. Of course, knowing that you have included all necessary elements is always risky. But, they are easy to use and all you have to do is determine the weights. However

UNCLASSIFIED 5 Bumper Sticker Metrics need to be organic to the plan, not appliquéd.

UNCLASSIFIED 6 Value Focused Metrics Use decomposition for only the highest level, fundamental or strategic elements (Fundamental Objectives Hierarchy or FOH) – be careful and don’t expect much. Use a Means Ends Objectives Network (MEON) to represent the intermediate, dependent objectives and exploit the relationships. This is difficult because it requires that you graph your understanding of the inherent theory of change. Identify, for some or all of the objectives, metrics that would indicate the level of achievement of that objective.

UNCLASSIFIED 7 Value Focused Metrics - 2 Objectives should be written as something you wish to achieve. If they can be caused to happen directly, they are tasks or actions, not objectives. Therefore actions are something that you identify and execute with the hope that they will assist in the achievement of the objectives. The MEON represents how you believe the objectives support one another and propagate the results of actions up to the achievement of the fundamental objectives.

UNCLASSIFIED 8 Fundamental Objective Hierarchy Means Ends Objective Network Controls, Actions, Programs, Projects The System Process Metrics Outcome Metrics Performance Metrics

UNCLASSIFIED 9 The Somalia Game A discovery game was held with multiple agencies of the US Government to look at the impact on US Africa Command’s End States and Objectives of various shocks or scenarios. Impacts on objectives and actions to be taken were collected from the teams. Impacts were represented by specific expectations on metrics for individual objectives. Prevention and response actions were modeled by applying these actions to one or more objectives and developing likelihood expectations based on player input. The actual game was held as For Official Use Only and is not currently fully releasable. However, the gist of the process can be seen in the following models and tables where the actual objective is represented by its coding identifier.

UNCLASSIFIED 10 Numbers The MEON requires that each local model, an objective and it’s contributing (parent) objectives, have a conditional probability model. These models were developed in the Joint Planning Team for Somalia at the Command. Several sessions augmented by modeler subject mater expertise resulted in a fully described model for the FOH, MEON and the Metrics. During the game, the players identified which objectives their actions would effect and to what degree and directions (negative, positive) that action would impact the objective. This information along with modeler input was used to provide the local models for prevention and response turns. The model results were not used in the game, but were part of the post game analysis. The Control Group, or White Cell, led all in-game discussions.

UNCLASSIFIED 11 A Model Base Model Fundamental Objectives Hierarchy Means Ends Objectives Network Developed over six months and three continents: Africa – Nairobi and Djibouti, Europe – Stuttgart, US – Washington Shows how the overall end states are achieved through a network of intermediate objectives.

UNCLASSIFIED 12 Base Model with Metrics Concurrent to the development of the Base Model, each objective was addressed by asking the question: How would we know that the objective was achieved? During the game, the impacts assessed by the player teams were reflected in values of the metrics.

UNCLASSIFIED 13 Base Model with Actions During the game, there were two turns that addressed needed actions: Prevention and Response. These actions were identified by which objectives they impacted and with what magnitude and direction.

UNCLASSIFIED 14

UNCLASSIFIED 15

UNCLASSIFIED 16

UNCLASSIFIED 17 Questions?