1. Defence Research and Development Canada Recherche et développement pour la défense Canada Canada Multinational Experiment 4 on Effects Based Approach to Operations A CFEC Report Philip S. E. Farrell, Ph.D. A presentation for IAMWG Meeting Friday 11 May 2007

2. Defence R&D Canada R & D pour la défense Canada Presentation Introduction – Human Experimentation MNE 4 CFEC Report Discussion – Can war-fighting experimentation be done in a controlled laboratory setting?

3. Defence R&D Canada R & D pour la défense Canada Introduction Not “The Bear” Now “The Ball of Snakes” New concepts need to be tested

4. Defence R&D Canada R & D pour la défense Canada Scientific Method High School Science Report –Background –Purpose –Hypothesis (related to concept) –Analysis –Apparatus –Procedures –Results –Discussion –Conclusions

5. Defence R&D Canada R & D pour la défense Canada Human Experimentation

6. Defence R&D Canada R & D pour la défense Canada Human Experimentation Job Similarity Index concept TTPs Recommendations Crew Selection Experiment (2006)

7. Defence R&D Canada R & D pour la défense Canada Human Experimentation in MNE 4 Too many people? Too many variables? Too many confounds? Too many constraints?

8. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Effects Based Approaches to Operations Refinement and Assessment of EBAO POT Desired End State Planning (desired effects) + _ Execution Assessment (effects) Operational Environment Current situations World states desired actions disturbances ++

9. Defence R&D Canada R & D pour la défense Canada Analysis Function KBD, SOSA, KM MNE 4 Experiment (CONOPS 0.9) + + ESAED RG ADRM SPRPREPOC&C AAMOP W AAMOE QCE MMA EBAP + _ + _ + _ IDI “action” comparators CA “situation” and “effects” comparators RsRs CsCs S RaRa ReRe CeCe CaCa AD ++ Planning Assessment Execution

10. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Experiment Aim Experiment Objectives Hypothesis Critical Operational Issues Measures Data Experiment Execution Data Collection

11. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Competing Aims! to explore concepts and supporting tools for EBAO to address the Canadian MNE 4 objectives

12. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Canadian Objectives –Refine, assess, and analyze elements of EBAO that Canada and the CF could apply to achieve integrated strategic effort in operations. –Determine whether EBAO represents a means of applying the Canadian 3-D approach to operations –Develop and refine a Multinational Knowledge Management Plan that can be tailored and applied to meet national operational requirements

13. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment The Hypothesis of this experiment was: If an operational commander and staff employ effects based operations, then they will have the ability to: Create a holistic understanding (KBD) Develop and refine an operational plan (EBP) Harmonize the full spectrum of DIME planning (MNIG) Coordinate, direct, and monitor operations (EBE) Assess actions, effects, and end state (EBA)

14. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment

15. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Experimental Design

16. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Observations –JOT (Joint Observation Tool) –JDCAT (Joint Battle Centre Data Collection Analysis Tool)

17. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses User surveys and network data

18. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Box Plots (KBD) Figure C.1 Distribution of correct responses for SA surveys, by organization

19. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Multi-Dimensional Scaling (KBD) Figure C.4: Multi-dimensional scaling of KBD3 answers by Group Reference Point (KBD 3) Reported same incorrect answers

20. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Count data (KM)

21. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Whitney – U (Culture) Q3.2b. To what degree is the decision of the leader contingent on the work and advice of others? (this question was for those who did not answer Leaderless Team in Q3.2a) Mann-Whitney U = 2212, p <.07 Not at allLittleSomewhatMuchCompletelyNot applicable 22 NRF0%5%44%36%11%4%  2 = 38.18, p <.001 CTF1%8%19%54%15%3%  2 = 110.13, p <.001

22. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Vector Method (Common Intent) DateNameVector 28 Febv1 (7, 5, 6, 7) 2 Marv2 (7, 6, 5, 7) 7 Marv3 (7, 6, 7, 7) 9 Marv4 (7, 7, 7, 6) 14 Marv5 (7, 6, 7, 7) 16 Marv6 (7, 7, 7, 6) Referencer (7, 7, 7, 7)

23. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Types of Analyses Vector Method (Common Intent) v1v2v3v4v5v6r v106.44.710.04.710.07.6 v26.407.810.07.810.07.6 v34.77.806.00.06.03.7 v410.0 6.00 0.03.7 v54.77.80.06.00 3.7 v610.0 6.00.06.003.7 r7.6 3.7 0 Table 3. Angle between vector pairs in degrees (Duncan & Farrell, 2007)

24. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Results, Discussion, Conclusions, and Recommendations are just as fuzzy as the objectives and hypothesis.

25. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Insights related to Canadian Objectives The notion of planning, execution, and assessment in an iterative fashion until the Current State reaches the End State is a robust concept. The relationship between MNIG, EBAO, and the 3- D concepts requires further development and experimentation. Some of the basic KM and KBD definitions and concepts still have not been resolved.

26. Defence R&D Canada R & D pour la défense Canada MNE 4 Experiment Recommendations More research is required!

27. Defence R&D Canada R & D pour la défense Canada Discussion Can war-fighting experimentation be done in a laboratory controlled setting?

28. Defence R&D Canada R & D pour la défense Canada Discussion Military buy-in vs. Scientific reputation Do you need SMEs to conduct the experiment?

29. Defence R&D Canada R & D pour la défense Canada Discussion Human Experimentation Ethics Is voluntary consent necessary?

30. Defence R&D Canada R & D pour la défense Canada Discussion political Considerations Who makes the decisions?

31. Defence R&D Canada R & D pour la défense Canada Discussion Concept complexity Are there too many variables?

32. Defence R&D Canada R & D pour la défense Canada Discussion Experimental Design Whatever happened to Hypothesis-testing?

33. Defence R&D Canada R & D pour la défense Canada Summary Human Experimentation is not new. MNE 4 has refined our processes for war-fighting experimentation. We need to pay attention to the scientific method if our recommendations are to be credible

34. Defence R&D Canada R & D pour la défense Canada Questions