1. © HVR Consulting Services Ltd What Influences a Decision? David Daniel HVR-CSL John Holt HVR-CSL Graham MathiesonDstl

2. © HVR Consulting Services Ltd Overview Background Background –Experimental work to determine the impact of information on decision-making Single decision game Single decision game –Overview of earlier work by Daniel Study comparisons Study comparisons –Review similar work over the last 25 years Recent gaming Recent gaming Conclusions Conclusions

3. © HVR Consulting Services Ltd Background A simple, single decision game used to investigate the impact of level of information on quality of decision (Daniel, 1977) A simple, single decision game used to investigate the impact of level of information on quality of decision (Daniel, 1977) –Based on pioneering work of Sutcliffe (1971) –Found that individual player’s ability had greater impact than information level –Result has been replicated a number of times –John Holt, a psychology student, joined the team, to help evaluate the behavioural factors –Graham Mathieson has been active in this research over the last few years –This presentation provides a re-appraisal, 25 years on, bringing together past and present perspectives

4. © HVR Consulting Services Ltd

5. Single Decision Game E. Quality of player’s decisions used to assess impact of information A. Use single player as advisor/ decision maker B. Player given varying data levels as determined by experimental design C. Player takes single decision D. No learning effect between games allowed A1 Questionnaires A2 Thinking aloud/ recording of pairs dialogue Cognitive Data Gathering

6. © HVR Consulting Services Ltd Single Decision Game 1.Differences between players far greater impact than differences in information 2.Effect of prior information twice as great as other information 3.Pairs effect 4.Tighten game control – contol of game pace; provide secondary tasks 5.Players perform differently if their role is changed Results

7. © HVR Consulting Services Ltd Single Decision Game to the Red advance

8. © HVR Consulting Services Ltd Single Decision Game to the Red advance

9. © HVR Consulting Services Ltd Single Decision Game to the Red advance

10. © HVR Consulting Services Ltd Study Comparisons Rose and Sutcliffe (1971) - Submarine Commander Rose and Sutcliffe (1971) - Submarine Commander Pickburn and Davis (1990) - ASW assessment Pickburn and Davis (1990) - ASW assessment Davis and Reimann (1991) - Task Force Cdr Davis and Reimann (1991) - Task Force Cdr Perry and Moffat (1994) - Task Force Cdr Perry and Moffat (1994) - Task Force Cdr McCarthy, Mathieson, Osborn, Rugg-Gunn and Payne (2001) - RN Principle Warfare Officer (PWO) McCarthy, Mathieson, Osborn, Rugg-Gunn and Payne (2001) - RN Principle Warfare Officer (PWO) Mathieson and Malish (2002) - Extend PWO results Mathieson and Malish (2002) - Extend PWO results

11. © HVR Consulting Services Ltd Study Comparisons A. Use single player as advisor/ decision maker B. Player given varying data levels as determined by experimental design C. Player takes single decision D. No learning effect between games allowed E. Quality of player’s decisions used to assess impact of information A1 Questionnaires A2 Thinking aloud/ recording of pairs dialogue Cognitive Data Gathering A.Used in all games B.Used in all games B.Used in all games C.Used in all games D.Used in all games E.Used in all except McCarthy – correlation between info and sets of COAs & geospatial A1 Two of the games used A2Moffat and Perry tape recorded dialogue

12. © HVR Consulting Services Ltd Study Comparisons 1.Differences between players far greater impact than differences in information 2.Effect of prior information twice as great as other information 3.Pairs effect 4.Tighten game control – contol of game pace; provide secondary tasks 5.Players perform differently if their role is changed Results 1.Significant for all games; McCarthy sig for COA but not for geospatial 2.Prior info was significant in two games 3.Not looked at in any study 4.McCarthy controlled time; secondary task not used again 5.Not looked at again

13. © HVR Consulting Services Ltd Study Comparisons Behaviour at least as important as information flows Behaviour at least as important as information flows Players acting as advisors Players acting as advisors –Need to be aware of in future experiments Correlation measures a useful innovation, Correlation measures a useful innovation, –Does no evaluation of players’ performance reduce the pay-off from the gaming? Effect of player prior experience important impact on use of information Effect of player prior experience important impact on use of information

14. © HVR Consulting Services Ltd Study Comparisons Moffat and Perry (1997) provide extensive gaming methodology to correlate information, decisions and outcome, based on players asking for information Moffat and Perry (1997) provide extensive gaming methodology to correlate information, decisions and outcome, based on players asking for information –Regardless of decisions, two information sets dominated the choice of questions asked –Regardless of information requested, players decided between two specific courses of action Not able to establish relationship between information requested and decisions taken Not able to establish relationship between information requested and decisions taken –Demonstrated importance of internal influences

15. © HVR Consulting Services Ltd Recent Gaming Mathieson and Malish (2002) Mathieson and Malish (2002) – Gary Klein’s Recognition-Primed Decision- making (RPD) model of decision making –Provides psychological perspective –Initial work based on field studies of fire chiefs in US Decision makers match the patterns of cues and indicators from past experienceDecision makers match the patterns of cues and indicators from past experience Recognition provides access to pre-learned response cuesRecognition provides access to pre-learned response cues

16. © HVR Consulting Services Ltd Recent Gaming Mathieson used Klein ideas to help structure game Mathieson used Klein ideas to help structure game –Placed subjects under time pressure –Get first reaction response Subject briefed, given information updates then askedSubject briefed, given information updates then asked –What action? –What is the situation? –What are the key indicators? –Having thought, do you wish to amend your action? Used Endsley measure of situation awareness Used Endsley measure of situation awareness

17. © HVR Consulting Services Ltd Recent Gaming Results and conclusions Results and conclusions –Players not just driven by information Information appears to drive where players actInformation appears to drive where players act Personality appears to dominate how they actPersonality appears to dominate how they act –No relationship between course of action and level of situation awareness –Demonstrated feasibility of studying links between command behaviour and aspects of situation awareness Future work Future work –Land based –Repertory grids (in-depth construct elicitation) Accesses hidden knowledge in a different way, similar to Moffat and PerryAccesses hidden knowledge in a different way, similar to Moffat and Perry

18. © HVR Consulting Services Ltd Conclusions Single decision gaming is a well established potential research tool, with large data set now available Single decision gaming is a well established potential research tool, with large data set now available –Wider use of the approach would be desirable –More research required on information and decision- making Research on behavioural aspects at least as important as information level Research on behavioural aspects at least as important as information level Need more methods to investigate information and decision-making (e.g. Continuing the approach of Moffat & Perry) Need more methods to investigate information and decision-making (e.g. Continuing the approach of Moffat & Perry)

19. © HVR Consulting Services Ltd Unanswered questions Could additional selecting or training for commanders be helpful? Could additional selecting or training for commanders be helpful? Does the apparent dominance of prior information diminish the value of digitization, and similar initiatives? Does the apparent dominance of prior information diminish the value of digitization, and similar initiatives? Should the C2 systems be reduced in scope to provide information that will actually be used by commanders? Should the C2 systems be reduced in scope to provide information that will actually be used by commanders? Is there a need to introduce information systems personalised to individual commanders? Is there a need to introduce information systems personalised to individual commanders? Conclusions

20. © HVR Consulting Services Ltd For further details Dr John Holt HVR Consulting Services Selborne House Mill Lane AltonHampshire GU34 2QJ Tel: 01420 87977 Email: john.holt@hvr-csl.co.uk john.holt@hvr-csl.co.uk Website: www.hvrgroup.com