Presentation is loading. Please wait.

Presentation is loading. Please wait.

Perspectives on Sensemaking Simon Attfield Visual Analytics Summer School, 2010 1.

Similar presentations


Presentation on theme: "Perspectives on Sensemaking Simon Attfield Visual Analytics Summer School, 2010 1."— Presentation transcript:

1 Perspectives on Sensemaking Simon Attfield Visual Analytics Summer School,

2 Picture by Ed Sanders 2

3 Sensemaking is... The integration of information and knowledge to form an interpretation (Attfield, last weekend) 3

4 4

5 Cognitive dissonance The fox and the grapes ‘adaptive preference formation’ 'Oh, you aren't even ripe yet! I don't need any sour grapes.' Picture credit - Milo Winter 5

6 A search for coherence information data knowledge expectation experience values aspirations 6

7 Cognition - How do people reason with information? How do they find meaning, reconstruct a domain, reach conclusions, and derive new puzzles and questions? Social interaction – How do people interact around information? How do they share, convince, question, agree, cajole and debate? Behaviour - How do people manage information? How do they adapt and shape their environments in order to achieve cognitive and social processes? Tools - What can all of this say about the resources and functionality that can provide the leverage to do these things more naturally, invisibly and effectively? Key questions 7

8 Expectation and language Battered Child Syndrome – Up to 1940’s no concept of battered child syndrome. – Late 1940’s - a surprise - a paediatric radiologist noticed a few cases of child injuries not adequately accounted for by parents medical ‘histories’. – Discrepancy explained away. – More cases noticed. – In 1953 a comprehensive report identified 749 cases. – Within a few years laws introduced mandating reporting. Weick,

9 How do we makes sense of the world with very little information? – schemata Data structures for representing generic concepts stored in memory Introduced into psychology by Bartlett ‘The building blocks of cognition’ (Rummelhart, 1980) Variables that can be bound to an instantiation – e.g. Buying A schema is a ‘theory’ – goes beyond our perception Comprehension is a process of hypothesis testing european-car-sharing-company/ 9

10 Eh? The procedure is actually quite simple. First you arrange things into different groups depending on their makeup. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set. It is important not to overdo any particular endeavor. That is, it is better to do too few things at once than too many. In the short run this may not seem important, but complications from doing too many can easily arise. A mistake can be expensive as well. The manipulation of the appropriate mechanisms should be self-explanatory, and we need not dwell on it here. At first the whole procedure will seem complicated. Soon, however, it will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the immediate future, but then one never can tell. From Bransford and Johnson,

11 The procedure is actually quite simple. First you arrange things into different groups depending on their makeup. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set. It is important not to overdo any particular endeavor. That is, it is better to do too few things at once than too many. In the short run this may not seem important, but complications from doing too many can easily arise. A mistake can be expensive as well. The manipulation of the appropriate mechanisms should be self-explanatory, and we need not dwell on it here. At first the whole procedure will seem complicated. Soon, however, it will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the immediate future, but then one never can tell. From Bransford and Johnson,

12 The data-frame theory of sensemaking Frame – An explanatory structure that links elements e.g. a story, a map, a script, a plan. – Accounts for data and guides the search for more data. – Inferred from a few key anchors (3 or 4 at most) – Functions as a hypothesis Data – Information from the world. – The interpreted signals of events. Process proceeds through fitting data to the frame while testing and improving the frame. Klein, Phillips, Rall, Peluso (2007) 12

13 Klein, Phillips, Rall, Peluso (2007) 13

14 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. 14

15 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. Add details. 15

16 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. Add details. Fundamental surprise (Lanir, 1991). Navigator’s frame- breaker. Emotional response Experts have more differentiated frames (Feltovich, 84) 16

17 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. Add details. Fundamental surprise (Lanir, 1991). Navigator’s frame- breaker. Emotional response Experts have more differentiated frames (Feltovich, 84) Explain away data. Knowledge shields. Fixation error produced by inaccurate anchor early on. 17

18 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. Add details. Fundamental surprise (Lanir, 1991). Navigator’s frame- breaker. Emotional response Experts have more differentiated frames (Feltovich, 84) Explain away data. Knowledge shields. Fixation error produced by inaccurate anchor early on. Nurses in ICU maintaining multiple frames. Tracking up to three frames. Logical competitor set. 18

19 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. Add details. Fundamental surprise (Lanir, 1991). Navigator’s frame- breaker. Emotional response Experts have more differentiated frames (Feltovich, 84) Explain away data. Knowledge shields. Fixation error produced by inaccurate anchor early on. Nurses in ICU maintaining multiple frames. Tracking up to three frames. Logical competitor set. Duncker’s radiation problem. 19

20 Klein, Phillips, Rall, Peluso (2007) Depends on data, goals, repertoire and stance. Pattern matching. Add details. Fundamental surprise (Lanir, 1991). Navigator’s frame- breaker. Emotional response Experts have more differentiated frames (Feltovich, 84) Explain away data. Knowledge shields. Fixation error produced by inaccurate anchor early on. Nurses in ICU maintaining multiple frames. Tracking up to three frames. Logical competitor set. Duncker’s radiation problem. Explicit search. 20

21 Reasoning in sensemaking Experts and novices reason in the same way, except novices find it difficult to distinguish signal from noise and are more reluctant to speculate. Experts have a richer repertoire of frames – greater variety, more coverage, finer differentiation. More precise expectations. Abductive reasoning predominates in sensemaking – ‘reasoning to the best explanation’ – based on plausibility. Klein, Phillips, Rall, Peluso (2007) 21

22 A symmetry of inference A)If it rains in the night the lawn will be wet in the morning. B)It rained last night. C)The lawn is wet (said in the morning) DeductionInductionAbduction 22

23 A symmetry of inference A)When someone with drugs in their pocket sees a policeman they put their hand in their pocket. B)A man with drugs in his pocket saw a policeman C)The man put his hand in his pocket DeductionInductionAbduction 23

24 24

25 25

26 26

27 Will you ever look at things the same again? 27

28 The rain example If it rains in the night, the lawn will be wet in the morning. If the children get up in the night and play with the hose, the lawn will be wet in the morning. If a bush-fire fighting helicopter on a training mission in the middle of the night accidently drops its load of water over our garden, the lawn will be wet in the morning. 28

29 What rule am I thinking of? I am thinking of a rule the describes the relationship between three numbers. The sequence satisfies the rule. Your job is to discover the rule by suggesting other number sets - I will tell you whether each one satisfies the rule. When you think you know the rule, tell me. 29

30 Confirmation bias Peter Wason Challenged subjects to identify a rule applying to triplets of numbers e.g Subjects seemed to test only positive examples, e.g. each number is two greater than its predecessor (11, 13, 15) 30

31 Confirmation bias Karl Popper Marxism and psychoanalysis explain everything? Worked with Alfred Adler, co- founder of psychoanalysis Alder analysed child in terms of theory of inferiority without seeing the child. Picture credit: Popper,

32 Hang on... how does this relate to visual analytics? Pirolli and Card’s model of intelligence analysis Pirolli and Card (2005) 32

33 Attfield and Blandford (in press) Externalises evidence and reasoning Structures thinking Reveals gaps Can enforce rigor Attfield and Blandford’s model of legal investigations 33

34 Representing the story with chronologies P1 I think it’s a very natural way for us to think here, we always use chronologies, our great organising basis. […] I had a team of five or six people and I allocated responsibilities to each of these people saying “Right you’re going to become the master of [issue a], I’m going to do [issue b], [issue c], [issue d]. Someone else is going to do [issue e]” 34

35 Representing the story with chronologies A met B at on … Evidence link B flew to on … B met C at on … ? 35

36 Support recursive decomposition Line of enquiry Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators 36

37 Support recursive decomposition Line of enquiry Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators 37

38 Support recursive decomposition Line of enquiry Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators 38

39 Support recursive decomposition Line of enquiry Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators Theories Questions Info. seeking strategies Evidence Knowledge reps Lines of enquiry Investigators 39

40 Analysis of Competing Hypotheses 40

41 Analysis of Competing Hypotheses 41

42 Analysis of Competing Hypotheses Starts with a set of possibilities rather than a most likely alternative for which the analyst seeks confirmation (avoids satisficing) Identifies the few items of evidence or assumptions that have greatest diagnostic value. Seeks evidence to refute hypotheses. Most probable hypothesis is one with least evidence against it (generally). 42

43 Analysis of Competing Hypotheses (ACH) 1. Identify hypotheses that merit consideration Heuer’s example... Q: Will Iraq retaliate for US bombing of its intelligence headquarters? H1 - Iraq will not retaliate. H2 - It will sponsor some minor terrorist actions. H3 - Iraq is planning a major terrorist attack. If you don’t generate the right hypothesis you wont get the right answer Use a group of analysts 43

44 Analysis of Competing Hypotheses (ACH) 2. List significant evidence for and against Heuer’s example... E1. Saddam public statement not to retaliate. E2. Absence of terrorist offensive during 1991 gulf war E3. Assumption that Iraq would not want to provoke another US attack E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts E5. Iraqi embassies instructed to take increased precautions. E6. Assumption that failure to retaliate would be unacceptable loss of face for Saddam Include assumptions Ask what should I expect to see or not see? Note the absence of evidence as well as its presence. 44

45 Analysis of Competing Hypotheses (ACH) 3. Prepare a matrix with hypotheses across the top and evidence down the side. Analyse how they relate. Consider diagnosticity Hypotheses Evidence No retaliation Minor actions Major attack E1. Saddam public statement not to retaliate. +++ E2. Absence of terrorist offensive during 1991 gulf war ++- E3. Assumption - Iraq would not want to provoke another US attack ++- E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts -++ E5. Iraqi embassies instructed to take increased precautions. -++ E6. Assumption - failure to retaliate would be unacceptable loss of face for Saddam

46 Analysis of Competing Hypotheses (ACH) 4. Refine the matrix. Reconsider hypotheses and delete evidence and arguments with no diagnostic value. Hypotheses Evidence No retaliation Minor actions Major attack E2. Absence of terrorist offensive during 1991 gulf war ++- E3. Assumption - Iraq would not want to provoke another US attack ++- E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts -++ E5. Iraqi embassies instructed to take increased precautions. -++ E6. Assumption - failure to retaliate would be unacceptable loss of face for Saddam --++ More hypotheses needed? Is your thinking influenced by evidence or assumptions not included? 46

47 Analysis of Competing Hypotheses (ACH) 5. Draw tentative conclusions about likelihood of hypotheses. Work down the matrix. Let them compete for favour. Hypotheses Evidence No retaliation Minor actions Major attack E2. Absence of terrorist offensive during 1991 gulf war ++- E3. Assumption - Iraq would not want to provoke another US attack ++- E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts -++ E5. Iraqi embassies instructed to take increased precautions. -++ E6. Assumption - failure to retaliate would be unacceptable loss of face for Saddam --++ Hypothesis with fewest minuses is probably best, BUT You can never prove a hypothesis true. 47

48 Analysis of Competing Hypotheses (ACH) 6. Consider how sensitive your analysis is to a few items of evidence. What if they were mistaken? Hypotheses Evidence No retaliation Minor actions Major attack E2. Absence of terrorist offensive during 1991 gulf war ++- E3. Assumption - Iraq would not want to provoke another US attack ++- E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts -++ E5. Iraqi embassies instructed to take increased precautions. -++ E6. Assumption - failure to retaliate would be unacceptable loss of face for Saddam --++ Question the linchpin assumptions – a common source of error. What about deception? 48

49 Analysis of Competing Hypotheses (ACH) 7. Report conclusions – discuss relative likelihood of all hypotheses. Hypotheses Evidence No retaliation Minor actions Major attack E2. Absence of terrorist offensive during 1991 gulf war ++- E3. Assumption - Iraq would not want to provoke another US attack ++- E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts -++ E5. Iraqi embassies instructed to take increased precautions. -++ E6. Assumption - failure to retaliate would be unacceptable loss of face for Saddam --++ There is always the possibility of being wrong. Decision makers need to evaluate. 49

50 Analysis of Competing Hypotheses (ACH) 8. Identify future observations that may indicate events are taking a different course than expected. Hypotheses Evidence No retaliation Minor actions Major attack E2. Absence of terrorist offensive during 1991 gulf war ++- E3. Assumption - Iraq would not want to provoke another US attack ++- E4. Increase in frequency/length of monitored Iraqi agent radio broadcasts -++ E5. Iraqi embassies instructed to take increased precautions. -++ E6. Assumption - failure to retaliate would be unacceptable loss of face for Saddam --++ Analytic conclusions should always be considered as tentative. 50

51 Summary Sensemaking – Integration of data and knowledge to form an interpretation. – Dependant upon knowledge, experience, values, aspirations, expectations. – Schemata and abduction can play a major role. – The process is powerful BUT fallible. – Schematisation can make sensemaking more systematic (e.g. ACH). 51

52 References Aesop (c 500BC) The Fox and the Grapes. Weick K. (1995) Sensemaking in Organisations. Sage, London, England. Klein G.A., Phillips J.K., Rall E.L., Peluso D.A.: A Data-frame Theory of Sensemaking. In: Expertise Out of Context: Proc. of the Sixth International Conf. on Naturalistic Decision Making (Pensacola Beach, Florida, May 15-17, 2003). Lawrence Erlbaum Assoc. Inc, US, (2007) pp , Popper K. (1963) Science as Falsification: In Conjectures and Refutations London: Routledge and Keagan Paul, 1963, pp , also at Pirolli P & Card S (2005) The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis, Proceedings of the International Conference on Intelligence Analysis (McLean, VA) https://analysis.mitre.org/proceedings/.https://analysis.mitre.org/proceedings/ Attfield S. & Blandford A. (in press) Making Sense of Digital Footprints in Team-based Legal Investigations: The Acquisition of Focus. Human Computer Interaction Journal, Special Issue on Sensemaking. Heuer R. J. (1999) The Psychology of Intelligence Analysis: Center for the Study of Intelligence, Central Intelligence Agency, available at https://www.cia.gov/library/center-for-the-study-of- intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/index.htmlhttps://www.cia.gov/library/center-for-the-study-of- intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/index.html Jones M.D. (1995) The Thinker’s Toolkit: 14 Powerful Techniques for Problem Solving. Three Rivers, New York. 52


Download ppt "Perspectives on Sensemaking Simon Attfield Visual Analytics Summer School, 2010 1."

Similar presentations


Ads by Google