Command Visualisation NATO Workshop on Visualisation of Massive Military Multimedia Datasets, DREV Justin G. Hollands Human-Computer Interaction Group, DCIEM

Defence and Civil Institute of Environmental Medicine (DCIEM) OVERVIEW Problem and Potential Solution Visualisation Philosophy--Human Centered –Consistent with NATO IST-05 Model –Empirical Basis Need for Multiple Views Command Visualisation Testbed Platform –Five Research Domains

Defence and Civil Institute of Environmental Medicine (DCIEM) INFORMATION VISUALISATION Problem: –Military systems do not allow users to understand what their various data and information sources mean Users cannot use engines to extract information from data Important information may be ignored due to deficient extraction techniques Strategic and tactical actions, simulation and training, are less efficient than they should be

Defence and Civil Institute of Environmental Medicine (DCIEM) –Provide information visualisation tools to aid in the appropriate comprehension of information –Military benefits:  Improved information accessibility, filtration, extraction, and understanding.  Support for joint activities through better interoperation and communication  Challenger Example  Erosion of O-Rings as detailed by Thiokol POTENTIAL SOLUTION

Defence and Civil Institute of Environmental Medicine (DCIEM)

Source: E. Tufte (1997). Visual Explanations Conclusion: The way in which information is presented affects the way it is interpreted.

Defence and Civil Institute of Environmental Medicine (DCIEM) VISUALISATION TOOLS for C2 CPoF (DARPA) –Large-scale –Multimedia: Maps, Charts, Graphs, Pictures, CNN –3D Electronic Sand-Table –Multi-display –$$$

Defence and Civil Institute of Environmental Medicine (DCIEM) CPoF Graphical representation of cross-correlation of radio activity provides insight Same data in tabular form as good as useless

Defence and Civil Institute of Environmental Medicine (DCIEM) Data have to be displayed just the “right way” –How can we increase P(Hit)? –Make available a wide variety of views In line with graphical perception literature –Performance with a given graphical format is task-dependent Two design implications are evident: –1) Successful C2 system must display data in multiple formats; –2) System must make transition from one format to the next as straightforward as possible MULTIPLE VIEW AND TASK DEPENDENCY

Defence and Civil Institute of Environmental Medicine (DCIEM) VISUALISATION PHILOSOPHY “Information Visualisation is the human's capacity to utilize effectively the output from the computer to understand the data.” –NATO IST-05 NATO IST-05 Reference Model Specific Research Goal: –Produce a testbed platform based on the principles and recommendations contained in the HAT report –Platform will be designed based on performance data from human observers

Defence and Civil Institute of Environmental Medicine (DCIEM) TIF (DND Technology Investment Fund) Proposal –Goal: To create an effective command visualisation platform based on empirical performance data collected from human observers, rather than upon a programmer’s “best guess” –Approach: 5 Stages: 1) Identification/Selection of Operational Tasks 2) Testbed Platform Setup 3) Experiments to Address Specific Research Issues (Five Domains) 4) Iterative Platform Design and Testing 5) Delivery of Results COMMAND VISUALISATION TESTBED

Defence and Civil Institute of Environmental Medicine (DCIEM) Selection of Operational Tasks –Cognitive Task Analysis (CTA) will be performed on task group and/or task force commanders –Anticipated that the visualisation system would be particularly useful for: battle planning and simulation; establishing the threat axis; assessing damage, weapons and supplies; checking weapon ranges correlation of sensor data across time (data fusion) –CTA will determine what kind of information should be displayed to commander at what time (in what context)

Defence and Civil Institute of Environmental Medicine (DCIEM) What are we Measuring? –Error (1 - Accuracy) Error Magnitude, Number of Errors Signal Detection Measures--sensitivity and bias –Response Time –Subjective Measures Preferences Situation Awareness, Workload –In tasks that are as similar to real operational tasks as possible (CTA) using military personnel as participants

Defence and Civil Institute of Environmental Medicine (DCIEM) FIVE RESEARCH DOMAINS –Frame of reference and visual momentum Easing the transition between egocentric and exocentric views –Perceptual bias and reference points Utility of reference points for reducing bias –Modeling mental operations Minimizing operations necessary for task –Preattentive processing Utility of “pop-out” in visual search –Mapping data to perceptual continua What’s good for what

Defence and Civil Institute of Environmental Medicine (DCIEM) FRAME OF REFERENCE AND VISUAL MOMENTUM –Easing the transition between general and specific views Egocentric and exocentric views Hierarchical: generalization vs. specialization Drill up vs. drill down Visual momentum techniques: –Continuously available global view –Gradual transitions –Brushing

Defence and Civil Institute of Environmental Medicine (DCIEM) Roth et al. (1997) Visage.

Defence and Civil Institute of Environmental Medicine (DCIEM) PERCEPTUAL BIAS –Utility of reference points for reducing bias, and thereby error Source: Hollands & Dyre (2000)

Defence and Civil Institute of Environmental Medicine (DCIEM) MODELING MENTAL OPERATIONS Minimizing operations necessary for task Two factors affect graph reading performance: –1. Number of operations necessary given a particular task-graph combination –2. Effectiveness of the perceptual features used as input for the operations Model yet to be extended to more realistic visualisation situation A B Ratio estimation(height[A], height[A+B]); A B Sum = Summation(height[A], height[B]); Ratio estimation (height[A],Sum);

Defence and Civil Institute of Environmental Medicine (DCIEM)

PREATTENTIVE PROCESSING Utility of “pop-out” in visual search Visual properties that are processed preattentively can be used to highlight important image characteristics –colour, shape, orientation, length, size, curvature, depth Preattentive dimensions can be used to segment, create textured areas

Defence and Civil Institute of Environmental Medicine (DCIEM) MAPPING DATA TO PERCEPTUAL CONTINUA Bertin--appropriate use of spatial dimensions versus, colour, shading, texture, orientation etc. Conceptual dimension (e.g., number of helicopters, state of alert) may be qualitative, ordered, quantitative Appropriate in the sense of mapping conceptual dimension to perceptual dimension Bertin’s ranking will be empirically assessed

Defence and Civil Institute of Environmental Medicine (DCIEM) SUMMARY Problem and Potential Solution Need for Multiple Views Visualisation Philosophy--Human Centered –Consistent with NATO IST-05 Model –Empirical Basis Command Visualisation Platform –Five Research Domains

Defence and Civil Institute of Environmental Medicine (DCIEM) QUESTIONS/COMMENTS? Justin G. Hollands DCIEM 1133 Sheppard Ave. W., P.O. Box 2000 Toronto, Ontario M3M 3B9 Canada (416) x3082