Ward Page CPOF Program Manager DARPA Command Post of the Future Limited Objective Experiment - One (LOE-1) Some Results William Wright Principal Investigator DARPA CPOF Visual Insights Inc. By Permission Ward Page CPOF Program Manager DARPA Richard Hayes President Evidence Based Research www.ebrinc.com

Acknowledgement Most of the presentation materials here were provided by Ward Page and Dick Hayes.

Overview Command Post of the Future Limited Objective Experiments Scenario Space Data Collection Significant Findings Summary

Command Post of Today (What the services are developing) Forward TAC Division Advanced Warfighting Experiment 4th Infantry Division Ft. Hood, TX Main TOC Planning Cell

Command Post of Today Current Limitations Characteristics 60+ Workstations, 100+ people People are flooded by individual data streams Disjointed data systems; fragmented pictures of the battlefield No portrayal of uncertainties, inconsistencies or unknowns Requires too many people, too much communication Consequences Disjointed systems can cause negative situational awareness Increased time to comprehend significance on information Incomplete, inaccurate understanding of the battlefield Delayed decisions while waiting for more data, understanding Mismatch between available data and Commander’s cognitive model.

Command Post of the Future Operational Impact Increased Operational Tempo Faster recognition and better understanding of significant battlefield changes Faster and more complete exploration of available courses of action More rapid and more accurate dissemination of commands Smaller, More Mobile Command Structures Fewer staff members Smaller support trail & reduced deployment requirements More mobile, distributed command organizations Increased Span of Control

Command Post of the Future Command Post of the Future (CPOF) is a DARPA program that aims to: Increase Speed and Quality of Command Decisions Faster recognition and better understanding of changing battlefield situation Faster and more complete exploration of available courses of action Provide More Effective Dissemination of Commands COA capture for dissemination of commander’s intent Status and capability feedback from deployed operators Enable Smaller, More Mobile and Agile Command Structures More mobile, distributed command element Smaller support tail & reduced deployment requirements The goal of CPOF is to shorten the commander’s decision cycle to stay ahead of the adversary’s ability to react.

“We Own The Last 18 Inches” Visualization and Human-Computer Interaction From Other Programs: Analysis Tools and Planning Aids; Information Management; Networking and Comms 18” CPOF Will Create Design Rules Enabling a New Interface Metaphor for C2 After Next

Command Post of the Future Tailored Visualizations Immediate Understanding Match user’s cognitive model Data => Information => Knowledge Intuitive visual presentations Abstract as well as geospatial Temporal as well as static Decision-Centered Information tailored to decisions Show decision-relevant details Highlight relevant changes, anomalies, exceptions Uncover battlespace patterns Portray uncertainties Tailored to User Current echelon, task & situation User’s functional role User’s background & preferences

CPOF Experiments x Users CPOF Technology Known Situation Noise CPOF Aces (8-15) Battle Lab Students (40) Users CPOF Technology x Known Situation Asymmetric Threat (IW or BIO) Chem/Bio attack Info warfare attack Guerrilla Dispersed guerilla force Threatened urban attack HADR Urban disaster Peace Keeping Stability Imminent attack Sustained Engagement Opportunity to attack Multiple avenues of attack Random forces Noise Pedigree Completeness Accuracy Consistency Perishability Decision Performance Time Situation Awareness Control Condition S M L C1 C2 T1 T2 T3 Trial Matrix Trial Conditions Conditions Time Experts Students Core Hypotheses Improve Decision Speed & Quality H1: Improve Situation Awareness H2: Improve COA Generation H3: Improve COA Selection H4: Improve COA Communication

Limited Objective Experiment - 1 (LOE -1) Hypothesis: Tailored visualizations will improve Situation Awareness MOE: Correctness of Situation Awareness comprehension Quality of Pattern Recognition Greater Entity Retention

LOE-1 Scenario Space Less Complex More Complex Situation 10 Force-on-Force Situation 4 Situation 5 Insurgency

F on F Treatment B Blobs, 3D

F on F Treatment A Color Coded Legend Bn - enemy Co - enemy Pt - enemy Bn - friend Co - friend Pt - friend Legend F on F Treatment A Color Coded

Haiti Sit4B - 5 Critical Events Insurgency Treatment B Time-Space-Event View Haiti Sit4B - 5 Critical Events

Insurgent activity in Haiti by category and region Civil Preparatory Military Insurgency Treatment A Drill Down Significant preparatory activity in Port-au-Prince Military activity only in the North Dormant South

Insurgent activity in Haiti by week and category 14 12 10 8 Civil Number of incidents Preparatory Military 6 4 2 C-140 C-133 C-126 C-119 C-112 C-105 C-98 C-91 C-84 C-77 C-70 C-63 C-56 C-49 C-42 C-35 C-28 C-21 C-14 C-7 Current Week Increase in preparatory activity (last 9 weeks) Decrease in civil activity Low level military activity

Preparatory incidents by week and type 2 4 6 8 10 12 14 Week Number of incidents Recruiting Leadership Logistics Arms shipment Propaganda Preparatory type C-140 C-133 C-126 C-119 C-112 C-105 C-98 C-91 C-84 C-77 C-70 C-63 C-56 C-49 C-42 C-35 C-28 C-21 C-14 C-7 Current Very active recruiting Increasing propaganda and logistics Leadership activity throughout the period

Significant Findings Visualization technologies generated better Situation Awareness CPOF strongest in complex situations CPOF strongest in force-on-force situations CPOF strongest in understanding adversary’s situation Different Strengths Emerged from Alternative CPOF technologies Treatment B strongest where force ratio is important in force-on-force scenarios Treatment A strongest in overall sketch scores in insurgency situations Treatment A strongest in overall Situation Awareness scores in insurgency situations

Significant Findings (cont.) Time Issues and Others Some changes due to control scores getting worse rather than CPOF scores greatly improving Time appeared to help in case where visualization technique introduced new concept Longer viewing time did not always result in higher scores

CPOF Technologies Significantly Outperform Control in Overall Scores Unprompted Prompted 33.86 25.62 x 25.80 23.40 17.77 21.41 22.30 23.89 Control CPOF Technologies s N=157 p=.058 N=157 p=.007 Interpretation CPOF Technologies generated: Better situation awareness (higher mean or x) CPOF Technologies performance improves for prompted

CPOF Technologies Significantly Outperform Control in Complex Situations Prompted Unprompted 19.96 x 18.28 17.24 18.53 5.10 8.85 3.95 s 6.28 Control CPOF Technologies N=78 p=.000 N=78 p=.000 Interpretation CPOF Technologies generated: Better situation awareness (higher mean or x) in complex situations

CPOF Technologies Significantly Outperformed Control in Force-on-force Situations Unprompted Prompted 29.69 x 21.51 18.55 16.73 16.95 19.37 10.10 11.87 Control CPOF Technologies s N=78 p=.020 N=78 p=.019 Interpretation CPOF Technologies generated: Significantly better situation awareness than Control for both prompted and unprompted in Force-on-Force situations

CPOF Treatments Vs.Control for Enemy Representation in Insurgency Treatment A Vs. Control 17.58 18.20 33.13 22.25 Treatment B Vs. Control 35.13 21.55 22.25 17.58 Control Treatment A/B Interpretation Treatments A and B significantly outperformed control in representing enemy force information

Treatment B outperforms Treatment A in Overall Situation Awareness in Situation 13 Unprompted Prompted 29.66 x 14.85 22.93 15.93 12.21 12.71 15.00 12.06 s Treatment B Treatment A N=29 p=.073 N=29 p=.002 Interpretation Treatment A used icon visualization scheme (color coded) that subjects stated was confusing.

Differences between Treatments A & B in Insurgency Situations Overall Sketch Score Prompted Situational Awareness 72.20 17.13 64.87 x 16.21 45.48 30.93 30.83 24.88 s Treatment B Treatment A N=59 p=.07 N= 59 p=.05 Interpretation In Insurgency Situations: Treatment A outperforms Treatment B in overall sketch score Treatment A significantly outperforms Treatment B in prompted overall Situational Awareness

More Time is Not Always Better (Percent of instances where time did not help, when significant differences between times were found) 39% 23% 20% 8% Situation 10 Situation 4 Situation 13 Situation 5 Interpretation In less complex situations, more reversals in performance between times were found (subjects performed worse when given 5 minutes when compared with 3 minutes) with force on force, situation 10, containing the highest percentage of instances.

More Time Helped Only in More Complex Situations

Situation 4

Situation 5

Situation 10

Situation 13

Summary CPOF technologies appear to make a difference CPOF experimental approach captures the strengths and weaknesses CPOF technologies appear to improve subjects’ overall Situation Awareness when compared to traditional methods CPOF experimental approach captures strengths and weaknesses of each treatment