Improving Collaboration in Unmanned Aerial Vehicle Operations March 27, 2007 Stacey D. Scott Humans & Automation Lab MIT Aeronautics and Astronautics

Teamwork in Unmanned Aerial Vehicle (UAV) Operations Intelligence Consumers (e.g., Ground Troops) UAV Operators Lawyers & PoliticiansGround Crew Shadow UAV

Supporting Futuristic UAV Teams Engaged in Time-Critical Operations Future UAV Teams

Current Collaboration Tools Insufficient for Time- Critical Operations Increasing reliance on collaboration tools -e.g., , instant messaging (chat) These tools introduce communication overhead, hindering management of tasking & team coordination (Scott et al., 2006) Trend in corporate and educational collaboration systems towards intelligently sharing activity information to improve planning & coordination in networked teams -Activity awareness design approach (Carroll et al., 2006) Promoting activity awareness in UAV operator interfaces may: -help UAV teams coordinate their actions -reduce the costs of obtaining activity-related information

Research Goals: Supporting Activity Awareness in Collaborative UAV Operations To develop and evaluate activity awareness tools for future teams of UAV operators engaged in time critical missions To investigate the software automation and interface requirements of UAV operations team members -what type of information is useful to know about one’s team members and their task activities? -what information do team members need to know about the UAVs’ activities? -how does one’s job duties / team role impact activity awareness requirements (e.g. mission commander, UAV operator, etc.)?

Initial Research Focus Future UAV Teams Technological Support for the Team Supervisor

Assisting Team Supervisors of UAV Teams Research Approach: 1.Develop representative collaborative, time-critical C2 task scenario 2.Conduct cognitive task analysis (CTA) to generate system requirements for supervisory decision support and activity awareness displays 3.Develop supervisory displays in a simulated task environment 4.Conduct exploratory experiment

Representative Collaborative UAV Task Scenario Ground Force Protection: Futuristic UAV Operations Team performing ISR in support of a time- sensitive targeting for a ground convoy traversing a hostile region Task Details: UAV Operations Team: -1 Mission Commander -3 UAV operators, each controlling multiple semi-autonomous UAVs UAVs have camera sensors only -Team must coordinate with external strike team UAV team can communicate with convoy & external intelligence sources

1 3 2 Hybrid Cognitive Task Analysis (Nehme et al., 2006): Step 1: Scenario Task Overview Step 2: Event Flow Diagram Step 3: Situation Awareness (SA) Requirements Step 4: Decision Ladders & Display Requirements Generating Requirements for Assisting Supervision 4 Produces Functional & Information Requirements

Supervisory Displays & Simulated Task Environment Large-Screen Wall Displays Situation Map DisplayMission Status Display TabletPC Display Mission Commander Display Simulation & Collaboration Server (Grouplab SharedDictionary)

Situation Map Display Mission Status Display Designing to Promote Activity Awareness UAV status & tasking Current & expected convoy safety Current & expected operator task performance, relative to convoy safety 4 (nominal) (down) 8 1 (reviewing ATR imagery)

Designing to Promote Activity Awareness Potential threat envelope Target strike indicators Known threat envelope Situation Map Display Mission Status Display UAV status & tasking Current & expected convoy safety Current & expected operator task performance, relative to convoy safety

Situation Map Display Mission Status Display Designing to Promote Activity Awareness UAV status & tasking Current & expected convoy safety Current & expected operator task performance, relative to convoy safety

Exploratory Study Subjects: 8 participants (including 6 NROTC) Protocol: -1 subject + 3 “confederates” per experiment -2 hrs training + 2 x 15 min trials -Retrospective interview 1 Subject as mCdr 3 “Confederates” as UAV Operators Situation Map Display Mission Status Display

Some Preliminary Results Convoy threat summary intuitive, useful for timing command decisions -Particularly, visualizing relationship between strike team activity and convoy’s progress UAV status & tasking useful for deciding when to assist UAV operators -However, lack of time-on-task cues required mental effort Operator performance displays rarely used, since surveillance progress was easily obtained from the Map (nominal)(down)(reviewing ATR imagery)

Ongoing Work: Test-bed for Exploring Other Collaborative TST Issues UAV Operations Team 1. Interruption recovery tools for the team supervisor

Ongoing Work: Test-bed for Exploring Other Collaborative TST Issues UAV Operations Team 2. Activity awareness interfaces for the UAV operators 3. New methods for deriving collaborative system requirements 1. Interruption recovery tools for the team supervisor

Expanded Simulated Task Environment Large-Screen Wall Displays Situation Map DisplayMission Status Display Mission Commander Display (TabletPC) Simulation & Collaboration Server (GroupLab SharedDictionary) Remote Assistance Display

Interruption Recovery Assistance for Team Supervisor Mission Commander Display (TabletPC) Interactive event timeline, containing event “bookmarks”

Interruption Recovery Assistance for Team Supervisor (cont’d) Clicking on an event bookmark highlights the location of the corresponding event on the map display, e.g., a convoy attack

Preliminary UAV Operator Display Designs Map Display Communications Display Tasking Display: TargetID & rerouting, reassigning UAVs

Improved Methodology for Deriving Collaborative System Requirements Project Goal: To develop techniques to identify dependencies in operator decision making to understand how to assist coordination of team member tasking

Behavioral pattern detection and performance predictions - Predictive algorithms (NN, HMM, Hybrid) Alert team supervising agent of sub-optimal cognitive strategies -Stress & time pressure -Rapidly evolving situation Is Operator using “good” strategies? Operator Is the team doing well? Supervisor Excessive workload Future Work: Team Performance Prediction

Behavioral pattern detection and performance predictions - Predictive algorithms (NN, HMM, Hybrid) Alert team supervising agent of sub-optimal cognitive strategies -Stress & time pressure -Rapidly evolving situation Is Operator using “good” strategies? Operator Is the team doing well? Supervisor Excessive workload Future Work: Team Performance Prediction Team

Conclusions Initial activity awareness designs show potential for facilitating supervisory-level decision making, and supporting teamwork in general -Particularly, designs aimed at conveying current & expected status of interdependent activities Activity awareness approach (i.e. intelligent sharing of activity data) appears to be an appropriate design technique for large-screen displays in time-critical C2 environments Further work is needed to understand what activity information should be shared (and at what level of detail) during different UAV mission types and mission phases

Contact and Sponsor Information Contacts: -Dr. Stacey Scott, -Prof. Mary (Missy) Cummings, -Humans and Automation Lab (HAL) website: Thanks to project sponsor: -Boeing Phantom Works