Presentation on theme: "Fall Meeting October 2013 International Technology Alliance in Network and Information Sciences Human-Machine Conversations to Support Coalition Missions."— Presentation transcript:
Fall Meeting October 2013 International Technology Alliance in Network and Information Sciences Human-Machine Conversations to Support Coalition Missions with QoI Trade-Offs Alun Preece (Cardiff) Dave Braines (IBM UK) Diego Pizzocaro (Cardiff) Christos Parizas (Cardiff) Tom La Porta (PSU)
Human-machine information interaction in coalition mission support High-level tasking of network resources in terms of mission objectives Enabling exploitation of soft (human) sources in addition to physical sensing assets Locate & track high value targets in Border Zone Information quality in coalition mission support networks is highly variable, both due to the nature of the sources and the network capacity Coalition mission support Conversational model for human-machine and machine- machine interactions Interactions flow from natural language (NL) to Controlled English (CE) and back Scenario-based experimentation Natural language-based approach Users request information from the network (ask), while also being sources of information (tell)
3 Human-machine conversations Human-machine example: Soldier on patrol reports a suspicious vehicle by means of a text message from their mobile device. Machine-human example: A software agent sends a briefgist report to a human analyst indicating the vehicle is associated with a known high-value target (HVT). Machine-machine example: A broker agent queries or updates a database agent managing HVT sightings. Contributions Natural language (NL) based conversational approach that includes support for: requests for information (with specified QoI requirements) provision of information (at specified QoI levels) human-machine reasoning and information fusion (with aggregated QoI levels) Show how the model can be used to support realistic information exchanges in a coalition mission support context, offering flexibility in dealing with trade-offs associated with quality-of-information (QoI).
A new approach using Controlled Natural Language TA6 context P4.3: Coalition Context-Aware Assistance for Decision Makers Link to P4.1: how coalition communications environment affects cognition Link to P4.2: advanced fact extraction & reasoning Link to P5.2: high-level CE- based policies for asset management Types of interaction NL to CE query or CE facts (confirm) CE query to CE facts (ask-tell) exchange of CE facts (tell) Gist to full CE (expand) CE to CE rationale (why)
Controlled English Conversation Cards (CE-Cards) We conceptualise a conversation as a series of cards exchanged between agents, including humans and software services A conversation unfolds through a series of primitive communicative acts (e.g. queries, assertions, requests) natural language Controlled English a form of template-based CE that provides the gist of complex sets of CE sentences for brevity and easier human-readability Cards are modelled in CE with types as shown and attributes that include: is from is to is in reply to has content has timestamp has resource (for linked non-text content) 3 kinds of card content CE card model
CE-Cards Conversation Policies Examples gist: the red SUV is a threat expand: red SUV tell: there is a vehicle named v12345 that has `red SUV as description and has XYZ456 as registration and… tell: there is a vehicle named v12345 that is a threat and is located at central junction and… why: v12345 is a threat tell: v12345 is owned by HVT John Smith and… Conversation sequence rules* *can be defined in CE as part of the model
Vignette I Interacting agents: human patrol (location A) human intelligence analyst broker software agent that mediates between humans and other agents tasker software agent that handles access to database and sensor resources Step 1 Patrol on North Road (A) reports a suspicious black saloon car, vehicle registration ABC123, moving south [confirm interaction between patrol & broker; QoI on certainty of observation]
Vignette II Interacting agents: human patrol (location A) human intelligence analyst broker software agent that mediates between humans and other agents tasker software agent that handles access to database and sensor resources Step 2 Broker sends the patrol's report to tasker agent, and a DB query reveals the vehicle is linked to a HVT [ask-tell interaction between broker & tasker; QoI on certainty that vehicle is linked to HVT]
Vignette III Interacting agents: human patrol (location A) human intelligence analyst broker software agent that mediates between humans and other agents tasker software agent that handles access to database and sensor resources Step 3 Broker sends a request to tasker to track the vehicle. Tasker assigns a UAV to perform this task [tell interaction between broker and tasker; QoI requirements on accuracy of tracking]
Vignette IV Interacting agents: human patrol (location A) human intelligence analyst broker software agent that mediates between humans and other agents tasker software agent that handles access to database and sensor resources Step 4 UAV reports that the vehicle stops near Central Junction (B); the broker sends an alert to analyst [tell interactions between tasker, broker, & analyst; aggregate QoI from steps 1-3]
Summary of interactions (Steps 1-4) there is a tell card named '#2b' that is from the agent tasker and is to the agent broker and is in reply to the card '#2a' and has content the CE content 'there is an HVT sighting named h00453 that has the vehicle v01253 as target vehicle and has the person p670467 as hvt candidate'.
Handling Quality of Information (QoI) Issues Example 1 – QoI from the patrol (step 1) If the patrol is a trained team, likely that the word suspicious has a codified meaning and can be directly mapped to some kind of certainty range (symbolic or numeric). For a message from an untrained team or social media, the subjective meaning ofsuspicious would need to be estimated and the certainty recorded. e.g. based on past performance. These cases can be handled by the receiver of the message – broker in our vignette – using rules (conversational pragmatics) Example 2 – quality of information retrieved from database (step 2) We may have uncertainty that the vehicle is associated with HVT John Smith (the information may be out of date or inaccurate). Subjective logic may play a role in combining different uncertainty representations (symbolic, numeric). CE rationale plus provenance information offers a means of communicating the compounded result, gist offers a way to make it easily- digestible while also being expandable to users on request (or during subsequent audit
Architecture for experimentation CE-Card server supports exchange & storage of cards Software agents representing human and machine parties can be implemented in any programming language We are experimenting with a speech-based interface, in conjunction with a Google Glass- style display for gist content We would envisage full CE being directed to a user's handheld device
Conclusions & future work The conversational model appears robust enough to handle realistic scenarios The model can be extended with pragmatics rules to handle a range of QoI cases Next: Incorporate (richer) QoI cases / pragmatics. Extend/enrich scenario. Extend and link approach across P4 and to P5. Develop and run experiments with human subjects.