8 Ruben Strenzke 2011 Requirement 1: The assistant system has to guide the attention of the human operator to the most urgent task. Requirement 2 : If the human operator cannot carry out the most urgent task because of overtaxing, then the assistant system has to take initiative to transfer this situation into one which can be handled normally by operator. Requirement 3: If there are tasks, the human operator is not capable to accomplish, or which are of too risky or costly, these tasks have to be allocated to the automation. Basic Requirements for Assistant Systems Cognitive Automation Approach [Onken&Schulte 2010] Assistant system shall provide: Situation awareness improvement, work load reduction, error avoidance, error elimination… but be silent and invisible most of the time – cannot be accessed by human initiative knows and follows the same work object as the human does
10 Ruben Strenzke 2011 Example for Mixed-Initiative Dialog Assistant System takes initiative: “UAV1 needs follow-up task” Operator presses proposal-button Assistant System proposes: “add task transit A B for UAV1” Operator presses accept-button Assistant System affirms: “added task transit A B for UAV1” asf. Assistant Sytem needs to solve these questions: - What is the operator planning? - What is he/she doing? - Is his/her plan good? - Does he/she follow the plan or make errors? - Is he/she overtaxed?
11 Ruben Strenzke 2011 Projects‘ Background –Manned-Unmanned Teaming Application –The Cooperative Automation Approach for Assistant Systems Assuming the Human‘s Cognitive State –Project 1: Mixed-Initiative Mission Planning –Project 2: Operator Workload Estimation Agenda
12 Ruben Strenzke 2011 Regarding the Human Operator from Outside Application-specific operator model: (disregarding intermediate goals in hierarchical plans, and top-level goals are known) goals plan action + e p + e a has knows measures estimates chooses
13 Ruben Strenzke 2011 Regarding Human Error and its Source workload goals plan action behavior + e b (e.g. + sweating) effects + e p + e a has knows measures assumes estimates Cognitive state shows measures generates suffers from chooses First project: Mixed-Initiative Planning
14 Ruben Strenzke 2011 Online Plan Recognition/Reasoning by Planning I generate a plan and enter the UAV tasks into the system. I cannot know what the human is planning because the entering of the plan is never explicitly finished, but I treat each planned UAV task as a constraint and I try to solve the mission problem while regarding these additional constraints. If there is no solution, the human must have made a mistake (or problem is unsolvable. If there is a solution, I can check its costs against my own solution and also check if the human forgot to enter any tasks. online possible plan set generation
15 Ruben Strenzke 2011 Mixed-Initiative Mission Planner (MMP) Concept 4. Assistant system takes initiative on basis of assumed human plan (action scheduling, missing plan elements) or reference plan (suboptimal assumed human plan, infeasible system plan) 5. Assistant system urges human to make system plan converge to one of its plans if necessary 1. System plan is entered incrementally partial or complete human plan 2. Assistant System has direct access to system plan 3. Assumed human plan is a possible completion of the system plan, i.e. it shall converge to human plan with every detail added to the system plan
16 Ruben Strenzke 2011 Slave MMP Free MMP Assistant Core Diverse information sources Mission Control There is a new objective! UAV Operator I want UAV1 to... and helo to… Ground vehicle spotted! The MMP is based on the PDDL 2.2 compatible planner „LPG-td“ [Gerevini et al. 2004] user goals, user plan! non-probabilistic, non-hierarchical, CWA
17 Ruben Strenzke 2011 Given and User Goals Example Free Slave
18 Ruben Strenzke 2011 Operator Workload Estimation workload goals plan action behavior + e b (e.g. + sweating) effects + e p + e a has knows measures assumes suffers from estimates Cognitive state shows generates chooses measures Second project: Workload Estimation
19 Ruben Strenzke 2011 Workload Estimation by HMMs [Donath et al. 2010, 2011] search for objectsidentify objectfurther tasks... normal (excl. e a /e b )HMM1HMM3... too high (incl. e a /e b )HMM2HMM4... task workload Video! -Workload can be estimated by regarding operator behavior -behavior differs between tasks and workload levels -Measuring of visual and manual interaction with the system -has been done offline -for one person:
20 Ruben Strenzke 2011 Challenges: -Online Generative Plan Recognition -Having Data-Fragments from Different Process Steps workload goals plan action behavior + e b effects + e p + e a has Cognitive state shows If we have assumptions about his plan, it‘s easier to assume what he is doing! If we have assumptions about his goal(s), it‘s easier to assume what he is planning! generates suffers from chooses knows measures assumes estimates measures
21 Ruben Strenzke 2011 The Obligatory Last Slide
Your consent to our cookies if you continue to use this website.