Learning HMM-based cognitive load models for supporting human-agent teamwork Xiaocong Fan, Po-Chun Chen, John Yen 소프트컴퓨팅연구실황주원

Overview Introduction HMM-based load models - A human-centered teamwork model - Computational cognitive capacity model - Agent processing load model - HAP’s processing load model Cognitive task design and data collection Learning cognitive load models - Learning procedure - The model space of cognitive load - Properties of ‘Good’ cognitive load models - The number of hidden states 1

Introduction Goal –How shared cognitive structures can enhance human-agent team performance –To develop a computational cognitive capacity model to facilitate the establishment of shared mental models Human-centered teamwork –Establishing situation awareness –Developing shared mental models 2

Introduction Human and autonomous agents –Human are limited by their cognitive capacity in information processing –Autonomous agents can learn expertise problem-solving knowledge Shared mental model –To predict others’ needs and coordinate behaviors –The establishment of shared mental models among human and agent team members –Concept of shared mental models include Role assignment and its dynamics Teamwork schemas and progresses Communication patterns and intentions 3

HMM-based load models –A human-centered teamwork model –Computational cognitive capacity model –Agent processing load model –HAP’s processing load model 4

5 HMM-based load models A human-centered teamwork model Human partner model –Human’s cognitive states (goals, intentions, trust) Processing Model & Communication Model –Dynamically updates models of other HAPs Assumption –An agent do not knows all the information/intentions –Agent’s processing capacity is limited by computing resources

6 HMM-based load models Computational cognitive capacity model Hidden Markov model –A statistical approach to modeling systems that can be viewed as a Markov process with unknown hidden parameters In this study –Cognitive load has a dynamic nature –HMM approach demands that the system being modeled (human’s cognitive capacity) Secondary task performance –Observable signals to estimate the hidden cognitive load state Miller’s 7 ± 2 rule –Observable state range : 0~9 5-state HMM model

Load state based –Resource-bounded agents -> a realistic information processing strategy Schema theory –Multiple elements of information can be chunked as single elements in cognitive schemas. 7 HMM-based load models Agent processing load model

The processing load of a HAP can thus be modeled as the co-effect of the processing load of the agent HMMs for HAP processing load The number of hypothetical hidden states is a critical parameter for modeling both human’s cognitive load and agent’s processing load. 8 HMM-based load models HAP’s processing load model

Cognitive task design and data collection 9 The goal of a team –To share information among members in a timely manner to develop global situation awareness Shared belief map –A table with color-coded info-cells –Row : model of one team member –Column : information type –Concept : development of global situation

10 Learning cognitive load models Learning procedure Subfigure –Top, middle, bottom components –3 log-likelihood log-likelihood in training log-likelihood in testing Standard deviation of log-likelihood in testing –Indicate Maxima of each model space (from 3 to 10) form a 3-layer structure Better trained models lead to better testing log-likelihood Better trained models incur lower deviations.

11 Learning cognitive load models Learning procedure

12 Learning cognitive load models Learning procedure

First –Each model space (from 3 to 10) has a 3-layer structure, which means the log-likelihood maxima are clustered in three levels Second –Better trained models performed better in testing: the trend of the log-likelihoods in fitting is consistent with the trend of the log-likelihoods in training Third –Better models produced lower deviation in testing. –Also, as the number of hidden states increased from 3 to 10, the fraction of models at the middle and bottom levels reduced with the fraction of models at the top level increased. 13 Learning cognitive load models The model space of cognitive load

14 Learning cognitive load models Properties of ‘Good’ cognitive load models ‘Good’ models -> Top-layer An example 5-state HMMTransition probability distributions

15 Learning cognitive load models Properties of ‘Good’ cognitive load models

16 Learning cognitive load models The number of hidden states How many hidden states are appropriate for modeling cognitive load using HMMS?

17 Learning cognitive load models The number of hidden states. Blue : human’s instantaneous cognitive loads. Red : processing loads of a HAP as a whole