April 28, 2004John C. Giordano – Masters Project Presentation1 Exploring the Constraints of Human Behavior Representation A Masters Project Presentation John C. Giordano Prof. Paul Reynolds - Advisor

April 28, 2004John C. Giordano – Masters Project Presentation2 Outline Introduction & problem statement Key terms Highlights of literature review Findings Proposed framework for considering human behavior representation (HBR) capabilities Conclusions

April 28, 2004John C. Giordano – Masters Project Presentation3 “We can only see a short distance ahead, but we can see plenty there that needs to be done.” Alan Turing Computing Machinery and Intelligence In Mind: 236, 1950

April 28, 2004John C. Giordano – Masters Project Presentation4 Introduction In 1950, Alan Turing proposes the Imitation Game Machines competing with or replacing humans Human behavior representation (HBR) refers to the portrayal of humans HBR is not Artificial Intelligence –More constrained –Still a challenge

April 28, 2004John C. Giordano – Masters Project Presentation5 What is the Problem? HBR is critical to many, but has proven elusive Several large-scale development failures with prominent HBR requirements –DoD’s Joint Simulation System (JSIMS) –NASA’s Air Traffic Management (ATM) simulation Shortcomings noted by many in the community

April 28, 2004John C. Giordano – Masters Project Presentation6 How We Attempt to Address It Examined successes and failures in research, design and implementation Describe what is currently attainable and propose what is unachievable Present a framework for assessing HBR capabilities Seeking publication of research conducted to date

April 28, 2004John C. Giordano – Masters Project Presentation7 Key Terms HBR: a computer-based model that mimics either the behavior of a single human or the collective action of a team of humans Intelligent Software Agent: an artificial agent that operates in a software environment and imitates human intelligence by mechanical means in pursuit of the goals of its clients Human Cognition: the process of receiving, processing, storing, and using information in humans

April 28, 2004John C. Giordano – Masters Project Presentation8 Literature Review Over 60 publications (papers, journal articles, texts, tech reports, requirements documents) Extended annotated bibliography Thorough, but not fully exhaustive

April 28, 2004John C. Giordano – Masters Project Presentation9 Literature Review Modeling Human and Organizational Behavior. Richard W. Pew and Anne S. Mavor (eds.). National Academy Press, Washington DC, Techniques for Modeling Human Performance in Synthetic Environments: A Supplemental Review. Frank E. Ritter, et al. Human Systems Information Analysis Center, Wright Patterson AFB, OH, A Taxonomy of Human Behavior Representation Requirements. Scott Y. Harmon. 11 th Conference on Computer Generated Forces and Behavior Representation, 2002.

April 28, 2004John C. Giordano – Masters Project Presentation10 Harmon’s Taxonomy Human Representation Non-Cognitive Factors Cognitive Capabilities Application Functions

April 28, 2004John C. Giordano – Masters Project Presentation11 Harmon’s Taxonomy

April 28, 2004John C. Giordano – Masters Project Presentation12 Harmon’s Taxonomy

April 28, 2004John C. Giordano – Masters Project Presentation13 Findings

April 28, 2004John C. Giordano – Masters Project Presentation14 Findings The tools used to model and simulate HBR are constrained The phenomena associated with HBR are highly complex At times, HBR requirements vastly exceed capabilities Capabilities and constraints should be clearly articulated to the community Some capabilities may (only) be attained with the emergence of a disruptive technology

April 28, 2004John C. Giordano – Masters Project Presentation15 Some Tools –Soar: general cognitive architecture for intelligent agents –COGNET/iGEN: emulator for human decision-making and problem-solving –ACT-R: architecture for human cognition

April 28, 2004John C. Giordano – Masters Project Presentation16 Three Categories of HBR Capabilities MatureDeveloping Unachievable in practice Constrained speech recognition, parsing and generation X Course of Action (COA) analysis, selection and implementation X Rudimentary emotions X Human physiological characteristics X Semi-automated coarse-grained behavior generation X Probabilistic human performance simulation and prediction X Autonomous, convincing group behavior X COA generation X Interdependence between physiology, emotion and cognition X Behavior adaptation appropriate to dynamic scenarios XX Speech generation w/ appropriate prosody XX Pattern recognition coupled w/ appropriate decision-making XX Generalized behavior prediction XX A single framework for modeling human behavior at multiple levels of resolution XX Complex cognition, reasoning and learning X Conversational dialogue X Synthesis of autonomous knowledge acquisition, planning and behavior X Complete integration between emotion, cognition and behavior X

April 28, 2004John C. Giordano – Masters Project Presentation17 Constrained speech recognition, parsing and generation Course of Action (COA) analysis, selection and implementation Rudimentary emotions Human physiological characteristics Semi-automated coarse-grained behavior generation Probabilistic human performance simulation and prediction Mature Capabilities

April 28, 2004John C. Giordano – Masters Project Presentation18  Autonomous, convincing group behavior  COA generation  Interdependence between physiology, emotion and cognition Developing Capabilities

April 28, 2004John C. Giordano – Masters Project Presentation19  Behavior adaptation appropriate to dynamic scenarios  Speech generation w/ appropriate prosody  Pattern recognition coupled w/ appropriate decision-making  Generalized behavior prediction  A single framework for modeling human behavior at multiple levels of resolution Developing and Unachievable

April 28, 2004John C. Giordano – Masters Project Presentation20 x Complex cognition, reasoning and learning x Conversational dialogue x Synthesis of autonomous knowledge acquisition, planning and behavior x Complete integration between emotion, cognition and behavior Unachievable in Practice

April 28, 2004John C. Giordano – Masters Project Presentation21 A Generational Framework for Considering HBR Capabilities Analogous to the generations of programming languages Generations –1 st : speech recognition, rudimentary emotions/physiology, probabilistic performance –2 nd : domain-independent speech, COA generation, adaptive behaviors –3 rd : single cognitive framework, architecture for multi-resolution behavior modeling, etc. –4 th : approaching human faculties

April 28, 2004John C. Giordano – Masters Project Presentation22 A Generational Framework for Considering HBR Capabilities General Modeled Phenomenon Measurable Not Measurable Model Specificity Speech recognitionIntegrated emotion and cognition Conversational dialogue Autonomous learning and planning Concrete

April 28, 2004John C. Giordano – Masters Project Presentation23 Conclusions Turing foresaw human-machine competition – HBR comprises portrayal Requirements development needs improvement Opportunities for continued research

April 28, 2004John C. Giordano – Masters Project Presentation24 Future Work Update and extend Pew and Mavor, Ritter et al. Focus on HBR successes, particularly with promise of generalization Rigorous, formalized HBR requirements

April 28, 2004John C. Giordano – Masters Project Presentation25 Questions? “It takes some philosophical discipline, in short, to resist specious blurrings of differences between simulations and the phenomena they simulate.” Larry Crockett In The Turing Test and the Frame Problem: AI’s Mistaken Understanding of Intelligence Intellect Books, 1994