1. Integrating Object-Oriented Technology with Cognitive Modeling Pamela E. Scott-Johnson, Ph.D. Department of Psychology, College of Liberal Arts Bheem Kattel, Ph.D., CPE Industrial, Manufacturing, & Information Engineering, School of Engineering LeeRoy Bronner, Ph.D., P.E. Industrial, Manufacturing, & Information Engineering, School of Engineering

2. Cognitive Models Anderson & Byrne o Carnegie Melon University o Currently used by HRED (Army) Overview of ACT-R Experiments o Face / Pattern Recognitions o Terrains o Eye Tracking Object Orienting

3. ACT-R A cognitive architecture: a theory about how human cognition works. Looks like a programming language. o Its constructs reflect assumptions about human cognition. These assumptions are based on numerous facts derived from psychology experiments. These assumptions can be tested by comparing the results of the model with the results of people doing the same tasks. By "results" we mean the traditional measures of cognitive psychology: o time to perform the task, o accuracy in the task, and, o (more recently) neurological data such as those obtained from FMRI.

4. Motivations for a Cognitive Architecture Philosophy: Provide a unified understanding of the mind. Psychology: Account for experimental data. Education: Provide cognitive models for intelligent tutoring systems and other learning environments. Human Computer Interaction: Evaluate artifacts and help in their design. Computer Generated Forces: Provide cognitive agents to inhabit training environments and games. Neuroscience: Provide a framework for interpreting data from brain imaging. Anderson & Byrne Tutorial

5. Approach: Integrated Cognitive Models Cognitive model = computational process that thinks/acts like a person Integrated cognitive models… User Model Driver Model

6. These Goals for Cognitive Architectures Require Integration, not just of different aspects of higher level cognition but of cognition, perception, and action. Systems that run in real time. Robust behavior in the face of error, the unexpected, and the unknown. Parameter-free predictions of behavior. Learning.

7. ACT-R is a production system theory that tries to explain human. Cognition through a model of knowledge structures. Two types of knowledge declarative and procedural. Declarative knowledge. Corresponds to things we know and can describe. Represented by structures call chunks. Chunk defined by a: type and slots ( ES –frame). Example: dog chased the cat – type: chase, agent: dog, object: cat. Procedural knowledge. Knowledge displayed by our behavior but we are not conscious of. No one can describe the rules by which we speak a language, but we do. Defined by production rules: Example: IF the goal is to classify a person and he is unmarried. Then classify him as a bachelor (action part). Chunks and productions are the basic building blocks of ACT-R. Atomic Components of Thought (ACT-R)

8. ETA Embodied Cognition Task Artefact Triad Embodied Cognition Interactive Behavior TaskArtefact Figure 1. The embodied cognition – task-artefact triad. Adapted from Byrne 2001 article. The user’s ability to interact with an interface is dependent upon the properties of the 1.The embodied cognition or cognitive, perceptual, and motor capacities of the user, 2.The task(s) that the user is engaged in, 3.The artefact or “device” that the user manipulates or utilizes in order to perform the task(s).

9. ACT-R/PM system diagram described by Byrne (2001). There are 4 perceptual-motor modules which communicate with the central components of cognition. Central cognition and each module are serial and use spreading activation process which work in parallel.

10. ACT-R 5.0 Environment Productions (Basal Ganglia) Retrieval Buffer (VLPFC) Matching (Striatum) Selection (Pallidum) Execution (Thalamus) Goal Buffer (DLPFC) Visual Buffer (Parietal) Manual Buffer (Motor) Manual Module (Motor/Cerebellum) Visual Module (Occipital/etc) Intentional Module (not identified) Declarative Module (Temporal/Hippocampus)

11. Face and Pattern Recognition

12. Face Recognition Purpose o Persons use particular characteristics or features of the face in order to determine if the face is familiar. The purpose of our study will be to determine which feature of the face (e.g. nose, eyes, forehead, mouth…) will allow participants to increase accuracy and reaction time in identifying familiar faces. o We will also determine how the length of time for which the stimulus (face) is view increases accuracy.

16. Signal Detection Theory (SDT) Defined as a mathematically based theory of signal (stimulus) detection Assumptions o The observer is not a passive receiver of stimulus active decision- maker who makes difficult perceptual judgment under conditions of uncertainty o The observer is an active decision- maker who makes difficult perceptual judgment under conditions of uncertainty

17. SDT Outcome Matrix Signal Response YesNo PresentHitMiss AbsentFalse AlarmCorrect negative When the signal is “present” and the response is “yes,” the observer has made a hit. If the observer responded “yes” when that signal was absent, then a false alarm has been made. missescorrect negatives The other cells are called misses and correct negatives for obvious reasons.

18. Industrial Engineering Research

19. Terrain Hazard Detection Objective o To collect data to study if providing depth in the field of vision would enhance the terrain hazard detection capability. Method and Procedure o Terrain scenes with and without hazards were video taped simultaneously by four video cameras mounted horizontally in line at the same level. The distances of center of lenses of the video cameras from the center of lens of the left most video camera (reference) were in multiples of 2.5 inches. This was done to give binocular parallax of 1X, 2X, and 3X.

20. Terrain Hazard Detection A computer with DPS Velocity software was used to prepare sets of images representing different viewing conditions. Images under different viewing conditions were projected by two DPL projectors on a special screen. A braking mechanism was designed in-house to stop the video stream as soon as the hazard was detected and also when the hazard was verified by the subjects. The response times were recorded from the time code on the monitor for different viewing conditions. The data collected were submitted to Army Research Laboratory, Aberdeen for further analysis.

21. Eye Tracking This is an ongoing experiment and will be integrated into OO technology. Objective o To study how much time eyes are focused on a given target with and without distractions. Method and Procedure o Video images of terrains with and without hazard will be presented separately on a video monitor. The eye tracking camera will track the movement of the eyes and record them in a file. Area of interest can be selected on the image to find how much time the eyes dwelled on the area. Thus by assigning a target as an area of interest the time eye dwelled on the target can be monitored.

22. Eye Tracking Experimental setup The following are the components of eye tracking device o Controller o Eye monitor o Pan/Tilt Optics o Scene monitor o Interface PC and monitor o PC with monitor for viewing by subject o Scan Converter o Artificial eye

23. Integrating Object-Oriented Technology with Cognitive Modeling

26. ACT-R is a production system theory that tries to explain human. Cognition through a model of knowledge structures. Two types of knowledge declarative and procedural. Declarative knowledge. Corresponds to things we know and can describe. Represented by structures call chunks. Chunk defined by a: type and slots ( ES –frame). Example: dog chased the cat – type: chase, agent: dog, object: cat. Procedural knowledge. Knowledge displayed by our behavior but we are not conscious of. No one can describe the rules by which we speak a language, but we do. Defined by production rules: Example: IF the goal is to classify a person and he is unmarried. Then classify him as a bachelor (action part). Chunks and productions are the basic building blocks of ACT-R. Atomic Components of Thought (ACT-R)

33. ACT-R Demonstration