2. SAGE ARIVA ARIVA PI Conference, November 2005 UNCLASSIFIED//FOR OFFICIAL USE ONLY

3. 2 SAGE Team Roles  Manage project/sustain vision  Intel and human factors expertise  Cognitive Science research  Build VizArch, simBorgs  Build Slate  HII Research –Network data –Image management  Novel Gaming Interfaces –CMU/Dynamix ARGUS  Cognitive Systems Engineering –Nutech ABEM

4. Selmer Bringsjord Andrew Shilliday, Joshua Taylor Konstantine Arkoudas Sangeet Khemlani, Eric Pratt, Gabe Mulley, Bettina Schimanski Rensselaer AI & Reasoning (RAIR) Laboratory Department of Cognitive Science Department of Computer Science Rensselaer Polytechnic Institute (RPI) Troy NY 12180 US 11.30.05 The Slate System: Four New Developments SAGE (Not based on ppt source.)

5. Original Concept

6. Current Release: v2.0

7. Reflects new theories of hypothesis generation E.g., MMOI-based hypo gen E.g., scenario generaion via MDF Automatic (v1) report generation Includes a proved-to-be sound system, S, for assembling proofs, arguments, composite arguments, meta-arguments — and these arguments can be automatically assessed Facilitates not just deduction, but abduction, induction, and model-based reasoning Seamless integration with the world’s best automated and interactive machine reasoning systems and model finders (Vampire, Paradox, Athena, etc.) IKL-compliant (empirically confirmed) Some Innovative Features

8. http://www.cogsci.rpi.edu/research/rair SAGE

9. http://www.cogsci.rpi.edu/research/rair/projects.php SAGE

12. All of CS #4 Translated into Common Logic So, we’re making progress, and will have this “mega” round trip working @ Nov/Dec 05 PI meeting SAGE

14. The Visualization “Hookup” Challenge We are swimming in an expanding sea of systems for exploring and visualizing data in the IA domain. We need interoperability here as well! SAGE

16. Challenge Reduced to Software Engineering: The Vivid family, mathematically, up to the challenge of visual data integration and interoperability. SAGE

17. The Uncertainty Challenge can be met; we have the formalism; simple example SAGE

20. 77% 82% 79% SAGE

21. Argument against an argument SAGE

22. Novel Methods of Human-Computer Interaction Network Data Visualization Catherine Plaisant, Benjamin B. Bederson Bongshin Lee, Hyunmo Kang Human-Computer Interaction Lab University of Maryland SAGE

23. Two applications for Network Data Visualization TreePlus Node-link graph visualization Tree-centered solution Improve incremental navigation NetLens Interactive coordinated overviews Dynamic filtering Bipartite graphs

24. SAGE Challenge of Network Data Visualization

25. SAGE TreePlus Project Goals Develop a good tree-centered solution for iterative inspection of large networks and in cooperation with the team of Wayne Gray Determine for which tasks and data sets different solutions works better Better understand how people use visualization tools to explore graphs Rensselaer ++ readability ++

26. SAGE TreePlus Extraction of spanning tree “Plant a seed and watch it grow” Dynamic root selection Incremental Exploration + Integrated Search Pan/Zoom + Animation Interaction techniques Highlight and preview of adjacent nodes Hints of graph structure Animated transitions

27. SAGE TreePlus Written in C# with Piccolo.NET Bars give a preview of how fruitful it would be to go down a path When cursor hovers over a node, a preview of all nodes connected to that node appear on the right

28. SAGE GraphPlus TreePlus Controlled experiment

29. SAGE Promising Results 28 subjects Compared TreePlus to GraphPlus Controlled interface “density” Study in collaboration with Wayne Gray - RPI

30. SAGE Two applications for Network Data Visualization TreePlus Node-link graph visualization Tree-centered solution Improve incremental navigation NetLens Interactive coordinated overviews Dynamic filtering Interaction history Bipartite graphs

31. SAGE NetLens – Papers (ACM DL) & Authors Papers on the left Authors on the right Overviews provided for all attributes (here for number of papers per year) Filtered to show only papers related to visua lization, and the people who wrote those papers are shown on the right side, aggregat ed by institution type.

32. SAGE NetLens –Email (Enron) & People Emails on the left People on the right Overviews provided for all attributes (here for emotional tone on emails side) Filtered to show only emails related to CA energy crisis; and the people who sent them are shown on the right side.

33. SAGE Rensselaer Cognitive Science Complex Visualizations of Massive Data: The Impact of Design on Mind ARDA ARIVA Workshop Orlando, Florida 30 November 2005

34. SAGE Rensselaer Cognitive Science Intelligently Designed Interfaces for Intelligence Analysts NIMD will bring a superhighway of information technology to the Intelligence Analyst But, by themselves the NIMD technologies will not bridge the crucial human-computer-information gap All of these technologies REQUIRE a user interface for the IA

35. SAGE Rensselaer Cognitive Science Interface = Visualization + Interaction Is this a good interface?

36. SAGE Rensselaer Cognitive Science Interface = Visualization + Interaction Is this?

37. SAGE Rensselaer Cognitive Science Interface = Visualization + Interaction How about this?

38. SAGE Rensselaer Cognitive Science Interface = Visualization + Interaction Or this?

39. SAGE Rensselaer Cognitive Science Design Mind Tasks Performanc e Understanding the Impact of Design on Mind A good interface is one that optimizes performance on a task that you care about Understanding what makes an interface good or bad requires understanding the constraints and affordances in the interactions between Mind, Design, and Tasks

40. SAGE Rensselaer Cognitive Science Cognitive Metric Profile -- showing dynamic changes in task demands on human cognitive, perceptual, and action resources Our Goals: Forecasting Dynamic Changes in Cognitive Workload Create a new generation of tools inspired by cognitive science theory that will enable us to predict the changing demands on human cognition, perception, and action imposed by an interface during task performance Apply these tools to aid the evaluation and design of the next generation of visualization and interaction techniques for software intended for use by Intelligence Analysts

41. SAGE Rensselaer Cognitive Science Our Approach 1 st – Play well with others (VIA) –Develop tools and techniques so that our data collection and modeling systems can interact with tools built by others 2 nd – Simulated Human Visualizers (simBorgs) –Develop generation of cognitive theory based models of human visualization and interaction 3 rd – Cognitive Metrics Profiling –Toolkit for accessing and predicting dynamic changes in demands that an interface makes on human cognition, perception, and action as the human user performs a typical task Human data – eye fixations, scanpaths, mouseclicks, performance time simBorg data – predictions of changes in internal use of memory, attention, and perception

42. SAGE Rensselaer Cognitive Science Current research focuses on three nested technologies CogWorks VIA: Visualization-Interaction Architecture

43. SAGE Rensselaer Cognitive Science VIA VIA release 0.0 used at UMd in August/September to evaluate an innovative design for visual display of abstract data VIA release 0.9 used to interact with a random application downloaded from the internet –required addition of two lines of code

44. SAGE Rensselaer Cognitive Science Demos VIA in action -- demo of ease of instrumenting C# applications to run with VIA Playback of logfile created during UMd session Logfiles analyzed for UMd dissertation Eye data analyzed for UMd dissertation simBorg written that uses VIA to perform tasks on UMd software Information scent implemented in simBorgs in real-time use

45. SAGE Rensselaer Cognitive Science Bridging the Human-Computer- Information Gap by... Engineer interfaces for the next generation tools developed for the IA by creating a new generation of human performance models focused on cognitive issues in usability To tailor the designed environment to best fit the ways in which the IA – thinks – perceives – and acts

46. SAGE Rensselaer Cognitive Science Questions?