1. Authoring and Critiquing COA’s in SHAKEN: A Storyboard showing work in progress UT May 2002

2. Disclaimers SHAKEN cannot be used to author some aspects of COA’s, e.g. –More sophisticated spatial knowledge (e.g. “Attack the enemy’s eastern flank”; “Direct the fires deep within the battlespace”; “The 3 rd Brigade Overtakes the Cavalry”) SHAKEN cannot simulate some aspects of COA’s, e.g. –Non-normative outcomes (e.g. “The attack might fail”)  Other components of the E2E system – such as the Umass Simulator and NuSketch – should contribute to these areas.

3. Our Focus at UT 1.Authoring COA’s: We’re expanding the component library with the Events, Entities, Roles, and relations that are required for describing BS situations and COA’s. We’re anticipating that spatial information will come from NuSketch. 2.Critiquing COA’s: Several sources can contribute here, including KANAL’s check of pre-req’s and expected affects, as well as Umass’s non-normative simulations. We’re planning to contribute pattern recognition methods which are applied to declarative descriptions of COA’s to identify their strengths and weaknesses. Also, we’re planning to contribute to methods for SME’s to author knowledge to improve these critiques. Let’s discuss each of these in turn …

4. 1. Authoring Battlespace Descriptions We’ve identified several areas of expansion for the component library –Goals things an agent intends to accomplish/prevent/etc. has-goal, objective, Goal, Intentional (cliché), Unauthorized (cliché) –Resources things used/consumed/output/etc. by Actions resource, by-product, relationship to participant slots –Views user-defined collections of participants viewpoint, View (Aggregate)

5. Library Expansion (cont.) –Time Event time, Entity “time”, relative time, Temporal Roles start-time, end-time, time-of-existence, duration, rate, frequency, etc. –Space Spatial frames (e.g. “geospace”), Spatial Roles, Granularity Conduit, Corridor, Landmark, Obstacle, Axis, Hide, Cross, Approach, etc. –Communication Information, Carrier, Medium, Express, Interpret, Transmit, etc. Encode, Decode, Language, Learn, Confirm, Monitor, Discover, Deceive, etc. –Aggression Trespass, Invade, Attack, Shoot, Impair, Damage Protect, Defend, Die/Kill, Live/Survive, etc.

6. Library Expansion (cont.) –“Meta-Events” Assist, Maintain, Begin, Stimulate, Stop, Control, Motivate, Oppose, Delay re-, counter-, Reciprocal, Intentional, Unauthorized http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentations/models.ppt `

7. An Example COA in SHAKEN We’ve encoded General Otstott’s COA#1 for Objective Jayhawk. A CMAP for it appears on the next slide. The CMAP is a good start, but it’s incomplete in two ways: –We could not well express some parts of the COA. We’re continuing to expand the component library to add this expressiveness. –We omitted all spatial aspects of the COA, anticipating that this information will come from NuSketch. Nevertheless, this COA is fully coded in SHAKEN (i.e. it’s not a mock-up) and it gives us a useful laboratory for further work.

8. Put the Cmap for the COA here Enemy Ally

9. A Brief Aside: Enabling SME’s to “speak loosely” while authoring knowledge in SHAKEN Loose Speak (LS): the phenomenon that human listeners are able to correctly interpret a speaker's imprecise utterance. Speaking loosely while authoring knowledge requires the correct interpretation of a SME's imprecise utterance. The next couple of slides show a few examples of loose speak in authoring knowledge in shaken. For more detail please visit: http://www.cs.utexas.edu/users/mfkb/RKF/MayWorksh op02/Presentations/Pete-slides.ppthttp://www.cs.utexas.edu/users/mfkb/RKF/MayWorksh op02/Presentations/Pete-slides.ppt

10. Example 1 of LS Given input: Following their attack, attack aviation will refuel and rearm SME encodes the following CMAP: Note: Attack-Aviation is encoded as the object of Refuel and Rearm, but the object of Refuel and Rearm should be Military-Unit

11. Example 1 of LS (Continued) The Loose-Speak mechanism recognizes the inconsistency, and asks: If the user clicks on the “(See CMAP)” link …

12. The Updated Version of CMAP

13. Example 2 of LS Given input: Fire support starts with the attack SME encodes the following CMAP: Note: The temporally-starts slot is incorrectly used to link the two events directly.

14. Example 2 of LS (Continued) The LS mechanism links the temporally-starts slot to the time-intervals of Fire-Support and Attack, and asks the user to “Accept” or “Cancel” the new version.

15. Example 3 of LS Given Input: Holding Attack SME enters the new concept:

16. Example 3 of LS (Continued) The LS mechanism attempts to interpret the new concept: If the user clicks on the “(See CMAP)” link …

17. Example 3 of LS (Continued)

18. 2. UT’s approach to enabling SHAKEN to critique COA’s Knowledge for critiquing takes the form of patterns to match against COA’s. Each pattern has associated conclusions to draw or actions to take when the match is successful. For example, General Otstott’s critique of COA#1 included this observation for fire support: We envision that SHAKEN will generate this observation by matching the patterns on the next two slides with our COA#1 CMAP. COA#1 makes effective use of attack aviation assets by attacking deep to neutralize the enemy artillery early. Attack aviation is available to assist attacking ground elements by the time the attack is joined.

19. An Example of a Pattern Attack Neutralize Artillery-Battery Aviation-Battalion Attack Time-Interval causes object agent before time Enemy plays An attack by air assets that is able to penetrate and neutralize the enemy artillery before the start of the main attack offers good fire support. “Main Attack” element-of object Penetratecauses

20. Another Example of a Pattern Attack Fire-Support Military-Unit Time-Interval Army-Division Time-Interval time supportsagent-of time temporally-starts A military unit that supports an army division when it begins its attack offers good fire support. agent

21. Pattern Matching The critiquing patterns are unlikely to match a COA exactly – flexible matching is required. We’re compiling knowledge to enable flexible matching. Some is sound, some is heuristic. Matching knowledge is of the form X  Y. X might match with the KB and Y with the user's input or vice versa. The next couple of slides show the matching knowledge that we have compiled so far. For more detail please visit –http://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentatio ns/Project-talk-final.ppthttp://www.cs.utexas.edu/users/mfkb/RKF/MayWorkshop02/Presentatio ns/Project-talk-final.ppt

22. Matching Knowledge Some slots are transitive, symmetric, and/or reflexive. For example, has-part is transitive, so Some slots ascend through parts. For example, object ascends through is-part-of, so We have compiled extensive (sound) knowledge of the form: For example,

23. More Matching Knowledge We have also compiled knowledge which encodes complex relationships. For example: Our matching knowledge also includes heuristic rules, such as:

24. Matching COA#1 through transitivity of causes

25. Matching COA#1 (cont.)

26. An Aside: Using the Pattern Matcher to Improve SHAKEN’s Q/A system The Q/A system does not look far for an answer to a query. The knowledge we’re compiling for the flexible matcher can broaden its search. For example, a SME might ask of COA #1 “What attacks did the 23 rd Battalion make?” This is expressed in KM as: (the Attack agent-of of 23 rd ) The current SHAKEN will return nothing, even though an answer exists. SHAKEN just needs to look a little harder…

27. Improving SHAKEN’s Q/A “23rd” Attack agent The flexible matcher can find an answer by applying the matching knowledge we have compiled. “What attacks did the 23 rd Battalion make?” End of Aside. Now back to the main theme of generating a critique…

28. Generating the Report Attack Neutralize Artillery-Battery Aviation-Battalion Attack Time-Interval causes object agent before time Enemy plays “Main Attack” element-of object Penetratecauses Whenever SHAKEN matches a pattern with a COA, it adds an entry to the final report. Associated with each pattern is knowledge about the dimension of COA critiquing (e.g. Simplicity) that the entry contributes to, and the strength of that contribution (these are somehow tallied to score the COA on the {--, -, +, ++} scale). In SHAKEN, this might be: Pattern Feasibility Strength-Value *high pattern contributes-to strength The pattern itself (rooted at “Attack”) might be enclosed in a big node

29. Generating the Entry in the Report Attack Neutralize Artillery-Battery Aviation-Battalion Attack Time-Interval causes object agent before time Enemy plays “Main Attack” element-of object Penetratecauses We could have “canned text” associated with each pattern. A nice alternative would be to have a text template associated with each pattern, then to instantiate the template using the particulars of the COA that got matched. Here’s an example template: Attack by Aviation-Battalion is expected to penetrate and neutralize Artillery-Battery before the start of Attack, thereby offering good fire support. Note: our current text generator would be used to generate phrases for the colored nodes. Those phrases would plug-in for the colored text.

30. Wouldn’t it be nice to have A Tool for Editing Patterns The SME’s main task this summer will be authoring knowledge to critique COA’s. “Basic SHAKEN” (i.e. component library, CMAPs) provides most of the functionality required for SME’s to enter patterns. A “Pattern Editor” would provide a bit more. It would help a SME to design a pattern that successfully matches a COA, then to generalize it.

31. CMAP Pattern SHAKEN proposes correspondences between a CMAP and a Pattern. The SME is allowed to revise the Pattern, generalizing nodes, editing correspondences, etc.