1. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 1 SOAR Prof. Dr. Andreas Wendemuth Lehrstuhl Kognitive Systeme / Sprachverarbeitung Institut für Elektronik, Signalverarbeitung und Kommunikationstechnik Fakultät für Elektrotechnik und Informationstechnik Otto-von-Guericke Universität Magdeburg http://www.kognitivesysteme.de

2. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 2 SOAR u Soar is used by AI researchers to construct integrated intelligent agents and by cognitive scientist for cognitive modeling. u It can basically be considered in three different ways:

3. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 3 u 1. A theory of cognition. As such it provides the principles behind the implemented Soar system. u 2. A set of principles and constraints on (cognitive) processing. Thus, it provides a (cognitive) architectural framework, within which you can construct cognitive models. In this view it can be considered as an integrated architecture for knowledge-based problem solving, learning and interacting with external environments. u 3. An AI programming language.

4. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 4 Soar incorporates u problem spaces as a single framework for all tasks and subtasks to be solved u production rules as the single representation of permanent knowledge u objects with attributes and values as the single representation of temporary knowledge u automatic subgoaling as the single mechanism for generating goals u and chunking as the single learning mechanism.

5. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 5 Rules (Productions) in SOAR u rules are if --> then u starts with the symbol “sp”, which stands for “Soar production.” u All conditions must match.

6. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 6 Execution of productions u A condition tests for the existence (or absence) of data in working memory. u Rules fire: all of the actions are performed in working mememory. u All rules are global.

7. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 7 Structure of production sp {rule*name (condition) … --> (action) …}

8. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 8 Working memory u Working memory is organized as graph structures in states. u Nodes that have links (non-terminals) are identifiers, terminals are constants. u Links are attributes which point to values. u An object is everything following its initial state, which is augmentation.

9. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 9 Objects: e.g. two blocks on table (s1 ^block b1 ^block b2 ^table t1) (b1 ^color blue ^name A ^ontop b2 ^type block) (b2 ^color yellow ^name B ^ontop t1 ^type block) (t1 ^color gray ^name Table ^type table) [also: table.color gray ]

10. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 10 Rules u Variables,, cf. Values 3, 1. u Commands write, halt. u Example: sp {hello-world (state ^type state) --> (write |Hello World|) (halt)}

11. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 11 Operators and Decisions u Soar must decide (select) which (acceptable) rules fire u The locus of this decision is working memory u Therefore, operators are proposed, selected and applied u This is a continuous cycle u Rules whose prerequisites do not match any more are retracted (exception: operators)

12. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 12 Proposing u A rule proposes an operator by creating an acceptable preference (+) for the operator. u Example sp {propose*hello-world (state ^type state) --> ( ^operator +) ( ^name hello-world)} u is an action variable which will be stored in working memory and examined later for application

13. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 13 Selecting u Since proposing and applying are kept separate, selection processes are possible u All acceptable proposed rules with matching conditions are always candidates for selection u If no other selection procedure is programmed, all these rules fire in parallel [in contrast to other programming languages]

14. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 14 Order of selection u Random order of multiple matching rules is given by „indifferent“ preference („= “ ) in proposal, avoiding an impasse u Example: sp {water-jug*propose*fill (state ^name water-jug ^jug ) --> ( ^operator + ^operator =)....

15. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 15 Applying (firing)  Example: sp {apply*hello-world (state ^operator ) ( ^name hello-world) --> (write |Hello World|) (halt)}  This rule finds all variables in working memory with matching conditions and fires all of them

16. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 16 Persistence in Working Memory- Truth Maintenance u Operator application creates persistent results in working memory (operator-support) [Operators are commitments of SOAR] u All other rules perform their actions, and retract from working memory if their conditions fail to match (instantiation-support, i.e. work only as long as that instantiation of the rule applies). E.g. proposal, comparison, elaboration.

17. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 17 Operator manipulations (I) u Compare and negate operators by - (negate),,>=, <> (not equal) u Example: sp {water-jug*propose*fill (state ^name water-jug ^jug ) ( -^empty 0) ( ^contents > 0)......

18. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 18 Operator manipulations (II) u Math in parantheses (), with prefix notation –Example 2 +vc: (+ 2 (* )) u Tests in parantheses {} –Example: sp {water-jug*propose*pour (state ^name water-jug ^jug ^jug { <> }) match and not equal to.

19. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 19 Manual removal of working memory elements by „-“ u Example: sp {water-jug*apply*fill (state ^name water-jug ^jug ) ( ^volume ^contents ) --> ( ^contents ) ( ^contents -)} u Replaces the old value with a new one and removes the old value from memory

20. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 20 Remembering retracted operators u Augment the current state by last-op u Example: sp {water-jug*apply*operator*record*last-operator*pour (state ^name water-jug ^operator ) ( ^name pour ^fill-jug ^empty-jug ) --> ( ^last-operator ) ( ^name pour ^fill-jug ^empty-jug )}

21. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 21 Suppress operator action by „<“ u Example: (if just emptied, do not fill) sp {water-jug*select*operator*avoid*inverse*fill (state ^name water-jug ^operator ^last-operator ) ( ^name fill ^fill-jug ) ( ^name empty ^empty-jug ) --> ( ^operator <)}

22. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 22 Removing operators u E.g. if last-op does not (-) have the same name and fill-jug, remove it sp {water-jug*apply*operator*remove*last-operator*pour (state ^name water-jug ^operator ^last- operator ) ( ^name pour ^fill-jug ^empty-jug ) - ( ^name pour ^fill-jug ^empty-jug ) --> ( ^last-operator -)}

23. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 23 Detecting Success and Halting u Match condition, write and halt. u Example: sp {water-jug*detect*goal*achieved (state ^name water-jug ^jug ) ( ^volume 3 ^contents 1) --> (write (crlf) |The problem has been solved.|) (halt)}

24. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 24 Operator Impasses - Types u Operator no-change impasse: an operator has been selected and there is no new decision to be made. u State no-change impasse: no operators are proposed for the current state. u Operator tie impasse: multiple operators proposed, but insufficient preferences to select between them. u Operator conflict impasse: multiple operators proposed, and the preferences conflict.

25. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 25 Impasse resolution (I) – learning in case of operator tie u Multiple operators are applicable in higher goal context = operator tie u Return to the pre-impasse environment (lower goal context) u Gather features in pre-impasse environment u Augment pre-impasse rules by those features u Look in memory for rules (partially) using these features u In those memorized rules, find additional conditions and preferred actions based on those features u Augment the lower goal context by these goal-relevant conditions and preferred actions

26. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 26 Example of augmenting u Options to get to the station: walk or take bus, are unresolved (operator tie) u In pre-impasse rules, a feature „available time“ was used u In memory, a situation was found where the goal was to get to the airport. Then, a condition was „available time < 30min“, and if true, a bus must be taken to catch the train. u Augment pre-impasse environment by condition „available time“ and corresponding actions. u If unavailable, get that information. Decide again.

27. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 27 Impasse resolution II - chunking u Look at our previous example of learning. u Divide the goal process (get to station) into subgoals: (assess available time) or (determine convenient travel mode) = chunking u Make these chunks new individual rules = deductive (bottom-up) learning u Use these rules for other goals, e.g. for (visit grandma) or (spend evening with Monica)

28. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 28 Impasse resolution (III) – alternative operators u If no (partially) matching previous memory examples exist: u Go to pre-impasse environment and gather all conditions and corresponding rules. u Follow alternative rules allowed (but e.g. not preferred) by these conditions

29. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 29 Example for alternative operators u Impasse: (if get to station -> walk or take bus ?) u Pre-impasse conditions & rules,in reverse order: (if get to work -> get to station) (if 08:00am -> get to work) u Other (non-preferred) rules on same conditions: (if get to work -> take car) (if 08:00am -> continue sleeping) u Use 1st matching of these rules in reverse order

30. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 30 Impasse Resolution IV in SOAR u Learning by abduction [„Seitliches Wegführen“] (Johnson, 1994) = Einbettung derselben Regel in anderen Kontext u Learning by instruction [„Unterweisung“] (Huffman, 1993) nach Anfrage von SOAR an Lehrer, oder nach Entscheidung einer „Aufsichtsperson“

31. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 31 Water Jug Problem u You are given two empty jugs. One holds five gallons of water and the other holds three gallons. u There is a well that has unlimited water that you can use to completely fill the jugs. u You can also empty a jug or pour water from one jug to another. There are no marks for intermediate levels on the jugs. u The goal is to fill the three-gallon jug with one gallon of water.

32. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 32 Solution Space u Define states (gallons in jug1, gallons in jug2) u Nr. of gallons can only be 0,1,2,3 (3 gallon jug) or 0,1,2,3,4,5 (5 gallon jug) -> 24 states u 4 possibilities of action: water 1->2, water 2->1, empty, fill. u All possibilities of actions on all states gives finite and complete diagram u If state (.,1) is in this diagram: problem solved

33. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 33

34. Andreas Wendemuth, Otto-von-Guericke-Universität Magdeburg, 18.01.2009 34..... M E R C I !!