1. A H IERARCHICAL G OAL -B ASED F ORMALISM AND A LGORITHM FOR S INGLE -A GENT P LANNING AAMAS ‘12 Utku Şirin 1560838

2. O UTLINE Planning and Domain Models Hierarchical Goal Network (HGN) Planner formalism and proof of its capabilities An algorithm for HGN planning, Goal Decomposition Planner (GDP) Experimental Results Comments and Conclusions

3. A UTOMATED P LANNING What is automated planning? There is goal and current situation, aim is to achieve the goal by executing possible actions Current situation is defined by states Repeatedly; execute an executable action, apply the changes to the state and check whether the goal is satisfied How to do these automatically, fast, and in less number of steps? Important ability for computurized agents Robotic Agents Game-Playing Agents Web-service Agents etc...

4. D OMAIN M ODELS Planner should have a domain model defining the states, actions and the relation between the actions and states How to build domain models? Hand-crafted planner module Huge development effort ! Domain-configurable planner Utilization of a domain-model file Most of the uses are Hierarchical Task Network (HTN) Planning There are methods dividing tasks into subtasks (will be analyzed deeper) Does not focus on goals, but tasks Just apply the tasks until there is no remaining tasks Easier with respect to hand-crafted planning module Problem of lacking of task and goal correspondence makes it hard to translate classical planning domains into HTN domains, hence to prove soundness Can we do better ? Hierarchical Goal Network (HGN) Planner Similar to HTN formalism, but easier to develop domain models More flexible Integrates domain-independent heuristics Decomposes goals rather than tasks Provably HGN has same expressivity power as HTN, is sound and complete

5. F ORMALISM Classical Planning Domain D is a finite-state transition system S is a set of states, each state is a finite set of ground atoms Ex: onTable(block1), on(block2,block1) G is the specification of the goal state comprised of a set of ground atoms O is a set of operators which is a triple (head(o), pre(o), eff(o)) Each action is a ground instance of an operator An action a is executable in a state s if s╞ pre(a) (s entails pre(a)) Meaning that s satisfies the preconditions of action a After execution an action a, the new state s’ is s’ = (s - eff - (a)) ∪ eff + (a) A plan  = is executable in s if each action a i is executable in the state produced by a i-1. A solution to a classical planning problem P = (D, s 0, g) is , if δ( s 0, ) ╞ g, where D is the domain, s 0 is the initial state and g is a goal definition

6. H IERARCHICAL G OAL N ETWORK (HGN) P LANNING Similar to classical planning but have methods additionally A HGN method m is a quadruple (head(m), pre(m), sub(m), post(m)) head(m) and pre(m) same as the ones in operators for classical planning sub(m) list of goals where each g i is a goal formula post(m) = g k ;if sub(m) is non-empty post(m) = pre(m) ; otherwise Relevance: An action a or a method m is relevant for a goal formula g if eff(a) or post(m) entails at least one literal in g. Provides smaller search space than a classical planner A HGN domain is D’ = (D,M) where D is a classical planning domain and M is the set of methods

7. P ROOFS HGN planning is sound and complete. These are proved by mapping HGN planning problem to classical planning problem Soundness: HGN planning domain is D = (D’,M), where D’ is a classical planning domain Every action executable in D is also executable in D’ Hence, every solution to problem P = (D, s 0, g) is also a solution to P = (D’, s 0, g) Hence, HGN planning is sound. Completeness For a path x in classical domain D, there can be constructed a method m that specifies each state in x as a sub-goal in its sub(m). Then a single action will achieve each subgoal completing the path Hence for each classical planning problem P = (D, s 0, g), there is a HGN planning problem P’ = ((D, M), s 0, g) where P and P’ have same set of solution

8. P ROOFS HGN formalism expressivity power is equal to HTN formalism From HGN formalism construct HTN formalism Map subgoals to subtasks with same preconditions mapped directly to In HTN, however, it is needed to define primitive tasks as well. So, define a new primitive task for each t gi having same precondition as g i and no subtasks (that’s why it is primitive, indeed). From HTN formalism construct HGN formalism Map subtasks to subgoals with same preconditions mapped directly to

9. A LITTLE B IT HTN Associate methods with networks Critics for different types of network

10. A LGORITHM, G OAL DECOMPOSITION PLANNING (GDP)

11. GDP IS SOUND AND COMPLETE Soundness, if GDP returns a plan, it is a solution indeed. Induction on length of the solution n For n = 0, it means s 0 ╞ g If  is a solution of length k < n returned by GDP Then  ’ of length k+1 returned by GDP is also a solution as line 11 appends a relevant action/method u to the plan Completeness, if there is a solution, then GDP will return it Induction on length of the solution n For n = 0, GDP will return it as s 0 ╞ g Assume there is a solution of length k and GDP returns it Then GDP returns solutions of length k+1 as at line 11 GDP appends relevant action/method

12. D OMAIN - INDEPENDENT HEURISTICS One of the most important contribution of HGN planning formalism Line 9-13 was choosing action/methods nondeterministically, however, it can be chosen based on a heruistic value So, line 9-13 will be replaced as below:

13. D OMAIN - INDEPENDENT HEURISTICS How to calculate heruistic value for each action/method: First propositional level in which p appears in Plannig Graph States-Levels Action-Levels PLANNING GRAPH

14. E XPERIMENTS An HTN planner SHOP2, a classical planner FF and the HGN planner GDP are compared in five different domains DOMAINs: Logistics Transportation Domain: There several cities. At each city there are several post-offices Aim is to move specified number of packages to different cities Intracity transportation is done via trucks Intercity transportation is done via airplanes Trucks and airplanes are unlimited Blocks-World: There are n-many blocks in a specified configuration Convert the initial configuration to goal configuration by obeying the following rules: Move one block at a time A block may be put on another block or table Depots: Combination of Logistics and Blocks-World domain Trucks have hoist just like the arms of robots in Blocks-World domain Stacking the crates becomes Blocks-World domain Towers of Hanoi: There are three sticks in which several disks are places on it Disks are put in such a way that each disk is smaller than the disk that it is put on it Move disks from one stick to other by obeying the following rules Move one disk at a time No disk may be put onto a smaller disk 3-City Routing The only newly written domain, hence it is a weak domain model There are 3 cities Each city has several locations and locations are connected with roads arbitrarily in the cities There is one random road connected city1 to city3 and one random road connected city2 to city3 Aim is to go from city1 or city3 to city2

15. R ESULTS Logistics Domain Results For n = 15, 20, …, 60 packages GDP-h does not bring much overhead for heuristic function calculation FF has strong heuristics

16. R ESULTS Blocks-World Domain Results For n = 10, 20, …, 100 blocks FF has known problems with Blocks-World GDP-h has heuristic value calcuation time overhead GDP-h results in a bit smaller plans

17. R ESULTS The Depots Domain Results For n = 10, 20, …, 80 crates FF cannot solve more than 24 crates GDP-h heuristic overhead is significant, also have almost same plans with the other planners

18. R ESULTS Towers of Hanoi Domain Results For n = 3, …, 14 rings SHOP2 could not solve problems for n > 12 and GDP and GDP-h cound not solve problems for n > 14 Both is due to stack overflow, hence thought as implementation issue, FF did not use a stack FF has very bad planning results while the others have almost optimal path results

19. R ESULTS 3-City Routing Domain Results All previous domains are strong and very well defined domains This one is constructed as a weak domain model having only one method for HGN and three corresponding methods for HTN For n = 10, 20, …, 100 cities GDP and SHOP2 could not solve except for n = 10 FF may solve the problems up to n = 60, after that point it even could not parse the problem file GDP-h solved all problems quickly and nearly optimal The reason for the success of the GDP-h is the guided search thanks to the heuristics As the model is weak, the other planners do not have enough information to constraint the search space and do a lot of backtrackings, however, GDP-h uses heuristic to be able to guide its search and narrow its search space As a conclusion, we can say that if there is a strong domain model, heuristic calculation most probably will result in a overhead and give not significantly better result; however, if the model is weak than contribution of heuristic function is crucial

20. R ESULTS Domain Authoring Subjective to developers Measures as number of lisp symbols and compared for GDP and SHOP2 planners GDP almost always have less number of symbols HTN specifies more than one task to achieve a goal formula. It defines a decomposition task, several primitive tasks and deletion-check conditions, while, GDP only needs to speficify those as goals and let the planner choose the appropriate action to do with respect to the goal. There is a need for different base cases for each method in HTN, however, GDP does not need such bases cases as the semantic of a goal provides to do nothing if a goal is true.

21. C OMMENTS AND C ONCLUSIONS No cross-domain explanations for the experiments. For example, why FF is unsuccessfull is not answered. Just the results are shown and it is said that GDP is capable enough the others, even produces better results for weak domain models. Almost everything is compared with HTN but HTN is not explained, at least in principle. Moreover, main difference is not shown algorithmically. What was doing HTN and now what is the thing that HGN is not doing, thereby resulting better. For example, can we use heuristics in SHOP2 planner. I guess we can, and if we can, it may also produce similar results. HGN is more intutive when comparing both, hence, seems good contribution to the literature (since 1974). However, HTN is being used many many years, hence more comprehensive comparison is expected So the only contribution of HGN is the easy development domain models, which is even a subjective criteria.

22. R EFERENCES V. Shivashankar, U. Kuter, D. S. Nau, and R. Alford. A hierarchical goal-based formalism and algorithm for single-agent planning. In Eleventh Internat. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), 2012 Kutluhan Erol, James A. Hendler, and Dana S. Nau. UMCP: A sound and complete procedure for hierarchical task-network planning. In Proceedings of the International Conference on AI Planning & Scheduling (AIPS), pages 249–254, 1994. J. Hoffmann and B. Nebel. The FF planning system. JAIR, 14:253–302, 2001. D. S. Nau, T.-C. Au, O. Ilghami, U. Kuter, J. W. Murdock, D. Wu, and F. Yaman. SHOP2: An HTN planning system. JAIR, 20:379–404, Dec. 2003. M. M. Veloso. Learning by analogical reasoning in general problem solving. PhD thesis CMU-CS-92-174, Carnegie Mellon University, 1992. F. Bacchus. The AIPS ’00 planning competition. AI Mag., 22(1):47–56, 2001. M. Fox and D. Long. International planning competition, 2002. http://planning.cis.strath.ac.uk/competition. http://planning.cis.strath.ac.uk/competition Hui Li. Technical Report: Relaxed Plan Graph Heuristic Cost Estimation. 2006.

23. A NY C OMMENTS OR Q UESTIONS ?