1. Presented by: Robert T. Effinger
4/20/2019
Qualifying Exam Research Presentation
Presented by: Robert T. Effinger
January 25, 2005
May 12, 2004

2. 4/20/2019
“Enabling Fast Flexible Planning through Incremental Temporal Reasoning with Conflict Extraction”
I’shiang Shu, Robert Effinger, Prof. Brian Williams
Model-Based Embedded and Robotic Systems Group
2005 International Conference on Automated Planning and Scheduling
May 12, 2004

3. Presentation Summary Importance of Automated Planning Research
Three Main Types of Automated Planners
A Simple Example
Generative Planner
Path Planner
Task Network Planner
Research Objectives of the Kirk Planner
Fast and Flexible Planning
Key Ideas and Concepts of the Kirk Planner
Simple Temporal Network (STN)
Incremental Update Algorithm (ITC)
Experimental Data
Key Assumptions and Limitations
Conclusions

4. Importance of Automated Planning
Airline Timetables
Assembly Lines
Radio Frequency Planning
Military Convoys
ISS Life Support Rations
MER Rover Planning
NASA’s Deep Space Probes

5. Three Main Types of Automated Planners
Generative Planners
Path Planners
Task Network Planners

6. A Simple Planning Example
Goal: Touchdown
Football Game: 20 10 G
Start Condition: 20 yardline
Library of Actions:
Center (C): - Hike - Block
Reciever (R):
- Run to Endzone
- Run Slant
- Catch Football
QuarterBack (QB)
- Pass to Endzone - Slant Pass - QB Sneak C R QB

7. Generative Planner Football Game: Goal: Touchdown 20 10 G
Start Condition: 20 yardline
Library of Actions:
Center (C): - Hike - Block
Reciever (QB):
- Run to Endzone - Run Slant
- Catch Football
QuarterBack (QB)
- Pass to Endzone - Slant Pass - QB Sneak
Path Planner C R QB
Generative Planning Advantage: Very Expressive
Generative Planning Disadvantage: Often Intractable (exponential growth) Can Generate Infeasible Plans Activity Times are not Flexible
Generate Forward until Goal Condition Found: (Graphplan, FF)
Generate Backward until Start Condition Found: (PoP)
Start Condition:
(R) Run to Endzone
Goal:
20 yardline
(C) Hike
(R) Catch Football
(QB) Pass to Endzone
Touchdown !

8. Task Network Planner Football Game: Goal: Touchdown20 10 G QB C R
Start Condition: 20 yardline
Goal: Touchdown
Library of Actions:
Center (C): - Hike - Block
Reciever (QB):
- Run to Endzone - Run Slant
- Catch Football
QuarterBack (QB)
- Pass to Endzone - Slant Pass - QB Sneak
Football Game:

9. Task Network Planning Goal: Touchdown Start Condition: 20 yardline
Library of Actions:
Center (C): - Hike - Block
Reciever (QB):
- Run to Endzone - Run Slant
- Catch Football
QuarterBack (QB)
- Pass to Endzone - Slant Pass - QB Sneak
Activity Plans: 20 10 G QB C R
Hail Mary
Slant Pass
QB Sneak
20 yardline
Hail Mary Activity Plan:
(C) Hike
(R) Run to Endzone
(QB) Pass to Endzone
(R) Catch Football
Touchdown !

10. Task Network Planning Goal: Touchdown Start Condition: 20 yardline
Library of Actions:
Center (C): - Hike - Block
Reciever (QB):
- Run to Endzone - Run Slant
- Catch Football
QuarterBack (QB)
- Pass to Endzone - Slant Pass - QB Sneak
Activity Plans: 20 10 G QB C R
Hail Mary
Slant Pass 20 10 G QB C R 20 10 G QB C R
QB Sneak
Task Network Advantages: Tractable in real-time Activity times can be flexible
Task Network Disadvantage: Less expressive

11. Research Objectives of the Kirk Planner
4/20/2019
Kirk’s Main Research Objective:
“To Enable Fast and Temporally Flexible Planning”
Translates into two technical research objectives:
1.) Allow flexible instead of fixed execution times on activities
2.) When replanning, try to reuse previous planning computations

12. “To Enable Fast and Temporally Flexible Planning”
Two Key Ideas
4/20/2019
Kirk’s Main Research Objective:
“To Enable Fast and Temporally Flexible Planning”
Translates into two technical research objectives:
1.) Allow flexible instead of fixed execution times on activities
- Kirk supports a lower and upper time-bound on each activity
2.) When replanning, try to reuse previous planning computations
- an incremental update algorithm (ITC) that utilizes previous computations

13. 1st Key Idea: Flexible Execution Times
Kirk supports flexible lower and upper time-bounds on activities
Touchdown !
20 yardline
(C) Hike
(R) Run to Endzone
(R) Catch Football
(QB) Pass to Endzone
Another Representation of the Hail Mary Activity Plan: 20 10 G QB C R
Hail Mary:
A Simple Temporal Network (STN):
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(C) Hike [0,5]

14. Determining Temporal Consistency
To determine temporal consistency, an STN is converted into a distance graph
STN:
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(C) Hike [0,5]
(20 yardline)
Distance Graph: -0 8 -5 9 -7 2 5 -0
Then the FIFO Label Correcting Algorithm is run on the new distance graph.
There are two possible outcomes:
1.) If the algorithm terminates, then the STN is temporally consistent
2.) If the algorithm enters an infinite loop, then the STN is temporally inconsistent
- an infinite loop is detected as soon as a nodes cost drops below zero
- the infinite loop will contain the timing constraints in conflict.

15. Determining Temporal Consistency
Update
FIFO Label -
Correcting Algorithm (
Graph G )
{01} for all s Î
V(G)
{02} d(s) = ∞
{03a} d(s
start
) = 0
{03b} insert(S
{04a} while !Q.empty()
{04b}
u = Q.pop()
{05a} for v
Succ(u)
{05b} dval = Update(u,v)
{06} if(dval) < 0
{07}
return false;
{08} return true;
value Update
(p,x)
{09} if (d(x) > d(p) + c(p,x))
{10} d(x) := d(p) + c(p,x);
{11} Q.Insert(x, d(x));
{12} return d(x);
Check for
Violating Arcs
Initialize
Run FIFO Label-Correcting Algorithm: 5 8 -5 9 -7 2 5 13 5 13 15 5 13 14
Terminates, so STN is consistent!!
Original STN:
(20 yardline)
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(C) Hike [0,5]

16. Detecting a Conflict Run FIFO Label-Correcting Algorithm: 1 5 9 13 1 5
Update
FIFO Label -
Correcting Algorithm (
Graph G )
{01} for all s Î
V(G)
{02} d(s) = ∞
{03a} d(s
start
) = 0
{03b} insert(S
{04a} while !Q.empty()
{04b}
u = Q.pop()
{05a} for v
Succ(u)
{05b} dval = Update(u,v)
{06} if(dval) < 0
{07}
return false;
{08} return true;
value Update
(p,x)
{09} if (d(x) > d(p) + c(p,x))
{10} d(x) := d(p) + c(p,x);
{11} Q.Insert(x, d(x));
{12} return d(x);
Check for
Violating Arcs
Initialize
Run FIFO Label-Correcting Algorithm: 5 8 -5 9 -7 2 1 5 9 13 1 5 9 13 15 1 -3 5 13 9 20
-12
Node cost dropped below zero! STN is Inconsistent !!
Following the backpointers reveals conflict!
Modified STN:
(QB) Pass to Endzone [5,8]
(C) Hike [0,5]
(R) Catch Football [0,2]
(20 yardline)
(R) Walk to Endzone [12,15]
(R) Run to Endzone [7,9]
( Touchdown ! )

17. 2nd Key Idea: Incremental Update Algorithm (ITC)
Use an Incremental Update Algorithm (ITC) to retain previous computations
Three Arc Update Rules:
1.) Arc Change With No Effect to Shortest-Path
2.) Arc Change Improves Shortest-Path
3.) Arc Change Invalidates Shortest-Path 5 8 -5 9 -7 2
Hail Mary Distance Graph: 5 13 5 13 15 5 13 14
Hail Mary STN:
(C) Hike [0,5]
(20 yardline)
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]

18. 1.) Arc Change Without Effect to Shortest-Path
Rule 1: If an arc change doesn’t affect the shortest-path, then no update is needed.
Hail Mary Distance Graph: 5 8 -5 9 -7 2 5 13 5 13 15 11 5 13 14
Hail Mary STN:
(C) Hike [0,5]
(20 yardline)
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(R) Run to Endzone [7,11]

19. 2.) Shortest-Path Improvement
Rule 2: If an arc change improves shortest-path, then just propagate the improved costs
Hail Mary Distance Graph: 5 8 -5 9 -7 2 7 5 12 13 5 13 12 15 14 5 13 14 12
Hail Mary STN:
(C) Hike [0,5]
(20 yardline)
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(QB) Pass to Endzone [5,7]

20. 3.) Invalidation of Shortest-Path
Rule 3: If an arc change invalidates shortest-path:
First, reset all invalidated nodes to infinity, and then propagate improved costs from the changed arc’s head node.
Hail Mary Distance Graph: 5 8 -5 9 -7 2 9 5 14 13 5 13 14 15 16 5 13 14 14
Hail Mary STN:
(C) Hike [0,5]
(20 yardline)
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(QB) Pass to Endzone [5,9]

21. Glass Box Behavior of ITC
Rule 1: No effect to shortest path
Rule 2: Shortest path improvement
Rule 3: Invalidation of shortest path
END
START

22. Experimental Results
The Incremental Update Algorithm (ITC) was tested on:
- UAV Scenarios Randomly Generated STNs
NFZ1
NFZ2
WaterA2
WaterA1
WaterB1
WaterB2
Fire1
Fire2
Seeker UAV
Water UAV
No-Fly Zone
Legend:
Fire
Water
UAV Base
Plan Goal: Extinguish All Fires
Vehicles: Two Seeker UAVs
One Water UAV Resources: Fuel & Water

23. Experimental Results: UAV Scenarios

24. Experimental Results: Random STNs

25. Key Assumptions and Limitations
Kirk’s pre-defined operators are sufficiently expressive Desired plan is expressible with Kirk’s pre-defined operators
Activities are controllable - dispatcher can pick any specific time between the lower bound and upper bound of an activity (e.g. Hike [0,5] )
There is no innovative problem solving or logical deduction when planning fails

26. Conclusions Automated Planning is Important to Aerospace Engineering
Kirk allows flexibility in planning through flexible time-bounds
With Incremental Update Rules, Kirk can utilize previous computations
Experimental Data shows an order of magnitude improvement

27. 4/20/2019
Questions ?
May 12, 2004

28. 4/20/2019
Additional Slides
May 12, 2004

29. Origins of Kirk Deep Space 1 mission Used the HSTS planner
NASA’s first fully autonomous spacecraft demonstration
Used the HSTS planner
Used flexible time (Temporal Constraint Graph)
Developed into Europa and then Kirk

30. Another Key Idea: Reducing Complexity
Kirk restricts activity plans to predefined operators.
Touchdown !
20 yardline
(C) Hike
(R) Run to Endzone
(R) Catch Football
(QB) Pass to Endzone
Another Representation of the Hail Mary Activity Plan: 20 10 G QB C R
Hail Mary:
Hail Mary:
( sequence
( C: hike )
( parallel
( R: Run to Endzone )
( QB: Throw to Endzone ) )
( R: Catch Football )

31. Using Conflicts to Guide the Search
A Simple Temporal Network formed from the pre-defined operators can be transformed into a Conditional CSP.
Then, Conflict-Directed Search Algorithms from the CSP literature such as Dynamic Backtracking and Conflict- Directed A* can be used to guide the search.
Sometimes the conflicts returned by ITC aren’t focused enough:

32. Figure 6 ITC Pseudo-Code

33. ITC Arc Update Rules (Pseudocode)
Three Arc Update Rules:
1.) Arc Change With No Effect to Shortest-Path
2.) Arc Change Improves Shortest-Path
3.) Arc Change Invalidates Shortest-Path

34. References
R. Ahuja, T. Magnanti, J. Orlin. Network Flows: Theory, Algorithms, and Applications. Prentice Hall, 1993.
R. Bellman. On a Routing Problem. Quaterly of Applied Mathematics, vol. 16, pp , 1958.
N. Chandrachoodan, S. Bhattachryya, and K.J. Liu. Adaptive Negative Cycle Detection in Dynamic Graphs. In Proc. of the International Symposium on Circuits and Systems, May 2001.
R. Dechter, I. Meiri, J. Pearl. Temporal Constraint Networks. Artificial Intelligence, 49:61-95, May 1991.
T. Estlin, G. Rabideau, D. Mutz, S. Chien. Using Continuous Planning Techniques to Coordinate Multiple Rovers. Electronic Transactions on Artificial Inttligence, 4:45-57, 2000.
P. Kim, B. Williams, and M. Abrahmson. Executing Reactive, Model-based Programs through Graph-based Temporal Planning. In Proceedings of IJCAI-2001, Seattle, WA, 2001.
S. Koenig and M. Likhachev. Incremental A*. In Adv. in Neural Information Processing Systems 14, 2001.
D. McAllester. Truth Maintenance. In Proceedings of AAAI-90, 1990,
G. Rabideau, R. Knight, S. Chien, A. Fukunaga, A. Govindjee. Iterative Repair Planning for Spacecraft Operations in the ASPEN System. ISAIRAS, Noordwijk, The Netherlands, June 1999.
Tsmardinos, N. Muscettola, and P.Morris. Fast transformation of temporal plans for efficient execution. In AAAI-98, 1998.
B.C. Williams and R.J. Ragno. Conflict-directed A* and its role in model-based embedded systems. Journal of Discrete Applied Math,
A. Stentz. Optimal and efficient path planning for partially known environments. In Proceedings of IEEE ICRA, May 1994.
N. Muscettola, P. Morris, B. Pell, and B. Smith. Issues in temporal reasoning for autonomous control systems. In Autonomous Agents, 1998.
G. Verfaillie and T. Schiex. Dynamic backtracking for dynamic constraint satisfaction problems. In Proceedings of the ECAI'94 Workshop on Constraint Satisfaction Issues Raised by Practical Applications, Amsterdam, The Netherlands, pages 1-8, 1994.
Matthew L. Ginsberg and David A. McAllester. Gsat and dynamic backtracking. In Principles of Knowledge Representation and Reasoning: Proceedings of the Fourth International Conference (KR `94), 1994.
P. Prosser, Hybrid algorithms for the constraint satisfaction problem, Computational Intelligence 3 (1993) 268—299.