1. Finding Nemo: Translating Reactive Tasks to Reactive Controllers Hadas Kress-Gazit joint work with Georgios Fainekos and George Pappas

2. “Look for Nemo. If he is in a red zone, sound the alarm and go to the office. If he is in another zone, take him to one of the parking lots”

4. Symbolic reactive high level task Low level sensor based robot control Automatic Correct Efficient “Look for Nemo. If he is in a red zone, sound the alarm and go to the office. If he is in another zone, take him to one of the parking lots”

5. Problem Known workspacerobot Actions Sensor inputs Correct robot motion and action high level task ?

6. Related work Known workspacerobot Actions Sensor inputs high level task What to do? robot motion and action How?

7. Related work Known workspacerobot Actions Sensor inputs high level task What to do? Planning and AI S. Russell and P. Norvig. Artificial Intelligence, A Modern Approach. Prentice Hall, second edition, 2003. P. Bertoli, A. Cimatti, M. Pistore, M. Roveri and P. Traverso. MBP : A model based planner. In In Proc. IJCAI’01 Workshop on Planning under Uncertainty and Incomplete Information, 2001. S. M. LaValle. Planning Algorithms. Cambridge University Press, Cambridge, U.K., 2006.

8. Related work robot motion and action How? Motion Planning R. R. Burridge, A. A. Rizzi, and D. E. Koditschek, “Sequential composition of dynamically dexterous robot behaviors,” International Journal of Robotics Research, vol. 18, no. 6, pp. 534–555, 1999. S. M. LaValle. Planning Algorithms. Cambridge University Press, Cambridge, U.K., 2006. (Sampling based, etc.)

9. Known workspacerobot Actions Sensor inputs high level task What to do? robot motion and action How? Automatic Correct Efficient

10. Formal methods: Verification, Model checking and Synthesis A short detour

11. System/Program/Design Requirements Synthesis Verification Correct ?

12. System/Program/Design Requirements Verification Correct ? Simulation and testing Deductive verification Model checking

13. Finite state systems Requirements Usually in Temporal Logic Requirements Usually in Temporal Logic Correct ? Model checking a,b a,c a,b a b,c Exhaustive search of the state space Algorithm is guaranteed to terminate with the correct answer. If there is a bug, (sometimes) a counter-example is given

14. Crash Course in Temporal Logic Propositional logic: – if a = b = c = True in our world, then f = True if a = b = True but c = False in our world, then f = False What if the “world” is not static? –We can discuss the truth of statements of the form: a is always True c is eventually True b and c are never True together If c is True, then c stays True forever It is possible that c will become True b is True in the next state a,b a,c a,b a b,c

15. Crash Course in Temporal Logic Linear Temporal Logic (LTL) –Truth is evaluated along infinite computation paths (a,b),a,a,a… (a,b),(a,b),(a,c),(a,c),… (a,b),a,a,a,(a,b),(a,b),(a,c),(a,c),… –Formulas: Atomic propositions (a,b,c…) Boolean combinations ¬p (not), p  q (and), p  q (or) Temporal operators O p (next), qUp (until), ◊p (eventually), []p (always) –Examples: []a = “a is always true” ◊(a  c) = “(a and c) is eventually true” O b = “b is true in the next state” a,b a,c a,b a b,c

16. Crash Course in Temporal Logic Robotic Task examples: “Visit rooms 1,2,3 while avoiding corridor 1”: [] ¬(corridor1)  ◊(room1)  ◊(room2)  ◊(room3) “ If the light is on, visit rooms 1 and 2 infinitely often”: []( (LightOn) -> ([]◊(room 1)  []◊(room 2)) ) “If you are in room 3 and Mika is there, beep” []( (room3)  (SeeMika) -> (Beep) ) And much more…

17. Model checking []a ◊(a  c) [](c → Oc) []a ◊(a  c) [](c → Oc) Correct ? a,b a,c a,b a b,c True False: (a,b)(a,b)(a,b)… True

18. Model checking Great book: Model Checking. Clarke, E.M., Grumberg, O., and Peled, D. MIT Press, 2000. Many tools (SPIN, NuSMV, UPPAAL…) Extensively used in the hardware design world

19. System/Program/Design Requirements Synthesis Given a formula, create the system Synthesis of the full LTL is double exponential in the size of the formula For a specific fragment, it is polynomial in the state space.

20. Back to Robotics

21. Known workspacerobot Actions Sensor inputs high level task What to do? robot motion and action How? Automatic Correct Efficient

22. Path 1 2 34 8 7 6 5 12 10 9 11 Known workspacerobot Actions Sensor inputs Hybrid Controller Correct robot motion and action high level task Binary Propositions Binary Propositions Discrete Abstraction Temporal logic formula φ Approach using Model Checking Non reactive Find a counter example for ¬ φ

23. Example “Go to room 4” φ = ◊(room4) Model check the formula ¬ ◊(room4) The formula is False and the counter example is: room1, room9, room12, room11, room4 Approach using Model Checking 1 2 34 8 7 6 5 12 10 9 11 123876591041112

24. Approach using Model Checking Advantages –Fast – example with 9250 regions took 55 seconds –Many tools NuSMV SPIN Disadvantages –Paths are not optimal –Result is a path – not a plan, so we can’t do reactive tasks. G. E. Fainekos, H. Kress-Gazit, and G. J. Pappas. Temporal logic motion planning for mobile robots. ICRA, 2005.

25. IEEE Robotics & Automation Magazine, March 2007: 61-70 Workshop on “Symbolic Approaches for Robot Motion Planning and Control”, Robotics: Science and Systems, 2006 Invited session “Symbolic Methods for Control of Mobile Robots ”, ICRA, 2007 G. E. Fainekos, H. Kress-Gazit, and G. J. Pappas. “Temporal logic motion planning for mobile robots”. ICRA, 2005. M. Kloetzer and C. Belta. “A fully automated framework for control of linear systems from ltl specifications”. HSCC, 2006. J. McNew and E. Klavins. “Locally interacting hybrid systems with embedded graph grammars”. CDC, 2006. A. Bicchi, A. Marigo and B. Piccoli. “Feedback encoding for efficient symbolic control of dynamical systems ”.IEEE Trans. Autom. Control, Jan 2006. M. M. Quottrup, T. Bak and R. Izadi- Zamanabadi. “Multi-robot planning : a timed automata approach”. ICRA, 2004. G. E. Fainekos, H. Kress-Gazit, and G. J. Pappas. “Hybrid Controllers for Path Planning: A Temporal Logic Approach”. CDC, 2005.

26. 1 2 34 8 7 6 5 12 10 9 11 Known workspacerobot Actions Sensor inputs Automaton Hybrid Controller Correct robot motion and action high level task Binary Propositions Binary Propositions Discrete Abstraction Temporal logic formula φ Approach using Synthesis Reactive ! H. Kress-Gazit, G. E. Fainekos and G. J. Pappas. Where’s Waldo? Sensor Based Temporal logic motion planning. ICRA, 2007.

27. “Nemo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” 1 2 34 8 7 6 5 12 10 9 11

28. Binary Propositions Binary Propositions Discrete Abstraction Temporal logic formula φ 1 2 34 8 7 6 5 12 10 9 11 Known workspacerobot Actions Sensor inputs Automaton Hybrid Controller Correct robot motion and action high level task Reactive ! Approach using Synthesis

29. 1 2 34 8 7 6 5 12 10 9 11 Constructing φ robot Actions Sensor inputs Discrete Abstraction Sensor (Input) propositions: X = {SenseNemo, SenseFire, HearBaby,… } = {s Nemo } Robot (Output) propositions: Y = {Room1, Room2,…,Room12, Beep, RecordVideo,… } = {r 1, r 2,…, r 12,Beep } Known workspace

30. Constructing φ “Nemo may be in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” “Nemo may be in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” Structured English (MURI SUBTLE) ??? H. Kress-Gazit, G. E. Fainekos and G. J. Pappas. From Structured English to Robot Motion, IROS, San Diego, CA, Oct 2007. To appear.

31. Structured English Interface “Nemo may be in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” “Nemo may be in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” Environment starts with not Nemo … You start in r12 If you are sensing Nemo then stay there Beep if and only if you are sensing Nemo If you are not sensing Nemo then go to r1 …

32. Constructing φ We consider LTL formulas of the form: Assumptions about the environment Desired robot behavior *Note that only if the assumptions are met ( is true), the desired behavior is guaranteed ( must be true).

33. Initial Conditions Transitions Goals Example Task: “Nemo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” Sensor (Input) propositions: X = {s Nemo } Robot (Output) propositions: Y = {r 1, r 2,…, r 12,Beep} Task: “Nemo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” Sensor (Input) propositions: X = {s Nemo } Robot (Output) propositions: Y = {r 1, r 2,…, r 12,Beep} Environment Assumptions Desired behavior

34. Why this structure? Can be efficiently synthesized into an automaton No significant loss of expressivity with respect to the full LTL Clear distinction between assumptions and desired behavior

35. Automaton robot high level task 1 2 34 8 7 6 5 12 10 9 11 Actions Sensor inputs Hybrid Controller Correct robot motion and action Binary Propositions Binary Propositions Discrete Abstraction Temporal logic formula φ Known workspace Approach using Synthesis

36. Automaton synthesis Synthesis algorithm due to Piterman, Pnueli and Sa’ar (VMCAI 2006) Polynomial (n 3 ) in the number of states (as opposed to double exponential in the length of the formula) Basically, solves a game between the environment and the robot. If the robot wins, an automaton is extracted.

37. Example Task: “Nemo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” r 12 r9r9 r8r8 r9r9 r 10 r5r5 r 11 r3r3 r 12 r9r9 Beep r 1 r1r1 s Nemo 3 4 8 7 6 5 12 10 9 11 1 2 Beep r 8 Beep r 3 Beep r 5

38. Hybrid Controller Automaton Correct robot motion and action robot high level task 1 2 34 8 7 6 5 12 10 9 11 Actions Sensor inputs Binary Propositions Binary Propositions Discrete Abstraction Temporal logic formula φ Known workspace Approach using Synthesis

39. Hybrid controller We employ a set of “simple” controllers that drive the robot from one region to an adjacent one, based on its dynamics. –Belta et. al. (CDC 2004) –Conner et. al. (IROS 2003, RSS 2006,IROS 2007) –Lindemann et. al. (RSS 2006, ICRA 2007) The automaton governs the sensor based composition of these controllers

40. Example Task: “Nemo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop and beep” 3 4 87 6 5 12 10 9 11 1 2 r 12 r9r9 r8r8 Beep r 5 Beep r 8 r9r9 r 10 r5r5 r 11 Beep r 3 r3r3 r 11 r 12 r9r9 Beep r 1 r1r1 s Nemo

41. When will this break? Logical inconsistency – “go to room 1 and always stay in room 4” Topologically impossible – “go to room 5 and always avoid room 10” → No automaton is synthesized Environment behaves badly – –Sensors inputs contradict assumptions ( is false) –“Violent” environment → Execution may be incorrect or terminated prematurely

42. D. C. Conner, H. Kress-Gazit, H. Choset, A. Rizzi and G. J. Pappas. Valet parking without a valet, IROS, San Diego, CA, Oct 2007. To appear. Extensions Complex Dynamics *Images courtesy of David C. Conner

43. Extensions Multi Robot –Naturally captured in a decentralized way –The environment of each robot contains all other robots –“ If robot 2 is in the kitchen, do not go there” X = {Robot2Kitchen,… }, Y = {Kitchen, Hall, Bedroom,… } …  []( Robot2Kitchen → ¬O( Kitchen) )  …

44. H. Kress-Gazit, D. C. Conner, H. Choset, A. Rizzi and G. J. Pappas. Courteous cars: Decentralized multi-agent traffic coordination, Special Issue of the IEEE Robotics and Automation Magazine on Multi-Agent Robotics. To appear. Extensions Multi Robot “Drive around the environment, while obeying traffic rules, until you find a free parking space, and then park” “Leave the block, while obeying traffic rules, through Exit i ”

45. Summery Method for transforming high level reactive tasks into low level robot control Advantages –Sensor based –Automatic –Efficient –Correct by construction –Interface between discrete planning and continuous control

46. Demo

47. What’s next? Natural language interface Feedback\Dialog with the user Unknown workspace (SLAM ?) Verify sensor models obey assumptions Incorporate cost On the fly extraction of automata Experiments

48. Thanks You