1. Ontologies Reasoning Components Agents Simulations Reasoning about actions, change, events and time Jacques Robin

2. Outline  Non-monotonic reasoning  When and where to reason about actions, change, event and time?  Reasoning services and illustrative case study  Roadmap of approach  The Situation Calculus (SC)  Key ideas  Representation  Reasoning  The frame problem  The ramification problem  The qualification problem  Limitations  The Event Calculus  Key ideas  Normal logic programs and negation as failure  Representation  Reasoning  Abductive logic programs and abductive planning  Solutions to the frame and ramification problems  Limitations and comparison with SC  Transaction Logic  Key ideas  Transaction logic program syntax  Transaction logic program semantics  Representing action and changes in transaction logic  Solution to the frame and ramification problems  Limitations and comparison with SC, EC

3. Non-monotonic reasoning  Classical logic reasoning is monotonic  If KB |= f, then KB  g |= f  Inference engine only performs ask and tell to the KB, never retract  Non-monotonic reasoning  Allows KB |= f, and then KB  g |  f  Previously derived facts can be retracted upon arrival (for example from sensors) of new, conflicting evidence  Two distinct sources of non-monotonicity  Ontological: in non-stationary environments E  KB must reflect environment changes as time goes by  when fact f true in E(t) no longer true in E(t+1), (i.e., the agent’s position)  if KB does not use a historically cumulative knowledge representation scheme, sentence s in KB(t) that represents f must retracted from KB(t+1) and so must all sentences in KB(t) proven using s!  and so must all sentences in KB(t) proven using s! (truth-maintenance)  Epistemological: in partially observable environments E  Decision making with partial knowledge requires using abduction instead of pure deduction  Some sentences in KB(t) were deductively derived from sensors and KB(0)  While others were merely abduced (i.e, hypothesized as true or false by default)  In the face of

4. Two orthogonal sources of non-monotonicity  Ontological:  Ontological: in non-stationary environment E  KB must reflect environment changes as time goes by  when fact f true in E(t) no longer true in E(t+1)  unless KB uses a historically cumulative knowledge representation scheme  sentence s in KB(t) that represents f must retracted from KB(t+1),  and so must all sentences in KB(t) proven using s! (truth-maintenance)  Epistemological:  Epistemological: in partially observable environment E  Decision making with partial knowledge requires using abduction in addition to deduction  KB must reflect changes of the agent’s beliefs as new evidence becomes available through sensing  KB(t) = D(t)  A(t,H(t)), where  D(t) are sentences derived purely deductively from percept sequence and KB(0)  A(t,H(t)) are sentences derived using at least one abductive step relying on some hypothesis in H(t)  When  a  D(t) and a  A(t-1,H(t-1)), a must be retracted from KB(t)  and so must all sentences in A(t-1,H(t-1)) derived using a!

5. When to reason about actions, change, events and time?  Non-stationary environments  Sequential (only actions and change)  Concurrent synchronous (actions, change and events) Percept Environment Agent Action State 1 Reasoning PerceptAction State 2 Reasoning PerceptAction State 3 Reasoning State N... Percept Agent Action State 1 Reasoning PerceptAction State 2 Reasoning State 4State 5 State 3 Environment

6. When to reason about actions, change, events and time?  Non-stationary environments  Concurrent asynchronous (actions, change, events and time)... Percept Agent Action State 1 Reasoning State 2State 4State 3 PerceptAction State 5 Reasoning State 6Environment

7. Where to reason about actions, change, events and time? Environment Sensors Actuators Environment model (past and present) Percept interpreter: percepts(t)  model(t-1)  model(t) Action chooser: model(t)  actions(t) Environment model updater: model(t-1)  model(t) model(t)  model’(t) actions(t)  model(t)  model(t+1) Automaton Agent

8. Where to reason about actions, change, events and time? Environment Sensors Actuators Model of past and current environments Percept interperter percepts(t)  model(t-1)  model(t) Action chooser result([action(t),...,action(t+n)]) = hyp(model(t+n))  hyp(model(t+n))  goal(t)  do(action(t)) Environment model updater model(t-1)  model(t) model(t)  model’(t) do(action(t-1))  model(t) Goal updater model(t)  goals(t-1)  goals(t) Goals Future environments predictor model(t)  hyp(action(t))  hyp(model(t+1)) model(t)  model(t+1) Model of hypothetical future environments Deliberative Agent

9. Reasoning services  Environment model decomposed in: Eternals  Eternals: properties and relations not affected by actions, events or the passing of time Fluents  Fluents: properties and relations that change as result of the execution of an action, the occurrence of an event or the passing of time  Single action (event) consequence:  Given current environment state model  Compute fluents resulting from the execution (occurrence) of a single action (event)  Temporal projection:  Given current environment state model  and a sequence of hypothetical actions to execute and events to occur  Compute fluents of the resulting environment state  Planning:  Given current environment model  and a set of goal fluents  Compute action sequence whose execution will turn these goal fluents true

10. Illustrative case study: the block world  Agent: robot with one arm building stacks from blocks on a table  Eternals:  isclear(table)  block(B)  Fluents:  on(Block,Loc)  isclear(Loc)  above(B1,B2)  Action: safeMove(B,L1,L2)  preconditions  on(B,L1)  isclear(L2)  isclear(B)  intended effects   on(B,L1)  on(B,L2)  side effects  block(L2)   isclear(L2)  Action: riskyMove(B,L1,L2)  preconditions  on(B,L1)  isclear(L2)  intended effects   on(B,L1)  on(B,L2)  side effects   X, above(X,B)  (  above(X,L1)  above(X,L2))  block(L2)   isclear(L2) a b c table a b c

11. Roadmap of the approaches: situation and event calculi Full Classical First-Order Logic (FCFOL) SC Inference Engine: Non-monotonic reasoning about actions and change FCFOL Theorem Prover: Monotonic deduction Situation Calculus (SC) added constructs (domain-independent): function: result predicates: preconds, posEffect, negEffect reuses EC Inference Engine: Non-monotonic reasoning about actions, events, change and time Tabled Prolog Engine: Negative abduction Event Calculus (EC) added constructs (domain-independent): predicates: holds, happens, initiates, terminates, clips, < clauses defining EC semantics on top of ALP/NLP semantics reuses (Pure) Definite Logic Program (DLP) ALP Engine: Positive and negative abduction reuses Abductive Logic Program (ALP) added constructs: abducible predicates integrity constraint clauses Normal Logic Program (NLP) added construct: connective Negation As Failure (NAF)

12. Roadmap of the approaches: transaction logic TL(HL,{BTI,BTD}) Inference Engine: Non-monotonic reasoning about and programming actions and change Tabled Prolog Engine: Negative abduction Transaction Logic w/ Data Oracle: Hilog & Data Update Oracle: btinsert(HiLogTerm), btdelete(HiLogTerm) (TL (HL,{BTI,BTD}) added constructs): backtrackable knowledge base update predicates: btinsert, btdelete sequential connectives: ,  reuses (Pure) Definite Logic Program (DLP) HiLog: Negative abduction and meta-abduction reuses Normal HiLog Program (NHP) added constructs: high-order functions high-order predicates reified clauses Normal Logic Program (NLP) added construct: connective Negation As Failure (NAF)

13. The situation calculus

14. FCFOLConnective FCFOLFormulaFCFOLAtomicFormulaFCFOLTerm FCFOLFunctionalTerm FCFOL NonFunctionalTerm Ontology of the Situation Calculus FunctorArg 1..2 Arg 1..* Arg1..* Fluent Atom Eternal Atom Fluent Term Eternal Term Result Term Action Term Situation Precond Atom Effect Atom NegEffect Atom PosEffect Atom Precond Axiom Effect Axiom Frame Axiom Successor State Axiom Ramification Axiom

15. Precondition and effect axioms  Precondition axioms   B,L1,L2,S block(B)  on(B,L1,S)  isclear(L2,S)  isclear(B,S)  precond(safeMove(B,L1,L2),S)   B,L1,L2,S block(B)  on(B,L1,S)  isclear(L2,S)  precond(riskyMove(B,L1,L2),S)  Effect axioms   B,L1,L2,S,S1 precond(safeMove(B,L1,L2),S)  (S1 = result(safeMove(B,L1,L2),S)  (S1 = S)   on(B,L1,S1)  on(B,L2,S1)  (block(L2)   isclear(L2,S1)))   B,L1,L2,S,S1 precond(safeMove(B,L1,L2),S)  (S1 = result(riskyMove(B,L1,L2),S)   (S1 = S)   on(B,L1,S1)  on(B,L2,S1)  (block(L2)   isclear(L2,S1)))

16. The frame problem  In most non-stationary environments:  Most fluents remain the same most of the time  Each action only affects a tiny percentage of all fluents  Effect axioms only specify the fluents changed by an action  But the execution of an action changes the situation  How to specify that fluents unchanged by the last executed action carry over to the new situation that it created? frame axioms  Naive approach: frame axioms   B,L1,L2,S,S1,B1,B2 precond(safeMove(B,L1,L2),S)  on(B1,B2,S)   (B = B1)   (B = B2)  (S1 = result(safeMove(B,L1,L2),S)   (S1 = S)  on(B1,B2,S1)  Leads to exponential number of frame axioms

17. Solving the frame problem in the situation calculus successor state axiom  Substituting implicative effect and frame axioms by successor state axiom using equivalence  Successor state axiom schema  (precond(A,S)  (f i (result(A,S))  ((posEffect(A,f i ))  f i (S)   negEffect(A,f i ))) ....  posEffect(a j,f i ) ...  negEffect(a k,f i ) ...  where S is a situation, A,a j,a k an action and f i are fluents  Example:  (  A,B,L,S precond(A,S)  (on(B,L,result(A,S))  posEffect(A,on(B,L))  (on(B,L,S)   negEffect(A,on(B,L))))   A,B,L1,L2,S posEffect(safeMove(B,L1,L2), on(B,L2))   A,B,L1,L2,S posEffect(safeMove(B,L1,L2), isclear(L1))   A,B,L1,L2,S negEffect(safeMove(B,L1,L2), on(B,L1))   A,B,L1,L2,S negEffect(safeMove(B,L1,L2), isclear(L2))   A,B,L1,L2,S posEffect(riskyMove(B,L1,L2), on(B,L1))...

18. The ramification problem  Actions have:  intended effects that satisfy the agent’s goal and justify their execution  side effects related to preconditions of other actions  Modeling side effects together with intended ones in successor state axioms link them to actions leading to combinatorial explosion ramification axioms  Side effects best modeled orthogonally as ramification axioms that relate fluents among themselves independently of actions in the same situation  Example:   B1,B2,B3,S above(B1,B2,S)  (on(B1,B2,S)  (on(B1,B3,S)  above(B3,B2,S)))  Execution of riskyMove(b,c) in S-1 w/ above(a,b,S-1)  Successor state axiom allows deducing: on(b,c,S)  above(a,b,S)  Ramification axiom then allows deducing: above(a,c,S)

19. The qualification problem  Because it is based on classical logic, the situation calculus requires explicitly modeling all preconditions and all side-effects of every action in every situation  In practice, rarely occurring combinations of fluents are easily overlooked when specifying preconditions and side-effects  This is the qualification problem: think of everything that can go wrong  Best solved by probabilistic knowledge representation, aggregating all unforeseen cases in one “catch-all” situation

20. Unique name axioms unique name axioms  Being based on classical logic, a KB using the situation calculus must contain unique name axioms stating that no two members of the Herbrand base are equal unless explicitly specified  Example:   a = b   a = c   a = table ......  (  B,L1,L2,S,B’,L1’,L2’,S’ safeMove(B,L1,L2,S) = safeMove(B1’,L1’,L2’,S’)  (B = B1, L1 = L1’, L2 = L2’, S = S’)) ...

21. Limitations of the situation calculus  Expressiveness:  Punctual and implicit representation of time by way of situations  All changes result from a single agent executing actions  Limited to sequential, mono-agent environments  Non-intuitive for being based on full classical first-order logic  Efficiency:  Very space inefficient  Guards the entire history of the environment  If a fluent changed only once in a 2001 steps process, the knowledge base contains 2000 copies of it, each one w/ a different situation argument  Time inefficient  Reasoning relies on theorem proving in full classical first-order logic  Determining the truth of a fluent requires regressing through the entire history of the environment

22. The Event Calculus (EC)  Exists in various version of increasing expressiveness  Most expressive versions overcome all the expressiveness limitations of the SC except the ramification problem  Also more time efficient than the SC for relying on abductive logic programming in Horn first-order logic instead of theorem proving in classical first-order logic  Explicitly represents time points, time intervals, punctual events, events with durations, sequential and concurrent events  An agent executing an action is a special case of event t not(holds(Fluent))holds(Fluent) initiates(Event,Fluent) terminates(Event,Fluent) not(holds(Fluent))

23. General logic programs not  General (or normal) logic programs extend pure (or definite) logic programs with Negation As Failure (NAF) connective not that can precede atoms in a clause premise  Inference engine is extended with step to prove premise not A  Generally it tries to prove goal A:  if it succeeds, then not A fails  if it finitely fails to do so, it assumes A false (negatively abduces A) and not A succeeds  abducing not A from failure to prove A it is a completely different inference than deductively proving  A in classical first-order logic  Tabled logic programming engine can detect loops through NAF such as query q with program {p :- not q. q :- not p.} Well-Founded Semantics (WFS)  It computes the Well-Founded Semantics (WFS) in ternary logic  When A neither succeeds nor can it finitely fail due to NAF loops, the WFS assigned truth value unknown to both A and not A

24. Representing a domain with EC  Domain-dependent knowledge represented as abductive logic program clauses using EC predicates:  holds(F,T): fluent F is true at time T  precond(E,T): precondition of event E are fulfilled at time T  do(A,T): action A is executed at time T  happens(E,T): event E occurs at time T  initiates(E,F,T): fluent F becomes true as a positive effect of event E  terminates(E,F,T): fluent F becomes false as a negative effect of event E  clipped(F,T1,T2): fluent F becomes false between time T1 and time T2  Domain-independent EC clauses that specify generic relations between EC predicates, that axiomatize their temporal reasoning semantics  holds(F,T2) :- happens(E,T1), initiates(E,F,T1), T1  T2, not clipped(F,T1,T2).  clipped(F,T1,T3) :- happens(E,T2), terminates(E,F,T2), T1  T2, T2  T3.  happens(E,T) :- precond(E,T), not action(E).  happens(E,T) :- precond(E,T), action(E), do(E,T).

25. Block world EC clauses  Initial state clauses:  holds(on(a,b),0). holds(on(b,c),0). holds(on(c,table),0). holds(isclear(a),0).  Precondition clauses:  precond(safeMove(B,L1,L2),T) :- holds(on(B,L1),T), holds(isclear(L2),T), holds(isclear(B),T).  Positive effect clauses:  initiates(safeMove(B,L1,L2), on(B,L2),T).  initiates(safeMove(B,L1,L2), isclear(L1), T).  Negative effect clauses:  terminates(safeMove(B,L1,L2), on(B,L1),T).  terminates(safeMove(B,L1,L2), isclear(L2), T).  Ramification clauses:  holds(above(B1,B2),T) :- holds(on(B1,B2),T).  holds(above(B1,B3),T) :- holds(on(B1,B2),T), holds(above(B2,B3),T).

26. Variations of the EC  Basic EC already handles concurrent actions and time intervals  Additional arguments to EC predicates and additional EC clauses allow extending basic EC to handle:  Events with durations: happens(E,T1,T2)  Multiple agents: do(Agent,Action,T)

27. Abductive logic programming  Extend general logic programs with:  Integrity constraints I abducibles  A distinguished subset of predicate symbols called abducibles  Atoms with an abducible functors (abducible atoms) are generally restricted to appear only in clauses premises  they have no definitions where they appear in conclusion  An abducible atom goal is assumed true (positive abduction) whenever:  Its assumption does not allow deducing conclusions that violate the integrity constraints  Abduction can be used to construct plans using an EC representation of the domain D:  holds atoms represent the goal G to be reached  do atoms represent the hypothesis H to abduce  The abductive proof procedure computes H such that:  D  H |= G  D  H  I |  false

28. Transaction Logic (TL): starting point  A set of Prolog clauses defining a predicate have two distinct semantics:  A declarative, logical one (Clark’s completion or least Herbrand model)  A procedural one (implicit)  Example  p :- q, r. p :- u.  Declarative semantics: p  ((q  r)  u)  Procedural semantics:  define p {{call q; if q = true then {call r; return r} else return fail}; {call u; return u}}  In the declarative semantics the order of q, r and u does not matter  In the procedural semantics, the order is crucial  Transaction logic unifies the two semantics by representing the procedural one explicitly and declaratively

29. Transaction logic connectives  Extends normal logic programs with backtrackable clause update predicates and two new connectives  btinsert{C1,..., Cn} premise has side effect to explicitly add clauses C1,..., Cn to the logic program  btdelete{C1,..., Cn} premise has side effect to explicitly delete all clauses that match C1 or... or Cn from the logic program  Serial conjunction  :  Declaratively captures procedural semantics of Prolog’s conjunction  Distinct from classical conjunction  In TL: a, b  b, a, but a  b  b  a  Serial disjunction  :  Declaratively captures procedural semantics of Prolog’s disjunction  Distinct from classical disjunction  In TL: a; b  b; a, but a  b  b  a  Transaction semantics:  If one element of a serial conjunctive premise fails  ex, r fails in c :- btinsert(a)  btdelete(b)  r  s  then b is put back in the KB and a is retracted from it

30. The block world in TL  Initial state clauses:  block(a). block(b). block(c). on(a,b). on(b,table). on(c,table). isclear(table).  Ramification clauses:  isclear(B) :- block(B), not(on(C,B)).  above(B1,B3) :- on(B1,B2) ; (on(B1,B2), above(B2,B3)).  Precondition clauses:  preconds(safeMove(B,L1,L2)) :- on(B,L1), isclear(B), isclear(L2).  Effect clauses:  effects(safeMove(B,L1,L2)) :- btdelete(on(B,L1))  btinsert(on(B,L2)).  Action execution clauses:  do(A) :- preconds(A)  effects(A).  Planning clauses:  achieve(Goal) :- Goal.  achieve(on(B,L)) :- do(safeMove(B,_,L)).

31. TL x EC x SC  TL far more space and time efficient than EC and SC  Because it does not keep the history of the environment  It only maintains its current state  But this history is available in the trace of its proof mechanism that can be backtracked upon demand  Like SC and unlike SC, TL does not explicitly represent time points and durations  With additional arguments and clauses it does concisely and efficiently support planning and temporal projection  Complex, chained ramifications require truth-maintenance which is not supported as built-in by available TL engines