Knowledge Repn. & Reasoning Lec #26: Filtering with Logic UIUC CS 498: Section EA Professor: Eyal Amir Fall Semester 2004

Last Time Dynamic Bayes Nets –Forward-backward algorithm –Filtering Approximate inference via factoring and sampling

Filtering Stochastic Processes Dynamic Bayes Nets (DBNs): factored representation s1s4s3s2s5s1s4s3s2s5s1s4s3s2s5s1s4s3s2s5

Filtering Stochastic Processes Dynamic Bayes Nets (DBNs): factored representation s4s3s2s5 s4s3s2s5 s4s3s2s5 s4s3s2s5

Filtering Stochastic Processes Dynamic Bayes Nets (DBNs): factored representation s4s3s5 s4s3s5 s4s3s5 s4s3s5

Filtering Stochastic Processes Dynamic Bayes Nets (DBNs): factored representation s4s5 s4s5 s4s5 s4s5 O(2 n ) space O(2 2n ) time

Filtering Stochastic Processes Dynamic Bayes Nets (DBNs): factored representation: O(2 n ) space, O(2 2n ) time Kalman Filter: Gaussian belief state and linear transition model s1s4s3s2s5s1s4s3s2s5s1s4s3s2s5

Filtering Stochastic Processes Dynamic Bayes Nets (DBNs): factored representation: O(2 n ) space, O(2 2n ) time Kalman Filter: Gaussian belief state and linear transition model s4s5 s4s5 s4s5 O(n 2 ) space O(n 3 ) time

Complexity Results Filtering for deterministic systems is NP- hard when the initial state is not fully known [Liberatore ’97] [Amir&Russell’03]: Every representation of [Amir&Russell’03]: Every representation of belief states grows exponentially for belief states grows exponentially for some deterministic systems some deterministic systems

Today Tracking and filtering logical knowledge Foundations for efficient filtering Compact representation indefinitely Possible projects

Logical Filtering Belief state = logical formula

Logical Filtering Belief state = logical formula Observations = logical formulae

Logical Filtering Belief state = logical formula Observations = logical formulae Actions = effect rules –e.g., “fetch(X,Y) causes has(X) if in(X,Y)”

Logical Filtering Belief state = logical formula Observations = logical formulae Actions = effect rules –e.g., “fetch(X,Y) causes has(X) if in(X,Y)” Actions may be nondeterministic Partial observations

Example: A Cleaning Robot Initial Knowledge: ? Apply action fetch(broom,closet) Resulting knowledge  in(broom,closet) Reason: –If initially  in(broom,closet), then still  in(broom,closet) –If initially in(broom,closet), then now  in(broom,closet)

Filtering with Possible Worlds Problem: n world features  2 n states

Filtering Possible Worlds Initially we are in {s1,…,sk} Action a Filter[a]({s1,…,sk})= {s’ | R(s1,a,s’) or … or R(sk,a,s’)} observing o Filter[o]({s1’,…,su’})= {s1’,…,su’}  {s | o holds in s}

Filtering with Logical Formulae Action-Definition(a) t,t+1    (Precond i (a) t  Effect i (a) t+1 )  i  Frame-Axioms(a)

Filtering with Logical Formulae Belief state S represented by  Actions: Filter[a](  t )  logical results t+1 of  t  Action-Definition(a) t,t+1 A t v B t B t  C t+1   A t+1 v (B t+1  C t+1 )

Filtering with Logical Formulae Belief state S represented by  Actions: Filter[a](  t )  logical results t+1 of  t  Action-Definition(a) t,t+1 Observations: Filter[o](  ) =   o  t+1 = Filter[o](Filter[a](  t )) A t v B t B t  C t+1   A t+1 v (B t+1  C t+1 )

Filtering with Logical Formulae Belief state S represented by  Actions: Filter[a](  t )  logical results t+1 of  t  Action-Definition(a) t,t+1 Observations: Filter[o](  ) =   o  t+1 = Filter[o](Filter[a](  t )) Theorem: formula filtering implements possible-worlds semantics

Contents Tracking and filtering logical knowledge Foundations for efficient filtering Compact representation indefinitely Possible projects

Distribution Properties Filter[a](  Filter[a](  Filter[a](  ) Filtering a DNF belief state by factoring

Distribution Properties Filter[a](  Filter[a](  Filter[a](  ) Filter[a](  Filter[a](  Filter[a](  )  Filter[a](   Filter[a](  Filter[a](TRUE) Filtering a DNF belief state by factoring

Distribution for Some Actions Filter[a](  Filter[a](  Filter[a](  ) Filter[a](  Filter[a](  Filter[a](  ) Filter[a](   Filter[a](  Filter[a](TRUE) Filter literals in the belief-state formula separately, and combine the results STRIPS Actions STRIPS Actions 1:1 Actions 1:1 Actions

Actions that map states 1:1 Examples: –flip(light) but not turn-on(light) –increase(speed,+10) but not set(speed,50) –pickUp(X,Y)but not pickUp(X) Most actions are 1:1 in proper formulation

Actions that map states 1:1 Reason for distribution over  Filter[a](  Filter[a](  Filter[a](  ) 1:1 Non-1:1

STRIPS Actions Possibly nondeterministic effects No conditions on effects Example: turn-on(light) Used extensively in planning

Distribution for Some Actions Filter[a](  Filter[a](  Filter[a](  ) Filter[a](  Filter[a](  Filter[a](  ) Filter[a](   Filter[a](  Filter[a](TRUE) Filter literals in the belief-state formula separately, and combine the results STRIPS Actions STRIPS Actions 1:1 Actions 1:1 Actions

Example: Filtering a Literal Initial knowledge: in(broom,closet) Apply fetch(broom,closet) Preconds: in(broom,closet)  locked(closet) Effects: has(broom)  in(broom,closet) Resulting knowledge: has(broom)  in(broom,closet)  locked(closet)

Example: Filtering a Formula Initial knowledge: in(broom,closet)  locked(closet) Apply fetch(broom,closet) Preconds: in(broom,closet)  locked(closet) Effects: has(broom)  in(broom,closet) Resulting knowledge: has(broom)  in(broom,closet)  locked(closet)

Filtering a Single Literal Closed-form solution: Filter[a](literal) =  (Eff 1  Eff u )  B(a) literal ╞ Pre 1  Pre u –a has effect rules (and frame rules) a causes Eff i if Pre i Eff 1  Eff u - effects of action a Pre 1  Pre u - preconditions of action a –Roughly, B(a)  Filter[a](TRUE)

Filtering a Literal Filter[a](literal) =  (Eff 1  Eff u )  B(a) literal╞ Pre 1  Pre u Belief state (  :  locked(closet) Action (a): fetch(broom,closet) with “fetch(X,Y) causes has(X)  in(X,Y) if  locked(Y)  in(X,Y)” Belief state after a: Filter[a](  locked(closet)   locked(closet)  in(broom,closet)

Filtering a Literal Filter[a](literal) =  (Eff 1  Eff u )  B(a) literal╞ Pre 1  Pre u Action (a): fetch(broom,closet) with “fetch(X,Y) causes has(X)  in(X,Y) if  locked(Y)  in(X,Y)” Belief state after a:  Filter[a](  locked(closet)   locked(closet)  in(broom,closet) Reason:  locked(closet)╞  locked(closet)  in(broom,closet))   in(broom,closet)

Algorithm for Permutation Actions Belief state (  :  locked(closet)  (in(broom,closet)  in(broom,shed)) Action (a): fetch(broom,closet) with “fetch(X,Y) causes has(X)  in(X,Y) if  locked(Y)  in(X,Y)” Resulting belief state: Filter[a](  Filter[a](  locked(closet)   Filter[a](in(broom,closet)  Filter[a](in(broom,shed))) Filter[a](  locked(closet)   locked(closet)  in(broom,closet)

Algorithm for Permutation Actions Belief state (  :  locked(closet)  (in(broom,closet)  in(broom,shed)) Filter[a](  locked(closet)   locked(closet)  in(broom,closet) Filter[a](in(broom,closet)  locked(closet)   in(broom,closet)  has(broom))  in(broom,closet) Filter[a](in(broom,shed)  in(broom,shed) Filter[a](  =  locked(closet)  in(broom,closet)   has(broom)  in(broom,shed))

Summary: Efficient Update Fast exact update with any observation formulae, if one of the following: –STRIPS action (possibly nondeterministic) –Action is a 1:1 mapping between states –Belief states include all their prime implicates

Talk Outline Tracking and filtering knowledge Tractability results Compact representation over time Discussion & Future work

Tractability and Representation Size Theorem 1 : Every propositional repn. of the belief state grows exponentially for some systems, even when initial belief state is compactly represented (follows from [Boppana & Sipser ’90]) 1 Rough statement. Complete one in [A. & Russell ’03].

Example: A Cleaning Robot Initial Knowledge: in(broom,closet)  in(broom,shed) Apply action fetch(broom,closet) Resulting knowledge  (has(broom)  locked(closet)  in(broom,closet))   (  has(broom)  locked(closet)  in(broom,closet))   (  has(broom)  in(broom,shed)) Reason for space explosion: uncertainty of action’s success and preconditions applied Small formula Big formula

Compact & Tractable Cases Compact belief state representation –STRIPS actions with belief state in k-CNF –1:1 actions with belief state in k-CNF … Observations in 2-CNF Theorem: Filtering  with STRIPS actions –k-CNF  k-CNF –time ~ O(|  | 2 #rules(a) ) Corollary: Filtering with STRIPS actions keeps belief state in O(n k ) size (k fixed).

STRIPS-Filter: Experiments Average time per step Filtering step Filter time (m.sec) ~270 features ~240 features ~210 features ~180 features ~150 features

STRIPS-Filter: Experiments Average space per step Filtering step Filter space (literals) ~210 features ~185 features ~160 features ~135 features ~110 features

Intuition for More Results Filtering with deterministic action a is equivalent to filtering with actions a1 (“a succeeds”) or a2 (“a fails”) successfully, –a1,a2 STRIPS with known success/failure Filter[a](φ)  Filter[a1](φ) v Filter[a2](φ) STRIPS with known success/failure: Filter[a](l 1  l u ) = (l 1  l u )  B(a) or B(a)

Recent Results: (unpublished) #1 Compact representation indefinitely for STRIPS, if failure leaves features unchanged, and effects are 2-clauses a causes (f v g) & (g v -h) if x & y Starting from belief state with r clauses we get at most max(r,n) clauses indefinitely, if effects are conjunction of at most two clauses

Recent Results: (unpublished) #2 Compact representation indefinitely for STRIPS, if failure has nondeterministic effect on affected features a causes f & g if x & y a causes (f v -f) & (g v -g) if (-x v -y) Belief state in k-CNF maintained indefinitely, if effects in k1-CNF, preconditions in k2-DNF, k=k1+k2

Related Work Stochastic filtering –[Kalman ’60], [Doucet et-al. ’00], [Dean & Kanazawa ’88], [Boyen & Koller ’98], … Action theories and semantics –[Gelfond & Lifschitz ’97], [Baral & Son ’01], [Doherty et-al. ’98], … Computation of progression –[Winslett ’90], [del Val ’92], [Lin & Reiter ’97], [Simon & del Val ’01], …

Possible Projects More families of actions/observations –Stochastic conditions on observations –Different data structures (BDDs? Horn?) Compact and efficient stochastic filtering Relational / first-order filtering Dynamic observation models, filtering in expanding worlds Logical Filtering of numerical variables

More Projects Filtering for Kriegspiel (partially observable chess) Autonomous exploration of uncharted domains Smart agents in rich environments

THE END

Example: Explosion of Space Initial Knowledge: in(broom,closet)  in(broom,shed) Apply action fetch(broom,closet) Resulting knowledge  (has(broom)  locked(closet)   in(broom,closet))   (  has(broom)  locked(closet)  in(broom,closet))   (  has(broom)  in(broom,shed))

Tractability Problem Formula filtering is NP-hard in general Actions: Filter[a](  t )  Cn(  t  (Precond(a) t  Effect(a) t+1 )  Frame-Axioms(a)) Cn() = Logical consequences of Specific cases? Approximation?

Example: A Cleaning Robot Initial Knowledge: ? Apply action fetch(broom,closet) Resulting knowledge  in(broom,closet) Reason: –If initially  in(broom,closet), then still  in(broom,closet) –If initially in(broom,closet), then now  in(broom,closet)

Filtering Beliefs Filtering: Update knowledge of the world after actions and observations Stochastic filtering examples: –Dynamic Bayes Nets (DBNs): factored representation –Kalman Filter: Gaussian belief state and linear transition model s1s4s3s2 World state

Agents Acting in The World Agents in partially observable domains –Cognitive, medical assistants –Cleaning, gardening robots –Space robots (exploration, repair, assist) –Game-playing/companion agents Knowledgeable agents –Use knowledge to decide on actions –Update knowledge about the world

Example: A Cleaning Robot Decides to clean the current room Knows the broom is in the closet Fetches the broom from the closet Now knows that the broom is in its hand and not in the closet

Filtering a Single Literal Closed-form solution: Filter[a](literal) =  (Eff 1  Eff u )  B(a) literal ╞ Pre 1  Pre u –a has effect rules (and frame rules) a causes Eff i if Pre i Eff 1  Eff u - effects of action a Pre 1  Pre u - preconditions of action a –Roughly, B(a)  Filter[a](TRUE)

Permutation Actions Actions that permute the states: –flip(light) but not turn-on(light) –increase(speed,+10) but not set(speed,50) –pickUp(X,Y)but not pickUp(X)

Results: Tractable Cases Filtering a single literal Permutation actions STRIPS actions Prime-implicate representation of belief state

Filtering Logical Formulae: STRIPS-Filter If every executed action was possible to execute (or we observed an error), and actions do not have conditional effects (but may have nondeterministic effects), and the belief state representation in PI-CNF, then Filter[a](  C i ) =  Filter[a](C i )

Summary: Tractable Cases Fast approximate update: propositional belief state represented in NNF,CNF,DNF Fast exact update (if one of the following): –Action is a 1:1 mapping between states –STRIPS action (unconditional, nondeterministic effects of actions; observations distinguish success from failure of action) –Belief states include all their prime implicates –Any observations

Sources of Difficulty for Compact Representation For action a with effect rule a causes Eff if Pre –We always know after the action that Eff   Pre –If we know (Pre  p), then after the action we know Eff  p

How Is the State Kept Compact? STRIPS (nondeterministic) actions –We always know that the precondition held (or we got a signal that the action failed) –There are no conditional effects Permutation actions –We restrict the preconditions and effects, e.g., All rules of the form a causes l 1 if l 2, or One of the preconditions is always satisfied, or … Observations (and obs. model): in 2-CNF

STRIPS-Filter: Experimental Results [A. & Russell ’03]

Tractability and Representation Size Theorem 1 : Every propositional repn. of the belief state grows exponentially for some systems, even when initial belief state is compactly represented (follows from [Boppana & Sipser ’90]) However, special cases can can be computed efficiently and represented compactly s in 2-CNF 1 Rough statement. Complete one in [A. & Russell ’03].

STRIPS-Filter: Experimental Results [A. & Russell ’03]

Applications Tractable filtering and tracking of the world in high-dimensional domains with many objects, locations and relationships Learn effects and preconditions of actions in partially-observable domains Autonomous exploration of uncharted domains

Related Work Stochastic filtering –[Kalman ’60], [Blackman & Popoli ’99], [Doucet et-al. ’00], … Action theories and semantics –[Gelfond & Lifschitz ’97], [Baral & Son ’01], [Doherty et-al. ’98], … Computation of progression –[Winslett ’90], [del Val ’92], [Lin & Reiter ’97], [Simon & del Val ’01], …

THE END

Filtering STRIPS Actions STRIPS: –Action was executed or we observed an error, –No conditional effects, and –Possibly nondeterministic effects

Logical Filtering: Progress Outlook 18 months: relational filtering, learning actions in partially-observable domains 36 months: dynamic observation models, Horn belief states, filtering in expanding worlds, autonomous agents in games 54 months: first-order filtering, factored belief states, continuous time, autonomous exploration of uncharted domains

Today 1.Probabilistic graphical models 2.Treewidth methods: 1.Variable elimination 2.Clique tree algorithm 3. Applications du jour: Sensor Networks

Contents 1.Probabilistic graphical models 2.Exact inference and treewidth: 1.Variable elimination 2.Junction trees 3. Applications du jour: Sensor Networks

Application: Planning General-purpose planning problem: –Given: Domain features (fluents) Action descriptions: effects, preconditions Initial state Goal condition –Find: Sequence of actions that is guaranteed to achieve the goal starting from the initial state

Application: Planning with partitions PartPlan Algorithm Start with a tree- structured partition graph Identify goal partition Direct edges toward goal In each partition –Generate all plans possible with depth d and width k –Pass messages toward goal

Planning with partitions PartPlan Algorithm Start with a tree-structured partition graph Identify goal partition Direct edges toward goal In each partition –Generate all plans possible with depth d and width k: “if you give me a block, I can return it to you painted”, “if you give me a block, let me do a few things, and then give me another block, then I can return the two painted and glued together.” –Pass messages toward goal: All preconditions/effects for which there are feasible action sequences

Factored Planning: Analysis Planner is sound and complete Running time for finding plans of width w with m partitions of treewidth k is O(m  w  2 2w+2k ) Factoring can be done in polynomial time Goal can be distributed over partitions by adding at most 2 features per partition

Next Time Probabilistic Graphical Models: –Directed models: Bayesian Networks –Undirected models: Markov Fields Requires prior knowledge of: –Treewidth and graph algorithms –Probability theory