1. CS 416 Artificial Intelligence Lecture 12 First-Order Logic Chapter 9 Lecture 12 First-Order Logic Chapter 9

2. Midterm October 25 th Up through chapter 9 (excluding chapter 5) October 25 th Up through chapter 9 (excluding chapter 5)

3. Review Inference in First-Order Logic Could convert first-order logic (FOL) to propositional logic (PL) and use PL inferenceCould convert first-order logic (FOL) to propositional logic (PL) and use PL inference –Must convert (reduce) Universal and Existential Quantifiers into PL –Potential problem with infinite number of substitutions Could lift Modus Ponens to FOLCould lift Modus Ponens to FOL –Unification required to make expressions look identical Inference in First-Order Logic Could convert first-order logic (FOL) to propositional logic (PL) and use PL inferenceCould convert first-order logic (FOL) to propositional logic (PL) and use PL inference –Must convert (reduce) Universal and Existential Quantifiers into PL –Potential problem with infinite number of substitutions Could lift Modus Ponens to FOLCould lift Modus Ponens to FOL –Unification required to make expressions look identical

4. How to perform inference in FOL Use a knowledge base Insert data with “STORE”Insert data with “STORE” Retrieve “unified” sentences using “FETCH”Retrieve “unified” sentences using “FETCH” –Search for substitutions that unite query with every sentence in KB How can se make FETCH fast? Only attempt unifications if they have a chance of succeedingOnly attempt unifications if they have a chance of succeeding Use a knowledge base Insert data with “STORE”Insert data with “STORE” Retrieve “unified” sentences using “FETCH”Retrieve “unified” sentences using “FETCH” –Search for substitutions that unite query with every sentence in KB How can se make FETCH fast? Only attempt unifications if they have a chance of succeedingOnly attempt unifications if they have a chance of succeeding

5. Predicate Indexing Index the facts in the KB Example: unify Knows (John, x) with KB:Example: unify Knows (John, x) with KB: –Knows (John, Jane) –Knows (y, Bill) –Brother (Richard, John) –Knows (y, Mother(y)) –Knows (x, Elizabeth) Index the facts in the KB Example: unify Knows (John, x) with KB:Example: unify Knows (John, x) with KB: –Knows (John, Jane) –Knows (y, Bill) –Brother (Richard, John) –Knows (y, Mother(y)) –Knows (x, Elizabeth) No need to check

6. Predicate Indexing Predicate indexing puts all the Knows facts in one bucket and all the Brother facts in anotherPredicate indexing puts all the Knows facts in one bucket and all the Brother facts in another –Might not be a win if there are lots of clauses for a particular predicate symbol  Consider how many people Know one another –Instead index by predicate and first argument  Clauses may be stored in multiple buckets Predicate indexing puts all the Knows facts in one bucket and all the Brother facts in anotherPredicate indexing puts all the Knows facts in one bucket and all the Brother facts in another –Might not be a win if there are lots of clauses for a particular predicate symbol  Consider how many people Know one another –Instead index by predicate and first argument  Clauses may be stored in multiple buckets

7. Subsumption lattice How to construct indices for all possible queries that unify with it Example: Employs (AIMA.org, Richard)Example: Employs (AIMA.org, Richard) How to construct indices for all possible queries that unify with it Example: Employs (AIMA.org, Richard)Example: Employs (AIMA.org, Richard)

8. Subsumption Lattice

9. Subsumption lattice Each node reflects making one substitutionEach node reflects making one substitution The “highest” common descendent of any two nodes is the result of applying the most general unifierThe “highest” common descendent of any two nodes is the result of applying the most general unifier Predicate with n arguments will create a lattice with O(2 n ) nodesPredicate with n arguments will create a lattice with O(2 n ) nodes Benefits of indexing may be outweighed by cost of storing and maintaining indicesBenefits of indexing may be outweighed by cost of storing and maintaining indices Each node reflects making one substitutionEach node reflects making one substitution The “highest” common descendent of any two nodes is the result of applying the most general unifierThe “highest” common descendent of any two nodes is the result of applying the most general unifier Predicate with n arguments will create a lattice with O(2 n ) nodesPredicate with n arguments will create a lattice with O(2 n ) nodes Benefits of indexing may be outweighed by cost of storing and maintaining indicesBenefits of indexing may be outweighed by cost of storing and maintaining indices

10. Forward Chaining Remember this from propositional logic? Start with atomic sentences in KBStart with atomic sentences in KB Apply Modus PonensApply Modus Ponens –add new sentences to KB –discontinue when no new sentences Hopefully find the sentence you are looking for in the generated sentencesHopefully find the sentence you are looking for in the generated sentences Remember this from propositional logic? Start with atomic sentences in KBStart with atomic sentences in KB Apply Modus PonensApply Modus Ponens –add new sentences to KB –discontinue when no new sentences Hopefully find the sentence you are looking for in the generated sentencesHopefully find the sentence you are looking for in the generated sentences

11. Lifting forward chaining First-order definite clauses –disjunction of literals with exactly one positive or –implication whose antecedent is a conjunction of positive literals and whose consequent is a single positive literal –all sentences are defined this way to simplify processing First-order definite clauses –disjunction of literals with exactly one positive or –implication whose antecedent is a conjunction of positive literals and whose consequent is a single positive literal –all sentences are defined this way to simplify processing Universal quantification assumed

12. Example The law says it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is AmericanThe law says it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American We will prove West is a criminalWe will prove West is a criminal The law says it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is AmericanThe law says it is a crime for an American to sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American We will prove West is a criminalWe will prove West is a criminal

13. Example It is a crime for an American to sell weapons to hostile nationsIt is a crime for an American to sell weapons to hostile nations Nono… has some misslesNono… has some missles –Owns (Nono, M1) –Missile (M1) All of its missiles were sold to it by Colonel WestAll of its missiles were sold to it by Colonel West It is a crime for an American to sell weapons to hostile nationsIt is a crime for an American to sell weapons to hostile nations Nono… has some misslesNono… has some missles –Owns (Nono, M1) –Missile (M1) All of its missiles were sold to it by Colonel WestAll of its missiles were sold to it by Colonel West

14. Example We also need to know that missiles are weaponsWe also need to know that missiles are weapons and we must know that an enemy of America counts as “hostile”and we must know that an enemy of America counts as “hostile” “West, who is American”“West, who is American” The country Nono, an enemy of AmericaThe country Nono, an enemy of America We also need to know that missiles are weaponsWe also need to know that missiles are weapons and we must know that an enemy of America counts as “hostile”and we must know that an enemy of America counts as “hostile” “West, who is American”“West, who is American” The country Nono, an enemy of AmericaThe country Nono, an enemy of America

15. Forward-chaining Starting from the facts find all rules with satisfied premisesfind all rules with satisfied premises add their conclusions to known factsadd their conclusions to known facts repeat untilrepeat until –query is answered –no new facts are added Starting from the facts find all rules with satisfied premisesfind all rules with satisfied premises add their conclusions to known factsadd their conclusions to known facts repeat untilrepeat until –query is answered –no new facts are added

16. First iteration of forward chaining Look at the implication sentences first must satisfy unknown premisesmust satisfy unknown premises We can satisfy this ruleWe can satisfy this rule –by substituting {x/M1} –and adding Sells(West, M1, Nono) to KB Look at the implication sentences first must satisfy unknown premisesmust satisfy unknown premises We can satisfy this ruleWe can satisfy this rule –by substituting {x/M1} –and adding Sells(West, M1, Nono) to KB

17. First iteration of forward chaining We can satisfyWe can satisfy –with {x/M1} –and Weapon (M1) is added We can satisfyWe can satisfy –with {x/Nono} –and Hostile {Nono} is added We can satisfyWe can satisfy –with {x/M1} –and Weapon (M1) is added We can satisfyWe can satisfy –with {x/Nono} –and Hostile {Nono} is added

18. Second iteration of forward chaining We can satisfyWe can satisfy –with {x/West, y/M1, z/Nono} –and Criminal (West) is added We can satisfyWe can satisfy –with {x/West, y/M1, z/Nono} –and Criminal (West) is added

19. Analyze this algorithm Sound? Does it only derive sentences that are entailed?Does it only derive sentences that are entailed? –Yes, because only Modus Ponens is used and it is sound Complete? Does it answer every query whose answers are entailed by the KB?Does it answer every query whose answers are entailed by the KB? –Yes, if the clauses are definite clause Semi-decidable? Algorithm returns “yes” for every entailed sentence, but does not return “no” for every nonentailed sentenceAlgorithm returns “yes” for every entailed sentence, but does not return “no” for every nonentailed sentenceSound? Does it only derive sentences that are entailed?Does it only derive sentences that are entailed? –Yes, because only Modus Ponens is used and it is sound Complete? Does it answer every query whose answers are entailed by the KB?Does it answer every query whose answers are entailed by the KB? –Yes, if the clauses are definite clause Semi-decidable? Algorithm returns “yes” for every entailed sentence, but does not return “no” for every nonentailed sentenceAlgorithm returns “yes” for every entailed sentence, but does not return “no” for every nonentailed sentence NatNum(0) NatNum(n)  NatNum(S(n)) NatNum(S(0)), NatNum(S(S(0))),…

20. Proving completeness Assume KB only has sentences with no function symbols What’s the most number of iterations through algorithm?What’s the most number of iterations through algorithm? Depends on the number of facts that can be addedDepends on the number of facts that can be added –Let k be the arity, the max number of arguments of any predicate and –Let p be the number of predicates –Let n be the number of constant symbols At most pn k distinct ground factsAt most pn k distinct ground facts Fixed point is reached after this many iterationsFixed point is reached after this many iterations A proof by contradiction shows that the final KB is completeA proof by contradiction shows that the final KB is complete Assume KB only has sentences with no function symbols What’s the most number of iterations through algorithm?What’s the most number of iterations through algorithm? Depends on the number of facts that can be addedDepends on the number of facts that can be added –Let k be the arity, the max number of arguments of any predicate and –Let p be the number of predicates –Let n be the number of constant symbols At most pn k distinct ground factsAt most pn k distinct ground facts Fixed point is reached after this many iterationsFixed point is reached after this many iterations A proof by contradiction shows that the final KB is completeA proof by contradiction shows that the final KB is complete

21. Complexity of this algorithm Three sources of complexity inner loop requires finding all unifiers such that premise of rule unifies with facts of databaseinner loop requires finding all unifiers such that premise of rule unifies with facts of database –this “pattern matching” is expensive must check every rule on every iteration to check if its premises are satisfiedmust check every rule on every iteration to check if its premises are satisfied many facts are generated that are irrelevant to goalmany facts are generated that are irrelevant to goal Three sources of complexity inner loop requires finding all unifiers such that premise of rule unifies with facts of databaseinner loop requires finding all unifiers such that premise of rule unifies with facts of database –this “pattern matching” is expensive must check every rule on every iteration to check if its premises are satisfiedmust check every rule on every iteration to check if its premises are satisfied many facts are generated that are irrelevant to goalmany facts are generated that are irrelevant to goal

22. Pattern matching Conjunct ordering Missile (x) ^ Owns (Nono, x) => Sells (West, x, Nono)Missile (x) ^ Owns (Nono, x) => Sells (West, x, Nono) –Look at all items owned by Nono, call them X –for each element x in X, check if it is a missile –Look for all missiles, call them X –for each element x in X, check if it is owned by Nono Optimal ordering is NP-hard, similar to matrix mult Good heuristic: group first according to whichever group is smallerGood heuristic: group first according to whichever group is smaller Conjunct ordering Missile (x) ^ Owns (Nono, x) => Sells (West, x, Nono)Missile (x) ^ Owns (Nono, x) => Sells (West, x, Nono) –Look at all items owned by Nono, call them X –for each element x in X, check if it is a missile –Look for all missiles, call them X –for each element x in X, check if it is owned by Nono Optimal ordering is NP-hard, similar to matrix mult Good heuristic: group first according to whichever group is smallerGood heuristic: group first according to whichever group is smaller

23. Incremental forward chaining Pointless (redundant) repetition Some rules generate new informationSome rules generate new information –this information may permit unification of existing rules some rules generate preexisting informationsome rules generate preexisting information –we need not revisit the unification of the existing rules Every new fact inferred on iteration t must be derived from at least one new fact inferred on iteration t-1 Pointless (redundant) repetition Some rules generate new informationSome rules generate new information –this information may permit unification of existing rules some rules generate preexisting informationsome rules generate preexisting information –we need not revisit the unification of the existing rules Every new fact inferred on iteration t must be derived from at least one new fact inferred on iteration t-1

24. Irrelevant facts Some facts are irrelevant and occupy computation of forward-chaining algorithm What if Nono example included lots of facts about food preferences?What if Nono example included lots of facts about food preferences? –Not related to conclusions drawn about sale of weapons –How can we eliminate them? Some facts are irrelevant and occupy computation of forward-chaining algorithm What if Nono example included lots of facts about food preferences?What if Nono example included lots of facts about food preferences? –Not related to conclusions drawn about sale of weapons –How can we eliminate them?

25. Magic Set Rewriting the rule set We have learned earlier that backward chaining limits the search spaceWe have learned earlier that backward chaining limits the search space We’re still using forward chaining, but we will add elements to premises of sentences that further restrict the forward inferences we will drawWe’re still using forward chaining, but we will add elements to premises of sentences that further restrict the forward inferences we will draw –added elements are based on goal (thus the backward influence) Rewriting the rule set We have learned earlier that backward chaining limits the search spaceWe have learned earlier that backward chaining limits the search space We’re still using forward chaining, but we will add elements to premises of sentences that further restrict the forward inferences we will drawWe’re still using forward chaining, but we will add elements to premises of sentences that further restrict the forward inferences we will draw –added elements are based on goal (thus the backward influence)

26. Magic Set Rewriting the rule set Let goal = Criminal (West)Let goal = Criminal (West) –American(x) ^ Weapon(y) ^ Sells(x,y,z) ^ Hostile(z)  Criminal(x) We know we are concerned with criminals and WestWe know we are concerned with criminals and West Let’s distinguish West as important by placing it in a Magic set: Magic(West)Let’s distinguish West as important by placing it in a Magic set: Magic(West) Let’s rewrite the rule to constrain our forward chaining to only include sentences related to WestLet’s rewrite the rule to constrain our forward chaining to only include sentences related to West –Magic(x) ^ American(x) ^ Weapon(y) ^ Sells(x, y, z) ^ Hostile(z) => Criminal (x) Rewriting the rule set Let goal = Criminal (West)Let goal = Criminal (West) –American(x) ^ Weapon(y) ^ Sells(x,y,z) ^ Hostile(z)  Criminal(x) We know we are concerned with criminals and WestWe know we are concerned with criminals and West Let’s distinguish West as important by placing it in a Magic set: Magic(West)Let’s distinguish West as important by placing it in a Magic set: Magic(West) Let’s rewrite the rule to constrain our forward chaining to only include sentences related to WestLet’s rewrite the rule to constrain our forward chaining to only include sentences related to West –Magic(x) ^ American(x) ^ Weapon(y) ^ Sells(x, y, z) ^ Hostile(z) => Criminal (x)

27. Backward Chaining Start with the premises of the goal Each premise must be supported by KBEach premise must be supported by KB Start with first premise and look for support from KBStart with first premise and look for support from KB –looking for clauses with a head that matches premise –the head’s body (premise) must then be supported by KB –Place premises on a stack with the goal A recursive, depth-first, algorithm Suffers from repetition and incompletenessSuffers from repetition and incompleteness Start with the premises of the goal Each premise must be supported by KBEach premise must be supported by KB Start with first premise and look for support from KBStart with first premise and look for support from KB –looking for clauses with a head that matches premise –the head’s body (premise) must then be supported by KB –Place premises on a stack with the goal A recursive, depth-first, algorithm Suffers from repetition and incompletenessSuffers from repetition and incompleteness

28. Resolution We saw earlier that resolution is a complete algorithm for refuting statements Must put first-order sentences into conjunctive normal formMust put first-order sentences into conjunctive normal form –conjunction of clauses, each is a disjunction of literals  literals can contain variables (which are assumed to be universally quantified) We saw earlier that resolution is a complete algorithm for refuting statements Must put first-order sentences into conjunctive normal formMust put first-order sentences into conjunctive normal form –conjunction of clauses, each is a disjunction of literals  literals can contain variables (which are assumed to be universally quantified)

29. Resolution - aside Mathematical theorem provers Resolution has been used to prove mathematical theoremsResolution has been used to prove mathematical theorems Kurt Gödel proved incompleteness of logic systems containing induction (a building block of discrete math)Kurt Gödel proved incompleteness of logic systems containing induction (a building block of discrete math) –There are sentences entailed by a knowledge base, but no finite proof exists  That is, generating all possible sentences, when induction is one possible operator, will not derive all sentences that are entailed Mathematical theorem provers Resolution has been used to prove mathematical theoremsResolution has been used to prove mathematical theorems Kurt Gödel proved incompleteness of logic systems containing induction (a building block of discrete math)Kurt Gödel proved incompleteness of logic systems containing induction (a building block of discrete math) –There are sentences entailed by a knowledge base, but no finite proof exists  That is, generating all possible sentences, when induction is one possible operator, will not derive all sentences that are entailed

30. First-order CNF For all x, American(x) ^ Weapon(y) ^ Sells(x, y, z) ^ Hostile (z) => Criminal(x)For all x, American(x) ^ Weapon(y) ^ Sells(x, y, z) ^ Hostile (z) => Criminal(x) ~American(x) V ~Weapon(y) V ~Sells(x, y, z) V ~Hostile(z) V Criminal(x)~American(x) V ~Weapon(y) V ~Sells(x, y, z) V ~Hostile(z) V Criminal(x) Every sentence of first-order logic can be converted into an inferentially equivalent CNF sentence (they are both unsatisfiable in same conditions) For all x, American(x) ^ Weapon(y) ^ Sells(x, y, z) ^ Hostile (z) => Criminal(x)For all x, American(x) ^ Weapon(y) ^ Sells(x, y, z) ^ Hostile (z) => Criminal(x) ~American(x) V ~Weapon(y) V ~Sells(x, y, z) V ~Hostile(z) V Criminal(x)~American(x) V ~Weapon(y) V ~Sells(x, y, z) V ~Hostile(z) V Criminal(x) Every sentence of first-order logic can be converted into an inferentially equivalent CNF sentence (they are both unsatisfiable in same conditions)

31. Example Everyone who loves all animals is loved by someone

32. Example

33. F and G are Skolem Functions arguments of function are universally quantified variables in whose scope the existential quantifier appearsarguments of function are universally quantified variables in whose scope the existential quantifier appears F and G are Skolem Functions arguments of function are universally quantified variables in whose scope the existential quantifier appearsarguments of function are universally quantified variables in whose scope the existential quantifier appears

34. Example Two clausesTwo clauses F(x) refers to the animal potentially unloved by xF(x) refers to the animal potentially unloved by x G(x) refers to someone who might love xG(x) refers to someone who might love x Two clausesTwo clauses F(x) refers to the animal potentially unloved by xF(x) refers to the animal potentially unloved by x G(x) refers to someone who might love xG(x) refers to someone who might love x

35. Resolution inference rule A lifted version of propositional resolution rule two clauses must be standardized aparttwo clauses must be standardized apart –no variables are shared can be resolved if their literals are complementarycan be resolved if their literals are complementary –one is the negation of the other –if one unifies with the negation of the other A lifted version of propositional resolution rule two clauses must be standardized aparttwo clauses must be standardized apart –no variables are shared can be resolved if their literals are complementarycan be resolved if their literals are complementary –one is the negation of the other –if one unifies with the negation of the other l i and m are complementary literals

36. Resolution l i and m j are complementary literals

37. Inference in first-order logic Our goal is to prove that KB entails a fact,  We use logical inferenceWe use logical inference  Forward chaining  Backward chaining  Resolution All three logical inference systems rely on search to find a sequence of actions that derive the empty clause Our goal is to prove that KB entails a fact,  We use logical inferenceWe use logical inference  Forward chaining  Backward chaining  Resolution All three logical inference systems rely on search to find a sequence of actions that derive the empty clause