Predicate Calculus Syntax Countable set of predicate symbols, each with specified arity  0. For example: clinical data with multiple tables of patient information. Each such table can be a predicate and each record (tuple) is an atomic formula. Table – Patient patient(1,3-3-03,f) – a predicate with arity = 3. IDDOBGender f --

Further Notation…. Variables – start with upper-case letter Predicate/atomic symbols – start with lower-case letter Constant – start with lower-case letter Logical connectives: , , , , ,  We will also use negation by failure (please see Skolemization slides) Limiting ourselves to atomic formulas and (if -then) rules (clauses) Learn Non-recursive rules – predicate in consequent cannot appear in antecedent Can have recursive rules elsewhere (background knowledge)

Examples… 1. p(a,b,c)^p(d,e,f) true if both atomic formulas are true 2. p(a,b,c)  p(d,e,f) true if first atomic formula is false and/or second atomic formula is true 3.  X  Y  Z (q(X,Y)^q(Y,Z)  p(X,Z)) non-recursive rule 4. If a and b are formulas then so are  a, a  b, a  b, a  b, a  b 5. If X is a variable and a is a formula then  Xa and  Xa are formulas. We say that X is quantified in the formulas  Xa and  Xa.

Some Notes Predicates of arity 0 are also called propositions, the only atomic formulas allowed in propositional logic An expression is an atomic formula A sentence is any formula in which all variables are quantified

Skolemization Process is applied to one sentence at a time and applied only to the entire sentence (so outermost quantifier first). Each sentence initially has empty vector of free variables. Replace  X A(X) with A(X), and add X to vector of free variables. Replace  X A(X) with A(x(V)) where x is a new function symbol and V is the current vector of free variables. If no function symbols, don’t need second bullet, and x is just a constant (nullary function) in third bullet

Learning … To learn, summarizing all information (eg. clinical data) and coming up with ,000 features could be one approach; however, we lose information in this process. Another option is a join, but the frequency of subjects in database alters the algorithm results. Inductive logic programming (ILP) can handle these issues by using predicate calculus to represent the database, and by learning rules.

Inductive Logic Programming: The Problem Specification Given: – Examples: first-order atomic formulas (atoms), each labeled positive or negative. – Background knowledge: definite clause (if-then rules) theory. – Language bias: constraints on the form of interesting new rules (clauses).

ILP Specification (Continued) Find: – A hypothesis h that meets the language constraints and that, when conjoined with B, implies (lets us prove) all of the positive examples but none of the negative examples. To handle real-world issues such as noise, we often relax the requirements, so that h need only entail significantly more positive examples than negative examples.

A Common Approach Use a greedy covering algorithm. – Repeat while some positive examples remain uncovered (not entailed): Find a good clause (one that covers as many positive examples as possible but no/few negatives). Add that clause to the current theory, and remove the positive examples that it covers. ILP algorithms use this approach but vary in their method for finding a good clause.

Scoring function MDL – minimum description length Score(R) = Positive R - Negative R -A (# atomic formulas) FOIL – uses information gain as scoring function. Example Labelpredicate +mother(john,sue) +mother(jane,sue) +mother(joe,jane) -mother(sue,jane) - mother(joe,john) -mother(fred,jane) - mother(jane,fred)

Background knowledge parent(john, fred) parent(john,sue) parent(jane,fred) parent(joel,jane) parent(joel,jack) parent(jane,sue) male(john) male(jack) male(fred) male(joel) female(sue) female(jane)

We want to learn predicate ‘mother’ with scoring function S_MDL = S(P,N,A) = P – N - A mother(X,Y) P N A S meaning S(3,4,2)=-2 mother(X,Y)  parent(X,Y) mother(X,Y)  female(X) mother(X,Y)  female(Y) P N A S P N A S P N A S Lattice can grow further by including substitution of variables but generalization is limited in this case and can be misleading. mother(X,sue) P N A S Now see if we can cover all +examples combination of clauses

Building Bottom clause (refinement) Start substituting variables in predicate to generate all ‘+ examples’ and write down atomic formulas from background knowledge which explains the predicate. For example: mother(X,Y)  X = john, Y = sue parent(X,Y)^ male(X)^female(Y) parent(X,Z), Z = fred etc Generate lattice to get these bottom clause for each ‘+ example’. Each example will provide a sub-lattice. Just search the sub-lattice above this bottom clause (only use atoms in this bottom clause).