1. Automating Common Sense Reasoning
Gopal Gupta
Elmer Salazar, Kyle Marple, Zhuo Chen, Farhad Shakerin
(and many others)
Department of Computer Science
The University of Texas at Dallas
Support from NSF is gratefully acknowledged

2. Artificial Intelligence
Intelligent reasoning by computers has been a goal of computer scientists ever since computers were first invented in the 1950s.
Intelligence has two broad components:
Acquiring knowledge (machine learning)
Applying knowledge that is known (automated reasoning)
+ Vision, Speech Processing, Speech Generation, etc., etc.
Our focus: automated reasoning
Reasoning is essential: machine learning algorithms learn rules that have to be employed for reasoning

3. Automated Reasoning
Automated reasoning has had its ups and down due to the difficulties involved in building deduction systems based on classical logic
The current AI excitement is driven by machine learning
To build effective AI systems, reasoning methods have to be simplified
Humans have simplified the burden of reasoning by using “common sense reasoning”
Use of probabilistic reasoning & fuzzy sets can be seen as other ways of simplifying reasoning
Classical logic as the basis for common sense reasoning has not worked due to its undecidability, incompleteness & monotonicity

4. Commonsense Reasoning (CSR)
Humans simplify the burden of reasoning by using
Defaults: E.g., Normally birds fly
Exceptions: penguins, ostriches, wounded birds
Commonsense reasoning also requires:
Nonmonotonicity: revise earlier conclusion in light of new information
allows us to jump to conclusions, but if contradictory information is discovered later, things don’t break down (as in classical logic)
jumping to conclusion == drawing conclusions in absence of info:
Can’t tell if it is cold outside. If I see no one wearing a jacket, it must be warm
You text your friend in the morning; He does not respond; normally he responds right away. You may conclude: he must be taking a shower.
Requirement: be able to reason with negation as failure

5. Classical Negation vs Negation as Failure
represented as –p
An explicit proof of falsehood of predicate p is needed
-robbed(sutton, bank_one) holds true only if there is an explicit proof of Willie Sutton not robbing Bank One
Negation as failure (NAF)
represented as not(p)
We try to prove proposition p, if we fail, we conclude not(p) is true
No evidence of p then conclude not(p)
not(robbed(sutton, bank_one)) holds true if we fail to find any proof that Willie Sutton robbed Bank_One
Answer Set Prog: paradigm that includes both classical negation and negation as failure

6. Answer Set Programming
Popular formalism for non monotonic reasoning
Rules of the form:
p :- a１ ,…, am, not b1 ,…, not bn. m≥0, n≥0 (rule)
p (fact)
Another reading: add p to the answer set if a１ ,…, am
are in the answer set and b1 ,…, bn are not
Applications to common sense reasoning, planning, constrained optimization, etc.
Semantics given via lfp of a residual program obtained after “Gelfond-Lifschitz” transform
Popular implementations: Smodels, DLV, CLASP, etc.
Almost 30 years of research invested

7. Negation as Failure Humans use negation as failure all the time.
Hard (for humans) to deal with nested negation though
Who all will go to Mexico?
Code this as: p :- not s.
s :- not r.
r :- not p.
r :- not s.
What is the semantics of this program?
Individual rules easy to understand; extremely hard to understand what the program means as a whole
Paul will go to Mexico if Sally will not go to Mexico
Sally will go to Mexico if Rob will not go to Mexico.
Rob will go to Mexico if Paul will not go to Mexico.
Rob will go to Mexico if Sally will not go to Mexico.

8. Answer Set Programming
Answer set programming (ASP)
Incorporates circular reasoning
Based on Possible Worlds and Stable Model Semantics
Given an answer set program, find its models
Model: assignment of true/false value to propositions to make all formulas true. Models are called answer sets
Captures default reasoning, non-monotonic reasoning, constrained optimization, exceptions, weak exceptions, preferences, etc., in a natural way
“Better way to build automated reasoning systems”
Caveats
p ⇐ a, b. really is taken to be p ⇔ a, b.
We are only interested in supported models: if p is in the answer set, it must be in the LHS of a ‘successful’ rule

9. ASP
Given an answer set program, we want to find out the knowledge (propositions) that it entails (answer sets)
For example, given the program:
p :- not q q :- not p.
the possible answer sets are:
1. {p} i.e., p = true, q = false
2 2. {q} i.e, q = true, p = false
Computed via Gelfond-Lifschitz Method (Guess & Check)
Given an answer set S, for each p  S, delete all rules whose body contains “not p”;
delete all goals of the form “not q” in remaining rules
Compute the least fix point, L, of the residual program
If S = L, then S is an answer set
p = Tom teaches DB
q = Mary teaches DB
Two worlds:
Tom teaches DB, Mary does not
Mary teaches DB, Tom does not

10. Finding the Answer Set Consider the program: p :- not q. t. r :- t, s.
q :- not p, r s h :- p, not h.
Is {p, r, t, s} the answer set?
Apply the GL Method
-- If x in answer set, delete all rules with not x
in body
-- Next, remove all negated goals from the
remaining program
{p, r, t, s, h}
-- Find the LFP of the program:
-- Initial guess {p, r, t, s} ≠ {p, r, t, s, h}
so {p, r, t, s} is not a stable model.

11. Finding the Answer Set Consider the program: p :- not q. t. r :- t, s.
q :- not p, r s h :- p, not h. r.
Is {q, r, t, s} the answer set?
Apply the GL Method
-- If x in answer set, delete all rules with not x
in body
-- Next, remove all negated goals from the
remaining program
{q, r, t, s}
-- Find the LFP of the program:
-- Initial guess {q, r, t, s} = LFP
so {q, r, t, s} is a stable model.

12. ASP: Example Consider the following program, A:
p :- not q t r :- t, s.
q :- not p s.
A has 2 answer sets: {p, r, t, s} & {q, r, t, s}.
Now suppose we add the following rule to A:
h :- p, not h (falsify p)
Only one answer set remains: {q, r, t, s}
Consider another program:
p :- not s.
s :- not r.
r :- not p.
r :- not s.
What are the answer sets?
h :- p, not h.
Paul will go to MX if Sally will not go to MX
Sally will go to MX if Rob will not go to MX.
Rob will go to MX if Paul will not go to MX.
Rob will go to MX if Sally will not go to MX.
{p, r}

13. Constraints The rules that cause problem are of the form:
h :- p, q, r, not h that implicitly declare p to be false
ASP also allows rules of the form:
:- p, q, r.
which asserts that the conjunction of p, q, and r should be false.
The two are equivalent, except that in the first case, not h may be called indirectly:
h :- p, q, s.
s :- r, not h.
Constraints are responsible for nonmonotonic behavior
A rule of the form
p :- not p
wrecks the whole knowledge base

14. Slide from Son Tran of NMSU

15. Slide from Son Tran of NMSU

16. Defaults and Exceptions
Slide from Chita Baral (ASU)
Defaults and Exceptions
AS = {flies(tweety), flies(sam)}
AS = {flies(tweety))}
flies(sam) does not hold any more
Our reasoning is aggressive: if we don’t know anything about a bird, it can fly

17. Closed World Assumption
Slide from Chita Baral (ASU)
Closed World Assumption
CWA: if we fail to prove, then take it as a definite
proof of falsehood
We humans use CWA all the time
Make it explicit in ASP via the use of classical negation

18. Open World Assumption OWA == No CWA; But now we can be more selective:
Slide from Chita Baral (ASU)
Open World Assumption
OWA == No CWA; But now we can be more selective:
CWA for some and OWA for others.
CWA about bird, penguin, ab
OWA about flies

19. Slide from Chita Baral (ASU)
Open World Assumption
Next, remove CWA about bird, penguin, ab (assume our
information about these concepts is incomplete)
But, now that we no longer have CWA
about being a penguin, it’s possible that et
might be a penguin. We may choose to be
more conservative in our reasoning: declare
et abnormal only if we fail to prove that et
is not a penguin for sure, i.e. X flies if X is a
bird and not a penguin for sure (-penguin(X))
flies(et) will now fail, as I fail to
establish that et is not a penguin
Does et fly? Answer is YES
Does et fly now?:
Answer is NO

20. Slide from C. Baral (ASU)
University Admission
To put all this in perspective, consider the college admission process:
Since we have no information about John being special or –special,
both eligible(john) and –eligible(john) fail.
So John will have to be interviewed

21. Incomplete Information
Consider the course database:
By default professor P does not teach course C, if teach(P,C) is absent.
The exceptions are courses labeled “staff”. Thus:
¬ teach(P,C) :- not teach(P,C), not ab(P,C).
ab(P,C) :- teach(staff, C).
Queries
?- teach(mike, pl) and
?- ¬ teach(mike,pl)
will both fail: we really don’t know if mike teaches pl or not
Professor
Course
mike ai
sam db
staff pl
Represented as:
teach(mike,ai).
teach(sam,db).
teach(staff,pl).

22. Combinatorial Problems: Coloring
Slide from S. Tran (NMSU)

23. Combinatorial Problems: Coloring
Slide from S. Tran (NMSU)

24. Current ASP Systems
Very sophisticated and efficient ASP systems have been developed based on SAT solvers:
CLASP, DLV, Cmodels, Smodels
These systems work as follows:
Ground the programs w.r.t. the constants present in the program
Transform the propositional answer set programs into propositional formulas and then find their models using a SAT solver
The models found are the stable models of the original program
Because SAT solvers require formulas to be propositional, programs with only constants and variables are feasible
Problem: Current systems limited to programs that are finitely groundable (lists and structures are not allowed)

25. Current ASP Systems: Issues
Problem 1: Program has to be finitely groundable
Not possible to have lists, structures, and complex data structures
Not possible to have arithmetic over reals
Problem 2: Grounding can result in exponential blowup
Given the clause: p(X, a) :- q(Y, b, c).
It turns into 3 x 3, i.e., 9 clauses
p(a, a) :- q(a, b, c). p(a, a) :- q(b, b, c). p(a, a) :- q(c, b, c).
p(b, a) :- q(a, b, c). p(b, a) :- q(b, b, c). p(b, a) :- q(c, b, c).
p(c, a) :- q(a, b, c). p(c, a) :- q(b, b, c). p(c, a) :- q(c, b, c).
Imagine a large knowledgebase with1000 clauses with 100 variables and 100 constants;
Programmers have to contort themselves while writing ASP code
Programs cannot contain lists structures and complex data structures: result in infinite-sized grounded program

26. Current ASP Systems: Issues
Problem 3: SAT solvers find the entire model
Entire model may contain lot of unnecessary information
I want to know the path from Boston to NYC, the model will contain all possible paths from every city to every other city (overkill)
Problem 4: Some times it may not even be possible to find the answer sought, as they are hidden in the answer set
Answer set of Tower of Hanoi program contains large # of moves
The moves that constitute the actual answer hard to identify
Problem 5: Minor inconsistency in the answer set will result in the system declaring that there are is no answer set
We want to compute an answer if it steers clear of the inconsistent part of the knowledge base
Impossible to have a large knowledge base that is 100% consistent

27. Why Goal-directed ASP?
Most of the time, given a theory, we are interested in knowing if a particular goal is true or not.
Top down goal-directed execution provides operational semantics (important for usability)
Most practical examples add a constraint to force the answer set to contain a certain goal.
E.g. Zebra puzzle: :- not satisfied.
Answer sets of non-finitely groundable programs computable
Constraints (CLP(R), CLP(FD)) incorporated in the style of current LP systems
Why check the consistency of the whole KB?
Inconsistency in some unrelated part will scuttle us

28. Solution
Develop goal-directed answer set programming systems that support predicates
Goal-directed means that a query is given, and a proof for the query found by exploring the program search space
Essentially, we need Prolog style execution that supports stable model semantics-based negation
Thus, part of the knowledge base that is only relevant to the query is explored
Predicate answer set programs are directly executed without any grounding: lists and structures also supported
Realized in the s(ASP) system developed at UT Dallas
Predecessor: Galliwasp (top-down propositional ASP)

29. s(ASP) System s(ASP) With s(ASP):
Prolog extended with negation based on stable-model semantics
general predicates allowed;
goal-directed execution: user writes ASP code, issues queries
With s(ASP):
Problem #1 & #2 (grounding and explosion) are irrelevant as there is no grounding
Problem #3: only the partial model to answer the query is computed
Problem #4: answer is easily discernible due to query driven nature
Problem #5: consistency checks can be performed incrementally so that if the knowledge base is inconsistent, consistent part of the knowledge base is still useful
Available on sourceforge; includes justification & abduction
s(CASP): incorporates constraints, tabling, DCC

30. Goal-directed execution of ASP
Key concept for realizing goal-directed execution of ASP:
coinductive execution
Coinduction: dual of induction
computes elements of the GFP
In the process of proving p, if p appears again, then p succeeds
Given:
there are two possibilities:
Both jack and jill eat (GFP)
Neither one eats (LFP)
eats_food(jack) :- eats_food(jill).
eats_food(jill) :- eats_food(jack).
?- eats_food(jack)
eats_food(jill).
eats_food(jack)
coinductive success
coinductive hypothesis set = {eats_food(jack), eats_food(jill)}

31. Goal-directed execution of ASP
To incorporate negation in coinductive reasoning, we need a negative coinductive hypothesis rule:
In the process of establishing not(p), if not(p) is seen again in the resolvent, then not(p) succeeds
In the process of establishing p, if not(p) is seen, computation fails & backtracking occurs
Also, not not p reduces to p.

32. Goal-directed execution of ASP
Distincition between even loops and odd loops:
Even loop: p recursive calls itself with even number of intervening negations between the two
Odd loop: ….. odd number …… (OLON rules; constraints)
Intuition: even loops generate worlds, odd loops kill worlds
Even-loops succeed by coinduction
p :- not q.
q :- not p.
?- p  not q  not not p  p (coinductive success)
The coinductive hypothesis set is the answer set: {p, not q}
For each constraint (OLON) rule, we extend the query
Given a query Q for a program that contains a constraint rule
p :- q, not p (q ought to be false)
extend the query to: ?- Q, (p ∨ not q)

33. Goal-directed ASP Consider the following program, A:
p :- not q t r :- t, s.
q :- not p s.
A has 2 answer sets: {p, r, t, s} & {q, r, t, s}.
Now suppose we add the following rule to A:
h :- p, not h (falsify p)
Only one answer set remains: {q, r, t, s}
Recall:
Even loops over negation produce possible worlds (generate)
Constraints or odd loops over negation remove them (test)

34. Goal-directed ASP Consider the following program, A’:
p :- not q t r :- t, s.
q :- not p, r s h :- p, not h.
Separate into constraint and non-constraint rules: only 1 constraint rule in this case.
Execute the query under co-LP, candidate answer sets will be generated.
Keep the ones not rejected by the constraints.
Suppose the query is ?- q. Execution: q  not p, r  not not q, r  q, r  r  t, s  s  success Ans = {q, r, t, s}
Next, we need to check that constraint rules will not reject the generated answer set: (not p ; h)
(it doesn’t in this case)

35. Augmenting the Query
In general, for the constraint rules of p as head, p1:- B1. p2:- B pn :- Bn., generate rule(s) of the form:
chk_p1 :- not(p1), B1.
chk_p2 :- not(p2), B2.
...
chk_pn :- not(p), Bn.
Generate: nmr_chk :- not(chk_p1), ... , not(chk_pn).
For each pred. definition, generate the dual:
not_p :- not(B1), not(B2), ... , not(Bn).
If you want to ask query Q, then ask ?- Q, nmr_chk.
Execution keeps track of atoms in the answer set (PCHS) and atoms not in the answer set (NCHS).
not(chk_p1) == p1 \/ not B1

36. Goal-directed ASP Consider the following program, P1:
(i) p :- not q (ii) q:- not r (iii) r :- not p. (iv) q :- not p.
P1 has 1 answer set: {q, r}.
Separate into: 3 constraint rules (i, ii, iii)
2 non-constraint rules (i, iv).
p :- not(q). q :- not(r). r :- not(p). q :- not(p).
chk_p :- not(p), not(q). chk_q :- not(q), not(r). chk_r :- not(r), not(p). nmr_chk :- not(chk_p), not(chk_q), not(chk_r).
not_p :- q. not_q :- r, p. not_r :- p.
Suppose the query is ?- r.
Expand as in co-LP: r  not p  not not q  q (  not r  fail, backtrack)  not p  success. Ans={r, q} which satisfies the constraint rules of nmr_chk.

37. Implementing Predicate ASP Systems
Aside from coinduction many other innovations needed to realize the s(ASP) system:
Dual rules to handle negation
Constructive negation support (domains are infinite)
Universally Quantified Vars (due to negation & duals)
A special infinite Herbrand Universe
s(ASP): Essentially, Prolog with stable model-based negation
Has been used for implementing non-trivial applications:
Check if an undergraduate student can graduate at a US university
Physician advisory system to manage chronic heart failure
Representing cell biology knowledge as an ASP program

38. Predicate ASP Systems: Challenges
Constructive negation:
For each variable carry the values it cannot be bound to
X \= a unifies with Y \= b: result X \= {a, b}
Disunification of such variables is complex (not allowed)
How to detect failure when a variable is constrained against every element of the universe?
Dual rules to handle negation
Need to handle existentially quantified variables p(X) :- q(X, Y).
Programs such as:
p(X) :- not q(X).
q(X) :- not p(X).
produce uncountable number of models
Computing Stable Models of Normal Logic Programs without
Grounding. Marple, Salazar, Gupta, UT Dallas Tech Rep.
s(ASP) available on sourceforge

39. Dual Rules Negation is handled using dual rules
The propositional case is simple: given the rules:
p :- not q.
p :- r, not s.
Rule for not p is computed (completion)
not p :- np1, np2.
np1 :- q.
np2 :- not r.
np2 :- s.
Whenever not p is called, dual rule executed
Each goal gets its own dual clause
p :- not q; (r, not s)

40. Dual Rules (cont’d)
Predicates complicate things because they can create bindings; Given the rules:
p(X) :- not q(X).
p(X) :- r(X), not s(X).
The call r(X) may modify X before not s(X) is called
The dual must account for this; the solution is to include the previous goals when computing the dual clause for a given goal:
not p(X) :- np1(X), np2(X).
np1(X) :- q(X).
np2(X) :- not r(X).
np2(X) :- r(X), s(X).
Nice innovation by Kyle & Elmer (tabling will help with efficiency, hence)

41. Universally Quantified Vars
Variables in a clause also have implicit quantifiers
Dual rules must handle these correctly. The rule:
p(X) :- not q(X, Y).
is equivalent to the formula:
(p(X) ← Y ¬q(X,Y ))
The dual of this formula is:
(¬p(X) ← Y q(X,Y ))
Thus, the dual clause has a universally quantified var in its body
Problem: all “body variables” are existentially quantified XA E XA A

42. Forall Mechanism the dual will be:
Our solution was to create a “forall” mechanism
Given the rule:
p(X) :- not q(X, Y).
the dual will be:
not p(X) :- forall(Y, np1(X, Y)).
np1(X, Y) :- q(X, Y).
To execute the forall, the goal np1 is called normally and the forall variable examined:
If the variable is unbound, the forall succeeds
If the variable is bound, failure and backtracking occur
If the variable is negatively constrained, the goal is executed for each constrained value, substituting the value for the variable
The forall will only succeed if the goal succeeds for all values of the variable

43. Special s(ASP) Universe
What would happen if a variable were constrained against every element of its domain?
To prevent this situation, s(ASP) uses a special Herbrand Universe (HU) that is an infinite superset of the HU
There will always be elements in the s(ASP) universe against which a variable cannot be constrained
These elements are purely theoretical
If a variable is now constrained against every element of the Herbrand Universe, s(ASP) can succeed using these elements
These new solutions will never be produced (and are not inconsistent with the result) from a grounded program
They represent different, equally correct information

44. s(ASP) Universe
AS = {d(1), not p(1)}

45. Applications Developed with s(ASP)
Grad audit system: figure out if an undergraduate student at a university can graduate
Complex requirements expressed as ASP rules
100s of courses that students could potentially choose from
Lists and structures come in handy in keeping the program compact
Developed over a few weeks by 2 undergraduates
Catalog changes frequently: ASP rules easy to update

46. Applications Developed
Physician advisory system: advises doctors for managing chronic heart failure in patients
Automates American College of Cardiology guidelines
Complex guidelines expressed as rules (60 odd rules in 80 pages)
Knowledge patterns developed to facilitate the modeling
Tested with UT Southwestern medical center;
found diagnosis that was missed by the doctor
Representing high school cell biology knowledge:
represented using ASP
Questions can be posed as ASP queries to the system
Natural lang. Q&A systems (make use of defaults)
Recommendation systems – birthday gift advisor

47. Recommendation Systems
Recommendation systems in reality need sophisticated modeling of human behavior
Current systems based on machine learning, and are not too precise, in our opinion
Birthday gift advisor:
model the generosity, wealth level of the person
model the interests of the person receiving the gift
use the information to determine the appropriate gift for one of your friends
very complex dynamics go in determining what gift you will get a person
In general, any type of recommendation system can be similarly built.

48. Intelligent IoT
Intelligent systems that take as input sensor signals and determine the action based on some logic can be developed
The situation where we take an action in the absence of a signal arises frequently
Example: intelligent lighting system
turn_on_light :- motion_detected, door_moved.
turn_on_light :- light_is_on, motion_detected.
turn_on_light :- light_is_on, not(motion_detected), not(door_moved).
-turn_on_light :- not(motion_detected), door_moved.

49. s(ASP) Hackathon
2 weeks of teaching ASP; 150 signed up; 18 projects

50. Conclusions
ASP: a comprehensive methodology for developing intelligent applications; common sense reasoning easily simulated
Incorporates both circular and well-founded reasoning
SAT-based implementations are fast, but face problems wrt building general purpose, large scale applications
s(ASP) adds stable model-based negation to Prolog, but now inherits all of Prolog’s problems (inefficient search mechanism)
Challenges: make efficiency competitive with SAT-based impl.
Integrate CLP(FD)
Integrate CLP(R)
Add tabled execution
Design an abstract machine; efficient (or-parallel) implementation
Develop more large-scale applications
Work by visiting PhD student Joaquin Arias from IMDEA, Spain. Available as s(CASP) system on github

51. Conclusions
Knowledge expressed as an answer set program captures the way humans think really well
One could learn answer set programs from examples and come closer to the human learning process
extension of inductive logic programming;
results produced are excellent;
Ongoing work:
Shakerin, Salazar, Gupta; A new algorithm to automate inductive learning of default theories; Theory and Practice of LP, 2017
Shakerin, Gupta: Learning Answer Set Programs with Multiple Stable Models; work in progress

52. THANK YOU QUESTIONS?