Lakatos-style Methods in Automated Reasoning Alison Pease University of Edinburgh Simon Colton Imperial College, London

Spin Mathematics textbooks are neat But mathematical research is scruffy After abortive attempts to prove conjecture Option 1: show independent of axioms E.g., parallel postulate, axiom of choice Option 2: modify conjecture in order to prove Often in the light of a counterexample E.g., Euler’s theorem, even perfect numbers

Spin 2 Automated theorem provers Not particularly robust or flexible Usually expect to be given a true theorem Always expect theorem to be as intended Very simple example: Child asks ATP: “Are all primes odd?” ATP says “no” A more flexible ATP would say: “If you mean all primes except 2 are odd, then yes”

Proofs and Refutations Imre Lakatos’ famous book Gives methods for fixing faulty theorems Uses Euler’s theorem as a running example In polyhedra, V – F + E = 2 Has a social setting Imagined situation of a teacher and class Respond to new counterexamples encountered Calls into question what we mean by polyhedra

PhD Project of Alison Pease To implement a model of reasoning Based on the notions in proofs and refutations Has a social aspect Implemented as a multi-agent system Has a reasoning aspect How to fix faulty conjectures Aims: Cognitive modelling Philosophical look at creativity Enhance automated theory formation Suggest possible applications to AI techniques

Automated Theory Formation in the HR System Cycle of invention, induction & deduction: Form concepts, make conjectures relating the concepts, prove conjectures, assess concepts Uses generic & specific concept production rules Conjectures are made empirically Uses a heuristic search New concepts are built from best old ones Concrete measures of interestingness Interacts with third party mathematical software Otter, MACE, Maple, Gap, MathWeb, SystemOnTPTP

Enhanced Theory Formation In other domains of science: An 80% conjecture is really quite interesting (just plain wrong in mathematics) HR now makes near conjectures Near equivalences: A  B Near implications: A  B User sets percentage threshold Includes both negative and positive for concept Working on positives only

Lakatos-Enhanced Automated Theory Formation Cycle: invent, induce, repair, prove Additional Implementation Progress measured on two axes Lakatos-inspired methods What to do in the event of a counterexample Sophistication of the Agency Teacher and student communication and actions

Lakatos-inspired Methods #1 Surrender Too many counterexamples to continue Until recently, only option for HR Important to know which ones to surrender Number and importance (size?) of examples which are counterexamples to conjecture considered “Small” counterexamples may be OK

Lakatos-inspired Methods #2 Concept barring Given near-equivalence P  Q If all counterexamples, C, are positive for P Then find concept X which exactly covers C Negate X and force the concept P  ¬ X This produces the exact conjecture: P  ¬ X  Q If some counterexamples are positive for P and Q Find concept X covering positive counters for P Find concept Y covering positive counters for Q Force production of concepts (i) P  ¬ X (ii) Q  ¬ Y This leads to conjecture P  ¬ X  Q  ¬ Y Very similar for near-implications Working on more flexibility in finding concepts

Lakatos-inspired Methods #3 Counterexample Barring for P  Q If an agent cannot find a concept to cover the (small number of) counters Then use entity_disjunct production rule Suppose [a,b] are counters to conjecture Invent concept X = n for which n=a  n=b Use negate to invent concept P  ¬ X Leads to the conjecture P  ¬ X  Q Split near-equivalences into two near-imps

Lakatos-inspired Methods #4 Strategic Withdrawal for P  Q If all counters are positives for P Find a concept X which covers most of the positives for both P and Q (different from Q) Make conjectures X  Q; X  Q; X  P If some counters positive for P, others for Q Find a concept Z which covers most of the examples which are positive for both Make conjectures Z  P; Z  Q

Other Lakatos Methods Not implemented yet: Monster adjusting Counterexample is re-interpreted Domains like number theory not good for this Lemma incorporation Looking at steps in proofs to identify problem Proofs and refutations Use proofs to suggest counterexamples Fix them using lemma incorporation

Levels of Agent Sophistication A single teacher, multiple student agents All have copies of HR, but different objects of interest Communicate via sockets Build up a global theory via local theories Characterisation of agency Problem: model social process of fixing conjs Knowledge: background to each copy of HR Motivation: to accept/reject/modify conjs Actions: Lakatos methods Communication: sending concepts,conjectures,examples Discussion: directed by teacher keeping group agenda

Complexity of Interactions Axis of complexity Max user input Puppet show Teacher uses students as references Dictatorship – teacher sets problems, specifies methods Students make decisions on method to apply Teacher still decides which conjectures to look at Students decide group agenda and global theory Democracy via voting Students allowed class discussion Can ignore the teacher, send requests themselves, etc. CRC Current?

Illustrative Example (Concept Barring) Three students and teacher Integers: 1-10; 11-50; 51-60; Divisors, multiplication; Forced: squares, even num divisors Teacher asks students for best equivalence conjecture Third student supplied this: n is an integer  has even number of divisors Has no counterexamples between 51 and 60 Other students supply counters: 1,4,9,16,25,… And find concept of squares to cover these Formed concept of non-squares Fixed conjecture to be: n is non-square  is has an even number of divisors

Illustrative Example (Counterexample Barring) Two sessions with two students Session 1: All primes except two are odd Session 2: User forced the concept of Integers which are the sum of two primes One agent said this was true of all integers Conjecture was fixed to: All integers except 2 are the sum of two primes Goldbach’s conjecture Done in anger: see ECAI’02 workshop paper

Illustrative Example (Strategic Withdrawal) Two students given integers 1 to 10 Less that or equal to; divisors; digit; *; +; Forced: prime numbers, odd numbers Asked to make near-equivs up to 60% First student: n is prime  n is odd Second student: 2 (prime, not odd), 1, 9 Found concept for 3, 5, 7 (pos for both sides) Odd non-squares Modified conjectures: N is odd  n is non-square  n is odd [tautology] N is odd  n is non-square  n is prime [false] Surrender probably better here!!

Conclusions Reparation techniques may be answer to: Lack of robustness in ATP (notion of noise) Model given by Lakatos is social in nature Partial model implemented Consideration one: reparation methods Consideration two: agency sophistication Lakatos-enhanced theory formation Advantages over normal theory formation

Further Work Improve along the two axes of development Get examples during autonomous sessions Evaluate the system Does it model Lakatos’ ideas? Does it improve theory formation? Richer set of conjectures/concepts: is this an improvement? Apply the techniques Machine learning: fix faulty learned hypotheses Automated reasoning: fix and prove near-theorems