1. A Theory of Theory Formation Simon Colton Universities of Edinburgh and York

2. Overview What is a theory? Four components of the theory of ATF – Techniques inside the components Cycles of theory formation – Case Studies Applications – Of both the theories and the process

3. What is a Theory? Theories are (minimally) a collection of: – Objects of interest – Concepts about the objects – Hypotheses relating the concepts – Explanations which prove the hypotheses Finite Group Theory: – All cyclic groups are Abelian Inorganic Chemistry: – Acid + Base  Salt + Water

4. So, We Require: Object Generator Concept Generator Hypothesis Generator Explanation Generator

5. In Principle, These Could Be: Database, CAS, CSP, Model Generator Machine Learning Program Data Mining Program ATP System, Pathway Finder, Visualisation

6. In Practice, Current Implementation: Database, Model Generator, (CAS, CSP nearly) The HR Program ATP Systems

7. Object Generation and Explanation Generation Object Generation: – Machine learning – reading a file, database In Mathematics – CSP (e.g., FINDER, Solver), CAS (e.g., Maple) – Davis Putnam method (e.g., MACE) – Resolution Theorem Proving (e.g., Otter) HR must be able to communicate – Read models and concepts from MACE’s output – Read proofs and statistics from Otter’s output

8. Concept Generation Build a new concept from old ones – 10 general production rules (demonstrated later) – Produce both a definition and examples Throw away concepts using definitions – Tidy definitions up – Repetitions, function conflict, negation conflict Decide which concepts to use for construction – Plethora of measures of interestingness – Weighted sum of measures

9. Concept Generation: Lakatos-inspired Techniques Monster Barring – Remove an object of interest from theory Counterexample Barring – Except a finite subset of objects from a theorem – E.g., all primes except 2 are odd Concept Barring – Except a concept from a theorem – All integers other than squares have an even number of divisors Credit to Alison Pease

10. Hypothesis Generation: Finding Empirical Relationships Equivalence conjectures – One concept has the same examples as another Subsumption conjectures – All examples of one concept are examples of other Non-existence conjectures – A concept has no examples Assessment of conjectures – Used to assess the concepts mentioned in them

11. Hypothesis Generation Extracting Prime Implicates Extract implications, then prime implicates Equivalence conjectures are split: – A & B & C  D & E & F becomes – A & B & C  D, A & B & C  E, etc. Non-existence conjectures are split: – ¬(A & B & C) becomes: A & B  ¬C, etc. Extract Prime implicates: – A & B & C  D, try A  D, then B  D, C  D, then A & B  D, etc.

12. Hypothesis Generation: Imperfect Conjectures User sets a percentage minimum, say 80% Near-subsumption conjectures – E.g., primes  odd (99% true) – Also returns the counterexamples: here, 2 Near-equivalence conjectures – Prime  odd (70% true) Applicability conjectures – A concept has a (small) finite number of examples – E.g., even prime numbers: 2 is only example

13. Cycles of Theory Formation How the individual techniques are employed Concept driven conjecture making – Finding conjectures to help understand concepts – Exploration techniques Conjecture driven concept formation – Inventing concepts to fix faulty conjectures – Imperfect conjectures, Lakatos techniques

14. Concept Driven Cycle (cut-down) Invent Concept EquivalenceNon ExistenceNew Concept Subsumptions Implications Reject

15. Concept Driven Cycle Continued Implications Counterexample Proof Prime Implicates CounterexampleProof

16. Conjecture Driven Cycle Invent ConceptReject Near EquivalenceApplicabilityNear Subsumption Concept Barring Counterex Barring New/Old Concept Equivalence Implications Monster Barring New Concept Counterex Barring Concept Barring New/Old Concept

17. Case Study: Groups Given: Group theory axioms

18. Case Study: Groups MACE model generator finds a model of size 1 Davis Putnam Method

19. Case Study: Groups Extracts concepts: Element, Multiplication, Identity, Inverse HR Reads MACE’s Output

20. Case Study: Groups Invents the concept: idempotent elements (a*a=a) Match Production Rule

21. Case Study: Groups Makes Conjecture: a*a=a  a is the identity element Equivalence Finding

22. Case Study: Groups Otter proves this in less than a second Resolution Theorem Proving

23. Case Study: Groups a*a = a  a=identity, a=identity  a*a=a End of cycle Extracts Prime Implicates

24. Case Study: Groups Later: Invents the concept of triples of elements (a,b,c) for which a*b=c & b*a=c Compose Production Rule

25. Case Study: Groups Invents concept of pairs (a,b) for which there exists an element c such that: a*b=c & b*a=c Exists Production Rule

26. Case Study: Groups Invents the concept of groups for which all pairs of elements have such a c: Abelian groups Forall Production Rule

27. Case Study: Groups Makes the Conjecture: G is a group if and only if it is Abelian Equivalence Finding

28. Case Study: Groups Otter fails to prove this conjecture Sorry

29. Case Study: Groups MACE finds a counterexample: Dihedral Group of size 6 (non-Abelian) Davis Putnam Method

30. Case Study: Groups Concept of Abelian groups allowed into theory Theory recalculated in light of new object of interest Assessment of Concepts

31. Case Study: Goldbach Given: Integers 1 to 100, Concepts: Divisors, Addition

32. Case Study: Goldbach Invents: Even Numbers (divisible by 2) Split Production Rule

33. Case Study: Goldbach Invents: Number of Divisors (tau function) Size Production Rule

34. Case Study: Goldbach Invents: Prime numbers (2 divisors) Split Production Rule

35. Case Study: Goldbach Half an hour later: Invents: Goldbach numbers (sum of 2 primes) Compose Production Rule

36. Case Study: Goldbach Conjectures: Even numbers are Goldbach numbers (with one exception, the number 2) Near Equivalence Finding

37. Case Study: Goldbach Forces: Concept of being the number 2 Counterexample Barring (Split)

38. Case Study: Goldbach Forces concept: Even numbers except 2 Counterexample Barring (Negate)

39. Case Study: Goldbach Conjectures: Even numbers except 2 are Goldbach Numbers (Goldbach’s Conjecture) Subsumption Finding

40. Case Study: Goldbach Passes the conjecture to an inductive theorem prover? Absolutely No Chance

41. Applications of Theories #1 Puzzle generation – Which is the odd one out: 4, 9, 18, 36? – Which is the odd one out: cat, platypus, bat, dolphin* – What’s next in the sequence: 21, 22, 24, 25, 26, 28? – See AISB’02 paper Problem generation – Find theorem to differentiate Spass, Bliksem & E – 12,000 theorems generated (at least one for each) – 200 problems in TPTP library (used at CASC’02) – See AI+Maths paper

42. Applications of Theories #2 Prediction tests: (e.g., Progol animals file) – P(mammal | has_milk) = 1.0 – P(mammal | habitat(water)) = 0.125 – Take average over all Bayesian probabilities – Moral Reasoner Dataset (200 people, 24 atts) – HR gets 91% accuracy Preliminary work, more to come

43. Applications of Theory Formation #1 Identifying concepts – Forward look ahead mechanism – Michalski trains, number sequences, – 1, 9, 25, ? (finds odd, finds square, knows to combine) – see ICML-00 paper Simplifying problems – Constraint generation for CSP – E.g., QG3 quasigroups: (a*b)*(b*a)=a – These are anti-Abelian and a*a = b  b*b=a – See CP-01 paper – Lemma generation for ATP (few positive results yet)

44. Applications of Theory Formation #2 Making conjectures – About Maple (CAS) functions – Discard any that a prover can prove – tau(sigma(n)) = 2  tau(tau(n)) = 2 – See Calculemus paper Inventing concepts – Refactorable numbers (1, 2, 8, 9, 12, 18, …) – Odd refactorables are square, perfects not refactorable – 20 sequences in the Encyclopedia – See AAAI-00 paper, Journal of Integer Sequences

45. Major Projects In Progress #1 Discriminating between algebraic structures – With Volker Sorge, Andreas Meier – HR part of a large system (8 programs) – HR discriminated 97% of 777 example pairs – exists x (x*x=x & (y*y=x  y*y=y)) 0 1 2 3 4 0 0 3 1 4 2 1 0 0 0 0 0 2 0 2 4 1 3 3 0 4 3 2 1 4 0 1 2 3 4 0 1 2 3 4 0 0 2 4 1 3 1 0 4 3 2 1 2 0 1 2 3 4 3 0 3 1 4 2 4 0 0 0 0 0

46. Major Projects In Progress #2 Interactive ATF – Human replaces one/all of the four modules – With Roy McCasland, mathematician – Work on discovery in Zariski spaces with HR Multiagent ATF – Another agent replaces one/all of the four modules – With Alison Pease’s (PhD) – Cognitive modelling of social discovery – Lakatos style reasoning and machine creativity

47. Conclusions Presented a snapshot of the theory of ATF – Autonomous – Four components, numerous techniques – Uses third party software – Concept driven and conjecture driven cycles Applies to many machine learning tasks – Concept identification, puzzle generation, – Predictions, problem simplification, – Conjecture making, discrimination finding

48. Theory Formation in Bioinformatics? Can work with non-maths data Can form near-conjectures Needs to relax notion of equality Multi-agent/interactive approaches http://www.dai.ed.ac.uk/~simonco/research/hr