1. FLAIRS 20011 Graph-Based Concept Learning Jesus Gonzalez, Lawrence Holder and Diane Cook Department of Computer Science and Engineering The University of Texas at Arlington

2. FLAIRS 20012 Outline Relational concept learning Graph-based concept learning Conceptual graphs and Galois lattice Graph-based discovery in Subdue SubdueCL Empirical results Conclusions

3. FLAIRS 20013 Relational Concept Learning Inductive Logic Programming (ILP) FOIL Progol First-order logic vs. graphs Expressiveness Interpretability Conceptual graphs

4. FLAIRS 20014 Conceptual Graphs Logic-based knowledge representation Object On Shape Triangle SquareObject Shape shape(X,triangle) shape(Y,square) on(X,Y)

5. FLAIRS 20015 Conceptual Graphs Graph Logic PAC-learning CGs [Jappy & Nock] Size of CG class and generalization (projection) operator polynomial in Number of relations Number of concepts Number of labels

6. FLAIRS 20016 Galois Lattice Each node consists of a description graph and set of subsumed examples Begins with positive examples Generalization operator Most specific generalization Union of example sets

7. FLAIRS 20017 Galois Lattice [ 1][ 2][ 3][ 4] [ 1, 2 ][ 3, 4 ] [ 1, 2, 3 ][ 1, 2, 4 ][ 1, 3, 4 ][ 2, 3, 4 ] [ 1, 2, 3, 4 ] [ 1][ 2][ 3][ 4] [ 1, 2 ][ 3, 4 ] [ 1, 2, 3 ][ 1, 2, 4 ][ 1, 3, 4 ][ 2, 3, 4 ] [ 1, 2, 3, 4 ] [ 1][ 2][ 3][ 4] [ 1, 2 ][ 3, 4 ] [ 1, 2, 3 ][ 1, 2, 4 ][ 1, 3, 4 ][ 2, 3, 4 ] [ 1, 2, 3, 4 ] [ 1][ 2][ 3][ 4] [ 1, 2 ][ 3, 4 ] [ 1, 2, 3 ][ 1, 2, 4 ][ 1, 3, 4 ][ 2, 3, 4 ] [ 1, 2, 3, 4 ] [ 1, 2 ][ 3, 4 ] [ 1, 2, 3 ] [ 1, 2, 4 ] [ 1, 3, 4 ] [ 2, 3, 4 ] [ 1, 2, 3, 4 ] triangle on square on circle on rectangle triangle on triangle on rectangle triangle on triangle on [ 1, 2 ] triangle on square on circle on rectangle triangle on triangle on rectangle triangle on triangle on [ 1, 2 ] triangle on square on triangle on square on circle on rectangle circle on rectangle triangle on triangle on triangle on rectangle triangle on rectangle triangle on triangle on

8. FLAIRS 20018 Galois Lattice Galois lattice creation O(n 3 p) n examples p nodes in lattice Tractable for poly-time generalization GRAAL system

9. FLAIRS 20019 Graph-Based Discovery Finding “interesting” and repetitive substructures (connected subgraphs) in data represented as a graph object triangle R1 C1 T1 S1 T2 S2 T3 S3 T4 S4 Input DatabaseSubstructure S1 (graph form) Compressed Database R1 C1 object square on shape

10. FLAIRS 200110 Graph-Based Discovery “Interesting” defined according to the Minimum Description Length principle min [DL(S) + DL(G|S)] General-to-specific beam search through substructure space Poly-time inexact graph match Subdue system S

11. FLAIRS 200111 Subdue System Graph-based… Discovery Concept learning Hierarchical conceptual clustering Background knowledge Parallel/distributed capability http://cygnus.uta.edu/subdue

12. FLAIRS 200112 Graph-Based Concept Learning object on triangle square shape

13. FLAIRS 200113 Graph-Based Concept Learning Extension to graph-based discovery Input now a set of positive graphs and a set of negative graphs Set-covering approach Iterate until all positive graphs and no negative graphs covered Result is a substructure DNF

14. FLAIRS 200114 Graph-Based Concept Learning Solution 1 Find substructure compressing positive graphs, but not negative graphs Compress graphs and iterate until no further compression Problem Compressing, instead of removing, partially-covered positive graphs leads to overly-specific hypotheses

15. FLAIRS 200115 Graph-Based Concept Learning Solution 2 Find substructure covering positive graphs, but not negative graphs Remove covered positive graphs and iterate until all covered Substructure value = 1 - Error

16. FLAIRS 200116 Empirical Results Comparison with ILP systems Non-relational domains from UCI repository GolfVoteDiabetesCreditTicTacToe FOIL66.6793.0270.6666.16100.00 Progol33.3376.9851.9744.55100.00 SubdueCL66.6794.8864.2171.52100.00

17. FLAIRS 200117 Empirical Results Comparison with ILP systems Relational domains: Chess endgame WKC WKR WRRWRC BKC BKR pos adj pos adj pos eq 012 0 1 2 WK WR BK lt

18. FLAIRS 200118 Empirical Results: Chess FOIL: 11 rules, 99.34% Progol: 5 rules, 99.74% SubdueCL: 7 rules, 99.74% WKC pos BKCWKRBKR adj lt WKC pos BKCWKRBKR adj WKC pos BKC WKR BKR adj lt WKC pos BKC WKR BKR adj

19. FLAIRS 200119 Empirical Results Relational domain: Cancer SubdueCL: 62% Progol: 64% [72%] compound amine p chromaberr has_group compound amine p chromaberr has_group compound amine p chromaberr has_group compound p drosophila_slrl compound p drosophila_slrl compound p drosophila_slrl

20. FLAIRS 200120 Empirical Results Relational domain: Web Professor (+) vs. student (-) websites Hyperlink structure and page content

21. FLAIRS 200121 Empirical Results Relational domain: Web Computer store (+) vs. professor (-) websites Hyperlink structure only

22. FLAIRS 200122 Conclusions Theoretical analysis of graph-based concept learning PAC-learning conceptual graphs Galois lattice Next step: Relax graph constraints Empirical analysis Competitive with other relational concept learners (ILP) Next step: More relational domains SubdueCL (http://cygnus.uta.edu/subdue)