Presentation is loading. Please wait.

Presentation is loading. Please wait.

Speeding Up Inference in Markov Logic Networks by Preprocessing to Reduce the Size of the Resulting Grounded Network Jude Shavlik Sriraam Natarajan Computer.

Published byMiles Winfred Snow Modified over 9 years ago

Similar presentations

Presentation on theme: "Speeding Up Inference in Markov Logic Networks by Preprocessing to Reduce the Size of the Resulting Grounded Network Jude Shavlik Sriraam Natarajan Computer."— Presentation transcript:

1 Speeding Up Inference in Markov Logic Networks by Preprocessing to Reduce the Size of the Resulting Grounded Network Jude Shavlik Sriraam Natarajan Computer Sciences Department University of Wisconsin, Madison USA

2 Markov Logic Networks (Richardson & Domingos, MLj 2006) A probabilistic, first-order logic Key idea compactly represent large graphical models using weight = w  x, y, z f(x, y, z) Standard approach 1) assume finite number of constants 2) create all possible groundings 3) perform statistical inference (often via sampling) Univ of WisconsinShavlik & Natarajan, IJCAI-092

3 The Challenge We Address Creating all possible groundings can be daunting A story … Given: an MLN and data Do:quickly find an equivalent, reduced MLN Univ of WisconsinShavlik & Natarajan, IJCAI-093

4 Computing Probabilities in MLNs Probability( World S ) = ( 1 / Z )  exp {  weight i x numberTimesTrue(f i, S) } i  formulae Univ of WisconsinShavlik & Natarajan, IJCAI-094

5 Counting Satisfied Groundings Typically lots of redundancy in FOL sentences  x, y, z p(x) ⋀ q(x, y, z) ⋀ r(z)  w(x, y, z) If p(John) = false, then formula = true for all Y and Z values Univ of WisconsinShavlik & Natarajan, IJCAI-095

6 Some Terminology Three kinds of literals (‘predicates’) Evidence: truth value known Query:want to know prob’s of these Hidden:other Univ of WisconsinShavlik & Natarajan, IJCAI-096

7 e Bi e B1 + … + e Bn Let A =weighted sum of formula satisfied by evidence Let B i =weighted sum of formula in world i not satisfied by evidence Prob(world i ) = e A + Bi e A + B1 + … + e A + Bn Factoring Out the Evidence Univ of WisconsinShavlik & Natarajan, IJCAI-097

8 Key Idea of Our FROG Algorithm Efficiently factor out those formula groundings that evidence satisfies Can produce many orders-of-magnitude smaller Markov networks Can eliminate need for approximate inference, if resulting Markov net small/disconnected enough Resulting Markov net compatible with other speed- up methods, such as lifted and lazy inference, knowledge-based model construction Univ of WisconsinShavlik & Natarajan, IJCAI-098

9 Worked Example  x, y, z GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x, z) ⋀ SameGroup(y, z)  AdvisedBy(x, y) 10,000People at some school 2000Graduate students 1000Professors 1000TAs 500Pairs of professors in the same group Total Num of Groundings = |x|  |y|  |z| = 10 12 10 12 The Evidence Univ of WisconsinShavlik & Natarajan, IJCAI-099

10 10 12 ¬ GradStudent(P2) ¬ GradStudent(P4) … 2 × 10 11 GradStudent(x) GradStudent(P1) ¬ GradStudent(P2) GradStudent(P3) … True False GradStudent(P1) GradStudent(P3) … 2000 Grad Students 8000 Others All these values for X satisfy the clause, regardless of Y and Z GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x,z) ⋀ SameGroup(y,z)  AdvisedBy(x,y) FROG keeps only these X values Instead of 10 4 values for X, have 2 x 10 3 Univ of WisconsinShavlik & Natarajan, IJCAI-0910

11 2 × 10 11 2 × 10 10 Prof(y) ¬ Prof(P1) Prof(P2) … Prof(P2) … 1000 Professors ¬ Prof(P1) … 9000 Others GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x,z) ⋀ SameGroup(y,z)  AdvisedBy(x,y) True False Univ of WisconsinShavlik & Natarajan, IJCAI-0911

12 2 × 10 10 2 × 10 9 GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x,z) ⋀ SameGroup(y,z)  AdvisedBy(x,y) >> Univ of WisconsinShavlik & Natarajan, IJCAI-0912

13 2 × 10 9 2 × 10 6 SameGroup(y, z) 10 6 Combinations SameGroup(P1, P2) … 1000 true SameGroup’s ¬ SameGroup(P2, P5) … 10 6 – 1000 Others GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x,z) ⋀ SameGroup(y,z)  AdvisedBy(x,y) True False 2000 values of X 1000 Y:Z combinations Univ of WisconsinShavlik & Natarajan, IJCAI-0913

14 TA(x, z) 2 × 10 6 Combinations TA(P7,P5) … 1000 TA’s ¬ TA(P8,P4) … 2 × 10 6 – 1000 Others ≤ 10 6 GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x,z) ⋀ SameGroup(y,z)  AdvisedBy(x,y) True False ≤ 1000 values of X ≤ 1000 Y:Z combinations Univ of WisconsinShavlik & Natarajan, IJCAI-0914

15 Original number of groundings = 10 12 10 12 10 6 GradStudent(x) ⋀ Prof(y) ⋀ Prof(z) ⋀ TA(x,z) ⋀ SameGroup(y,z)  AdvisedBy(x,y) Final number of groundings ≤ 10 6 Univ of WisconsinShavlik & Natarajan, IJCAI-0915

16 Some Algorithmic Details Initially store 10 12 groundings with 10 4 space Storage needs grow because literals cause variables to ‘interact’ P(x, y, z) might require O(10 12 ) space Order literals ‘reduced’ impacts storage needs Simple heuristic (see paper) chooses literal to process next – or try all permutations Can merge inference rules after reduction After reduction, sample rule only has advisedBy(x,y) Univ of WisconsinShavlik & Natarajan, IJCAI-0916

17 Empirical Results: CiteSeer Fully Grounded Net FROG’s Reduced Net Univ of WisconsinShavlik & Natarajan, IJCAI-0917

18 Empirical Results: UWash-CSE FROG’s Reduced Net without One Challenging Rule FROG’s Reduced Net Fully Grounded Net advisedBy(x,y)  advisedBy(x,z)  samePerson(y,z)) Univ of Wisconsin18Shavlik & Natarajan, IJCAI-09

19 Runtimes On Full UWash-CSE (27 rules) FROG takes 4.2 sec On CORA (2K rules) and CiteSeer (8K rules) FROG takes less than 700 msec per rule On CORA Alchemy’s Lazy Inference takes 94 mins to create its initial network FROG takes 30 mins and produces small enough network (10 6 nodes) that lazy inference not needed Univ of WisconsinShavlik & Natarajan, IJCAI-0919

20 Related Work Lazy MLN inference Singla & Domingos (2006), Poon et al (2008) FROG: precompute instead of lazily calculate Lifted inference Braz et al (2005), Singla & Domingos (2008), Milch et al (2008), Riedel (2008), Kisynski & Poole (2009), Kersting et al (2009) Knowledge-based model construction Wellman et al (1992) FROG also exploits KBMC Univ of WisconsinShavlik & Natarajan, IJCAI-0920

21 Future Work Efficiently handle small changes to truth values of evidence Combine FROG with Lifted Inference Exploit commonality across rules Integrate with weight and rule learning Univ of WisconsinShavlik & Natarajan, IJCAI-0921

22 Conclusion MLN’s count the satisfied groundings of FOL formula Many ways a formula can be satisfied P(x)  Q(x, y)  R(x, y, z)  ¬ S(y)  ¬ T(x, y) Our FROG algorithm efficiently counts groundings satisfied by evidence FROG can reduce number of groundings by several orders of magnitude Reduced network compatible with lifted and lazy inference, etc Univ of WisconsinShavlik & Natarajan, IJCAI-0922

Download ppt "Speeding Up Inference in Markov Logic Networks by Preprocessing to Reduce the Size of the Resulting Grounded Network Jude Shavlik Sriraam Natarajan Computer."

Similar presentations

Joint Inference in Information Extraction Hoifung Poon Dept. Computer Science & Eng. University of Washington (Joint work with Pedro Domingos)

Joint Inference in Information Extraction Hoifung Poon Dept. Computer Science & Eng. University of Washington (Joint work with Pedro Domingos)
Discriminative Training of Markov Logic Networks

Discriminative Training of Markov Logic Networks
Efficient Inference Methods for Probabilistic Logical Models

Efficient Inference Methods for Probabilistic Logical Models
University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Discriminative Structure and Parameter.

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Discriminative Structure and Parameter.
Bayesian Abductive Logic Programs Sindhu Raghavan Raymond J. Mooney The University of Texas at Austin 1.

Bayesian Abductive Logic Programs Sindhu Raghavan Raymond J. Mooney The University of Texas at Austin 1.
Bayesian Networks CSE 473. © Daniel S. Weld 2 Last Time Basic notions Atomic events Probabilities Joint distribution Inference by enumeration Independence.

Bayesian Networks CSE 473. © Daniel S. Weld 2 Last Time Basic notions Atomic events Probabilities Joint distribution Inference by enumeration Independence.
Online Max-Margin Weight Learning for Markov Logic Networks Tuyen N. Huynh and Raymond J. Mooney Machine Learning Group Department of Computer Science.

Online Max-Margin Weight Learning for Markov Logic Networks Tuyen N. Huynh and Raymond J. Mooney Machine Learning Group Department of Computer Science.
CPSC 322, Lecture 30Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 30 March, 25, 2015 Slide source: from Pedro Domingos UW.

CPSC 322, Lecture 30Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 30 March, 25, 2015 Slide source: from Pedro Domingos UW.
Undirected Probabilistic Graphical Models (Markov Nets) (Slides from Sam Roweis Lecture)

Undirected Probabilistic Graphical Models (Markov Nets) (Slides from Sam Roweis Lecture)
10-803 Markov Logic Networks Instructor: Pedro Domingos.

Markov Logic Networks Instructor: Pedro Domingos.
Department of Computer Science The University of Texas at Austin Probabilistic Abduction using Markov Logic Networks Rohit J. Kate Raymond J. Mooney.

Department of Computer Science The University of Texas at Austin Probabilistic Abduction using Markov Logic Networks Rohit J. Kate Raymond J. Mooney.
Undirected Probabilistic Graphical Models (Markov Nets) (Slides from Sam Roweis)

Undirected Probabilistic Graphical Models (Markov Nets) (Slides from Sam Roweis)
Markov Logic: Combining Logic and Probability Parag Singla Dept. of Computer Science & Engineering Indian Institute of Technology Delhi.

Markov Logic: Combining Logic and Probability Parag Singla Dept. of Computer Science & Engineering Indian Institute of Technology Delhi.
Islam Beltagy, Cuong Chau, Gemma Boleda, Dan Garrette, Katrin Erk, Raymond Mooney The University of Texas at Austin Richard Montague Andrey Markov Montague.

Islam Beltagy, Cuong Chau, Gemma Boleda, Dan Garrette, Katrin Erk, Raymond Mooney The University of Texas at Austin Richard Montague Andrey Markov Montague.
Random Regression: Example Target Query: P(gender(sam) = F)? Sam is friends with Bob and Anna. Unnormalized Probability: Oliver Schulte, Hassan Khosravi,

Random Regression: Example Target Query: P(gender(sam) = F)? Sam is friends with Bob and Anna. Unnormalized Probability: Oliver Schulte, Hassan Khosravi,
Review Markov Logic Networks Mathew Richardson Pedro Domingos Xinran(Sean) Luo, u0866707.

Review Markov Logic Networks Mathew Richardson Pedro Domingos Xinran(Sean) Luo, u
Adbuctive Markov Logic for Plan Recognition Parag Singla & Raymond J. Mooney Dept. of Computer Science University of Texas, Austin.

Adbuctive Markov Logic for Plan Recognition Parag Singla & Raymond J. Mooney Dept. of Computer Science University of Texas, Austin.
Efficient Weight Learning for Markov Logic Networks Daniel Lowd University of Washington (Joint work with Pedro Domingos)

Efficient Weight Learning for Markov Logic Networks Daniel Lowd University of Washington (Joint work with Pedro Domingos)
Markov Logic: A Unifying Framework for Statistical Relational Learning Pedro Domingos Matthew Richardson

Markov Logic: A Unifying Framework for Statistical Relational Learning Pedro Domingos Matthew Richardson
Speaker:Benedict Fehringer Seminar:Probabilistic Models for Information Extraction by Dr. Martin Theobald and Maximilian Dylla Based on Richards, M., and.

Speaker:Benedict Fehringer Seminar:Probabilistic Models for Information Extraction by Dr. Martin Theobald and Maximilian Dylla Based on Richards, M., and.

Similar presentations

Ads by Google