Download presentation
Presentation is loading. Please wait.
1
Learning Statistical Models From Relational Data Lise Getoor University of Maryland, College Park Joint work with: Nir Friedman, Hebrew U. Daphne Koller, Stanford Avi Pfeffer, Harvard Ben Taskar, Stanford
2
Outline Motivation and Background PRMs w/ Attribute Uncertainty PRMs w/ Link Uncertainty PRMs w/ Class Hierarchies Statistical Relational Models
3
Discovering Patterns in Structured Data Patient Treatment Strain Contact
4
Learning Statistical Models Traditional approaches –work well with flat representations –fixed length attribute-value vectors –assume independent (IID) sample Patient flatten Problems: –introduces statistical skew –loses relational structure incapable of detecting link-based patterns –must fix attributes in advance Contact
5
Outline Background: »Bayesian Networks (BNs) [Pearl, 1988] –Probabilistic Relational Models (PRMs) Learning PRMs w/ Attribute Uncertainty PRMs w/ Link Uncertainty PRMs w/ Class Hierarchies Statistical Relational Models
6
Bayesian Networks nodes = random variables edges = direct probabilistic influence Network structure encodes independence assumptions: XRay conditionally independent of Pneumonia given Infiltrates XRay Lung Infiltrates Sputum Smear TuberculosisPneumonia
7
Bayesian Networks XRay Lung Infiltrates Sputum Smear TuberculosisPneumonia Associated with each node X i there is a conditional probability distribution P(X i |Pa i : ) — distribution over X i for each assignment to parents –If variables are discrete, P is usually multinomial –P can be linear Gaussian, mixture of Gaussians, … 0.8 0.2 p t p 0.6 0.4 0.010.99 0.2 0.8 tp t t p T P P(I |P, T )
8
BN Semantics Compact & natural representation: –nodes have k parents 2 k n vs. 2 n params conditional independencies in BN structure + local probability models full joint distribution over domain = X I S TP
9
Queries Full joint distribution specifies answer to any query: P(variable | evidence about others) XRay Lung Infiltrates Sputum Smear TuberculosisPneumonia XRay Sputum Smear
10
BN Learning BN models can be learned from empirical data –parameter estimation via numerical optimization –structure learning via combinatorial search. BN hypothesis space biased towards distributions with independence structure. Inducer Data X I S TP
11
Outline Background: Bayesian Networks (BNs) »Probabilistic Relational Models (PRMs) [Pfeffer, 2000] Learning PRMs w/ Attribute Uncertainty PRMs w/ Link Uncertainty PRMs w/ Class Hierarchies Statistical Relational Models
12
Probabilistic Relational Models Combine advantages of relational logic & Bayesian networks: –natural domain modeling: objects, properties, relations; –generalization over a variety of situations; –compact, natural probability models. Integrate uncertainty with relational model: –properties of domain entities can depend on properties of related entities; –uncertainty over relational structure of domain.
13
Relational Schema Strain Unique Infectivity Infected with Interacted with Describes the types of objects and relations in the databaseClasses Relationships Contact Close-Contact Skin-Test Age Patient Homeless HIV-Result Ethnicity Disease-Site Attributes Contact-Type
14
Probabilistic Relational Model Close-Contact Transmitted Contact-Type Disease Site Strain Unique Infectivity Patient Homeless HIV-Result POB Contact Age Cont.Contactor.HIV Cont.Close-Contact Cont.Transmitted | P 4.06.0 3.07.0 2.08.0 1.09.0,,,,, tt ft tf ff P(T | H, C) CH
15
Relational Skeleton Fixed relational skeleton –set of objects in each class –relations between them Uncertainty over assignment of values to attributes PRM defines distribution over instantiations of attributes Strain s1 Patient p2 Patient p1 Contact c3 Contact c2 Contact c1 Strain s2 Patient p3
16
A Portion of the BN P1.Disease Site P1.Homeless P1.HIV-Result P1.POB C1.Close-Contact C1.Transmitted C1.Contact-Type C1.Age C2.Close-Contact C2.Transmitted C2.Contact-Type true falsetrue 4.06.0 3.07.0 2.08.0 1.09.0,,,,, tt ft tf ff P(T | H, C) CH 4.06.0 3.07.0 2.08.0 1.09.0,,,,, tt ft tf ff CH C2.Age
17
PRM: Aggregate Dependencies sum, min, max, avg, mode, count Disease Site Patient Homeless HIV-Result POB Age Close-Contact Transmitted Contact-Type Contact Age.. Patient Jane Doe POB US Homeless no HIV-Result negative Age ??? Disease Site pulmonary A. Contact #5077 Contact-Type coworker Close-Contact no Age middle-aged Transmitted false Contact #5076 Contact-Type spouse Close-Contact yes Age middle-aged Transmitted true Contact #5075 Contact-Type friend Close-Contact no Age middle-aged Transmitted false mode
18
PRM with AU Semantics Attributes Objects probability distribution over completions I : PRM relational skeleton += Strain Patient Contact Strain s1 Patient p1 Patient p2 Contact c3 Contact c2 Contact c1 Strain s2 Patient p3
19
Learning PRMs w/ AU Database Patient Strain Contact Relational Schema Patient Contact Strain Parameter estimation Structure selection
20
Parameter Estimation in PRMs Assume known dependency structure S Goal: estimate PRM parameters –entries in local probability models, is good if it is likely to generate the observed data, instance I. MLE Principle: Choose so as to maximize l As in Bayesian network learning, crucial property: decomposition separate terms for different X.A
21
ML Parameter Estimation Contact CloseContact Transmitted Patient HIV DiseaseSite Count Query for counts: Patient table Contact table P ?? ?? ?? ??,,,,, tt ft tf ff P(T | H, C) CH Cont.Contactor.HIV Cont.Close-Contact Cont.Transmitted | P
22
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: –legal models –scoring models –searching model space
23
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: »legal models –scoring models –searching model space
24
Legal Models author-of PRM defines a coherent probability model over a skeleton if the dependencies between object attributes is acyclic How do we guarantee that a PRM is acyclic for every skeleton? Researcher Prof. Gump Reputation high Paper P1 Accepted yes Paper P2 Accepted yes sum
25
Attribute Stratification PRM dependency structure S dependency graph Paper.Accecpted Researcher.Reputation if Researcher.Reputation depends directly on Paper.Accepted dependency graph acyclic acyclic for any Attribute stratification: Algorithm more flexible; allows certain cycles along guaranteed acyclic relations
26
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: –legal models »scoring models –searching model space
27
Scoring Models Bayesian approach: Standard approach to scoring models; used in Bayesian network learning
28
Structure Selection Idea: –define scoring function –do local search over legal structures Key Components: –legal models –scoring models »searching model space
29
Searching Model Space Contact Strain Patient score Delete C.C C.T Contact Strain Patient score Add S.I S.U Strain Contact Patient Phase 0: consider only dependencies within a class
30
Contact Strain Patient score Add S.I P.D score Add P.H C.T Contact Strain Patient Contact Patient Strain Phase 1: consider dependencies from “neighboring” classes, via schema relations Phased Structure Search
31
score Add S.I C.T score Add C.P S.I Phase 2: consider dependencies from “further” classes, via relation chains Contact Strain Patient Contact Strain Patient Contact Strain Patient
32
Experimental Evaluation
33
Synthetic Data Simple ‘genetic’ domain Construct training set of various sizes Compare the log-likelihood of test set of size 100,000 –‘gold’ standard model –Learn parameters (model structure given) –Learn model (learn both structure and parameters)
34
Blood Type M-chromosome P-chromosome Person Result Contaminated Blood Test Blood Type M-chromosome P-chromosome Person Blood Type M-chromosome P-chromosome Person (Father) (Mother)
35
Error on Test Set
36
Error Variance
37
Errors in Learned Structure
38
TB Cases in SF Patient ( 2300) Ethnicity Homeless Age @ diagnosis HIV result Disease-site X-ray Contact ( 20000) Contact-type Age Care Infected Strain ( 1000) Unique Drug-Resistance
39
hivres # contacts result transmitted infectivity smrpos care closecont ageatdx closecont hh_oohh ethnic # infected % infected hh_oohh contype homeless gender contype disease site contage xray pob Contact Strain Subcase Patient TB PRM
40
total assets # roles rtn earn assets age rtn assets fired # employees top_role total_assets retired salary Company Role Prev- Role Person SEC PRM 20,000 120,000 40,000
41
Outline Motivation and Background PRMs w/ Attribute Uncertainty »PRMs w/ Link Uncertainty PRMs w/ Class Hierarchies Statistical Relational Models
42
Introduction Topic TheoryAI Agent Theory papers Cornell Scientific Paper Topic TheoryAI Attributes of object Attributes of linked objects Attributes of heterogeneous linked objects Collective Classification
43
Our Approach Motivation: relational structure provides useful information for density estimation and prediction Construct probabilistic models of relational structure that capture link uncertainty Here we propose two new mechanisms: –Reference uncertainty –Existence uncertainty
44
PRMs w/ AU: another example Vote Rank Movie IncomeGender Person AgeGenre PRM consists of: Relational Schema Dependency Structure Vote.Person.Gender, Vote.Person.Age Vote.Movie.Genre, Vote.Rank | P Local Probability Models
45
Fixed relational skeleton : –set of objects in each class –relations between them Movie m1 Vote v1 Movie: m1 Person: p1 Person p2 Person p1 Movie m2 Uncertainty over assignment of values to attributes PRM w/ Attribute Uncertainty Vote v2 Movie: m1 Person: p2 Vote v3 Movie: m2 Person: p2 Primary Keys Foreign Keys
46
PRM w/ AU Semantics Attributes Objects Ground BN defining distribution over complete instantiations of attributes I : PRM relational skeleton += Patient p2 Vote Movie Person Movie Vote Vote Person Movie Vote
47
Issue PRM w/ AU applicable only in domains where we have full knowledge of the relational structure Next we introduce PRMs which allow uncertainty over relational structure…
48
PRMs w/ Link Uncertainty Advantages: –Applicable in cases where we do not have full knowledge of relational structure –Incorporating uncertainty over relational structure into probabilistic model can improve predictive accuracy Two approaches: –Reference uncertainty –Existence uncertainty Different probabilistic models; varying amount of background knowledge required for each
49
Citation Relational Schema Wrote Paper Topic Word1 WordN … Word2 Paper Topic Word1 WordN … Word2 Cites Count Citing Paper Cited Paper Author Institution Research Area
50
Attribute Uncertainty Paper Word1 Topic WordN Wrote Author... Research Area P( WordN | Topic) P( Topic | Paper.Author.Research Area Institution P( Institution | Research Area)
51
Reference Uncertainty Bibliography Scientific Paper ` 1. ----- 2. ----- 3. ----- ? ? ? Document Collection
52
PRM w/ Reference Uncertainty Cites Cited Citing Dependency model for foreign keys Paper Topic Words Paper Topic Words Naïve Approach: multinomial over primary key noncompact limits ability to generalize
53
Reference Uncertainty Example Paper P5 Topic AI Paper P4 Topic AI Paper P3 Topic AI Paper M2 Topic AI Paper P1 Topic Theory Cites Cited Citing Paper P5 Topic AI Paper P3 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P1 Topic Theory Paper.Topic = AI Paper.Topic = Theory P1 P2 Paper Topic Words P1 P2 3.07.0 P1 P2 1.09.0 Topic 99.001.0 Theory AI
54
PRMs w/ RU Semantics PRM-RU + entity skeleton probability distribution over full instantiations I Cites Cited Citing Paper Topic Words Paper Topic Words PRM RU Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? Reg Cites entity skeleton
55
Learning PRMs w/ RU Idea: just like in PRMs w/ AU –define scoring function –do greedy local structure search Issues: –expanded search space construct partitions new operators
56
Learning Idea: –define scoring function –do phased local search over legal structures Key Components: –legal models –scoring models –searching model space PRMs w/ RU model new dependencies new operators unchanged
57
Structure Search: New Operators Cites Cited Citing Paper Topic Words Paper Topic Words Citing Papers 1.0 Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Topic = AI Δscore Refine on Topic Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Paper Δscore Refine on Author.Instition Author Institution Institution = MIT
58
PRMs w/ RU Summary Define semantics for uncertainty over foreign-key values Search now includes operators Refine and Abstract for constructing foreign-key dependency model Provides one simple mechanism for link uncertainty
59
Existence Uncertainty Document Collection ? ? ?
60
PRM w/ Exists Uncertainty Cites Dependency model for existence of relationship Paper Topic Words Paper Topic Words Exists
61
Exists Uncertainty Example Cites Paper Topic Words Paper Topic Words Exists Citer.Topic Cited.Topic 0.9950005 Theory FalseTrue AI Theory0.9990001 AI 0.9930008 AI Theory0.9970003
62
PRMs w/ EU Semantics PRM-EU + object skeleton probability distribution over full instantiations I Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? Paper P5 Topic AI Paper P4 Topic Theory Paper P2 Topic Theory Paper P3 Topic AI Paper P1 Topic ??? object skeleton ??? PRM EU Cites Exists Paper Topic Words Paper Topic Words
63
Learning PRMs w/ EU Idea: just like in PRMs w/ AU –define scoring function –do greedy local structure search Issues: –efficiency Computation of sufficient statistics for exists attribute Do not explicitly consider relations that do not exist
64
Structure Selection Idea: –define scoring function –do phased local search over legal structures Key Components: –legal models –scoring models –searching model space PRMs w/ EU model new dependencies unchanged
65
Experiment I: EachMovie + thriller action horrorgender theater_status gender video_status age animation art_foreignclassic personal_income comedydrama rank household_income familyromance Movie Person Movie Actor MOVIE ROLE VOTE PERSON ACTOR education * © 1999 -2000 Internet Movie Database Limited † http://www.research.digital.com/SRC/EachMovie Size: 1600 Size: 35,000 Size: 50,000 Size: 25,000 Size: 300,000 * †
66
EachMovie + PRM-RU thriller action horror gender theater_status gender video_status age animation art_foreign classic personal_income comedy drama rank household_income family romance Movie Person Movie Actor MOVIE ROLE VOTE PERSON ACTOR education M F 8.02.0 Action 7.03.0 true false Typical Voter: male, young adult, college w/o degree, middle income
67
EachMovie + PRM-EU age comedy drama rank gender family personal_income horror romance exists household_income thriller exists gender theater_status video_status action education animation art_foreign classic MOVIE ROLE VOTE PERSON ACTOR + - Men much more likely to vote on action movies
68
Experiment II: Prediction Paper P506 Paper P516 Topic Reinforcement Learning Words … Paper P1309 Topic Probabilistic Reasoning Words … Paper P289 Topic Reinforcement Learning Words … Cited Papers Paper P134 Topic Reinforcement Learning Words … Paper P1067 Topic Reinforcement Learning Words … Citing Papers Topic ?? w1wN...
69
Domains Cites Exists Paper Topic w1wN... Paper Topic w1wN... cited paperciting paper Cora Dataset, McCallum, et. al Link Exists Web Page Category w1wN... Category w1wN... From PageTo Page Web Page WebKB, Craven, et. al
70
Prediction Accuracy
72
Experiment III: Collective Classification Paper#2 Topic Paper#3 Topic WordN Paper#1 Word1 Topic... Author#1 Area Ins t #1-#2 Author#2 Area Inst Exists #2-#3 Exists #2-#1 Exists #3-#1 Exists #1-#3 Exists WordN Word1 WordN Word1 Exists WordN Word1 WordN Word1 WordN Word1 Exists Ins t Topic Area Topic Area Topic Area #3-#2
73
Inference in Unrolled BN Prediction requires inference in “unrolled” network –Infeasible for large networks –Use approximate inference for E-step Loopy belief propagation (Pearl, 88; McEliece, 98) –Scales linearly with size of network –Guaranteed to converge only for polytrees –Empirically, often converges in general nets (Murphy,99) Local message passing –Belief messages transferred between related instances –Induces a natural “influence” propagation behavior Instances give information about related instances
74
... From-Page Category Word1 WordN Exists From To Link Hub To-Page Word Anchor Has... Category Word1 WordN Hub Web Domain
75
WebKB Results* * from “Probabilistic Models of Text and Link Structure for Hypertext Classification”, Getoor, Segal, Taskar and Koller in IJCAI 01 Workshop Text Learning: Beyond Classification
76
Outline Motivation and Background PRMs w/ Attribute Uncertainty PRMs w/ Structural Uncertainty »PRMs w/ Class Hierarchies Statistical Relational Models
77
From Instances to Classes in Probabilistic Relational Models Compare two approaches –Probabilistic Relational Models (PRMs) –Bayesian Network (BNs) PRMs with Class Hierarchies (PRM-CH) –bridge gap between BNs and PRMs Learning PRM-CHs –hierarchy supplied –discovering hierarchy
78
Vote Program Voter Ranking PRM for Collaborative Filtering Vote Program Voter Ranking Income 1.06.03.0 5.04.01.0 4.05.01.0 1.04.05.0 bs hs sitcom bs doc hs doc hml E G sitcom + Dependency Model TV-Program Genre Budget Time-slot Network TV-Program Genre Budget Time-slot Network Relational Schema Person Age Gender Education
79
PRM Instantiation TV-Program Nova Genre doc Budget low Timeslot primetime Network PBS Person Jane Doe Age elderly Gender female Education bs Income medium Vote #5630 Ranking ? TV-Program Seinfeld Genre sitcom Budget high Timeslot rerun Network ABC TV-Program Frasier Genre sitcom Budget medium Timeslot primetime Network ABC Person John Deer Age middle-aged Gender male Education hs Income low Vote #5631 Ranking ? Vote #5632 Ranking ? Vote #5632 Ranking ? Vote #5633 Ranking ? Income 1.06.03.0 5.04.01.0 4.05.01.0 1.04.05.0 bs hs sitcom bs doc hs doc hmlEG sitcom
80
BN for Collaborative filtering Law & Order Frasier NBC Monday Night Movies Mad about you Beverly Hills 90210 Seinfeld Friends Melrose Place Models Inc. Breese, et al. UAI-98
81
Limitations of PRMs In PRM, all instances of the same class must use the same dependency mode, it cannot distinguish: –documentaries and sitcoms –“60 Minutes” and Seinfeld PRM cannot have dependencies that are “cyclic” –ranking for Frasier depends on ranking for Friends
82
Limitations of BNs In BN, each instance has its own dependency model, cannot generalize over instances –If John tends to like sitcoms, he will probably like next season’s offerings –whether a person enjoys sitcom reruns depends on whether they watch primetime sitcoms BN can only model relationships between at most one class of instances at a time –In previous model, cannot model relationships between people –if my roommate watches Seinfeld I am more likely to join in
83
Desired Model Allows both class and instance dependencies WWWF Person Age Gender Education Income Soap Genre Budget Time-slot Network Genre Budget Time-slot Network Documentary Sitcom-Vote Program Voter Ranking Doc-Vote Program Voter Ranking Vote Program Voter Ranking TV-Program Genre Budget Time-slot Network
84
PRMs w/ Class Hierarchies Allows us to: Refine a “heterogenous” class into more coherent subclasses Refine probabilistic model along class hierarchy –Can specialize/inherit CPDs –Construct new dependencies that were originally “acyclic” Provides bridge from class-based model to instance-based model
85
PRM-CH Person Age TV-Program Genre Budget Time-slot Network Gender Education Income Vote Program Voter Ranking Relational Schema Class Hierarchy SoapOpera TV-Program SitComDocumentaryDrama Legal-DramaMedical-Drama Dependency Model Budget SoapOpera Budget TV -Program Budget SitCom Budget Documentary Budget Drama Budget Legal-Drama Budget Medical -Drama Koller & Pfeffer 1998 Pfeffer 2000
86
Learning PRM-CHs Relational Schema Database: TVProgram Person Vote Person Vote TVProgram Instance I Class hierarchy provided Learn class hierarchy
87
Bayesian Model Selection for PRMs Idea: –define scoring function –do phased local search over legal structures Key Components: –scoring models –searching model space PRM-CHs new operators unchanged
88
Guaranteeing Acyclicity with Subclasses Soap-Vote Program Voter Ranking Doc-Vote Program Voter Ranking Soap-Vote.Ranking Doc-Vote.Ranking Vote Program Voter Ranking Vote.Ranking Vote.Class
89
Scenario 1: Class hierarchy is provided New Operators –Specialize/Inherit Learning PRM-CH Budget SoapOpera Budget TV -Program Budget SitCom Budget Drama Budget Legal-Drama Budget Medical -Drama Budget Documentary
90
Learning Class Hierarchy Issue: partially observable data set Construct decision tree for class defined over attributes observed in training set TV-Program.Genre sitcom drama class1 class3 documentary class2 class4 English TV-.Network.Nationality class5 French class6 American New operator – Split on class attribute – Related class attribute
91
MOVIE Animation Famil y Dram a Comed y Romance Actio n Horror Thriller Theater Status Video Status Art/Foreig n Classic VOTE Rating PERSON Gender Age Personal Income Household Income Education EachMovie + PRM 1400 Movies 5000 People 240,000 Votes http://www.research.digital.com/SRC/EachMovie
92
Theater Status Video Status Art/Foreign Classic Drama ROMANCE-MOVIE Animation Family Horror Thriller Gender Age Personal Income Household Income Education PERSON ROMANCE-VOTE Rating OTHER-VOTE Rating COMEDY-VOTE Rating ACTION-VOTE Rating Theater Status Video Status Art/Foreign Classic Drama ACTION-MOVIE Animation Family Horror Thriller Theater Status Video Status Art/Foreign Classic Drama COMEDY-MOVIE Animation Family Horror Thriller PRM-CH Theater Status Video Status Art/Foreign Classic OTHER-MOVIE Thriller Drama Horror Animation Family
93
Comparison 5 Test Sets: 1000 votes, ~100 movies, ~115 people –PRM Mean LL: -12,079, std 475.68 –PRM-CH Mean LL: -10558, std 433.10 Using standard t-test, PRM-CH model outperforms PRM model with over 99% confidence
94
PRM-CH Summary PRMs with class hierarchies are a natural extension of PRMs: –Specialization/Inheritance of CPDs –Allows new dependency structures Provide bridge from class-based to instance- based models Learning techniques proposed –Need efficient heuristics –Empirical validation on real-world domains
95
Outline Motivation and Background PRMs w/ Attribute Uncertainty PRMs w/ Structural Uncertainty PRMs w/ Class Hierarchies »Statistical Relational Models Summary
96
Statistical Relational Models Capture the frequency information, rather than probabilistic information about individuals Application to selectivity estimation, SIGMOD01 Here, development of theory
97
A Comparison of Two Aproaches Possible WorldsDomain Frequency Generalization Compression Syntax - (almost) same Learning - (almost) same Semantics - (very) different Inference - (very) different
98
Crucial for –Multirelational data mining –cost-based query optimization –query profilers Key: joint frequency distribution F D (A 1,…,A n ) Application: Query Result Size Estimation F D (X,Y) = x1x2x3x1x2x3 y 1 y 2 y 3 y 4 F D ( x 2, y 3 ) Problem: exponential in # of attributes v n representing distribution exactly is infeasible
99
Traditional Approaches to Selectivity Estimation Approximate joint distribution by making several key independence assumptions: –Attribute Value Independence: joint distribution is product of single attribute distributions –Join Uniformity Assumption: tuple in one relation is equally likely to join with any tuple in the other relation
100
SRMs Use graphical models to compactly represent joint distribution –over single table for select selectivity –over multiple tables for join selectivity Provides a unified framework for estimating the selectivity of select-join queries over multiple tables
101
System Architecture Model Constructor Database offline Selectivity Estimator execution time Query Q Size(Q) Methods for incremental maintenance of BN Friedman and Goldszmidt, 1997
102
BN Construction* Heuristic search over graph and tree structure at nodes Learn more complex networks when required, simpler networks when possible; subject to storage size restrictions Key computational step: –computation of sufficient statistics - frequency of different instantiations of a node and its parents in DB H I C AE Database Construct BN B s.t. P D (A 1,…,A n ) F D (A 1,…,A n )/ |R| * Cooper and Herkovits, 1992; Heckerman, 1995
103
BNs for Selectivity Estimation Query: select * from R where R.A 1 = a 1 and … and R.A k = a k Size(Q) = |R| P D (a 1,…,a k ) Use Bayesian inference algorithm* to compute P D (a 1,…,a k ) Algorithm complexity depends on BN connectivity; efficient in practice * Pearl, 1988; Lauritzen and Spiegelhalter, 1988
104
Foreign-key Join Selectivity PersonPurchase Uniform Join Assumption Size(Purchase Person) = | Purchase | Assuming referential integrity
105
Correlated Attributes PersonPurchase Income = high Income = low Type = luxury Type = necessity
106
Skewed Join PersonPurchase Income = high Income = low Type = luxury Type = necessity
107
Join Indicator S R Query: select * from R, S where R.F = S.K and R.A = a and S.B = b P(J F ) = prob. randomly chosen tuple from R joins with a randomly chosen tuple from S size(Q) = | R | | S | P(J F, a, b)
108
Universal Foreign Key Closure A DB schema is table-stratified if we can order the tables s.t. if R.F refers to S.K, S precedes R.F in the stratification ordering The universal foreign key closure is the query constructed by introducing a tuple variable for each leaf in the stratification, and, introducing, for each foreign key, a new tuple variable
109
Universal Foreign Key Closure Schema: R, S, T R.F refers to S, S.F refers to T stratification: T < S < R r s t r s t r.F 1 = s.K s.F 2 = t.K Schema: R, S R.F 1 refers to S, R.F 2 refers to S stratification: S < R r s1 s2 r r.F 1 = s 1.K s1s1 s2s2 r.F 2 = s 2.K
110
Statistical Relational Models Model distribution of attributes across multiple tables Allow attribute values to depend on attributes in the same table (like a BN) Allow attribute values to depend on attributes in other tables along a foreign key join Can model the join probability of two tuples using join indicator variable
111
Example SRM Person Income Age School Prestige J school Purchase J person Type 0.9998, 0.0002 Type=necessity false true false true 0.999, 0.001 Income = high 0.99, 0.01 Attended Bought-by
112
Path Dependency Graph Construct path dependency graph T C S JTJT U D JSJS 2 R A JSJS 1 B Q JRJR E JUJU t 1.C s 1.J T q.J R q.E q.J U u.D u.J S 2 r.A r.J S 1 s 1.B t 2.C s 2.J T s 2.B 21
113
SRM Semantics Theorem: If D is a model of then P U (V) = P (V) Definition: D is a model of if over same table-stratified schema I U (V,nondescendants(V)|Pa(V),J * =T) P U (V|Pa(V),J * =T) = P (V|Pa(V),J*=T)
114
Answering Queries Using SRMs Construct Query Evaluation BN for Query: select * from Person, Purchase where Person.id = Purchase.buyer-id and Person.Income = high and Purchase.Type = luxury Compute upward closure of query attributes by including all parents as well Person Income Age School Prestige Purchase J person Type J person Income Type J school Age Prestige J school
115
SRM Learning Learn parameters & qualitative dependency structure Extend known techniques for learning Bayesian networks from data and learning PRMs Database Patient Strain Contact
116
Structure selection Idea: like in BNs –define scoring function Generalization: Bayesian Score, MDL Compression: modified LL –do greedy local structure search Issues: –immense set of structures searching over large space –efficiency sufficient statistics harder to compute: associated with multiple entities requires intelligent use of DB technology
117
closecont hiv age smear care us_born hhoohh J_patient infected disease_site J_strain gender contage treatment race contype homeless unique Strain Contact Patient SRM for TB Database
118
hiv us_born J_strain gender race homeless unique Query Evaluation BN for TB Query: select * from patient, strain where patient.strain = strain.id and patient.homeless = true
119
Experimental Setup Compare –Sampling –BN w/ Uniform Join –SRM with same storage restrictions Two realworld databases: TB database and financial database* Relative error on three different query sets; each query joins three relations, select on one attribute from each relation * http://lisp.vse.cz/pkdd99/chall.htm, 1999
120
Results on Select-Join Queries Account 4.5K tuples Transaction 106K tuples District 77 tuples 0 10 20 30 40 50 60 70 80 90 100 123 Query Set Average Relative Error (%) SAMPLE BN+UJ SRM Patient 2.3K tuples Contact 20K tuples Strain 1K tuples Construction Time: 124 sec Estimation Time: 0.004 sec Construction Time: 157 sec Estimation Time: 0.002 sec 0 200 400 600 800 1000 1200 1400 123 Query Set Average Relative Error (%) SAMPLE BN+UJ SRM
121
SRMs vs PRMs Syntax - (almost) same Learning - (almost) same Semantics - (very) different Inference - (very) different
122
Conclusions PRMs can represent distribution over attributes from multiple tables PRMs can capture link uncertainty PRMs allow inferences about individuals while taking into account relational structure (they do not make inapproriate independence assuptions) SRMs provide a unified framework for selectivity estimation for both select and join operations SRMs provide extremely compact model that captures frequency information in multirelational data
123
Selected Publications “Learning Probabilistic Models of Link Structure”, L. Getoor, N. Friedman, D. Koller and B. Taskar, JMLR 2002. “Probabilistic Models of Text and Link Structure for Hypertext Classification”, L. Getoor, E. Segal, B. Taskar and D. Koller, IJCAI WS ‘Text Learning: Beyond Classification’, 2001. “Selectivity Estimation using Probabilistic Models”, L. Getoor, B. Taskar and D. Koller, SIGMOD-01. “Learning Probabilistic Relational Models”, L. Getoor, N. Friedman, D. Koller, and A. Pfeffer, chapter in Relation Data Mining, eds. S. Dzeroski and N. Lavrac, 2001. –see also N. Friedman, L. Getoor, D. Koller, and A. Pfeffer, IJCAI-99. “Learning Probabilistic Models of Relational Structure”, L. Getoor, N. Friedman, D. Koller, and B. Taskar, ICML-01. “From Instances to Classes in Probabilistic Relational Models”, L. Getoor, D. Koller and N. Friedman, ICML Workshop on Attribute-Value and Relational Learning: Crossing the Boundaries, 2000. Notes from AAAI Workshop on Learning Statistical Models from Relational Data, eds. L.Getoor and D. Jensen, 2000. Notes from IJCAI Workshop on Learning Statistical Models from Relational Data, eds. L.Getoor and D. Jensen, 2003. See http://www.cs.umd.edu/~getoor
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.