Bayesian Nets and Applications Next class: machine learning C. 18.1, 18.2 Homework due next class Questions on the homework? Prof. McKeown will not hold office hours today

2 Bayesian Networks A directed acyclic graph in which each node is annotated with quantitative probability information –A set of random variables makes up the network nodes –A set of directed links connects pairs of nodes. If there is an arrow from node X to node Y, X is a parent of Y –Each node X i has a conditional probability distributionP(X i |Parents(X i ) that quantifies the effect of the parents on the node

3 Example Topology of network encodes conditional independence assumptions

4 Smart Good test taker Understands material Hard working Exam GradeHomework Grade

5 Smart Good test taker Understands material Hard working Exam GradeHomework Grade Smart TrueFalse.5 Hard Working TrueFalse.7.3 SGood Test Taker TrueFalse True False SHWUM TrueFalse True TrueFalse.6.4 FalseTrue.6.4 False.2.8

6 Conditional Probability Tables Smart TrueFalse.5 Hard Working TrueFalse.7.3 SGood Test Taker TrueFalse True False SHWUM TrueFalse True TrueFalse.6.4 FalseTrue.6.4 False.2.8 GTTUMExam Grade ABCDF True TrueFalse FalseTrue False Homework Grade UMABCDF True False

7 Compactness A CPT for Boolean X i with k Boolean parents has 2 k rows for the combinations of parent values Each row requires one number p for X i =true (the number for X i =false is just 1-p) If each variable has no more than k parents, the complete network requires O(nx2 k ) numbers Grows linearly with n vs O(2 n ) for the full joint distribution Student net: =11 numbers (vs. 26-1)=31

8 Global Semantics/Evaluation Global semantics defines the full joint distribution as the product of the local conditional distributions: P(x 1,…,x n )=∏ i n =1 P(x i | Parents(X i )) e.g., P(EG=AΛGTΛ⌐UMΛSΛHW)

9 Global Semantics Global semantics defines the full joint distribution as the product of the local conditional distributions: P(X 1,…,X n )=∏ i n =1 P(X i |Parents(X i )) e.g., Observations:S, HW, not UM, will I get an A? P(EG=AΛGTΛ⌐UMΛSΛHW) = P(EG=A|GT Λ⌐UM)*P(GT|S)*P(⌐UM |HW ΛS)*P(S)*P(HW)

10 Conditional Independence and Network Structure The graphical structure of a Bayesian network forces certain conditional independences to hold regardless of the CPTs. This can be determined by the d- separation criteria

11 a b c a b c b a c Linear Converging Diverging

12 D-separation (opposite of d- connecting) A path from q to r is d-connecting with respect to the evidence nodes E if every interior node n in the path has the property that either It is linear or diverging and is not a member of E It is converging and either n or one of its decendants is in E If a path is not d-connecting (is d-separated), the nodes are conditionally independent given E

13 Smart Good test taker Understands material Hard working Exam GradeHomework Grade

14 S and EG are not independent given GTT S and HG are independent given UM

Medical Application of Bayesian Networks: Pathfinder

16 Pathfinder Domain: hematopathology diagnosis –Microscopic interpretation of lymph-node biopsies –Given: 100s of histologic features appearing in lymph node sections –Goal: identify disease type malignant or benign –Difficult for physicians

17 Pathfinder System Bayesian Net implementation Reasons about 60 malignant and benign diseases of the lymph node Considers evidence about status of up to 100 morphological features presenting in lymph node tissue Contains 105,000 subjectively-derived probabilities

18

19 Commercialization Intellipath Integrates with videodisc libraries of histopathology slides Pathologists working with the system make significantly more correct diagnoses than those working without Several hundred commercial systems in place worldwide

20 Sequential Diagnosis

21 Features Structured into a set of 2-10 mutually exclusive values Pseudofollicularity Absent, slight, moderate, prominent Represent evidence provided by a feature as F 1,F 2, … F n

22 Value of information User enters findings from microscopic analysis of tissue Probabilistic reasoner assigns level of belief to different diagnoses Value of information determines which tests to perform next Full disease utility model making use of life and death decision making Cost of tests Cost of misdiagnoses

23

24

25 Group Discrimination Strategy Select questions based on their ability to discriminate between disease classes For given differential diagnoisis, select most specific level of hierarchy and selects questions to discriminate among groups Less efficient Larger number of questions asked

26

27

28 Other Bayesian Net Applications Lumiere – Who knows what it is?

29 Other Bayesian Net Applications Lumiere Single most widely distributed application of BN Microsoft Office Assistant Infer a user’s goals and needs using evidence about user background, actions and queries VISTA Help NASA engineers in round-the-clock monitoring of each of the Space Shuttle’s orbiters subsystem Time critical, high impact Interpret telemetry and provide advice about likely failures Direct engineers to the best information In use for several years Microsoft Pregnancy and Child Care What questions to ask next to diagnose illness of a child

30 Other Bayesian Net Applications Speech Recognition Text Summarization Language processing tasks in general