1. 1 © 1998 HRL Laboratories, LLC. All Rights Reserved Development of Bayesian Diagnostic Models Using Troubleshooting Flow Diagrams K. Wojtek Przytula: HRL Laboratories & Don Thompson: Pepperdine University Malibu, California

2. 2 © 1998 HRL Laboratories, LLC. All Rights Reserved The Troubleshooting Problem Given a malfunctioning system initial observations (symptoms, error messages in archive) Derive a sequence of tests to provide more observations a diagnosis of one or more defects

3. 3 © 1998 HRL Laboratories, LLC. All Rights Reserved Sample System Test Initial Observations

4. 4 © 1998 HRL Laboratories, LLC. All Rights Reserved Troubleshooting with Software Assistants Quantitative measures Flexible assistance (not restrictive in recommendations) Use of all available initial evidence Recommendation of next best test as well as alternatives (or ranked list of tests) Suggestion of most likely defects (or ranked list of defects) Suggestion of when to stop troubleshooting, allowing for continuation with next best observation

5. 5 © 1998 HRL Laboratories, LLC. All Rights Reserved Three Approaches to Software Assistants Troubleshooting Flow diagrams (TFD) Case Based Reasoning (CBR) Bayesian Networks (BN)

6. 6 © 1998 HRL Laboratories, LLC. All Rights Reserved Troubleshooting Flow Diagram Example Initial Evidence T-Test D-Defect

7. 7 © 1998 HRL Laboratories, LLC. All Rights Reserved Troubleshooting Flow Diagram Characteristics Natural representation of troubleshooting knowledge & process Single fault diagnosis Inflexible—no alternative next test Simple to execute Hard to create and modify Provides sequencing of tests and stopping of troubleshooting

8. 8 © 1998 HRL Laboratories, LLC. All Rights Reserved Case Based Example

9. 9 © 1998 HRL Laboratories, LLC. All Rights Reserved Application of Case Bases to Troubleshooting Given initial evidence, find the best case base Recommend next best test using separate sequencing algorithm, e.g. decision tree algorithm: ID3 End of troubleshooting determined by exhausting all tests

10. 10 © 1998 HRL Laboratories, LLC. All Rights Reserved Characteristics of Case Based Approach Natural representation of diagnostic knowledge (diagnostic cases) Single fault diagnosis Flexible choice of next test at expense of rebuilding decision trees Hard to create and modify large case bases (e.g. handling of conflicts)

11. 11 © 1998 HRL Laboratories, LLC. All Rights Reserved Bayesian Networks Example Prior Probability Distribution: P(D i ) for all i Conditional Probability Distribution: P(T i | D j ) for all i,j Structure: Causal Dependencies Between Defects and Tests Parameters:

12. 12 © 1998 HRL Laboratories, LLC. All Rights Reserved Application of Bayesian Networks to Troubleshooting Given initial evidence, provides list of defects ranked by probability Recommends ranked list of next test using separate sequencing algorithm – e.g. Value of Information (VOI) algorithm Recommends end of troubleshooting using separate stopping algorithm – e.g. heuristic evaluation of ranked lists of defects and tests

13. 13 © 1998 HRL Laboratories, LLC. All Rights Reserved Characteristics of Bayesian Network Approach Troubleshooting knowledge represented as causal probabilistic model Multiple and single fault diagnosis Flexible quantitative choice of next test Easy to modify and maintain (e.g. learning) Requires probabilities – obtained by expert estimation or learning from data

14. 14 © 1998 HRL Laboratories, LLC. All Rights Reserved Conversion and Comparison of Three Approaches TFDCBRSequencing BN Algorithm Learning Algorithm

15. 15 © 1998 HRL Laboratories, LLC. All Rights Reserved Flow Diagrams to Conversion to Case Bases Conversion of Flow Diagrams to Case Bases Trace each path of TFD to create one entry in CB (ordering of tests lost)

16. 16 © 1998 HRL Laboratories, LLC. All Rights Reserved Conversion of Flow Diagrams to Bayesian Networks Represent all defects as states of single defect node Represent tests as separate children of defect node (sequencing of tests lost) Probabilities of defects: 1/n. Conditional probabilities: 0, 1, ½

17. 17 © 1998 HRL Laboratories, LLC. All Rights Reserved Conclusions Flow diagrams are natural and popular, but The user cannot modify test sequence Time-consuming to create Different experts = Different diagrams Impractical for complex systems

18. 18 © 1998 HRL Laboratories, LLC. All Rights Reserved Conclusions – Case Bases Seed case base can be created from flow diagrams and then augmented by new cases For troubleshooting, case base needs to be combined with sequencing algorithms For sequencing, tests and cases can be weighted or repeated to express test cost or defect frequency More expressive and flexible than TFD

19. 19 © 1998 HRL Laboratories, LLC. All Rights Reserved Conclusions – Bayesian Networks Seed BN can be automatically obtained from flow diagram BN can be learned from case base For troubleshooting, BN needs to be combined with sequencing algorithm Encode failure rates explicitly in the model and express uncertainty about impact of defect on test results BN can be easily converted from single-fault to multiple- fault diagnostic tool More flexible solution than TFD or CBR