COMP 538 Reasoning and Decision under Uncertainty Introduction Readings: Pearl (1998, Chapter 1 Shafer and Pearl, Chapter 1

COMP 538 Introduction / Slide 2 Objectives n Course objectives n Course contents

COMP 538 Introduction / Slide 3 Uncertainty  Uncertainty: the quality or state of being not clearly known.  Uncertainty appears in many tasks  Partial knowledge of the state of the world  Noisy observations  Phenomena that is not covered by our models  Inherent randomness

COMP 538 Introduction / Slide 4 Probability and Decision Theory  Well-known and well-understood framework for uncertainty  Clear semantics  Provides principled answers for:  Combining evidence  Predictive & Diagnostic reasoning  Incorporation of new evidence  Intuitive (at some level) to human experts  Can be learned

COMP 538 Introduction / Slide 5 Course Objectives n When applied to real-world problems, probability theory and decision theory suffer from n Complexity of model construction n Complexity of problem solving n This course covers methodologies developed recently in AI community for dealing with those complexity problems. n The methodologies combine ideas from several disciplines Artificial Intelligence, Machine Learning Decision Theory, Theory of Computer Science Statistics, Information Theory, Operations Research

COMP 538 Introduction / Slide 6 Complexity Problem of Applying Probability Theory n Example: n Patients in hospital are described by several attributes: n Background: age, gender, history of diseases, … n Symptoms: fever, blood pressure, headache, … n Diseases: pneumonia, heart attack, … n A joint probability distribution needs to assign a number to each combination of values of these attributes, exponential model size. 20 attributes require ( roughly 10 6 ) numbers n Real applications usually involve hundreds of attributes

COMP 538 Introduction / Slide 7 n Because of the exponential model size problem, it was believed that probability theory is not practical for dealing with uncertainty in AI. n Alternative uncertainty calculi were introduced: uncertainty factors, non-monotonic logic, fuzzy logic, etc. Complexity Problem of Applying Probability Theory

COMP 538 Introduction / Slide 8 n Bayesian networks alleviate the exponential model size problem. n Key idea: use conditional independence to factorize model into smaller parts. n Example: Alarm network 37 variables Model size: 2 37 Size of factored model: 509 PCWP CO HRBP HREKG HRSAT ERRCAUTER HR HISTORY CATECHOL SAO2 EXPCO2 ARTCO2 VENTALV VENTLUNG VENITUBE DISCONNECT MINVOLSET VENTMACH KINKEDTUBE INTUBATIONPULMEMBOLUS PAPSHUNT ANAPHYLAXIS MINOVL PVSAT FIO2 PRESS INSUFFANESTHTPR LVFAILURE ERRBLOWOUTPUT STROEVOLUMELVEDVOLUME HYPOVOLEMIA CVP BP Complexity Problem of Applying Probability Theory Model construction and problem solving becomes possible

COMP 538 Introduction / Slide 9 Advantages of Bayesian Networks n Semantics n Probability theory provides the glue whereby the parts are combined, ensuring that the system as a whole is consistent. n Alternative approaches suffer from several semantic deficiencies (Pearl 1988, Chapter 1). n Model construction by expert n Appealing graphical interface allows experts to build model for highly interacting variables. n Model construction from data n Probability foundation allows model construction from data by well established statistical principles such as maximum likelihood estimation and Bayesian estimation.

COMP 538 Introduction / Slide 10 Fielded Applications  Expert systems  Medical diagnosis  Fault diagnosis (jet-engines, Windows 98)  Monitoring  Space shuttle engines (Vista project)  Freeway traffic  Sequence analysis and classification  Speech recognition  Biological sequences  Information access  Collaborative filtering  Information retrieval See tutorial by Breese and Koller (1997) and online resources for application samples.

COMP 538 Introduction / Slide 11 Course Content/Bayesian Networks n Concept and semantics of Bayesian networks n Inference: How to answer queries efficiently n Learning: How to learn/adapt Bayesian network models from data n Causal models: How to learn causality from statistical data Detailed discussion of those topics can take one whole course (DUKE). We will focus on the main ideas and skip the details so that we can study other related topics.

COMP 538 Introduction / Slide 12 n Special cases of Bayesian networks: many of the classical multivariate models from n statistics, systems engineering, information theory, pattern recognition and statistical mechanics n Examples: mixture models, factor analysis, hidden Markov models, Kalman filters, Ising models. n Bayesian networks provide a way to view all of these models as instances of a common underlying formalism. Bayesian networks and classical multivariate models

COMP 538 Introduction / Slide 13 Course Content/Special models n Latent class analysis: n Statistical method for finding subtypes of related cases. n With proper generalization, might provide a statistical foundation for Chinese medicine diagnosis. n Hidden Markov models: n A temporal model widely used in pattern recognition: handwriting recognition, speech recognition.

COMP 538 Introduction / Slide 14 Decision Making under Uncertainty n Typical scenario: whether to take umbrella n Decision theory provides basis for rational decision making: Maximum expected utility principle. n Decision analysis: applying of decision theory. n Suffers from exponential model size.

COMP 538 Introduction / Slide 15 Course Content/Simple Decision Making n Influence diagrams n Generalization of Bayesian network. n Alleviate the complexity problem of decision analysis n Topics: n Evaluation: How to compute optimal decisions in an influence diagram. n Value of information: Whether it is worthwhile to collect more information to reduce uncertainty.

COMP 538 Introduction / Slide 16 n Agent/robot needs to execute multiple actions in order to achieve a goal n Uncertainty originates from noisy sensor and inaccurate actuators/uncontrollable enviroment factors n What is the best way to reach goal with minimum cost/time? Sequential Decision Making under Uncertainty

COMP 538 Introduction / Slide 17 Course Content/Sequential Decision Making n Models: n Markov decision processes, consider only uncertainty in actuators/environment factors n Partially observable decision processes, consider uncertainty in sensors and actuators/environment factors. n Solution methods: n Value iteration n Policy iteration n Dealing with model complexity using dynamic Bayesian networks n Learning sequential decision models n Model-based reinforcement learning

COMP 538 Introduction / Slide 18 Course Content/Summary n Bayesian networks n Concept and semantics, inference, learning, causilty n Special models: Hidden Markov models, latent class analysis n Influence diagrams: n Evaluation, value of information n Markov decision processes and partially observable Markov decision processes n Solution methods: value iteration, policy iteration, dynamic Bayesian networks n Model-based reinforcement learning