1. Nov. 13th, 20031 Causal Discovery Richard Scheines Peter Spirtes, Clark Glymour, and many others Dept. of Philosophy & CALD Carnegie Mellon

2. Nov. 13th, 20032 Outline 1.Motivation 2.Representation 3.Connecting Causation to Probability (Independence) 4.Searching for Causal Models 5.Improving on Regression for Causal Inference

3. Nov. 13th, 20033 1. Motivation Non-experimental Evidence Typical Predictive Questions Can we predict aggressiveness from the amount of violent TV watched Can we predict crime rates from abortion rates 20 years ago Causal Questions: Does watching violent TV cause Aggression? I.e., if we change TV watching, will the level of Aggression change?

4. Nov. 13th, 20034 Causal Estimation Manipulated Probability P(Y | X set= x, Z=z) from Unmanipulated Probability P(Y | X = x, Z=z) When and how can we use non-experimental data to tell us about the effect of an intervention?

5. Nov. 13th, 20035 2. Representation 1.Association & causal structure - qualitatively 2.Interventions 3.Statistical Causal Models 1.Bayes Networks 2.Structural Equation Models

6. Nov. 13th, 20036 Causation & Association X is a cause of Y iff  x 1  x 2 P(Y | X set= x 1 )  P(Y | X set= x 2 ) Causation is asymmetric: X Y  Y X X and Y are associated (X _||_ Y) iff  x 1  x 2 P(Y | X = x 1 )  P(Y | X = x 2 ) Association is symmetric: X _||_ Y  Y _||_ X

7. Nov. 13th, 20037 Direct Causation X is a direct cause of Y relative to S, iff  z,x 1  x 2 P(Y | X set= x 1, Z set= z)  P(Y | X set= x 2, Z set= z) where Z = S - {X,Y}

8. Nov. 13th, 20038 Causal Graphs Causal Graph G = {V,E} Each edge X  Y represents a direct causal claim: X is a direct cause of Y relative to V Chicken Pox

9. Nov. 13th, 20039 Causal Graphs Not Cause Complete Common Cause Complete

10. Nov. 13th, 200310 Modeling Ideal Interventions Ideal Interventions (on a variable X): Completely determine the value or distribution of a variable X Directly Target only X (no “fat hand”) E.g., Variables: Confidence, Athletic Performance Intervention 1: hypnosis for confidence Intervention 2: anti-anxiety drug (also muscle relaxer)

11. Nov. 13th, 200311 Sweaters On Room Temperature Pre-experimental SystemPost Modeling Ideal Interventions Interventions on the Effect

12. Nov. 13th, 200312 Modeling Ideal Interventions Sweaters On Room Temperature Pre-experimental SystemPost Interventions on the Cause

13. Nov. 13th, 200313 Interventions & Causal Graphs Model an ideal intervention by adding an “intervention” variable outside the original system Erase all arrows pointing into the variable intervened upon Intervene to change Inf Post-intervention graph ? Pre-intervention graph

14. Nov. 13th, 200314 Conditioning vs. Intervening P(Y | X = x 1 ) vs. P(Y | X set= x 1 ) Teeth Slides

15. Nov. 13th, 200315 Causal Bayes Networks P(S = 0) =.7 P(S = 1) =.3 P(YF = 0 | S = 0) =.99P(LC = 0 | S = 0) =.95 P(YF = 1 | S = 0) =.01P(LC = 1 | S = 0) =.05 P(YF = 0 | S = 1) =.20P(LC = 0 | S = 1) =.80 P(YF = 1 | S = 1) =.80P(LC = 1 | S = 1) =.20 P(S,YF, L) = P(S) P(YF | S) P(LC | S) The Joint Distribution Factors According to the Causal Graph, i.e., for all X in V P(V) =  P(X|Immediate Causes of(X))

16. Nov. 13th, 200316 Structural Equation Models 1. Structural Equations 2. Statistical Constraints Statistical Model Causal Graph

17. Nov. 13th, 200317 Structural Equation Models zStructural Equations: One Equation for each variable V in the graph: V = f(parents(V), error V ) for SEM (linear regression) f is a linear function zStatistical Constraints: Joint Distribution over the Error terms Causal Graph

18. Nov. 13th, 200318 Structural Equation Models Equations: Education =  ed Income =    Education  income Longevity =    Education  Longevity Statistical Constraints: (  ed,  Income,  Income ) ~N(0,  2 )  2  diagonal - no variance is zero Causal Graph SEM Graph (path diagram)

19. Nov. 13th, 200319 3. Connecting Causation to Probability

20. Nov. 13th, 200320 Causal Structure Statistical Predictions The Markov Condition Causal Graphs Independence X _||_ Z | Y i.e., P(X | Y) = P(X | Y, Z) Causal Markov Axiom

21. Nov. 13th, 200321 Causal Markov Axiom If G is a causal graph, and P a probability distribution over the variables in G, then in P: every variable V is independent of its non-effects, conditional on its immediate causes.

22. Nov. 13th, 200322 Causal Markov Condition Two Intuitions: 1) Immediate causes make effects independent of remote causes (Markov). 2) Common causes make their effects independent (Salmon).

23. Nov. 13th, 200323 Causal Markov Condition 1) Immediate causes make effects independent of remote causes (Markov). E || S | I E = Exposure to Chicken Pox I = Infected S = Symptoms Markov Cond.

24. Nov. 13th, 200324 Causal Markov Condition 2) Effects are independent conditional on their common causes. YF || LC | S Markov Cond.

25. Nov. 13th, 200325 Causal Structure  Statistical Data

26. Nov. 13th, 200326 Causal Markov Axiom In SEMs, d-separation follows from assuming independence among error terms that have no connection in the path diagram - i.e., assuming that the model is common cause complete.

27. Nov. 13th, 200327 Causal Markov and D-Separation In acyclic graphs: equivalent Cyclic Linear SEMs with uncorrelated errors: D-separation correct Markov condition incorrect Cyclic Discrete Variable Bayes Nets: If equilibrium --> d-separation correct Markov incorrect

28. Nov. 13th, 200328 D-separation: Conditioning vs. Intervening P(X 3 | X 2 )  P(X 3 | X 2, X 1 ) X 3 _||_ X 1 | X 2 P(X 3 | X 2 set= ) = P(X 3 | X 2 set=, X 1 ) X 3 _||_ X 1 | X 2 set=

29. Nov. 13th, 200329 4. Search From Statistical Data to Probability to Causation

30. Nov. 13th, 200330 Causal Discovery Statistical Data  Causal Structure Background Knowledge - X 2 before X 3 - no unmeasured common causes Statistical Inference

31. Nov. 13th, 200331 Representations of D-separation Equivalence Classes We want the representations to: Characterize the Independence Relations Entailed by the Equivalence Class Represent causal features that are shared by every member of the equivalence class

32. Nov. 13th, 200332 Patterns & PAGs Patterns (Verma and Pearl, 1990): graphical representation of an acyclic d-separation equivalence - no latent variables. PAGs: (Richardson 1994) graphical representation of an equivalence class including latent variable models and sample selection bias that are d-separation equivalent over a set of measured variables X

33. Nov. 13th, 200333 Patterns

34. Nov. 13th, 200334 Patterns: What the Edges Mean

35. Nov. 13th, 200335 Patterns

36. Nov. 13th, 200336 PAGs: Partial Ancestral Graphs What PAG edges mean.

37. Nov. 13th, 200337 PAGs: Partial Ancestral Graph

38. Nov. 13th, 200338 Tetrad 4 Demo www.phil.cmu.edu/projects/tetrad_download/

39. Nov. 13th, 200339 Overview of Search Methods Constraint Based Searches TETRAD Scoring Searches Scores: BIC, AIC, etc. Search: Hill Climb, Genetic Alg., Simulated Annealing Very difficult to extend to latent variable models Heckerman, Meek and Cooper (1999). “A Bayesian Approach to Causal Discovery” chp. 4 in Computation, Causation, and Discovery, ed. by Glymour and Cooper, MIT Press, pp. 141-166

40. Nov. 13th, 200340 5. Regession and Causal Inference

41. Nov. 13th, 200341 Regression to estimate Causal Influence Let V = {X,Y,T}, where - Y : measured outcome - measured regressors: X = {X 1, X 2, …, X n } - latent common causes of pairs in X U Y: T = {T 1, …, T k } Let the true causal model over V be a Structural Equation Model in which each V  V is a linear combination of its direct causes and independent, Gaussian noise.

42. Nov. 13th, 200342 Regression to estimate Causal Influence Consider the regression equation: Y = b 0 + b 1 X 1 + b 2 X 2 +..… b n X n Let the OLS regression estimate b i be the estimated causal influence of X i on Y. That is, holding X/Xi experimentally constant, b i is an estimate of the change in E(Y) that results from an intervention that changes Xi by 1 unit. Let the real Causal Influence X i  Y =  i When is the OLS estimate b i an unbiased estimate of  i ?

43. Nov. 13th, 200343 Linear Regression Let the other regressors O = {X 1, X 2,....,X i-1, X i+1,...,X n } b i = 0 if and only if  Xi,Y.O = 0 In a multivariate normal distribuion,  Xi,Y.O = 0 if and only if X i _||_ Y | O

44. Nov. 13th, 200344 Linear Regression So in regression: b i = 0  X i _||_ Y | O But provably :  i = 0   S  O, X i _||_ Y | S So  S  O, X i _||_ Y | S   i = 0 ~  S  O, X i _||_ Y | S  don’t know (unless we’re lucky)

45. Nov. 13th, 200345 Regression Example b 2 = 0 b 1  0 X 1 _||_ Y | X 2 X 2 _||_ Y | X 1 Don’t know ~  S  {X 2 } X 1 _||_ Y | S  S  {X 1 } X 2 _||_ Y | {X 1 }  2 = 0

46. Nov. 13th, 200346 Regression Example b 1  0 ~  S  {X 2,X 3 }, X 1 _||_ Y | S X 1 _||_ Y | {X 2,X 3 } X 2 _||_ Y | {X 1,X 3 } b 2  0 b 3  0 X 3 _||_ Y | {X 1,X 2 } DK  S  {X 1,X 3 }, X 2 _||_ Y | {X 1 }  2 = 0 DK ~  S  {X 1,X 2 }, X 3 _||_ Y | S

47. Nov. 13th, 200347 Regression Example X 2 Y X 3 X 1 PAG

48. Nov. 13th, 200348 Regression Bias If X i is d-separated from Y conditional on X/X i in the true graph after removing X i  Y, and X contains no descendant of Y, then: b i is an unbiased estimate of  i See “Using Path Diagrams ….”

49. Nov. 13th, 200349 Applications Parenting among Single, Black Mothers Pneumonia Photosynthesis Lead - IQ College Retention Corn Exports Rock Classification Spartina Grass College Plans Political Exclusion Satellite Calibration Naval Readiness

50. Nov. 13th, 200350 References Causation, Prediction, and Search, 2 nd Edition, (2000), by P. Spirtes, C. Glymour, and R. Scheines ( MIT Press) Causality: Models, Reasoning, and Inference, (2000), Judea Pearl, Cambridge Univ. Press Computation, Causation, & Discovery (1999), edited by C. Glymour and G. Cooper, MIT Press Causality in Crisis?, (1997) V. McKim and S. Turner (eds.), Univ. of Notre Dame Press. TETRAD IV: www.phil.cmu.edu/projects/tetradwww.phil.cmu.edu/projects/tetrad Web Course on Causal and Statistical Reasoning : www.phil.cmu.edu/projects/csr/ www.phil.cmu.edu/projects/csr/