1. Topic Outline Motivation Representing/Modeling Causal Systems
Estimation and Updating
Model Search
Linear Latent Variable Models
Case Study: fMRI

2. Richard Scheines Carnegie Mellon University
Discovering
Pure Measurement Models
Richard Scheines Carnegie Mellon University
Ricardo Silva* University College London
Clark Glymour and Peter Spirtes Carnegie Mellon University

3. Outline Measurement Models & Causal Inference
Strategies for Finding a Pure Measurement Model
Purify
MIMbuild
Build Pure Clusters
Examples
Religious Coping
Test Anxiety

4. Goals: What Latents are out there?
Causal Relationships Among Latent Constructs
Depression
Relationship
Satisfaction
Depression
Relationship
Satisfaction or
or ?

5. Needed: Ability to detect conditional independence
among latent variables

6. Lead and IQ e2 e3 Lead _||_ IQ | PR PR ~ N(m=10, s = 3)
Parental Resources
Lead
Exposure IQ
Lead _||_ IQ | PR
PR ~ N(m=10, s = 3)
Lead = *PR + e2
e2 ~ N(m=0, s = 1.635)
IQ = *PR + e3
e3 ~ N(m=0, s = 15)

7. Psuedorandom sample: N = 2,000
Parental Resources
Lead
Exposure IQ
Regression of IQ on Lead, PR
Independent
Variable
Coefficient Estimate
p-value
Screened-off
at .05? PR
0.98
0.000 No
Lead
-0.088
0.378
Yes

8. Measuring the Confounder
Lead Exposure
Parental Resources IQ X1 X2 X3 e1 e2 e3
X1 = g1* Parental Resources + e1
X2 = g2* Parental Resources + e2
X3 = g3* Parental Resources + e3
PR_Scale = (X1 + X2 + X3) / 3

9. Scales don't preserve conditional independence
Lead Exposure
Parental Resources IQ X1 X2 X3
PR_Scale = (X1 + X2 + X3) / 3
Independent
Variable
Coefficient Estimate
p-value
Screened-off
at .05?
PR_scale
0.290
0.000 No
Lead
-0.423

10. Indicators Don’t Preserve Conditional Independence
Lead Exposure
Parental Resources IQ X1 X2 X3
Regress IQ on: Lead, X1, X2, X3
Independent
Variable
Coefficient Estimate
p-value
Screened-off
at .05? X1
0.22
0.002 No X2
0.45
0.000 X3
0.18
0.013
Lead
-0.414

11. Structural Equation Models WorkX1 X2 X3
Parental Resources
Lead Exposure IQ b
Structural Equation Model
(p-value = .499)
Lead and IQ “screened off” by PR

12. Local Independence / Pure Measurement Models
For every measured item xi:
xi _||_ xj | latent parent of xi

13. Local Independence Desirable

14. Correct Specification Crucial

15. Strategies Find a Locally Independent Measurement Model
Correctly specify the MM, including deviations from Local Independence

16. Correctly Specify Deviations from Local Independence

17. Correctly Specifying Deviations from Local Independence
is Often Very Hard

18. Finding Pure Measurement Models -
Much Easier

19. Tetrad Constraints Fact: given a graph with this structure
it follows that L
W = 1L + 1
X = 2L + 2
Y = 3L + 3
Z = 4L + 4 1 4 2 3 W X Y Z
WXYZ = WYXZ = WZXY
tetrad
constraints
CovWXCovYZ
= (122L) (342L) = = (132L) (242L) =
CovWYCovXZ

20. Early Progenitors g rm1 * rr1 = rm2 * rr2 Charles Spearman (1904)
Statistical Constraints  Measurement Model Structure g m1 m2 r1 r2
rm1 * rr1 = rm2 * rr2 1

21. Impurities/Deviations from Local Independence
defeat tetrad constraints selectively
rx1,x2 * rx3,x4 = rx1,x3 * rx2,x4
rx1,x2 * rx3,x4 = rx1,x4 * rx2,x3
rx1,x3 * rx2,x4 = rx1,x4 * rx2,x3
rx1,x2 * rx3,x4 = rx1,x3 * rx2,x4
rx1,x2 * rx3,x4 = rx1,x4 * rx2,x3
rx1,x3 * rx2,x4 = rx1,x4 * rx2,x3

22. Purify
True Model
Initially Specified Measurement Model

23. Purify
Iteratively remove item whose removal most improves measurement model fit (tetrads or c2)
– stop when confirmatory fit is acceptable
Remove x4
Remove z2

24. Purify Detectibly Pure Subset of Items
Detectibly Pure Measurement Model

25. Purify

26. How a pure measurement model is useful
Consistently estimate covariances/correlations among latents - test conditional independence with estimated latent correlations
Test for conditional independence among latents directly

27. 2. Test conditional independence relations among latents directly
Question: L1 _||_ L2 | {Q1, Q2, ..., Qn}
b21
b21 = 0  L1 _||_ L2 | {Q1, Q2, ..., Qn}

28. MIMbuild Input: - Purified Measurement Model
- Covariance matrix over set of pure items
MIMbuild
PC algorithm with independence tests
performed directly on latent variables
Output: Equivalence class of structural models
over the latent variables

29. Purify & MIMbuild

30. Goal 2: What Latents are out there?
How should they be measured?

31. Latents and the clustering of items they measure
imply tetrad constraints diffentially

32. Build Pure Clusters (BPC)
Input:
- Covariance matrix over set of original items
BPC
1) Cluster (complicated boolean combinations of tetrads)
2) Purify
Output: Equivalence class of measurement models over a pure subset of original Items

33. Build Pure Clusters

34. Build Pure Clusters Qualitative Assumptions Quantitative Assumptions:
Two types of nodes: measured (M) and latent (L)
M L (measured don’t cause latents)
Each m  M measures (is a direct effect of) at least one l  L
No cycles involving M
Quantitative Assumptions:
Each m  M is a linear function of its parents plus noise
P(L) has second moments, positive variances, and no deterministic relations

35. Build Pure Clusters Output - provably reliable (pointwise consistent):
Equivalence class of measurement models over a pure subset of M
For example:
True Model
Output

36. Build Pure Clusters Output
Measurement models in the equivalence class are at most refinements, but never coarsenings or permuted clusterings.
Output

37. Build Pure Clusters Algorithm Sketch:
Use particular rank (tetrad) constraints on the measured correlations to find pairs of items mj, mk that do NOT share a single latent parent
Add a latent for each subset S of M such that no pair in S was found NOT to share a latent parent in step 1.
Purify
Remove latents with no children

38. Build Pure Clusters + MIMbuild

39. Case Studies Stress, Depression, and Religion (Lee, 2004)
Test Anxiety (Bartholomew, 2002)

40. Case Study: Stress, Depression, and Religion
Masters Students (N = 127) item survey (Likert Scale)
Stress: St1 - St21
Depression: D1 - D20
Religious Coping: C1 - C20
Specified Model
p = 0.00

41. Case Study: Stress, Depression, and Religion
Build Pure Clusters

42. Case Study: Stress, Depression, and Religion
Assume Stress temporally prior:
MIMbuild to find Latent Structure:
p = 0.28

43. Case Study : Test Anxiety
Bartholomew and Knott (1999), Latent variable models and factor analysis
12th Grade Males in British Columbia (N = 335)
20 - item survey (Likert Scale items): X1 - X20:
Exploratory Factor Analysis:

44. Case Study : Test Anxiety
Build Pure Clusters:

45. Case Study : Test Anxiety
Build Pure Clusters:
Exploratory Factor Analysis:
p-value = 0.00
p-value = 0.47

46. Case Study : Test Anxiety
MIMbuild
p = .43
Uninformative
Scales: No Independencies or Conditional Independencies

47. Limitations
In simulation studies, requires large sample sizes to be really reliable (~ ).
2 pure indicators must exist for a latent to be discovered and included
Moderately computationally intensive (O(n6)).
No error probabilities.

48. Open Questions/Projects
IRT models?
Bi-factor model extensions?
Appropriate incorporation of background knowledge

49. References Tetrad: www.phil.cmu.edu/projects/tetrad_download
Spirtes, P., Glymour, C., Scheines, R. (2000). Causation, Prediction, and Search, 2nd Edition, MIT Press.
Pearl, J. (2000). Causation: Models of Reasoning and Inference, Cambridge University Press.
Silva, R., Glymour, C., Scheines, R. and Spirtes, P. (2006) “Learning the Structure of Latent Linear Structure Models,” Journal of Machine Learning Research, 7,
Learning Measurement Models for Unobserved Variables, (2003). Silva, R., Scheines, R., Glymour, C., and Spirtes. P., in Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence , U. Kjaerulff and C. Meek, eds., Morgan Kauffman