1. Causal Effect Estimation with Observational Data: Methods and Applications Part II
Michael Lamm and Yiu-Fai Yung
SAS Institute
2018 Iowa and Nebraska SAS Users Groups
Hi everyone!
I am Yiu-Fai Yung. And this is Michael Lamm.
Today we are going to give a workshop on causal treatment effect estimation.
Because both of use work for SAS as software developers, we are going to illustrate all the analyses by SAS software.
This is a basic course for causal effect analysis, and so I think that you can use other software to do similar analyses covered by this workshop.
Hopefully, even if you are not a SAS user, this workshop can still offer you a good introduction of this important topic.

2. Outline
Part I
Issues of causal inference from observational data
Introducing the propensity score
Theories and assumptions
Matching methods
Part II
Weighting methods
Doubly robust methods
Limitations
Summary and conclusions
Here is the outline of this workshop.
First, I introduce the issues about causal inferences from observational data.
Then I am going to illustrate propensity score matching method and briefly describe the propensity score stratification method.
I will spend some time on the theories and assumptions behind the causal analysis techniques.
After that, I will show you the importance of checking covariate balance.
In the middle , we will then take a ten-minute break.
Michael will then take over and covers the weighting, regression adjustment, and doubly robust methods.
He will also use an example to demonstrate different kinds of limitations of causal analysis that you might encounter in practice.
Finally, I will make a summary and some conclusions.
During the workshop, you are welcome to ask questions that clarify definitions and presentations.
If you have complicated issues that need more time to discuss, we can do that after the workshop.

3. (Academic Performance)
Confounding Variables complicate the estimation of causal effect from an observational study
Sports
Music
(Music Training)
GPA
(Academic Performance)
Confounding variables are pretreatment characteristics associated with both the treatment and the outcome variables
Confounding variables explain parts of the observed treatment outcome association and can bias causal effect estimates
Now look at case of observational studies.
Here, you do not control whether the students receive music training or not.
You simply collect data about whether they have taken music training in the past year and their current GPAs.
Without random assignment to the treatment or control conditions, the pre- treatment characteristics of the subjects are not controlled.
These pre-treatment characteristics become the confounding variables in the observational study.
In this causal diagram, Gender acts as a confounding variable for the observed music training effect on academic performance.
In observational studies, the treatment effects are usually confounded by some pretreatment or background characteristics.

4. The propensity score is commonly used as the basis of matching methodsNo
(Re-) Specify a propensity score model
Good covariate balance?
Yes
Outcome
analysis
A propensity score is the probability of receiving treatment given 𝑋=𝑥: 𝑒 𝑥 =Prob 𝑇=1|𝑋=𝑥

5. Causal effects are defined by using potential outcomes
Potential outcomes are used to describe what outcome would occur for a subject under every possible treatment scenario
𝑌 1 : potential outcome in the treatment condition
𝑌(0): potential outcome in the control condition
You can estimate the ATE: E(𝑌 1 – 𝑌 0 ) or the ATT: E 𝑌 1 −𝑌 0 ) 𝑇=1)
The stable unit treatment value assumption (SUTVA) ensures that causal effects are well-defined
The consistency assumption relates the observed outcomes to the potential outcomes: 𝑌=𝑇∙𝑌 1 +(1−𝑇)∙𝑌 0
No unmeasured confounding is assumed to enable the identification of treatment effects: 𝑌 𝑖 ⫫𝑇 | 𝑋 , 𝑖=0, 1

6. Illustration of the Potential Outcomes Framework
Let us now try to understand the logic of the matching method under the potential outcomes framework.

7. If you can observe all the potential outcomes ...
Y(1) –Y(0) 1 3 4 2 5 7 −1 6 8 9 10
A hypothetical perfect sample in which you can observe all potential outcomes Y(1), Y(0)
Average treatment effect =
Mean(Y(1)) − Mean(Y(0)) = 6 − 4 =2
This simple example contains 10 observations in an observational study.
First, suppose you can observe all their potential outcomes Y(1) and Y(0).
The estimation of the causal effect would be very easy.
You can compute average treatment effect by two ways.
One way is to compute the treatment effect of each individual and take the average.
Another way is compute the marginal mean difference in the potential outcome columns.
Either way, you get ATE=2.

8. The fundamental problem of causal inference
Obs T Y
Obs. Y(0)
Obs. Y(1) 1 4 ? 2 3 7 5 6 8 9 10
Holland (1986)
Each Y indicates only one of the potential outcomes
The other potential outcome is always missing
Observed mean of (Y|T=0) = 2
Observed mean of (Y|T=1) = 6
Observed effect = 4
A better way to look at these observational data is to split outcome Y into two columns of observed potential outcomes.
Then you will see that causal analysis can be viewed as a missing data problem.
That is, for each individual one of potential outcome is observed and the other is always missing.
The question now is how we can try to get unbiased estimation of causal effect from this data set?

9. Propensity Scores Weighting Methods
Let’s now come back from the theory to an example of the propensity score weighting methods.
Instead of doing matching based on the propensity scores, the weighting methods use propensity scores for adjustment.

10. Inverse probability weighting can create a pseudo population with comparable treatment conditions
Inverse probability weighting is a common approach for handling missing data
A patient with covariates 𝑥 and treatment 𝑇=𝑖 will have a weight based on the propensity score e x =𝑃𝑟𝑜𝑏 𝑇=𝑖 | 𝑋=𝑥

11. Observational studies have a large amount of missing potential outcomes
Only a single outcome is observed for each subject, so at least half of the potential outcomes are “missing”
This is a much larger amount of missing data than typically encountered in an experiment of RCT

12. Like experiments, observational studies should be carefully designed to ensure proper analyses
Designing an experiment helps ensure that you are examining a well- defined causal question that satisfies the SUTVA
What is the target population?
When is treatment assigned and how long is the treatment period?
Is the outcome being properly measured?
The same questions should be considered when designing and analyzing observational studies
Clear answers to design questions are necessary to state the conditions under which claims of causality are valid

13. Example 3. Propensity Scores Weighting Methods
Let’s now come back from the theory to an example of the propensity score weighting methods.
Instead of doing matching based on the propensity scores, the weighting methods use propensity scores for adjustment.

14. Does quitting smoking lead to weight change?
Data: A subset (N=1,746) of NHANES I Epidemiologic Follow-Up Study (NHEFS) in Hernán and Robins (2016)
Collect medical and behavioral information in an initial physical examination
Follow-up interviews were done approximately 10 years later
Treatment variable Quit: quit smoking during the 10-year period
Outcome variable Change: change in weight (in kg)
Confounders include: Activity, Age, BaseWeight, Education, Exercise, PerDay, Race, Sex, Weight, YearsSmoke
The data set here contains 1746 observations of an epidemiology study.
The treatment variable is Quit, representing whether an individual quit smoking during a 10-year period.
The outcome variable is Change, representing the change in weight.
Hence, the central question is whether quitting smoking will cause weight change.
Other background and behavioral information were also measured in an initial physical examination.

15. PROC PSMATCH: Estimating ATT through output weights
proc psmatch data=smokingweight; class Sex Race Education Exercise Activity Quit ; psmodel Quit(Treated='1') = Sex Age Education Exercise Activity YearsSmoke PerDay; output out= smokeATTWeights attwgt=attwgt; run; proc ttest data=smokeATTWeights; class Quit; var Change; weight attwgt;
Here is an example that uses the PSMATCH procedure.
In the PSMATCH statement, you specify the data set. The REGION=ALLOBS option instructs the procedure to use all observations.
In this example, you still specify the propensity score model in the PSMODEL statement. The propensity scores will be estimated from this model and then the weights are computed from the estimated propensity scores.
Notice that you no longer use the MATCH statement because you are not requesting the propensity score matching method.
The PSMATCH procedure produces weights in the output data set outPSWeights.
You will then use this data set to do outcome analysis for the causal effect.
This example still estimates the ATT because the weight being used in the WEIGHT statement of the t-test is _attwgt_.

16. Prob 𝑇=1 𝑋=𝑥) Prob 𝑇=𝑡 𝑋=𝑥 )
ATT weights correct for bias and target the analysis to the population of interest
For an individual with observed treatment 𝑇=𝑡 and covariates 𝑥, the ATT weight equals to
Prob 𝑇=1 𝑋=𝑥) Prob 𝑇=𝑡 𝑋=𝑥 )
The propensity score is, 𝑒 𝑥 = 𝑃𝑟𝑜𝑏 𝑇=1 𝑋=𝑥)
For subjects in the treatment condition,
ATT Weight = 𝑒 𝑥 𝑒 𝑥 = 1
For subject in the control condition,
ATT Weight = 𝑒 𝑥 (1 −𝑒 𝑥 )

17. The output data set contains the original data, propensity scores, and ATT weights
Obs sex age ... _PS_ attwgt
Let’s look at the output data set created by PROC PSMATCH.
It contains most of the original data in the School data set.
But now PROC PSMATCH appends two more columns at the end.
The _PS_ column shows the propensity scores.
The _ATTWHT_ column shows the so-called ATT weights. These are the weights that you want to apply in any subsequent outcome analysis.

18. Inverse propensity score weighting method for estimating ATT
T-test with weights based on the sample created from the PSMATCH procedure
Variable: Gpa
Quit Method Mean % CL Mean Std Err
Diff (1-2) Pooled
Diff (1-2) Satterthwaite
Here are the results by the two procedures for the ATT estimation.
The T-test results based on the data that are created by the PSMATCH shows the essentially the same POMS and ATT as that of PROC CAUSALTRT.
Notice that you need to flip the results of the diff in the t-test.
The difference is the estimate of standard error.
Which standard error should you trust?

19. PROC CAUSALTRT: Estimating ATT with METHOD=IPWR
proc causaltrt data=School method=ipwr att; class Sex Race Education Exercise Activity Quit ; psmodel Quit(Event='1') = Sex Age Education Exercise Activity YearsSmoke PerDay / plots = pscovden(effects(age YearsSmoke)); model gpa; run;
The model statement and psmodel statement are both required when you use PROC CAUSALTRT
You use the method= option to select an estimation method
By default PROC CASUALTRT estimates the ATE, to you request estimation of the ATT you use the att option
Before jumping into the results of the T-test, let us look at how the same kind of propensity score method can also be done in PROC CAUSALTRT.
When you use PROC CAUSALTRT, you can just input the School data set directly because the procedure would compute weights and the causal effect internally.
Instead, you use some options to specify the estimation method.
In this example, METHOD=IPWR requests the inverse probability weighting method with ratio adjustment.
The ATT option specifies the target estimand is average treatment effect for the treated.
The PSMODEL specification is the same as that of the PSMATCH procedure because you are fitting exactly the same propensity score model.
However, the MODEL statement specify the GPA variable here. This simply indicates that GPA is the outcome variable in this analysis and that you are supplying any outcome model here.

20. Estimation of ATT by the IPWR method of the CAUSALTRT procedure
Inverse propensity score weighting method for estimating ATT
Estimation of ATT by the IPWR method of the CAUSALTRT procedure
Analysis of Causal Effect
Treatment Robust Wald 95%
Parameter Level Estimate Std Err Confidence Limits Z Pr > |Z|
POM <.0001
POM <.0001
ATT <.0001
Here are the results by the two procedures for the ATT estimation.
The T-test results based on the data that are created by the PSMATCH shows the essentially the same POMS and ATT as that of PROC CAUSALTRT.
Notice that you need to flip the results of the diff in the t-test.
The difference is the estimate of standard error.
Which standard error should you trust?

21. Which PS-Weighting method in the CAUSALTRT and PSMATCH procedures should you use?
Both yield the same estimates of ATT in this example
PROC CAUSALTRT produces standard error estimates that takes the estimation of propensity scores into account
The catch: You must be certain that your propensity score model is correct
In this specific example, both produce insignificant difference. In general, the standard error by the CAUSALTRT is more trustable because it takes into account of the estimation of the propensity scores. However, this also assumes that your propensity score model is correct and you are not doing exploring steps to arrive at this propensity score model.
In practice, it seems to me that most researchers do not view the two-step method as a big disadvantage in estimating the standard error.

22. CAUSALTRT or PSMATCH
Same theoretical foundations: Potential outcomes framework (Neyman 1923; Rubin 1974)
Some overlap in functionalities (e.g., weighting methods)
PROC PSMATCH motto: “Do not involve the outcome variables when you do propensity score analysis---stratification, matching or weighting”
Advantage: separation of design from analysis enables exploratory analysis in propensity score analysis
PROC CAUSALTRT: Results from the propensity score model and outcome model can be “combined”--- AIPW or IPWREG
Advantage: more efficient point estimates and more accurate standard error estimates

23. Regression Adjustment and Doubly Robust Methods
So far we have discussed only the propensity score methods for causal effect estimation.
The propensity score methods basically rely on the modeling of the treatment variable.
Another approach would be the outcome variable modeling that the regression adjustment methods rely on.

24. Methods for Causal Effect Estimation in PROC CAUSALTRT
Treatment Model No
Yes
Outcome
Model
Weighting methods
Regression adjustment methods
Doubly robust methods
This table summarizes these approaches that PROC CAUSALTRT uses
If you model only the treatment variable in PROC CAUSALTRT, it uses propensity score weighting methods for estimating causal effects.
If you model only the outcome variable, PROC CAUSALTRT uses regression adjustment methods.
If you specify both treatment and outcome modeling, PROC CAUSALTRT uses doubly robust methods.

25. Regression Adjustment Method
Estimation by regression adjustment performs the following steps:
Fit models for the outcome separately within each of the treatment conditions
Compute predicted outcomes for each subject from these models
Use the predicted values to estimate the treatment effect of interest
METHOD=REGADJ option in PROC CAUSALTRT
Let’s first build some intuition about the regression adjustment methods.
You can use METHOD=REGADJ option in PROC CAUSALTRT.
There are two main steps of the regression adjustment method.
First, it estimates the potential outcomes for each individual from fitting the outcome models separately for each treatment level.
Then, the causal treatment effect can be computed as if all the potential outcomes have been observed.

26. Considerations when using regression adjustment
The method is dependent on a correctly specified outcome model
Incorrectly specified outcome models can lead to biased model estimates and biased treatment effects
Extrapolation might be a concern if covariate distributions in treatment conditions are systematically different

27. Doubly Robust Methods of PROC CAUSALTRT
Augmented inverse probability weighting
Estimate the propensity score and perform weighing
Augment weighting by using predicted outcome values
METHOD=AIPW in PROC CAUSALTRT
Inverse probability weighted regression
Estimate the propensity scores
Fit outcome models with inverse probability weights
Estimate the causal effects by using predicted values from
METHOD=IPWREG in PROC CAUSALTRT
Conceptually, the AIPW method computes the causal effect in two ways.
One is by using the weighting method that is based on the propensity scores.
Another is by using the regression adjustment method.
The estimation from these two methods are then combined in a formula to estimate the causal treatment effect.
Another doubly robust method is the IPWREG method, which also estimate the propensity scores and computes the inverse probability weights
These weights are then used in the fitting the weighted regression for the outcome model.
We will illustrate the AIPW method only.

28. Main Idea of Doubly Robust Methods
You can get unbiased estimation of causal treatment effects if either or both of the following models that you specify are true:
Propensity score model for the treatment variable
Regression model for the outcome model
“Doubly” robust: You have two chances to get it right
AIPW formulas:
𝜇 0 𝑎𝑖𝑝𝑤 = 1 𝑛 𝑖=1 𝑛 1− 𝑡 𝑖 𝑦 𝑖 1− 𝑒 𝑖 + 𝑦 𝑖0 ( 𝑡 𝑖 − 𝑒 𝑖 1− 𝑒 𝑖 )
𝜇 1 𝑎𝑖𝑝𝑤 = 1 𝑛 𝑖=1 𝑛 𝑡 𝑖 𝑦 𝑖 𝑒 𝑖 − 𝑦 𝑖1 ( 𝑡 𝑖 − 𝑒 𝑖 𝑒 𝑖 )
Analytic formulas for computing standard errors (Lunceford & Davidian 2004)
The main benefit of the doubly robust methods is that you got two chances to get it right.
This means that you can get unbiased estimation of causal treatment effect if either your propensity score model or your outcome model is specified correctly.
But the “double” does not mean that it is robust against two model misspecification.
The AIPW formulas for computing potential outcome means are like these.
Clearly, the first component is due to the propensity score weighting method. The second component is due to regression adjustment.
Another potential advantage of the AIPW method is that standard error estimates can be computed analytically.

29. Example 4. AIPW Example
This example illustrate the AIPW method of PROC CAUSALTRT.

30. Estimating ATE by the AIPW Method
proc causaltrt data=smokingweight method=aipw covdiffps plots=all;
class Sex Race Education Exercise Activity Quit /descending;
psmodel Quit = Sex Age Education Exercise Activity
YearsSmoke PerDay;
model Change = Sex Age Exercise Activity BaseWeight;
run;
The aipw estimation method requires models for both the treatment and the outcome
You can request measures of covariate balance by using the covdiffps option
This example demonstrates the AIPW method for estimating the weight change effect of quitting smoking.
The METHOD=AIPW option specifies the AIPW method.
The COVDIFFPS option requests the assessment of the covariate balance of the propensity score model.
The PSMODEL model specifies the propensity score model
The MODEL statement specifies the outcome model.

31. Estimation of ATE by the AIPW Method
Treatment Robust Wald 95%
Parameter Level Estimate Std Err Confidence Limits Z Pr > |Z|
POM <.0001
POM <.0001
ATE <.0001
The results for potential outcome means and the ATE are not much different from that of the regression adjustment method.

32. Assessing Covariate Balance
Covariate Differences for Propensity Score Model Standardized Difference Variance Ratio Parameter Unweighted Weighted Unweighted Weighted Sex Sex 0 Age Education Education Education Education Education 1 Exercise Exercise Exercise 0 Activity Activity Activity 0 YearsSmoke PerDay
You could also assess the appropriateness of the propensity score model by checking the covariate balance.
The weighted columns represent the weights that are applied in the propensity score model.
A good balance between the treatment conditions is indicated by close to zero standardized mean differences and close to one variance ratio.
This tables indicates acceptable balance. The propensity score model seems to be quite appropriate.

33. Propensity Score Clouds
The propensity score clouds, which is also known as jitter plots, of the propensity scores, for the treatment and control groups are shown here.
It seems like they do have some good overlap.

34. Example 4. Limitations
The PSMATCH procedure fits the propensity score model in the form of logistic regression.
Logistic regression is the most popular form of propensity score model. But it is certainly not the only way.
Recall that the propensity score model is a model that predicts the probability of receiving treatment given the background characteristics X.
Therefore, theoretically, any type of model that could model the probability of receiving treatment can produce propensity scores.
This example illustrates how you can use the PSMATCH procedure to input propensity scores and do a propensity score analysis and causal effect estimation.

35. Family Aid and Child Development
A subset of data from the 1997 Child Development Supplement to the Panel Study of Income Dynamics (Hofferth et al. 2001; Guo and Fraser 2015)
Treatment variable AFDC: Receiving welfare benefit
Outcome variables Lwi: child’s development, as measured by the age- normalized letter-word identification portion of the Woodcock-Johnson Tests for Achievement
N=1,003 children whose primary caregiver was less than 36 years old
This example uses a subset of data from the 1997 Child Development Supplement to the Panel Study of Income Dynamics.
The primary purpose of this analysis is to see if receiving welfare benefit would help child’s development.
The treatment variable is called AFDC and the outcome variable is Lwi (letter-word identification) score measured some years after 1997.
1,003 children whose primary caregiver was less than 36 years old were included in the analysis.

36. Other Variables Age: Age of the child in 1997
PcgAFDC: Indicator for whether the child’s primary caregiver received support from a public assistance program when the primary caregiver was between the ages of 6 and 12
PcgEd: Number of years of schooling for the child’s primary caregiver
Race: Indicator for whether the child is African-American
Ratio: Ratio of family income to the poverty threshold in 1996
Sex: Indicator for whether the child is male
There are many other background characteristics in the analysis.

37. Data Set data Children;
input Sex Race Age Ratio PcgEd PcgAFDC AFDC Lwi;
datalines;
... more lines ... ;

38. Estimating Welfare Effect on Child Development
proc causaltrt data=Children covdiffps nthreads=2;
class AFDC PcgAFDC Race Sex;
psmodel AFDC(ref='0') = Sex Race Age PcgEd PcgAFDC/
plots=(pscovden weightcloud);
model Lwi = Sex PcgEd Ratio;
bootstrap seed=1776;
run;
Both the PSMODEL and MODEL statements are used.
This means that both the propensity scores and outcome are modeled.
Notice that these two models do not necessarily use the same set of predictor variables.
Note that the COVDIFFPS option requests the assessment of covariate balance of the propensity score model.

39. AIPW Estimation of ATE by PROC CAUSALTRT
Analysis of Causal Effect
Treatment Robust Bootstrap Wald 95%
Parameter Level Estimate Std Err Std Err Confidence Limits
POM
POM
ATE
Bootstrap Bias
Treatment Corrected 95%
Parameter Level Confidence Limits Z Pr > |Z|
POM <.0001
POM <.0001
ATE
This shows the results of the AIPW estimation.
The POMs shows the estimation of the potential outcome means in the treatment and control conditions.
‘1’ means the receiving welfare and ‘0’ means not receiving welfare
The ATE shows that receiving welcome has a detrimental causal effect!
However, when the bootstrap confidence interval is used. It shows the effect covers the zero point, meaning that the effect is not significant.
These results seem to be inconsistent and unreliable.

40. Assessing Covariate Balance
Covariate Differences for Propensity Score Model
Standardized
Difference Variance Ratio
Parameter Unweighted Weighted Unweighted Weighted
Sex
Sex
Race
Race
Age
PcgEd
PcgAFDC
PcgAFDC
Let’s take a look at the assessment of covariate balance of the propensity score model.
The ‘weighted’ columns represent the adjustments that are brought by the propensity score model.
If the propensity score model is appropriate, the standardized mean differences of the weighted column should be closer to zero and the variance ratio of the weighted column should be closer to one.
This table shows that the propensity score model does result in more comparable control and treatment groups, in terms of their background characteristics.
However, the variance ratio of primary care giver’s education level are even farther away from 1 after the weighting.
This casts doubts on the appropriateness of the propensity score model.

41. Covariate Densities
For continuous covariates, PROC CAUSALTRT also displays the unweighted ad weighted densities. The treatment and the control groups are more comparable in terms of their age and primary after weighted by the propensity scores.

42. Other Types of Propensity Score Models
PROC PSMATCH uses logistic regression models for estimating propensity scores
If a propensity score model does not lead to good covariate balance, what can you do?
Use another set of predictors in the logistic model
Use another type of model

43. Can another modeling techniques yield better propensity scores?
/* Use of HPSPLIT to fit PSMODEL */
proc hpsplit data=children seed=12345 ;
class AFDC PcgAFDC Race Sex;
model AFDC = Sex Race Age PcgEd PcgAFDC;
output out=smpred;
id AFDC PcgAFDC Race Sex Age PcgEd Lwi;
run;
A decision tree that uses the same covariates offers a non-parametric alternative for predicting the propensity scores
Because there are doubts about the appropriateness of the propensity score model, you should attempt to get a better one to estimate the causal treatment effect.
You could try to fit another propensity score model by using another sets of background characteristics as predictors.
You can also fit another type of model to investigate possibilities.
This example uses the HPSPLIT procedure to fit a propensity score model.
Essentially, the HPSPLIT procedure creates a decision tree to predict whether the children receives the welfare.
In the model statement, AFDC is modeled by the same set of covariates as in the original logistic regression.
The OUT= option in the OUTPUT statement saves an output data set that contains the propensity scores that are estimated by the decision tree analysis.

44. HPSPLIT Output Data Set
Pcg Pcg
Obs AFDC AFDC Race Sex Age Ed Lwi _Node_ _Leaf_ P_AFDC0 P_AFDC1
… More lines …
Here is what the output data set looks like.
The last column contains the probability of receiving welfare.
This column is also the propensity scores obtained by the HPSPLIT procedure.

45. Full matching with propensity scores from HPSPLIT
proc psmatch data=smpred;
class afdc;
psdata treatvar=afdc(treated='1') ps=p_afdc1;
match method=full(kmax=5) stat=ps caliper=. ;
assess ps var=(age PcgEd) / plots=all weight=matchatewgt;
output out=fullMatchTree matchid=_MID_ matchatewgt=atewgt;
run;
You specify the psdata statement when the input data set includes precomputed propensity scores
The treatvar= option identifies the treatment variable
The ps= option identifies the variable that contains the propensity scores
To use those newly created propensity scores, you can use the PSDATA statement of the PSMATCH statement.
In the PSDATA statement, you define the treatment variable as AFDC and the propensity score as p_afdc1.
In this analysis, you use the optimal FULL matching method---meaning that not only each treated unit is matched by control unit; but also that each control unit would be matched by treated units.

46. No improvement is seen in the standardized mean differences

47. Assessing balance of categorical covariates of the data set created by full matching
proc freq data=fullMatchTree;
table afdc*Sex afdc*race afdc*pcgAFDC;
weight atewgt;
run;
You can use PROC FREQ to assess the balance of the categorical variables. Here is the code.
It is essential to use the weights created from the PSMATCH procedure.

48. Distribution of Sex and Race for the treatment conditions
Let us look at the Sex variable first.
The distribution seems to be balance between the control and treated group.
However, the frequency distribution of race are not comparable between the two groups even after doing propensity score matching.

49. Distribution of PcgAFDC for the treatment conditions
The balance of primary care-giver’s educational level is also not achieved.

50. Estimation of the ATE proc ttest data=fullMatchATE; class afdc;
var lwi;
weight atewgt;
run;
Variable: Lwi
AFDC Method Mean % CL Mean
Diff (1-2) Pooled
Diff (1-2) Satterthwaite
Here is the code to compute the ATE from the output data set.
Notice that there is no option (like the DECENDING option in PSMATCH) to flip the group, you need to flip the ATE estimate to interpret the results.
Here it shows that the control group performed better than the treatment group and with a significant effect.
The modeling here confirms that of the CAUSALTRT procedure.

51. Limitations of the Study
What about SUTVA?
AFDC might have very different levels for families
What about the positivity assumption?
High-income families would not receive welfare
What about the “no unmeasured confounding” assumption?
Both propensity score models fail to achieve covariate balance
See the SAS global forum paper of Lamm and Yung for possible limitations of the study.

52. Summary of causal effect estimation
Confounding in observational studies must be dealt with
Under the potential outcomes framework, causal interpretations of the effects are possible if assumptions are satisfied
You can use various methods for estimating average treatment effect (ATE) and average treatment effect for the treated (ATT)
Assessing covariate balance is important
Causal methodology enables you to get unbiased estimation of causal effects, ATE or ATT.
You have to be sure what practical questions that you are asking before choosing ATE or ATT.
There are strong assumptions behind the causal methods.
Strong ignorability is one of them. This assumption cannot be affirmed easily.
You have to assume a set of background characteristics that could account for the mechanism of the treatment assignments so that the potential outcomes and the treatment assignment are independent given these background characteristics.
Practically, failing to achieve covariate balance tells you that either the strong ignorability assumption or your propensity score model is wrong; or both are wrong.
If covariate balance is achieved, there would be no obvious violations of the strong ignorability assumption and you might get unbiased estimation of the causal effects under the assumption.
We have also shown how two new SAS procedures can help you do these causal analysis.

53. Introductory Books and Articles
Austin, P. C. (2011). “An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies.” Multivariate Behavioral Research 46:399–424.
Hernán and Robins (2016). Causal Inference. Boca Raton, FL: Chapman & Hall/CRC. Forthcoming.
Guo, S., and Fraser, M. W. (2015). Propensity Score Analysis: Statistical Methods and Applications (2nd ed.). Thousand Oaks, CA: Sage Publications.
Morgan, S. L., and Winship, C. (2015). Counterfactuals and Causal Inference: Methods and Principles for Social Research (2nd ed.). New York: Cambridge University Press.
Pan, W., and Bai, H (2015). Propensity Score Analysis: Fundamentals and Developments. New York: The Guilford Press.
Stuart, E. A. (2010). “Matching Methods for Causal Inference: A Review and a Look Forward.” Statistical Science 25:1–21.
SAS Global Forum Papers
Lamm, M., and Yung, Y. F. (2017). “Estimating Causal Effects from Observational Data with the CAUSALTRT procedure.” In Proceedings of the SAS Global Forum 2017 Conference. Cary, NC: SAS Institute Inc.
Yang, Y., Yung, Y. F. , and Stokes, M. (2017). “Propensity Score Methods for Causal Inference with the PSMATCH Procedure.” In Proceedings of the SAS Global Forum 2017 Conference. Cary, NC: SAS Institute Inc.
Yung, Y. F., Lamm. M, and Zhang. W (2018). “Causal Mediation Analysis with the CAUSALMED Procedure.” In Proceedings of the SAS Global Forum 2017 Conference. Cary, NC: SAS Institute Inc.