Presentation on theme: "Causality in Social Work Research via Instrumental Variable Estimation Roderick A. Rose Senior Research Associate Carolina Institute for Public Policy."— Presentation transcript:
Causality in Social Work Research via Instrumental Variable Estimation Roderick A. Rose Senior Research Associate Carolina Institute for Public Policy The University of North Carolina at Chapel Hill
Summary Rationale: endogeneity & causality Instrumental variable estimation The exclusion restriction: justification and identification; logic; example Recommendations
Rationale: Endogeneity & Causality Endogeneity is pervasive throughout non- experimental research.
What is Endogeneity? XY Rental housing assistance Employment Career-relevant instructionMath achievement Health insuranceHealth at time t This is a Directed Acyclic Graph, or DAG. Causality flows in the direction of the arrow. This DAG represents what we might want to show (3 examples), representing a causal relationship between X and Y.
What is Endogeneity? XY Rental housing assistance Employment C Education Level Endogeneity occurs as an undesired relationship between variables in this causal relationship and other variables not accounted for in the model. Eliminating endogeneity requires removing one of the arrows.
What is Endogeneity? XY Career-relevant instructionMath achievement C Teaching quality The arrows corresponding to the confounder may go in any direction; endogeneity has no regard for the direction of the arrow. Note that any relationships not modeled occur through the error term. This is the definition of endogeneity: a correlation between an effect an the error.
What is Endogeneity? XY C Public Health insuranceHealth at time t Health at time t-1 When simultaneity or reverse causality are needed, they are represented in a DAG by lagged versions of variables. b < 0 b > 0
A Common Risk in Social Work Research? XY Public Health insuranceHealth b > 0 or b < 0 When simultaneity occurs in the context of cross-sectional data collection (i.e., one time point), we cannot be sure which effect we are observing. Note that this is not technically a DAG (because of the two headed arrow).
Description vs. Causality XY C Public Health insuranceHealth at time t Race/ethnicity It is entirely appropriate to use regression methods to describe relationships among variables without inferring causality. This curved line represents an unspecified relationship between two variables; implicit in this is that the relationship occurs through or because of other variables.
Instrumental Variable Estimation Notation X = EV = endogenous variable Y = DV = dependent variable Z = IV = instrumental variable or instrument
The Instrumental Variable or IV XY C Public Health insuranceHealth at time t Health at time t-1 Not measured (x-sect) Z In large samples, the instrument is a variable Z that is highly correlated with X and uncorrelated with the error. The IV must effectively randomize participants to conditions of X. Like random assignment, it cannot be confounded by any C.
IV Estimator using Covariance (Diagram) Circles represent variance; Source: Kennedy, 2003. Y = DV X = Endog Var Z = Instr Var error in Y No overlap with error No overlap with Y outside of X Used to estimate β OLS Used to estimate β IV Note overlap with error Note the loss of efficiency.
IV Estimator (Diagram) Circles represent variance; Source: Kennedy, 2003. Y = DV X = Endog Var Z = Instr Var Predicted X Predict X’s covariance with Y
IV Estimator (Diagram) Circles represent variance; Source: Kennedy, 2003. Y = DV X = Endog Var Z = Instr Var X r
Common Instruments “Natural” phenomena including births, weather events, geographical boundaries. Katrina is a good example. Randomized treatments. These are a particular class of IVE that are used when doing mediation or dosage type models (local average treatment effects). This is also (under certain assumptions) the Bloom Estimator (Bloom, 1984): Divide the ITT of the outcome by the ITT of the dosage. Functions and aggregates of observed characteristics. A person’s ethnic background will not satisfy exclusion, but membership in a “cluster” defined by ethnicity, age, income, job and other characteristics might. See Cutler & Gruber, 1996. Taking the cluster idea one step further, multilevel data yield “natural” clusters that can be used, under certain assumptions, to produce useful instruments. This is often referred to as a “Wald instrument” because it accords with Wald’s hypothesis. See Cobb-Clark & Crossley, 2003.
The Wald Estimator and IVE “If Z is a dummy variable indicating membership in one of two groups, then zy′/n captures the difference in the average outcomes of the two groups and zd′/n captures the difference in the treatment intensity across the two groups. Thus the IV estimator simply scales the difference in average outcomes by the difference in treatment intensities, as our intuition suggests.” -Cobb-Clark and Crossley, 2003. page 504.
The Exclusion Restriction: Justification and Identification
Controversy Surrounding IVs The most problematic aspect of IVE: search for a variable satisfying the requirements of no correlation with error and high correlation with endogenous variable. Even some economists are skeptical of the value of certain approaches taken to estimating IVs: Heckman (1990; 1997). Heckman even questions random assignment as an instrument! Inadequate attention to ensuring exclusion is a frequent problem. This is changing particularly as researchers outside of economics begin to see the value of the method.
Can Exclusion be Proven? XY C Public Health insuranceHealth at time t Health at time t-1 Not measured (x-sect) Z The (flawed) logic of a test for exclusion is that if Z is associated with Y after conditioning on X, then it is not an IV, because that means it must operate through C (or any other unmeasured variable). The problem is that this test always fails, because conditioning on X creates an artifactual association between Z and C. Exclusion must be “proven” by inductive methods only. Food stamp receipt
Propensity to receive food stamps Health at time t-1 0 1 Can Exclusion be Proven? Green: does not have public health insurance pink: has public health insurance Refer to: Morgan & Winship, 2007; p. 67.
Exclusion Logic: Example A school district recently implemented a policy change, promoting students to middle school in 7 th grade rather than 6 th. You are an evaluator given the task of determining whether this policy is beneficial to students, based on 7 th grade exams. Promotion is endogenous because it is determined partly by performance on end-of-grade exams. Having been hired after implementation, randomization is not an option. If a valid IV can be identified, you might have a causal estimate. Birth date is random, right? (Right?)
Exclusion Logic: Example of Potential IV Blue cohort: advances in 6 th grade. Green cohort: advances in 7 th grade. Students in different cohorts nearest the star are most alike, and thus less subjected to random differences. In calendar Year B, birth date determines cohort membership. Correlation between birth date and cohort determined by dispersion over year-long period before/after star. Quality of IV (correlation with cohort; influence of error) determined by the distance of each student’s birth from the star. Sept 1Aug 31Dec 31 Jan 1 Yr A Yr B Yr C
Discussion: Heintze, Berger, Nadich & Meyers (2006) Question: Is housing assistance directly associated with employment among low-income single mother households? Endogeneity problem? “The instruments used here are lagged measures of state-level housing assistance generosity. They are constructed from HUD data (HUD 1998). The lagged measure of housing assistance generosity for the 1997 sample is the ratio of the number of federally subsidized units to the number of poor households in the state in 1993.” (p 649) How is exclusion assumed here? How might exclusion fail? Did the authors sufficient examine exclusion? What alternatives might exist?
Recommendations: Endogeneity Never include a known endogenous variable as an intermediary between your focal causal variable and your outcome (e.g., do not include dosage when modeling treatment). The seeming exception to this is mediation, though in this case the mediator is the focal causal variable. There are many methods for dealing with endogeneity, including fixed effects, interrupted time series, PSM, regression discontinuity; some of these remain relatively untapped by social work researchers.
Recommendations: IVE Knowing what an IV is and how it works will be important for social work evaluators and researchers as the disciplines converge on scarce grant funds for research projects. When doing mediation or dosage (efficacy; i.e., not intent-to-treat) analysis, IVE is a potentially valuable tool, particularly when the treatment (as an antecedent to dosage) is randomly assigned. Look for next paper on local average treatment effects. This method also appropriate when ethics prevents randomization directly to treatment. Think about what instruments might be useful in your work; look for examples of instruments by other researchers in the literature. Be critical; a lousy instrument does not promote good science.
Recommended Reading Morgan, S. L. & Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principals for Social Research. Cambridge University Press. There are about a dozen articles in the social work literature that use or refer to IVE. These are listed in the appendix of the Stone article on endogeneity. The papers I’ve referred to: Cutler, D.M., and J. Gruber. 1996. The Effect of Expanding the Medicaid Program on Public Insurance, Private Insurance, and Redistribution. American Economic Review: Papers and Pro-ceedings 86(2): 368–373. Cobb-Clark, D. A. & Crossley, T. (2003). Econometrics for evaluations: An introduction to recent developments. The Economic Record, 79(247), 491-511. A three part series in Social Science and Medicine by myself & Susan Parish & colleagues, and Michael Foster: Rose, R. A., Parish, S. L., Yoo, J. (2010). Observation, correlation and policy analysis: A response to Foster. Social Science & Medicine, 70, 1274-1276. Foster, E. M. (2010). A commentary on Rose et al. Social Science & Medicine, 70, 1271-1273. Rose, R. A., Parish, S. L., Yoo, J., Grady, M.D., Andrews, M. E., Powell, S. E., & Hicks- Sangster, T. K. (2010). Medicaid suppresses racial disparities in health care among children with special health care needs: Evidence from a multi-level mediation modeling analysis. Social Science & Medicine, 70, 1263-1270.
SSWR in Washington DC, Jan 2012 I will be presenting a paper using LATE via IVE methods in a symposium for CareerStart that has several papers by myself, Mike Woolley and Jay Unick. While my time for explaining the intricacies of LATE will be limited, I will have (by this time) briefs and probably a short report. The paper I am presenting will constitute one of my dissertation papers.