Presentation on theme: "Moving to Opportunity TOWN HALL Abt Associates, June 26, 2003 Background and Challenges: Judie Feins Findings: Robin Jacob Implications: Larry Orr Abt."— Presentation transcript:
Moving to Opportunity TOWN HALL Abt Associates, June 26, 2003 Background and Challenges: Judie Feins Findings: Robin Jacob Implications: Larry Orr Abt Associates Inc.
1 WHY MTO? Program Objective: Provide poor families living in high-poverty public or assisted housing with the opportunity to move to low-poverty neighborhoods with a Section 8 rent subsidy and counseling. Research Objective: Measure the impact of moving to lower poverty neighborhoods on the outcomes of adults and children.
2 Precursors of MTO 1)Long history of research showing harmful effects of bad neighborhoods 2)Chicagos Gautreaux Program: court-ordered mobility of poor, African-American families seemed to show positive effects of suburban life especially on the children 3)Trend of increasingly concentrated poverty in American cities suggested growing harm, unless a remedy could be found to reverse these neighborhood effects.
3 The Demonstration Program The MTO program ran from 1994 to1998 in five cities: Baltimore, Boston, Chicago, L.A. and N.Y. Families with children were recruited from public and assisted housing in concentrated-poverty neighborhoods. Many of the housing developments were very distressed and dangerous places. Any resident who wanted the chance to get a mobile housing subsidy and move out could sign up, as long as the family met Section 8 eligibility rules. Some 5,300 families volunteered; 4,608 were found eligible.
4 The Research Design For research purposes, families were randomly assigned into one of three groups: The experimental group received special Section 8 vouchers that could be used only in census tracts with poverty rates below 10 percent. Nonprofit counseling agencies in each city helped the experimental group families to locate and lease suitable housing in low-poverty areas. The Section 8 group received regular Section 8 vouchers, which could be used anywhere they found a suitable unit with a willing landlord and rent below the program cap. These families did not receive any mobility counseling. The control group received no vouchers but continued to be eligible for project-based assistance.
CBS News Correspondent John Roberts. From Ghetto To White Picket Fence BALTIMORE 6/5/ /17/2000 The Los Angeles Times Housing, Not School, Vouchers Are Best Remedy for Failing Schools - Jan 31, 1999, Larry Cuban A Social Experiment in Pulling Up Stakes; Aid: Does neighborhood affect economic and school success? Five cities relocate poor families to find out.- Sep 23, 1997; pg. 1, Larry Gordon A Fresh Start Housing: The Moving to Opportunity program will take families out of the projects to see if a new environment helps them succeed. Nov 8, 1994; pg. 1, Larry Gordon Washington Post " In Baltimore, Getting a Lease on Middle-Class Life. " May 10, 2000; pg. 1 Amy Goldstein. New York Times " Better Than a Voucher, a Ticket to Suburbia. " October 18, Richard Rothstein Chicago Tribune "Foes kill housing plan funds" - Dec 15, 1994; Laurie Abraham "Where Should Poor Families Live?" - Jul 23, 1994; Lori Montgomery "Hostility Toward Relocating the Poor is a Matter of Race" - Apr 27, 1994; Clarence Page
6 A Long and Challenging Effort
7 THE EARLY CHALLENGES Abt Associates began working on the MTO design with HUD in September Our initial job: Convincing HUD to shift to a 3-group design Developing uniform (but flexible) procedures Managing implementation (started July 1994) Surviving a political fire-storm Quietly growing the demonstration Finishing intake and lease-up (March 1999).
8 AND THEN… Database construction: We designed and built sophisticated relational database, which continues to evolve; Sample tracking: We kept up with 4,608 families (19,000+ persons) by active and passive means, despite high mobility rates and the disruption of public housing revitalization and demolition.
9 CHALLENGES OF THE INTERIM EVALUATION Research team: Abt partnered with a group of academics, the Urban Institute, and two data subcontractors to win the interim evaluation contract in July Broad scope: HUDs interest extended well beyond the usual housing and neighborhood issues to education, health, delinquency, employment and earnings, income and self-sufficiency. Complex data collection: The study required that several kinds of data be gathered from almost 11,000 sample members in a compressed period.
10 Evaluation Team ANALYSTS FROM: Abt Associates (design, implementation, data collection, impact analysis) Urban Institute (qualitative analysis) National Bureau of Economic Research (impact analysis) Georgetown University (collection and analysis of crime data)
11 Interim Evaluation Funders U.S. Department of Housing & Urban Development Grants via National Bureau of Economic Research National Institute for Child and Health Development (NICHD) National Science Foundation (NSF) National Institute of Mental Health (NIMH) Robert Wood Johnson Foundation Smith Richardson Foundation Russell Sage Foundation W.T. Grant Foundation Spencer Foundation MacArthur Foundation Grants via Georgetown University National Consortium on Violence Research (NSF) Brookings Institution
12 Interim Evaluation Data Sources Pre-Survey Qualitative Data In-depth semi-structured interviews with adult heads of household and children informed survey design. Quantitative Data Structured surveys with adults, teens, and children Achievement tests of teens and children Measurement of adult blood pressure, child height and weight Administrative data from state and local agencies (earnings, TANF, food stamps, arrest records)
13 Interim Evaluation Surveys Full Sample (adults, youth, children) Fielded from January through June Achieved survey response rates of 80% of adults, 77% of children, 76% of youth. 3-in-10 Sub-sample Random draw late in June from non-complete cases. Focused field resources on a sub-set of hard-to-find cases; included travel to remote sites. Increased response rates considerably and reduced the risk of non-response bias.
14 Interim Evaluation Response Rates High Effective Response Rates Attained a weighted adult response rate of 90 percent. Attained youth/child surveys and achievement test response rates of 86 to 90 percent. Group Differences Difference in response rates between random assignment groups was less than 1 percent Very low risk of non-response bias between groups
15 Interim Evaluation Findings What did we learn?
16 Estimation Methods All estimates regression-adjusted with standard set of covariates, including (where available) pre-RA value of outcome ITT = intention to treat – impact on entire treatment group, including those who did not lease up TOT = treatment on treated – impact on those who leased up only Tests of significance at.05 level
17 Mobility Outcomes Poverty rates of current locations are substantially reduced (entire experimental and Section 8 groups) Fraction minority population in experimental group locations is reduced, but more than half moved to areas 80%+ minority Almost half of experimental lease-ups were in tracts with increasing poverty from 1990 to 2000 More than half of experimental group lease-ups moved again to somewhat higher poverty tracts
18 Impacts on Neighborhood Outcomes Significant positive impacts for both experimental and Section 8 groups on: Feeling safe in the neighborhood (day and night) Police coming when called All measures of neighborhood quality Significant reductions for both experimental and Section 8 groups in: Witnessing drug activity in the neighborhood See public drinking, groups hanging out Crime victimization over last six months
19 Impacts on Housing Outcomes Significant increases for both experimental and Section 8 groups in: Most measures of housing quality Utility payment problems Prevalence of housing assistance receipt No significant impacts on current total housing cost
20 A Focus on Educational Outcomes The following factors were hypothesized to be mediators of educational outcomes: Community-Level Quality of Schools Community Norm and Values Social and Physical Environment Student and Family-Level Parent Attitudes and Behaviors Student Attitudes and Behaviors
21 School Characteristics Modest improvements for both groups 70% of Experimental lease-ups remained in the same large urban school district C MeanE-CS8-C Percent free lunch Percent white Percent limited English proficient ns Pupil-teacher ratio14.6ns Percentile rank on State Assessments
22 School Climate No significant effects on school climate C MeanE-CS8-C There is a lot of cheating in tests & assignments.416ns Discipline in school is fair.724ns Disruptions from other students inhibit learning.640ns Child feels safe in school.775ns Teachers are interested in students.810ns
23 Other mediators of educational outcomes Community-Level Mediators Significant impacts on social and physical environment Some evidence of impact on community norms and values but not on peer role models Some evidence of positive impact on economic opportunities but not on earnings or employment of sample adults. Student- and Family-Level Mediators No significant effects on parental monitoring No significant effects on parental involvement in school No significant effects on student school-related behaviors
24 Impacts on Education Outcomes No significant impact on student achievement No significant impacts on grades, coursework, special ed placement, graduate rates or college attendance.
25 Other Outcomes: Health Adult Physical and Mental Health Physical health & substance use: largely insignificant impacts Mental Health: significant E-C impacts Obesity: significant E-C impacts Youth Physical and Mental Health Physical health: no significant impacts Mental health: improvement for girls
26 Other Outcomes: Youth Behavior Youth Delinquency & Risky Behavior Behavior Problems Index: significant increase in self- reported behavior problems among boys Delinquency: no significant impact Arrests & Risky Behavior: substantial gender differences Employment and School Attendance No significant effects for boys, but an increase in full-time school attendance for girls in the experimental group and a reduction in full-time employment for girls in Section 8
27 Other outcomes: Income & Earnings Receipt of Public Assistance No significant impacts on current receipt of public assistance, either for full sample or subgroups by ethnicity and barriers to employment Household Income & Poverty Status No significant impacts on income, poverty, food security and self-sufficiency
28 Implications for Policy
29 Does this mean there are no impacts on these outcomes? On many outcomes, only fairly large impacts can be detected with confidencee.g., to be 80% sure of detecting impacts as significant: Adult earnings would have to be increased by about 40% in the experimental group, 30% in the Section 8 group TANF benefits would have to be reduced by 50% Youth asthma attacks would have to be reduced by 67% The fact that an impact estimate is not statistically significant does not mean there was no impactit means we dont know if there was an impact.
30 Policy Implications - Context The worst concentrations of urban poverty are usually HUD- subsidized. In US, high-poverty neighborhood almost always means high- crime neighborhood. There are legitimate concerns about relocating the ghetto. These might lead to restricted mobility programs.
31 Will mobility programs work? Can voluntary restricted mobility programs move extremely low-income public housing tenants to middle-class neighborhoods? Yesat least, for 48%. However, geographic restrictions come at some cost to lease-up (60% of Section 8 comparison group moved) Those who move are likely to follow the path of least resistance. In MTO lease-ups were predominantly in minority neighborhoods, where poverty rates trending up.
32 Who benefits, and for how long? Those who move enjoy substantially better housing, safer neighborhoods, less stress, better mental health, and lower obesity. There is some indication that girls behavior and motivation to succeed improves. These benefits are still significant 4-7 years after the initial move. Benefits to the rest of society are less clear. There is some indication of lower rates of criminal behavior among girls, but higher rates of crime and delinquency among boys. There are no reductions in welfare costs.
33 What do these results imply about choice of policies for improving the lives of low-income families? Are the problems of low-income families environmental, or due to attributes of the families themselves? These results imply that a change in environment can improve the familys sense of well-being directly, and have some salutary effects on the behavior of youth. But physical health, educational performance and attainment, employment, earnings, and welfare dependence do not appear to be sensitive to residential environment, at least within 4-7 years. To improve these outcomes within that time frame requires policies designed to deal directly with these specific problemssuch as educational improvements, employment and training, or welfare-to-work programs.
34 Dr. Larry Orr, Abt Associates Dr. Judith Feins, Abt Associates Dr. Jeffrey Kling, Princeton University Dr. Jens Ludwig, Georgetown University Dr. Robin Tepper Jacob, Abt Associates Dr. Barbara Goodson, Abt Associates Dr. Lisa Sanbonmatsu, NBER Dr. Lawrence Katz, Harvard University Dr. Jeffrey Liebman, Harvard Univsersity Dr. Erik Beecroft, Abt Associates Dr. Rhiannon Patterson, Abt Associates Dr. Alvaro Cortes, Abt Associates Ms. Carissa Climaco, Abt Associates Ms. Debi McInnis, Abt Associates Mr. Robert Teitel, Abt Associates Dr. Susan Popkin, Urban Institute Analysts: