1. National Alliance to End Homelessness Family Conference Assessment Tools Roundtable Charlene Moran Flaherty State and Local Policy February 7, 2008

2. 1 CSH helps communities create permanent housing with services to prevent and end homelessness. CSH’s Mission

3. 2 Family Connections Prevent children from entering child welfare system Reduce TANF caseload Non-recurrence of abuse and neglect Increase in stability/self-sufficiency

4. 3 Evaluation Goals Conscious shift from focus on outputs to investing in outcomes Reduce the gap between knowledge and practice Which programs are effective? What interventions produce results? Where do we target investment?

5. 4 Evaluation Goals Support activities that make a real difference in lives of families Evaluate the effectiveness of Family Connections Implement method of understanding what works & what doesn’t Identify best practices

6. 5 Arizona Self Sufficiency Matrix Engagement tool Basic Assessment Evaluation

7. 6 Arizona Self Sufficiency Matrix DomainScore12345 Income no incomeInadequate income and/or spontaneous or inappropriate spending Can meet basic needs with subsidy; appropriate spending Can meet basic needs and manage debt without assistance Income is sufficient, well managed; has discretionary income and benefits Employment No jobTemporary, part-time or seasonal; inadequate pay, no benefits. Employed full time; inadequate pay; few or not benefits Employee full time with adequate subsidized housing. Maintains permanent employment with adequate income and benefits. Housing Homeless or threatened with eviction In transitional, temporary or substandard housing; and/or current rent/mortgage payment is unaffordable (over 30%of income) In stable housing that is safe but only marginally adequate. Household is in safe, adequate, subsidized housing. Household is safe, adequate, unsubsidized housing. Food No food or means to prepare it. Relies to a significant degree on other sources of free or low- cost food. Household is on food stampsCan meet basic food needs, but requires occasional assistance. Can meet basic food needs without assistance. Can choose to purchase any food household desires. Childcare Needs childcare, but none is available/accessible and/or child is not eligible. Child case is unreliable or unaffordable, inadequate supervision is a problem for child care that is available Affordable subsidized childcare is available, but limited. Reliable, affordable childcare is available, no need for subsidies Able to select quality childcare of choice SafetyHome or residence is not safe; immediate level of lethality is extremely high; possible CPS involvement Safety is threatened/temporary protection is available; level of lethality is high Parenting SkillsThere are safety concerns regarding parenting skills Parenting skills are minimal.Parenting skills are apparent but not adequate. Parenting skills are adequate.Parenting skills are well developed.

8. 7 Arizona Self Sufficiency Matrix Children's Education One or more eligible children not enrolled in school. One or more eligible children enrolled in school, but not attending classes. Enrolled in school, but one or more children only occasionally attending classes. Enrolled in school and attending classes most of the time. All eligible children enrolled and attending on a regular basis Adult Education Literacy problems and/or no high school diploma/GED are serious barriers to employment. Enrolled In literacy and/or GED program and/or has sufficient command of English to where language is not a barrier to employment. Has high school diploma/GEDNeeds additional education/training to improve employment situation and/or resolve literacy problems to where they are able to function effectively in society. Has completed educational/training needed to become employable. No literacy problems Legal Current outstanding tickets or warrants. Current charges/trial pending, noncompliance with probation/parole. Fully compliant wit probation/parole terms. Has successfully completed probation/parole within past 12 months, no new charges filed. No active criminal justice involvement in more than 12 months and/or no felony criminal history Health Care No medical coverage with immediate need. No medical coverage and great difficulty accessing medical care when needed. Some household members may be in poor health. Some members (eg Children on AHCCCS All members can get medical care when needed, but may strain budget. All members are covered by affordable, adequate health insurance. Life Skills Unable to meet basic needs such as hygiene, food, activities of daily living. Can meet a few but not all needs of daily living without assistance. Can meet most but not all daily living needs without assistance. Able to meet all basic needs of daily living without assistance Able to provide beyond basic needs of daily living for self and family. Mental Health Danger to self or others; recurring suicidal ideation; experiencing severe difficulty in day-to-day life due to psychological problems. Recurrent mental health symptoms that may affect behavior, but not a danger o self/others; persistent problems with functioning due to mental health symptoms. Mild symptoms may be present but are transient; only moderate difficulty in functioning due to mental health problems. Minimal symptoms that are expectable response to life stressors; only slight impairment in functioning Symptoms are absent or rare; good or superior functioning in wide range of activities; no more than every day problems or concerns.

9. 8 Using the Matrix Periodicity of Data Collection At program entry and exit Three month interim All adults over the age of 18 Complete by client together with case manager Score to the deficit

10. 9 Models – Measurement Is this assessment technique valid and reliable? Using factor analysis, the instrument was found to measure overall self-sufficiency and 2 subscales: Economic self-sufficiency Social/emotional self-sufficiency Total composite score

11. 10 Models – Predictive Equations are generated from data submitted to determine the predictors of change in social emotional and economic self-sufficiency and total composite self-sufficiency score Each individual’s predicted change is uniquely determined based upon the client’s individual characteristics These predicted changes constitute the expected change

12. 11 Client Data 568 unduplicated adult clients 2/3 completed entry matrix 84 clients with entry and exit matrices. 24 have been in the program 3 months 60 terminated with less than 3 months The differences between those who “stick” and those who terminate are dramatic with longer term clients benefiting and the shorter term having minimal gains.

13. 12 Change Entry to Exit by Time in Program

14. 13 Level of Functioning More than 1/3 of clients either homeless or being evicted. More than ½ recently or currently homeless Two out of every five clients had parenting skills which ranged from “inadequate to dangerous” Approaching 1/3 of clients assessed to be functioning at a crisis level School age children of one in seven clients not enrolled in school

15. 14 Entry Problem Ranked Problem AreaScore Employment1.69 Income2.19 Food2.21 Shelter2.54 Childcare2.59 Community Involvement2.66 Family Relations2.67 Mobility2.90 Adult Education2.96 Life Skills3.34 Health Care3.38 Parenting Skills3.49 Mental Health3.76 Safety3.96 Children’s Education4.22

16. 15 Change in individual Clients Change in individual clients over time initially encouraging. 3 month matrices –Typical client improved in social-emotional self sufficiency more than a quarter (.28) of a standard deviation –Improvement in economic self-sufficiency was even more dramatic approaching half (.42) of a standard deviation The overall change in self-sufficiency also approached half of a standard deviation (.44)

17. 16 Change in Individual Clients Changes in client scores not the result of focusing on clients with fewer challenges. Rather, the lowest functioning clients made the greatest changes. Hence, not only were the lowest functioning clients changing the most, but the increased correlations over time indicate that the impact is greater at 6 months than at 3

18. 17 Change in Individual Clients Entry to 6 months – The service most related to positive changes in economic self-sufficiency is rent assistance. The best predictors of positive outcomes in social-emotional self-sufficiency are utility assistance and education and training. Changes in overall self-sufficiency - Same three variables occur, with utility assistance having as much positive impact as the education/training and rent assistance combined.

19. 18 Application What are the client characteristics that are indicative that an individual is more likely to benefit from a program? Do individual workers tend to be more effective with specific types of families? What is the differential impact of the varying services offered by DES and contracting agencies to clients and what is their cost effectiveness? Can we construct an “early warning system” which alerts us that based on client outcome that a worker, service delivery site or supervisor needs assistance in being effective?

20. 19 Application Is there an optimum time frame to work with families to maximize demonstrable effectiveness of intervention? Is there a substantive difference between professional and paraprofessional staff in client outcomes? How important is the time lag between first referral to FC and first interview with program staff in terms of eventual program impact?

21. 20 Benefits/Challenges Benefits –Client centered practice –Continuous program improvement –Target resources more effectively –Increased investment in –Incubator for new activity Challenges –Provider buy-in –Data collection/scrubbing –Evaluation expertise –Cost

22. Charlene Moran Flaherty charlene.flaherty@csh.org charlene.flaherty@csh.org visit www.csh.orgwww.csh.org