September 18-19, 2006 – Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development Using HMIS to Benchmark Progress on Our Regional Plan to End Homelessness Brande Mead Maricopa Association of Governments Continuum of Care Regional Committee on Homelessness

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 2 Homelessness in Maricopa County 7,000 – 10,000 homeless persons on any given night PIT Count :2,063 on the streets & 5,416 sheltered Homelessness among families is the fastest growing subpopulation Need for additional resources and affordable housing Hot summers lead to heat-related illnesses and death

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 3 MAG Continuum of Care Regional Committee on Homelessness Established at a Council of Governments (COG) Maintaining balance through membership –Public sector –Private sector –Non-profit providers –Homeless individuals Role of the CoC in the community Homeless Immersion Project AZ Evaluation Project

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 4 Goals of the AZ Evaluation Project Support activities that make a real difference in lives of homeless people Evaluate the effectiveness of homeless projects Implement method of understanding what works & what doesn’t Identify best practices

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 5 Goals of the AZ Evaluation Project (cont.) Conscious shift from focus on outputs to investing in outcomes. Reduce the gap between knowledge and practice. Which programs are effective? What interventions produce results? Where do we target investment?

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 6 Partners AZ Department of Economic Security AZ Department of Housing Continua of Care HUD Virginia G. Piper Charitable Trust Providers

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 7 Self-Sufficiency Matrix DomainScore12345 Income no incomeInadequate income and/or spontaneous or inappropriate spending Can meet basic needs with subsidy; appropriate spending Can meet basic needs and manage debt without assistance Income is sufficient, well managed; has discretionary income and benefits Employment No jobTemporary, part-time or seasonal; inadequate pay, no benefits. Employed full time; inadequate pay; few or not benefits Employee full time with adequate subsidized housing. Maintains permanent employment with adequate income and benefits. Housing Homeless or threatened with eviction In transitional, temporary or substandard housing; and/or current rent/mortgage payment is unaffordable (over 30%of income) In stable housing that is safe but only marginally adequate. Household is in safe, adequate, subsidized housing. Household is safe, adequate, unsubsidized housing. Food No food or means to prepare it. Relies to a significant degree on other sources of free or low-cost food. Household is on food stampsCan meet basic food needs, but requires occasional assistance. Can meet basic food needs without assistance. Can choose to purchase any food household desires. Childcare Needs childcare, but none is available/accessible and/or child is not eligible. Child case is unreliable or unaffordable, inadequate supervision is a problem for child care that is available Affordable subsidized childcare is available, but limited. Reliable, affordable childcare is available, no need for subsidies Able to select quality childcare of choice SafetyHome or residence is not safe; immediate level of lethality is extremely high; possible CPS involvement Safety is threatened/temporary protection is available; level of lethality is high Parenting SkillsThere are safety concerns regarding parenting skills

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 8 Self-Sufficiency Matrix (cont.) Children's Education One or more eligible children not enrolled in school. One or more eligible children enrolled in school, but not attending classes. Enrolled in school, but one or more children only occasionally attending classes. Enrolled in school and attending classes most of the time. All eligible children enrolled and attending on a regular basis Adult Education Literacy problems and/or no high school diploma/GED are serious barriers to employment. Enrolled In literacy and/or GED program and/or has sufficient command of English to where language is not a barrier to employment. Has high school diploma/GEDNeeds additional education/training to improve employment situation and/or resolve literacy problems to where they are able to function effectively in society. Has completed educational/training needed to become employable. No literacy problems Legal Current outstanding tickets or warrants. Current charges/trial pending, noncompliance with probation/parole. Fully compliant wit probation/parole terms. Has successfully completed probation/parole within past 12 months, no new charges filed. No active criminal justice involvement in more than 12 months and/or no felony criminal history Health Care No medical coverage with immediate need. No medical coverage and great difficulty accessing medical care when needed. Some household members may be in poor health. Some members (eg Children on AHCCCS All members can get medical care when needed, but may strain budget. All members are covered by affordable, adequate health insurance. Life Skills Unable to meet basic needs such as hygiene, food, activities of daily living. Can meet a few but not all needs of daily living without assistance. Can meet most but not all daily living needs without assistance. Able to meet all basic needs of daily living without assistance Able to provide beyond basic needs of daily living for self and family. Mental Health Danger to self or others; recurring suicidal ideation; experiencing severe difficulty in day-to-day life due to psychological problems. Recurrent mental health symptoms that may affect behavior, but not a danger o self/others; persistent problems with functioning due to mental health symptoms. Mild symptoms may be present but are transient; only moderate difficulty in functioning due to mental health problems. Minimal symptoms that are expectable response to life stressors; only slight impairment in functioning Symptoms are absent or rare; good or superior functioning in wide range of activities; no more than every day problems or concerns.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 9 Elements of the Project Measurement Model –what is being measured? Predictive Model –what is the anticipated change?

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 10 Measurement Model Answers the question: Is this assessment technique valid and reliable? Yes, the instrument was found to measure overall self-sufficiency and 3 subscales: –Level of Independence Skills –Level of Dysfunction –& Total Score

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 11 Measurement Model While the Arizona Self Sufficiency Matrix was found to be reliable and valid within all three Continua of Care, the typical client in each Continuum was found to score differently. At program entry: –Pima and Maricopa produce similar dysfunction scores with Rural clients demonstrating greater dysfunction. –Rural and Maricopa produce similar Independence scores with Pima clients demonstrating greater independence skills.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 12 Predictive Model Equations are generated from the data to determine the predictors of change in Independence, Dysfunction, and Total Self Sufficiency These equations are then used to predict the amount of change in each individual client if randomly assigned to a homeless program. Each individual’s predicted change is uniquely determined based upon the client’s individual characteristics These predicted changes constitute the expected change

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 13 Predictive Model The predicted change for each individual is then compared with the actual change. Agencies whose clients consistently do better than predicted are the most successful Agencies whose clients consistently under perform the predicted change are the least successful Agencies whose clients perform consistently with predictions are typical

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 14 Predictive Model All these steps have been taken with the data gathered between Oct 2005 and June Powerful results have been obtained and as increased data is collected going forward the accuracy will increase with each quarterly iteration.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 15 Compliance Rate For the predictive model to be truly accurate a high response rate/compliance is critical. A compliance rate (the percent of clients from each agency for whom full data is obtained) of 80 % is fair, 90 % is good, 95 % is excellent.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 16 Response Rate Within the three Continua of Care the compliance rate has been far below what is considered the “Fair” response. The potential for this instrument to help address the needs of the homeless and to strengthen programs highly depends on the quality of the data gathered.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 17 Additional Variables Maricopa Continuum of Care has supplemented the data set with 2 multiple choice questions measuring the history of homelessness and primary reason for homelessness. These additional variables have been shown to allow richer feedback.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 18 Referral Profiles The predictive model can produce fair, objective rankings capable of assisting Rating and Ranking committees. Profiles can be generated describing the clients most likely and least likely to benefit from a given agency

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 19 Profiles allow: Identification of agency strengths Diagnosis of relative agency weaknesses Objective feedback to assist agencies in determining if they want to specialize in a subpopulation niche or what needs strengthening to be more effective with the general homeless population.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 20 Profile A This agency positively impacts longer term placements. Particularly effective with homeless families and individuals homeless because of natural disasters. The following clients tend not to benefit from this agency: –Victims of domestic violence –Parents with high potential for child abuse –Those homeless primarily because of financial problems

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 21 Profile B This agency is highly successful in having positive impact with those recently released from jail or prison. Agency is among the least effective in working with homeless Hispanics.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 22 Profile C This agency is very effective in meeting the needs of homeless families who were relatively high functioning prior to homelessness. Quite ineffective agency in having positive impact with homeless Native Americans. While fairly effective with mentally ill homeless, fairly ineffective with homeless substance abusers.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 23 Profile D This agency primarily serves older and/or disabled homeless. Highly effective with older clients. Much less effective with disabled. Needs to determine if they should further develop the older niche or make programmatic changes to be more effective with the disabled.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 24 Feedback Reports In August all agencies received a report with feedback based upon their first 9 months of participation. Some agencies with larger amounts of data submitted received richer reports. SAMPLE REPORT

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 25 AZ Evaluation Project Next Steps Improve project compliance (data submission) Identify and replicate best practice Technical assistance and training Utilize results – Continua of Care, rating and ranking, State contracting, etc.

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 26 Regional Plan to End Homelessness Developed in 2002 Based on work of more than 150 individuals Identified themes and developed recommendations Update to Plan published in 2005 Provides a benchmark 80% of goals achieved or engaged

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 27 Key Themes Increase Funding Prevent Homelessness Remove Barriers to Accessing Services Improve Data Collection and Outcomes

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 28 How HMIS is Utilized to Measure Goals Goal: Increase Funding AZ Evaluation Project agency level reports used for grant applications HMIS reports used for grant applications Quality monitoring and compliance standards improves workflow and produces good data quality Increased capacity among agencies –Example: CONTACS Shelter Hotline on HMIS

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 29 Information on who is homeless –Agency level data (HMIS agency generated reports) –Continuum level data (AZ Evaluation Project) –Statewide level data (Statewide HMIS Reporting System) Risk assessment using HMIS data elements Best practices shared among providers –Next step of the AZ Evaluation Project How HMIS is Utilized to Measure Goals Goal: Prevent Homelessness

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 30 How HMIS is Utilized to Measure Goals Goal: Remove Barriers to Accessing Services Standardized Reports –Generated by City of Phoenix for CDBG funding Eligibility Module –Within the HMIS network Referrals among providers –Track and monitor referrals through HMIS –CONTACS Shelter Hotline Data Sharing among providers –Piloting with some agencies

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 31 How HMIS is Utilized to Measure Goals Goal: Improve Data Collection & Outcomes Statewide HMIS Reporting System –Statewide data on number & type of people accessing housing and services. AZ Evaluation Project Using HMIS to generate APR’s –Data and outcomes used in HUD application Annual Shelter Count –Done through HMIS rather than a manual survey. Data Quality Plan

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 32 HMIS and the Continuum of Care Important relationship Regular HMIS updates at Continuum of Care meetings CoC involvement at HMIS User Group, Data Quality, and Advisory Board level

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 33 Next Steps Related to HMIS Expand data sharing among agencies Share best practices among agencies Increase number of agencies using Continuum-wide referral function Identify and track unmet needs AZ Evaluation Project used for HUD application Ranking & Review Real-time bed reservations

September 18-19, Denver, Colorado Sponsored by the U.S. Department of Housing and Urban Development 34 Questions? Brande Mead Human Services Planner Maricopa Association of Governments (602)