1. ORGANIZATIONAL CLIMATE, DIVERSITY, AND TREATMENT OUTCOMES AT COMMUNITY MENTAL HEALTH AGENCIES: AN ATYPICAL CASE STUDY Christopher R. Larrison, Ph.D., Susan L. Schoppelrey, Ph.D., Eric Hadley-Ives, Ph.D., Barry J. Ackerson, Ph.D. University of Illinois at Urbana-Champaign School of Social Work Organizational Factors % of non-white staff Organizational Climate -Level of involvement, task orientation, clarity, innovation, physical comfort Location: urban or rural Case Manager Factors Demographic Characteristics Orientation to job Cultural Competence Client Factors Demographic characteristics Perceptions of access & quality of services Treatment dose & duration Treatment Outcomes Change over time in functioning and symptoms The two part case study began as an evaluation of services conducted by the lead author for the state agency overseeing the CMHA. The process of the evaluation, which created long term interactions with staff, clients, and administrators of the agency led to the observation that the climate at the agency was strongly set by the administrative staff and positively permeated most aspects of agency life. This observation was confirmed by a number of other professionals and state officials that had knowledge of the agency. In the first part of the case study, the existence of racial disparities in treatment outcomes was examined by observing the patterns in clients’ symptoms over a nine- month period. In the second part of the case study, the organizational climate of the agency was examined using both quantitative and qualitative descriptive data. INTRODUCTION COMMUNITY MENTAL HEALTH AGENCY (CMHA) ♦ Served six counties in the rural Southeast. ♦ Licensed staff & agency administrators were entirely white. ♦ Unlicensed staff was approximately 60% African American. ♦ The racial distribution of clients within the agency was 55% African American and 45% white. ♦ Approximately 40% of the adult clients were diagnosed with depression, 30% with schizophrenia and related disorders and the remaining 30% had a range of diagnoses, including bipolar disorder, psychotic disorders, adjustment disorders, PTSD, and impulse control disorder. ♦ Multiple interventions were provided including individual counseling, group therapy, medication, psychiatrists’ services, day treatment, client run peer groups, case management, and supportive services. ♦ A variety of therapeutic models shaped interventions including cognitive behavioral, solution focused therapy, family therapy, and insight-oriented psychotherapy. METHODS Part One ♦ Convenience sample consisting of nearly 17 % (n = 130) of adult clients. ♦ Symptoms were measured four times over a nine-month period (initial status and three follow-up measures) using BASIS-32 (Eisen, Dill & Grob, 1994) ♦ HLM (Byrk & Raudenbush, 2002) was used to determine whether differences existed between African American and white clients in treatment outcomes. The model consisted of an individual growth model, modeling change in each client’s observed level of symptomatology over time, and a between-client model that examined the possible differences between client-level change trajectories created by race, diagnosis, socioeconomic status, and gender. Part Two ♦ Purposeful sample of 20 African American clients and 22 agency staff. ♦ Clients completed the 28-item adult version of the Mental Health Statistical Improvement Program consumer survey (MHSIP) and staff completed the Work Environment Scale (WES) to estimate the CMHA’s organizational climate. ♦ The client sample was compared to a sixteen-state sample of mental health clients receiving outpatient services. The staff and administrators were compared to a group of 4,879 healthcare workers used to norm the WES (Moos, 1994). ♦ Clients and staff also participated in separate semi-structured focus groups about the agency’s organizational climate and its relationship to treatment outcomes for African American clients. THEORETICAL MODEL Overall, clients’ symptomatology either decreased or remained constant over the nine- month period of the study, depending primarily on their diagnosis. The average change over time was approximately 1/3 of a point on the four-point scale, which means that, although change over time was statistically significant, it may not be clinically significant. The only variable significantly related to variation in symptomatology patterns was diagnosis. Given that schizophrenia and major depression are likely to have different courses of treatment and patterns of outcomes, the findings make sense from an empirical standpoint and fit with the wide range of research concerning treatment of mental health problems. The results of this study suggest that in this agency client outcomes did not differ as a function of client level factors of race, gender, age, or method of payment (Larrison et al., 2004). FINDINGS: PART ONE Study Sample 16-State Study Median Access85.0%81.8% Quality & appropriateness 85.0%80.1% Positive change95.0%71.1% Consumer participation in treatment planning 70.0%72.3% General satisfaction 95.0%83.8% Note: Figures for the 16-state study come from Hall (2002) and Lutterman et al., (2003). FINDINGS: PART TWO Clients Staff The African American clients consistently cited the high quality of care, professional appearance of the facilities, and personalized treatment received from staff at all levels of the organization, which is consistent with the high levels of satisfaction reported on the MHSIP survey. They reported that staff were able to act in culturally appropriate ways, and also noted that staff were sensitive to the delicate balance between the needs of the client and the needs of the client’s family. Treatment in general was perceived as having a significant positive impact upon functioning and quality of life. The agency was characterized by staff as having a positive top-down administrative structure that orient staff towards agency goals, which centered on improving clients’ quality of life. African American and white staff reported feeling comfortable asking each other questions pertaining to cultural issues. All staff reported receiving cultural competency training as part of their employment experience. There was support for innovative and unique treatment as long as standard treatments had been exhausted and proven ineffective. African American staff reported that they sometimes observed African American clients being treated differently than white clients - Typical incidents cited were minor. All staff stated that there was recognition that client treatment was varied and individualized, and that this variation was determined by a number of factors beyond race. PROPOSED RESEARCH LITERATURE Hypotheses 1)CMHAs that demonstrate low levels of disparities in treatment outcomes between African American and white clients will have high levels of involvement, task orientation, clarity, innovation, and physical comfort as measured from the perspective of agency staff. 2)CMHAs that demonstrate low levels of disparities in treatment outcomes between African American and white clients will have easy access to services, good quality of care, and generally positive treatment outcomes from the perspective of clients. Methods ♦ Sample of 780 clients and 300 staff at seven CMHAs with 13 offices located in Upper Mississippi River Basin. The southern region of this geographical area has one of the highest concentrations of rural African Americans outside the deep South (U.S. Census Bureau, 2001). Assuming a fixed alpha level of 0.05, a sample size of 50 subjects per site, and an effect size of 0.40, a minimum of eleven sites are necessary to ensure adequate statistical power (β≥0.80; Raudenbush & Liu, 2001). ♦ A four level hierarchical linear model, which can partition the total variability in the repeated outcome measures into its constituent components of clients within staff (Level 2); among staff within agencies (Level 3); and among agencies (Level 4), will be used (see Analytic Model). Analytic Model ♦ The limited research regarding predictors of disparate treatment outcomes for African American clients has focused on client and staff level factors producing little consistent knowledge about why disparities occur (USDHHS, 2001). ♦ Organizational climate has a significant effect on staff performance, their emotions and attitudes about their workplace, and the quality of their relationships with clients (Glisson, 2000). These factors, in turn, are likely to influence the quality of services provided and the outcomes of those services. ♦ No research to date has systematically examined the relationship between organizational climate and racial or ethnic disparities in outcomes among CMHA clients (Snowden, 2001, 2003). Measurement Points 321 0 1.6 1.5 1.4 1.3 1.2 1.1 African American White BASIS-32 Scores Agency % of non- white staff Organi- zational Climate (WES) Urban or rural location Case Manager Case manager characteristics Age Gender Ethnicity Languages spoken Years of practice exp. Education Licensure Professional orgs. Continuing education Orientation to Job Maslach Burnout Inventory Job satisfaction Intention to quit Cultural Competence Universal Orientation Scale (UOS) Multicultural Counseling Inventory Client Demographics Age Gender SES Education Diagnosis Ethnicity Perceptions of access to & quality of services (MHSIP) Treatment dose and duration BASIS-32 WHO-DAS II Repeated Measures