Presentation is loading. Please wait.

Presentation is loading. Please wait.

Measuring Interorganizational Collaboration in Communities: Baltimores Strategy to Improve Birth Outcomes as a Case Study Donna Strobino and Meredith Matone.

Similar presentations

Presentation on theme: "Measuring Interorganizational Collaboration in Communities: Baltimores Strategy to Improve Birth Outcomes as a Case Study Donna Strobino and Meredith Matone."— Presentation transcript:

1 Measuring Interorganizational Collaboration in Communities: Baltimores Strategy to Improve Birth Outcomes as a Case Study Donna Strobino and Meredith Matone Department of Population, Family and Reproductive Health Johns Hopkins Bloomberg School of Public Health

2 Infant Mortality Rates: Maryland & Baltimore

3 Gaps in Current Efforts to Improve Birth Outcomes Individual initiatives are funded inadequately and separately Individual initiatives are funded inadequately and separately Poor coordination of services Poor coordination of services Lack of adequate primary health care Lack of adequate primary health care Lack of health insurance Lack of health insurance Lack of minimum standard of care Lack of minimum standard of care Limited community mobilization to promote healthy behaviors prior to or between pregnancies Limited community mobilization to promote healthy behaviors prior to or between pregnancies

4 General Overview The Strategy to Improve Birth Outcomes (SIBO) is a citywide and community-led strategy Reframes the way people, providers, services, and community agencies interrelate around the wellbeing of women throughout their reproductive life cycle (preconception, pregnancy and postpartum) and their infants Takes a population-based approach to solving intractable disparities in birth outcomes

5 High Impact Service Areas (1) Primary health care in a medical home (7) Smoking cessation (2) Obstetric care(8) Family planning (3) Home visiting(9) Nutrition support (4) Drug and alcohol treatment(10) Breastfeeding promotion (5) Domestic violence Interventions (11) Safe sleep education (6) Mental health care

6 Family League of Baltimore City RFP SIBO challenges communities to mobilize around improving birth outcomes in a community and increasing demand for and utilization of the eleven high-impact services. The Family League is funding: a citywide communications campaign an evaluation and performance monitoring strategy resident and community mobilization around infant mortality and collaboratives capable of connecting all women in a community with appropriate care and services.

7 Five Critical Components: 1 Policy mandate emphasizing Improved birth outcomes as a priority of the Baltimore public health landscape. Funding streams to facilitate systems and services to address gaps in coverage, improve effectiveness of practices and policies, and replace inefficient implementation of existing practices with evidenced-based alternatives.

8 Five Critical Components: 2 Reorganize or retool health care delivery systems and practices for mothers and infants (including fathers, partners, families and communities who care for and support them). Assess and refine service provider practices and policies, especially related to risk assessment and create new operational models of service provision, where needed.

9 Five Critical Components: 3 City-wide social marketing intervention to influence the behavior of community residents, with a focus on expecting mothers and their partners and families, caregivers, service providers, and policy makers. Target all citizens

10 Five Critical Components: 4 Community-led focused initiative designed to coordinate already existing, culturally relevant, local health and social services within target areas Customized approach requiring strong community input and collaboration, leading to transferable models for other neighborhoods. NO new services

11 Five Critical Components: 5 A collaborative strategy driven by a set of partnership principles defining the quality and character of stakeholder interactions which guide the tenor of policies, agreements and protocols. Important in defining the community led component Applies to CIT and Steering Committee

12 FLBC RFP for Community Collaboratives: Must Demonstrate the Capacity to: Outreach to all pregnant women in the community to connect them with appropriate care and services Participate fully in the new citywide triage system for pregnant women into home visiting programs; Link pregnant women with appropriate supports and services (home and center based) 12 targeted communities with 3 awards of $400,000 - 600,000 for the 21 month period April 2010 – December 2011

13 Expected Impact in RFP SIBOs long term success: improvements in rates for pre-term birth, LBW, and deaths from unsafe sleep in targeted communities and in Baltimore City Tracking of several indicators related to the activities of collaborative teams that contribute to the achievement of the key outcomes. Success also measured by whether communities develop partnerships collaborations to assure access to the 11 high-impact services and link the various components of the health care system and human resources in the community to assure this access.

14 Evaluation Objectives Assess implementation of SIBO core components related to policy, services, communities, and families and individuals; Monitor implementation throughout the duration of SIBO to allow for modifications and trouble-shooting in real time (including the collaboration strategy); and Produce data to understand the impact of the first four core components on infant mortality in Baltimore City.

15 SIBO Evaluation Design Quasi-experimental design to evaluate changes in indicators in communities funded for the community-led initiative with the other communities with high infant mortality rates not initially funded: funded communities compared to remaining targeted communities

16 Data Source for Collaboratives Data from community partners about collaborations among agencies and community leaders involved in implementation of the community-led focused intervention in targeted communities

17 Overview Significance Measurement problem Collaboration definition, models, research Evaluation strategies for collaboration Case Study: The Strategy to Improve Birth Outcomes in Baltimore City

18 Significance Collaboration between health and social service agencies within a community that creates a population-based system of care are ideal because complex issues are inseparable from social and physical determinants of health, which cluster by community.

19 Significance Proposed benefits of collaboration: Promote cost-effectiveness Reduce duplication of services Improve communication Comprehensive service provision Increase community capacity What is the evidence to support these claims?

20 A Measurement Problem: What is collaboration? Conceptually: logical and feel-good results indicator Reality: plagued with measurement issues

21 Collaboration Definition Collaboration occurs when a group of autonomous stakeholders of a problem domain engage in an interactive process, using shared rules, norms, and structures, to act or decide on issues related to that domain. Wood and Gray 1991 Collaboration in public health… A relationship between two organizations that involves exchanging information and sharing resources in order to coordinate services for mutual benefit and to achieve common goals, such as facilitating care of treatment of clients. Friedman et al. 2007

22 Collaboration Models Level of Collaboration models Attempt to classify the stage, or intensity, of relationships between organizations Indicators generally include purpose (goals), structures (central body, leadership) and behaviors (communication, budget)

23 Levels of Community Linkage Model, Hogue 1993

24 State of the Research Most community-level research to date has focused on costs and benefits of collaboration to organizations and/or accounts of the collaboration formation process Stressors of collaboration in public sector Characterization of nonprofit organizations likely to participate in formal collaboration Effective arrangements of collaborative efforts

25 State of the Research Collaboration research in community health promotion, specifically grant-funded initiatives and public and nonprofit organizations, is underdeveloped Done little to advance measurement tools

26 Evaluation Strategies Social Network Analysis Survey Thomson, Perry, and Miller Method Qualitative Analysis


28 Social Network Analysis Social network analysis is a method of studying the linkages among individuals, groups, organizations, and other units (termed actors) Patterns of relationships are assessed and used to understand the structure, function, and performance of a network

29 SNA in Public Health Broad applications: Transmission networks: disease and information Social networks: social support and social capital Organizational networks: health systems

30 SNA Methodology: Respondents Create sampling frame of organizations of relevance in evaluation Known upfront, or Formative research: snowball sampling, etc. Designate organization representatives: individual or group Validity **This methodology is applicable to all evaluation methods

31 SNA Content Rating scales of relationship type or intensity Questionnaires often draw heavily from the collaboration models for indicators of relationships

32 SNA Sample Questionnaire

33 SNA Data Longitudinal Numeric Network density, centrality, geodesic distances, directionality Graphical Sociograms: visual representations of numeric data Matrices

34 Sociograms: Cross et al., 2009


36 Survey Content Measures more variable in type and complexity Like SNA, can draw heavily from the collaboration models for indicators of linkage levels or, Frequency of action measures e.g. number of communications in the past year

37 Survey Data Longitudinal Numeric Conventional statistical methods Graphical Microsoft PowerPoint: models using circles to represent partners and lines to represent linkages (Frey et al. 2003)


39 Thomson, Perry, and Miller 2007 Goal: to make collaboration quantifiable with an empirically tested measurement tool Built on the Wood and Gray (1991) definition Categorize collaboration into five dimensions: Governance, Administration, Mutuality, Norms, Organizational Autonomy

40 TPM Content 56 item closed-ended Likert scale questions on organizational attitudes and behaviors surrounding each of the five dimensions of collaboration From the 56-indicator questionnaire, 17 indicators were empirically validated in statistical structural equation modeling 17 indicators cross-validated in a 2004 study of 133 collaborators in a Family Preservation Program

41 TPM Data The Thomson, Perry, and Miller strategy measures solely the presence of collaboration using a fixed model of attitudinal and behavioral indicators developed as proxies for the dimensions of an operational definition of collaboration Collaboration is standardized More readily compared across initiatives


43 Qualitative Methodology Facilitated focus groups or in-depth interviews Standard data analysis methods

44 Qualitative Data Formative Who are the players? Process/Outcome Lists of collaborators, collaborative process info Contextual information: costs/benefits, barriers/facilitators to collaboration Mixed-method approaches In addition to SNA, survey, or TPM Validate numeric data Provide explanatory data

45 Degree of Complexity: instrument development, data analysis techniques and software, and required expertise Cost: material and non-material resources required by the data collection strategy, and the associated burden on the research team and respondents Evaluation Strategies Matrix

46 The Strategy to Improve Birth Outcomes in Baltimore City

47 Choosing an Evaluation Method Data Ideal…and, Realistic Timeframe Degree of complexity Cost

48 Data Matches ideal data Timeframe Case studies of similar scale Complexity SNA Evaluator on evaluation team Cost Software for large analyses Quick rating-scale questionnaire administered at time of other evaluation activities (e.g. community surveys) SIBO Community Collaboration Component Proposed Evaluation Design

49 How will we measure success?

50 Cognitive Mapping Formative Who are the players? What should the system look like? What should the exchanges between agencies look like? 3 Perspectives Core Implementation Team Service Providers Steering Committee

51 Cognitive Mapping 1.Inform sampling frame 2.Inform data instrument construction 3.Inform Technical Assistance Toolkit for Collaboration

52 Evaluation Next Steps Finalize questionnaire Finalize sampling frame Select agency representative respondents Issue baseline survey


Download ppt "Measuring Interorganizational Collaboration in Communities: Baltimores Strategy to Improve Birth Outcomes as a Case Study Donna Strobino and Meredith Matone."

Similar presentations

Ads by Google