1 Situational Analysis as a Process Tool in Human Services Programming Presented By Gary Bess, PhD Jim Myers, MSW Gary Bess Associates School of Social Work, California State University, Chico

2 Presentation Overview 1.Process evaluation defined, and its contribution to developing programs 2.Evaluator’s role in process evaluation in developing programs 3.SWOT assessment defined 4.Using the SWOT in a pilot program 5.Longitudinal tracing of themes within the developing program

3 Process Evaluation Process evaluation assesses reasons for successful or unsuccessful performance, and provides information for potential replication [italics added]. Process Evaluation focuses on how a program was implemented and operates. It identifies the procedures undertaken and the decisions made in developing the program. It describes how the program operates, the services it delivers, and the functions it carries out. * *

4 Process Evaluation As a newer form of evaluation research, process evaluation is generally associated with qualitative methods, in that “process” is by definition a qualitative exercise. Concurrent assessments, however, of quantitative results (e.g., number of clients serviced or changes in functioning) -- above or below projections -- are used as measures of the program having met its designers’ expectations.

5 Process Evaluation Process evaluation is a method of recording and documenting salient ideas, concerns, activities, administrative and management structures, staffing patterns, products, and resources that emerge during the program’s developmental cycle.

6 Process Evaluation Similar to outcome evaluation that measures the results of a project’s implementation against programmatic projections. Process evaluation assesses a priori assumptions, such as staff qualifications and training, usefulness of resources provided by other agencies, family or stakeholder receptivity to engagement, client acceptance criteria, and other planned activities and anticipated responses.

7 Evaluator's Role in Process Evaluation The evaluator’s relationship with program staff is key. Given the intimacy of interaction, a trust level must be developed between the evaluation team and the program team. It is essential that evaluators convey their support for the project’s success, tempered by a constructive objectivity, which translates into formative reflection and input.

8 SWOT Assessment Having addressed the evaluator’s role in process evaluation, we wish to introduce an experimental technique for documenting developmental processes within programs. It is an approach that creates a new functional role for the evaluator.

9 SWOT Assessment The technique is a facilitated SWOT analysis, conducted by the evaluator. SWOT is an acronym for Strengths, Weaknesses, Opportunities, and Threats (Barry, 1986). In a conventional SWOT analysis, variations of following questions are sequentially asked...

10 SWOT Assessment 1.What are the program’s internal and external strengths? 2.What are the program’s internal and external weaknesses? 3.What opportunities are present, within the program and in the environment that surrounds the program? 4.What threats exist – either internally or externally – that could adversely affect the program’s efforts?

11 Overview of the Pilot Program We used the SWOT as part of a 30-month pilot initiative in which the authors were engaged by a northern California county consortium of public agencies to evaluate a California Board of Correction’s (BOC) funded Mentally-Ill Offender Crime Reduction Project (MIOCR) – known on the county level as FOREST: Forensic Resource Team. Process evaluation data was obtained between May 2002 and June 2004, during which time the authors documented the developmental model.

12 Overview of the Pilot Program The partners were: County’s Superior Court Department of Behavioral Health Sheriff’s Department Probation Department District Attorney’s Office Public Defender’s Office

13 SWOT Assessment: Pilot Program The SWOT assessment’s dual focus on internal and external dynamics was important, in that the developmental process could have been influenced by environmental changes (external) as well as program-related issues (internal).

14 The SWOT was conducted approximately every three months in a office or meeting room. Key participants – project staff and managers – were invited, often as part of, or in place of, a regularly scheduled staff meeting. Supplies included one or two large newsprint pads an easels, markers, and tape. As sheets were completed, they were taped on walls so participants were able to review previously cited perceptions. SWOT Assessment: Pilot Program

15 Ground rules pertained to: Allowing everyone to speak Not challenging others’ ideas, but offering your own perceptions, even if they may differ No labeling of responses as either correct or incorrect Focus on issues and not solutions SWOT Assessment: Pilot Program

16 Sometimes clarification was requested as statements were ambiguous or incomplete, such as when “resources” were noted as a weakness, which could refer to monetary or programmatic materials or professional expertise. SWOT Assessment: Pilot Program

17 An assessment of content specific themes identified during early SWOTs, and their progression throughout the program’s lifecycle are presented in Figures 1 through 7. Themes Manifesting

18

19

20

21

22

23

24

25 Thank You!