Presentation is loading. Please wait.

Presentation is loading. Please wait.

SWK 7401. Evaluating Social Work Practice Alvin Mares, PhD, LSW Autumn 2012.

Similar presentations


Presentation on theme: "SWK 7401. Evaluating Social Work Practice Alvin Mares, PhD, LSW Autumn 2012."— Presentation transcript:

1

2 SWK 7401. Evaluating Social Work Practice Alvin Mares, PhD, LSW Autumn 2012

3 Week 1 (Aug 22 nd & 24 th ) Introductions – Instructor – Students and memorable client from 1 st yr field placement Organization of Textbook – Overview (Chpts. 1-2) – Measurement (Chpts. 3-10) – Evaluation Designs & Decision Making (Chpts. 11-15) Review of Syllabus – Assignments – Schedule

4 Introductions Instructor – Practice, program & policy-level interventions for at- risk, TAY (18-29) to prevent chronic homelessness Emphasis on post-secondary education & vocational training Students – Name – BSSW/BSW Field Placement (OR) Work/Volunteer Exp. Memorable client – First name – Age – Gender – Presenting problem – Assistance provided by you (i.e., intervention) – Outcome/disposition

5 Wednesday Evening Section StudentAgencyMemorable Client AletaMarion Area Counseling Center Alisha BrittanyNorth Central Mental Health Services, Inc. ChelseaUnited Methodist Children's Home EmilyOSU-Hospital East-Talbot Recovery Svcs JaneUnion County Dept Job and Family Svcs Jon JulieOhio Department of Youth Services

6 Wednesday Evening Section (cont) StudentAgencyMemorable Client KimOSU-Hospital East-Talbot Recovery Svcs KrystleDirections for Youth and Families Mara Marissa MurielDirections for Youth and Families NicoleRosemont Center, Inc. Shannon TaylorFranklin County Children Services

7 Friday Morning Section StudentAgencyMemorable Client JasonLoving Care Hospice, Inc Julie KatieFranklin County Children Services MandyAmethyst Inc. MichaelMarion Area Counseling Center MichelleNorth Central Mental Health MollyColumbus Public Health-Community Outreach Assistance Team Sarah TylerVolunteers of America, Greater Ohio YohanaOSU Multicultural Center

8 Organization of Textbook

9 Review of Syllabus Assignments Attendance ………. 15 pts. – 1 pt./class Discussion ?s …… 15 pts. – 1 pt./chpt. – Download from Carmen Quizzes …………….. 40 pts. – 20 pts. each – Drawn from Discussion ?s Evaluation plan … 30 pts. – Rubric forthcoming Total ………………. 100 pts. Schedule Wk. 1: Intro/overview Wk. 2: Chpts. 1 & 2 Wk. 3: Chpts. 3 & 4 Wk. 4: Chpt. 5 Wk. 5: Chpt. 6 Wk. 6: Chpts. 7 & 8 (Quiz 1) Wk. 7: Chpts. 9 & 10 Wk. 8: Chpt. 11 Wk. 9: Chpts. 12 & 13 Wk. 10: Chpts. 14 & 15 (Quiz 2) Wks. 11-13: Evaluation plan Wk. 14: Off Wk. 15: Wrap-up

10 Homework Due Aug. 29 th & 31st Read Chapters 1 & 2 Answer Discussion Questions for Chpts. 1 & 2 downloaded from Carmen – Highlight & type answers to questions in MS Word Bold correct answer for T/F and MC questions Type answers for essay questions – Brief answers (3-5 sentences) sufficient for most essay questions – As much as possible, apply concepts to your work last year with your memorable client when answering essay questions – Bring printed copy of questions with answers to class to share your thoughts with others and turn in at end of class; pass/fail grading (0 pts. if late or incomplete)

11 Week 2 (Aug 29 th & 31 st ) Chapter 1 Introduction to Client-Centered Evaluation of Practice (C-CEP) – 8 steps/components Case study (Phillip) Graph (Fig. 1.1, p.8) Chapter 2 Conceptualization: Naming What We See in the Client Situation – Concepts – Propositions – Theories: General vs. Local – Intervention Design, C-CEP & Logic Model Elements Case study (Ben & Katherine)

12 CHAPTER 1: Introduction to Client-Centered Evaluation of Practice

13 Chapter 1 Introduction to Client- Centered Evaluation of Practice C-CEP framework: 8 steps in evaluating practice – Case study example of Phillip, 10 y/o boy with diabetes who is feeling depressed 1. Client defines goals – feel pretty good most of the time [positive self-image] – feeling ladder from 1 (worst) to 5 (best); 3=ok 2. Select intermediary objectives and their specific targets – objective: weight loss – targets (empirical proxies) » exercise: riding bike around block 3 times » healthy diet: snack on carots & fruit

14 Chapter 1 Introduction to Client- Centered Evaluation of Practice C-CEP framework (cont) 3. ID evidence-based general practice from literature + evaluation-informed specific practice from work with client – review of literature to find promising or best practices – tailoring/application of general best practices to clients situation – feedback from client over time on effectiveness of practice/intervention 4. Collect and plot data on graphs – Y-axis: goals, objectives, targets (e.g., 1-5 feeling about self) – X-axis: time (e.g., days) – pattern of targeted events (improvement, decline, no change) during various periods (baseline (A), intervention (B), maintenance (M), follow-up (F))

15 Chapter 1 Introduction to Client- Centered Evaluation of Practice C-CEP framework (cont) 5. Select benchmarks defining success – client-designated (e.g., 4=positive most of the time) and/or – norms from standardized scales, agency/admin standards, etc. 6. Review and discuss with client at end of intervention phase (B) extent to which goals attained, helpfulness of intervention 7. Transition to maintenance phase (M) – Client assumes responsibility for ongoing intervention his/herself; drives solo (rather than w/ you in passenger seat) 8. Analyze empirical data (graphs) & compare with qualitative data (sharing during intervention sessions)

16 C-CEP Framework Summary 1.Identify the Client Who Defines the Goals in the Case 2.Select Intermediary Objectives and Their Specific Targets 3.Identify Evidence-Based General Practice From the Literature and Evaluation-Informed Specific Practice From Working With the Client 4.Collect Data in an Ongoing Basis and Plot Data on Graphs 5.Construct Clear Benchmarks to Know When the Results Are (or Are Not) Successful 6.Engage the Client in Determining If the Clients Goals Have Been Attained During Intervention 7.Introduce a Maintenance Phase in Which the Client Is in Complete Control of the Intervention on His or Her Own 8.Analyze Patterns of Data to Coordinate With the Client's Statements

17 C-CEP Graph

18 Chapter 1 Introduction to Client- Centered Evaluation of Practice 1-1. Single-system designs are used only for individual clients, especially in institutional settings. A. True B. False 1-2. Single-system designs can be used in many different professional settings where clear targets can be defined and clear actions can be taken to attain these targets. A. True B. False 1-3. Evaluation is built into the codes of ethical practice of many helping professions. A. True B. False

19 Chapter 1 Introduction to Client- Centered Evaluation of Practice 1-5. Evidence-based general practice is: A. The same as evaluation-informed specific practice, when used with a given client. B. The same as empirical research, using control groups and experimental groups. C. The truth, and so practitioners ought to use it exactly as discussed in the literature. D. The best available information on some general topic, but it ordinarily cannot be used directly with your specific client.

20 Chapter 1 Introduction to Client- Centered Evaluation of Practice 1-6. When you use evaluation-informed specific practice, it means: A. That you have used have used an evidence-based model, without varying from the protocol or trying to alter the model to fit your specific client. B. That the information obtained in your single-system evaluation specifically fits this particular client, and you can make meaningful statements as to how well that client is doing. 1-7. It is easy to mix up goals and objectives in evaluation. Take a case example from your own practice experience (or make one up), and distinguish goals and objectives in the case.

21 Chapter 1 Introduction to Client- Centered Evaluation of Practice 1-8. Identify at least three basic ingredients for all single-system designs used in evaluation of practice from this list of terms: a specified target (problem or desired goal) operational definitions of targets, interventions, benchmarks establishing a baseline different phases (baseline, intervention, maintenance, follow-up) take repeated measures in both baseline and intervention phases use of an evaluation design, at least the basic design analyses of data decision making

22 Chapter 1 Introduction to Client- Centered Evaluation of Practice 1-9. We defined targets as empirical proxies. Explain what this means in your own language, but include in your discussion the idea of operational definitions. (That is, illustrate this phase, empirical proxies, by describing a client situation in which such proxies are involved.) 1-10. Why do we assert that you need to collect data on a repeated basis, even at those times when you are not even doing any intervention? 1-11. You are working with a group of young women (ages 15-20 years), all of whom are pregnant with their first child. They have determined, among themselves, to continue their education. Give one example of a benchmark in these situations. (Hint: is it measurable? How can we know whether or not each client has achieved this benchmark?)

23 BREAK

24 CHAPTER 2: Conceptualization: Naming What We See in the Client Situation

25 Chapter 2 Conceptualization: Naming What We See in the Client Situation Concept – Also known as a construct – An arbitrarily constructed general term derived from some class of events – Serves as a building block for larger conceptual terms (e.g., propositions, theories) and as a link to the information network where you can find evidence-based practice ideas (Britner & Bloom)

26 Chapter 2 Conceptualization: Naming What We See in the Client Situation Propositions – Sets of concepts related in such ways as to describe: (1)what we assume to exist, so we can get to the second proposition; (2)what we hypothesize to exist and which we can test in the empirical world; or (3)something we prefer or value, for which there is no empirical test.

27 Chapter 2 Conceptualization: Naming What We See in the Client Situation Theories – Systems of concepts and propositions that focus on, describe, explain, and predict a limited domain of the world. – General theories are abstract enough to apply to broad topics (e.g., behaviorism) – Local theories deal with specific situations (for a given client and their context)

28 Chapter 2 Conceptualization: Naming What We See in the Client Situation Case study: Ben & Katherine (Kat) – Young couple w/o children arguing about… not having enough money (primary presenting problem) having children sex (secondary presenting problem) religion – Assessment and Problem Formation phase of the Social Work Problem-Solving Process (Dr. Bean) Problem definition = primary presenting problem(s) – A problem well-defined is a problem half-solved

29 The Social Work Problem-Solving Process

30 Chapter 2 Conceptualization: Naming What We See in the Client Situation Case study: Ben & Katherine (Kat) – Given primary concerns: financial problems & sex – C-CEP goals: financial solvency & sexual satisfaction – Concepts and propositions Concepts relating to money – work; earnings; recurring bills to pay; other expenses; part-time vs. full-time employment Propositions – Prolonged imbalanced of earnings compared to expenses leads to psychological stresses that may affect the continued well-being or survival of the group [marital relationship]. – When the stronger partner does not earn enough money to pay the bills, that partner will compensate his or her weakened situation by becoming more demanding in other domains (e.g., sex), which may be unwelcomed by the other partner.

31 Chapter 2 Conceptualization: Naming What We See in the Client Situation Case study: Ben & Katherine (Kat) – Theories Local theory – Therefore, only when former (economic) difficulty is resolved will the latter (sexual conflict, etc.) be resolved. General theory(ies) – Family Life Model (establishment)? – Family Life Cycle for families (new couple)? – Ericksons developmental stages (industry vs. inferiority)? – Levinsons developmental phases (early adulthood)?

32 Chapter 2 Conceptualization: Naming What We See in the Client Situation Case study: Ben & Katherine (Kat) – Action plan based on… Presenting problems of financial troubles (primary) & sex (secondary) Goals of financial solvency & sexual satisfaction Local & general theories General theory(ies) – Ben to find higher paying job to increase HH income Fig. 2.1: Graph of job status (old vs. new) Fig. 2.2: Graph of HH income ($ range per week) – Kat to extend educational career

33 Chapter 2 Conceptualization: Naming What We See in the Client Situation Case study: Ben & Katherine (Kat) – C-CEP and Logic Model Elements Decreased psychological stress Operational measure? Increased sexual satisfaction Operational measure? Financial insolvency New job HH income >300/wk Increased well- being of marriage relationship Operational measure? Problem GoalObjective 2 Objective 1 Intermediate OutcomesLong-term Outcome Condition Targets Assumptions Inputs, Activities & Outputs Propositions & Theories Intervention-related activities

34

35 The Social Work Problem-Solving Process

36 Remember that good theory will have direct relevance to practice There will be a translation process where you will using concepts and propositions of the theory to frame your practice model or intervention Being aware of your theory of change and communicating that to your client(s) is an essential component of the helping process Objectives and targets, guided by local theory Empirical evidence + theories (informed by review of literature & knowledge of client) are bases of plans of action [and revisions to action plans over time] Thoughts On Interventions

37 Chapter 2 Conceptualization: Naming What We See in the Client Situation 2-1. A concept is a general and abstract term, for which there may be many instances in the real world. A. True B. False 2-2. Two concepts combined with a linking term create a proposition. A. True B. False 2-3. Theories are either true or false. A. True B. False

38 Chapter 2 Conceptualization: Naming What We See in the Client Situation 2-4. What if, in the process of evaluating your client, some concept gains local empirical support? Does this make the concept true? Explain why or why not. 2-5. Explain in your own terms why propositions can be tested in the real world and be found to be true or false. How can we go from two or more abstract concepts linked together (i.e., a proposition) to generate some factual evidence with regard to the clients actions? 2-6. Clearly distinguish between local and general theory, illustrated with one of your own cases, if possible. 2-8. How are conceptual definitions related to operational definitions, if at all?

39 Caring Instructor Intervention

40 Homework Due Sept 5 th & 7 th Read Chapters 3 & 4 Answer Discussion Questions for Chpts. 3 & 4; print & bring to next class Provide memorable client brief description (i.e., first name, age, gender, presenting problem, assistance provided by you & outcome/disposition at last contact) Develop C-CEP and Logic Model Elements (1-pg. diagram/figure) for your memorable case, using Ben & Kat case study as an example Draw or print memorable client description and C- CEP/Logic Model elements diagram/figure and include with Discussion Question answers for Chpts. 3 & 4

41 Week 3 (Sept 5 th & 7 th ) Measurement issues (Chpt. 3) – Validity – Reliability – Error – Sustainability Graphing (Chpt. 4) – Rules – Interpretation

42 CHAPTER 3: The Issues: Measurement Theory; Validity; Reliability; Error Messages; Sustainability

43 Validity Definition – True, accurate measure of a concept Example: digital scale is an accurate measure of weight Types – Face: appears to be accurate measure (faith) – Content: comprehensive set of items/questions – Criterion: correlation with gold standard measure or behavior – Construct: measure supports proposition or theory – Client: achieved state = desired state Assessed at end of intervention and maintenance phases

44 Reliability Definition – Consistency Example: digital scale also highly reliable measure of weight Types – Interobserver: among two or more observers Agreements / (Agreements + Disagreements) = reliability score – Test-retest: same measurement under same circumstances – Alternate forms: different questions measuring same concept – Internal consistency: high correlation among set of questions

45 Error & Sustainability Measurement Error – Random: natural variability among people – Systematic: consistent bias due to setting, time of day/season, helping professional, etc. Sustainability – Length of maintenance period long enough to be sure that gains realized during intervention period are retained post-intervention

46 Chapter 3 Discussion Questions 3-2. Validity means consistency in reporting what we see in the world A. True B. False 3-3. Reliability means how truthful a measure is in reflecting what is occurring in the world. A. True B. False 3-5. Which of the following is a true statement? A. If a measure is reliable, it will also give valid information. B. If a measure is valid, it will also give reliable information. C. Neither A or B is true.

47 Chapter 3 Discussion Questions 3-9. We have introduced the idea of client validity, meaning that whatever is true in a situation has to be true for the client in that situation, and only the client can tell us whether or not this is so…What are the limits or risks in depending on a clients assessment of his or her own outcome at any given stage of practice? 3-11. Give an example from your client case, if possible, or make up a client case, in which you illustrate random and systematic errors. 3-12. Why is sustainability in client outcome important in client-centered evaluation of practice?

48 BREAK

49 CHAPTER 4: Graphing: The Basics; Exceptions to the Rules

50 The Rules Draw a graph with horizontal and vertical axes Label the graph – x-axis (time intervals); y-axis (target) – Phases, divided by vertical lines Reconstructed baseline (A) -- if retrospective data used Baseline A Intervention B Maintenance M Follow-up – as needed, at less regular intervals Add data points!

51 Exceptions Follow the rules unless there is a compelling reason not to do so (e.g., missing data). Justify your reasoning for an exception and document it. As appropriate, make an annotation on the graph.

52 Case Study: Rick and his walking support group (Figure 4.1)

53 Figures 4.2 and 4.3

54 Intervention Theory 1.Increase Bens income (Fig. 2.1 & 2.2) 1.Extend Kats schooling Reduced stress Improved sex Ben & Kat Increase attendance Improved grades Increased self-efficacy Ricky Objective(s) Shorter-term goal(s) Longer-term goal(s) Increased marital well- being Problem Financial problems Truancy (Fig. 4.1) (Fig. 4.2)(Fig. 4.3)

55 Chapter 4 Discussion Questions 4-1. A graph is a means of representing client data over time so as to observe the changes (positive or negative or none at all) and to respond to these changes with appropriate professional helping. A. True B. False 4-3. A target for intervention is described by naming the concept referring to that client concern and giving its operational definition. A. True B. False

56 Chapter 4 Discussion Questions 4-6. Describe an on-going progress of a case. At what point do you decide: – to begin intervention after a sufficient baseline period? – change the intervention, when either improvement or deterioration appear to be occurring? – terminate the case? Explain each of the three decisions. 4-7. Look at Figure 4.1 (from the text; or substitute another example). Find the annotation. Why is it helpful (or not) to have annotations directly written/printed on graphs?

57 Accessing Adobe Connect iPhone & iPad App: http://itunes.apple.com/us/app/adobe-connect-mobile-for-ios/id430437503?mt=8&ign-mpt=uo%3D2 Android App: https://play.google.com/store/apps/details?id=air.com.adobe.connectpro BlackBerry App: https://appworld.blackberry.com/webstore/content/36989/

58 Homework Due Sept 12 th & 14th Read Chapter 5 and answer Discussion Questions Diagram Intervention Theory describing your work with your memorable client – See Ben & Kat and Ricky slide from today Develop 1-3 graphs summarizing your work with your memorable client, including data as best you can remember [postponed to Wk. 5] – Should related to Intervention Theory diagram and resemble Figs. in Chpts. 2 and 4

59 Week 4 (Sept 12 th & 14 th ) Feedback on Homework from Wks. 2 & 3 Caring Instructor Intervention Update Information Retrieval (Chpt. 5) – Search Engines and Databases – Boolean Search Strategies Overview of Client Evaluation Plan Assignment [Postponed to Wk. 5] – Sample Practice Evaluation Article/Report – Guidelines and Rubric

60 Homework Feedback (Chpts. 1-4) Great start! All +4/4 (for those who submitted) A few missing Client Descriptions and/or C-CEP/Logic Model Diagrams … submit ASAP Memorable case data compiled and appended throughout semester for reference & illustrative purposes – See 7401 memorable client case studies PP slides on CarmenContentOther Next step for most: specify general theory(ies) of human behavior or lifespan development providing rationale/justification/support for (local) intervention theory (recall Ricky, Ben & Kat, Caring Instructor slides)

61 Caring Instructor Intervention (Local) Intervention Theory Objective(s) Shorter-term goal(s) Longer-term goal(s) Problem Increased social support Increased well-being Stress Beginning Data & Graph SectionWk 1Wk 2 Wk 3 Wed eve3.883.223.50 Fri am3.893.503.45 General Theory Forthcoming

62 CHAPTER 5: Information Retrieval: Finding general evidence-based practice information

63 Search Engines What search engines have you used in your research and/or daily life? Why do you use that search engine? What are its advantages and disadvantages? How would those advantages and disadvantages apply to a search in which you wanted to find help creating an intervention for your client? Source: Bloom & Britner (2012)

64 Academic Search Engines and Databases Identify key words or search terms. Compare a search using a free, full-text search engine (like Google Scholar) with one using a subscription database (like Social Work Abstract, PsycINFO, or MEDLINE). Try an appropriate (for the field or topic) what works? clearinghouse site for leads on valid (for your client) interventions. Source: Bloom & Britner (2012)

65 Searching OSU Databases from Home Go to Library home page: library.osu.edu Click on Off Campus Sign-in; log on with OSU username and password Click on Subject Guides, then Social Work Select database of interest… – Academic Search Complete … journal articles – Article Express … request journal articles not available online – OhioLINK … books – Campbell Collaboration … systematic reviews Note: After logging on to OSU library network, able to access many full-text publications from Google Scholar

66 Information Clearinghouses Child welfare … Child Welfare Information Gateway – http://www.childwelfare.gov http://www.childwelfare.gov Youth development … HHS Administration for Children and Families National Clearinghouse on Families and Youth – http://ncfy.acf.hhs.gov/publications/ydfactsh.htm http://ncfy.acf.hhs.gov/publications/ydfactsh.htm Substance abuse & mental health … SAMHSA Treatment, Prevention & Recovery – http://store.samhsa.gov/facet/Treatment-Prevention-Recovery http://store.samhsa.gov/facet/Treatment-Prevention-Recovery Homelessness … U.S. Interagency Council on Homelessness Research and Evaluation – http://www.usich.gov/usich_resources/research_and_evaluation http://www.usich.gov/usich_resources/research_and_evaluation Aging … NIH National Institute on Aging A-Z Health Topics – http://www.nia.nih.gov/health/topics http://www.nia.nih.gov/health/topics Others?

67 Boolean Search Strategies AND (intersection) OR (union) A … Self-concept B … Academics C … Practice Fig. 5.1 Ricky case illustration Source: Bloom & Britner (2012)

68 Searching the Literature A Pecking Order of Proof – Systematic Reviews and meta-analyses – Randomized controlled trial studies (true experiments) – Non-randomized comparison studies (quasi- experimental designs) – Cross-sectional studies with retrospective measurements (surveys) – Case studies Source: Bean (2012)

69 Caring Instructor Intervention Lit Review General theory article on social support – Cohen, S., & Wills, T.A. (1985). Stress, social support, and the buffering hypothesis. Psychological Bulletin, 98(2):310-357. Cited by 6,571 Examines whether the positive association between social support and well-being is attributable more to an overall beneficial effect of support (main- or direct-effect model) or to a process of support protecting persons from potentially adverse effects of stressful events (buffering model)…It is concluded that there is evidence consistent with both models…Implications for theories of social support processes and for the design of preventive interventions are discussed.

70 Caring Instructor Intervention Lit Review Local intervention theory article: social support among graduate students – Goplerud, E.N. (1980). Social support and stress during the first year of graduate school. Professional Psychology, 11(2):283-290. Cited by 64 Describes the effects of varying levels of social interaction on 22 1st-yr graduate students' reports of stressful events and on their health and emotional problems during the 1st 6 mo of graduate study…An inverse relationship was found between frequency of social interactions with peers and faculty during the 1st 10 wks of study and the incidence of stressful life events and the number of reported physical and psychological disturbances throughout the next 6 mo. The quality of faculty–student interactions also emerged as an important factor that moderated the negative consequences of the major life changes associated with beginning graduate school. Conclusion Social support appears to be a crucial variable that moderates the negative consequences of the unavoidable life changes that occur during students' first year of graduate work. Frequency of social interactions with other students and with faculty was found to be inversely related to the intensity and duration of life disruptions and also to susceptibility to psychological and physical disturbances. Developing faculty awareness of their critical influence on graduate students' health and emotional well-being and helping new students expand their socially supportive contacts appear to be important primary and secondary prevention strategies to reduce graduate students' risk for stress-related problems.

71 (Local) Intervention Theory Diagram for Caring Instructor Intervention Increased [faculty] social support Increased well-being Key intervention elements 1.Express interest 2.Validate feelings 3.Share thoughts, suggestions 4.Encourage contact with field liaison 5.Improve teaching, class as-needed Decreased stress 1-5 rating of overall well- being

72 Chapter 5 Discussion Questions 5-2. A search question puts together all of the major concepts under consideration as entry points into information space, recognizing that depending on the number of hits resulting from this very specific question, we may have to remove one or more terms until we reach some useful answers. A. True B. False 5-3. A key term is to information retrieval as concepts are to theory. A. True B. False

73 Chapter 5 Discussion Questions 5-5. What is the importance of a chapter on information retrieval in this book on evaluating client outcomes? 5-6. Lets say we wanted information on depression, children under the age of 10, and African Americans. How would we create a Venn Diagram that included just these ideas and excluded others? How would you write that search, in Boolean terms? 5-8. What if you locate information through your retrieval searches that indicate that the therapy or intervention you are using has been shown to be relatively ineffective with clients very much like your own? How does this information influence your practice methods?

74 BREAK

75 Demonstration of Searching Literature to Examine Level of Support for Local Intervention Theory

76 Homework Due Sept 19 th & 21st Read Chapter 6 and answer Discussion Questions Develop 1-3 graphs summarizing your work with your memorable client, including data as best you can remember – Should related to Intervention Theory diagram and resemble Figs. in Chpts. 2 and 4 Search the literature to find 1-3 scholarly articles/publications providing strongest level of support for your local intervention theory for your memorable client (ie, diagram included with Chpt. 5 Discussion Question answers) – Include full citation(s) and abstracts only – NOT FULL TEXT OF ARTICLES/PUBLICATIONS – next week with Chpt. 6 Discussion Question answers

77 Week 5 (Sept. 19 th & 21 st ) Examine practice evaluation article – Sample final report for capstone project due end of spring semester Review rubric for Evaluation Plan paper due Nov. 16 th (8 wks. from now) – Background & Methods sections of capstone final report – Results & Discussion sections added during spring semester Break Data collection options – Individualized rating scales & structured logs (Chpt. 6) – Observation (Chpt. 7) – Standardized rating scales (Chpt. 8)

78 Practice Evaluation Article CarmenContentOther – Sample clinical eval report – Title: On the use of progressive relaxation in the treatment of bulimia: A single-subject design study Background section – Paragraph 1: Description of "negative affect" model (i.e., theory of change explaining binge/purge aspect of bulimia) – P2: Description of stress-reduction function (i.e., intervention theory)

79 Practice Evaluation Article Background section (cont) – P3,4: Goal of study/project: examine effectiveness of progressive relaxation (i.e., advanced practice method) Methods section – P1: Patient (client) description – P2: Treatment (description of baseline phase and data collection methods) – P3: Description of intervention phases and clarification of key outcome measures (e.g., # of binge urges and binge episodes)

80 Practice Evaluation Article

81 Rubric for Evaluation Plan Paper CarmenContentOther – Rubric for eval plan paper Highlights – Timeline Memorable client & text readings … Wks 1-10 Develop of eval plan for current clients … Wks 11-13 – Draft eval plan paper due Fri., Nov. 16 th Receive feedback on draft from field instructor & myself (over Winter/Christmas break) Revise plan as-needed, based on feedback, and obtain final approval to begin collecting data by Jan. 28 th Complete data collection by Apr. 5 th (10 wks.) Submit final eval report by Apr. 19 th & present findings next week in class (2 wks. before Commencement)

82 Rubric for Evaluation Plan Paper Highlights (cont) – Sections and sub-sections Background section – Practice setting – Problem statement – Intervention – Theory of change Methods section – Clients – Measures – Design – Data collection – Analysis – Format (e.g., body 10 pg. max., single-spaced) – Checklist

83 BREAK

84 Data Collection Options Individualized rating scales & structured logs (Chpt. 6) Observation (Chpt. 7) Standardized rating scales (Chpt. 8) --- End of material covered on Midterm Quiz --- Midterm Quiz – Chpts. 1-8 of text – Take-home, completed on Carmen, opens Sept. 29 th at 10 AM and closes Oct. 2 nd at 11:59 PM (4 days to complete) – T/F, MC questions mentioned in lecture – Essay questions mentioned in lecture, applied to memorable client

85 Data Collection Options (Cont) Individualized rating scales & structured logs (Chpt. 6) – Co-constructed with client – Single item, Likert 1-5 (or so) responses – Unknown validity (beyond face) and reliability – Most useful (in my view) Observation (Chpt. 7) – Useful when able to observe client in setting where and when problem presents (seldom)

86 Data Collection Options (Cont) Standardized rating scales (Chpt. 8) – Existing, multiple item measure with known psychometric properties and available norms against which clients scores may be compared – Administered and scored by helping professionals (sans co-construction with client) – Known validity and reliability, thus most rigorous and scientific option – However, of limited use post-intervention (maintenance & follow-up periods) given unlikely self- administration and scoring by client Homework due Sept 26 th & 28 th : Discussion questions from chpts. 7 & 8

87 Week 6 (Sept. 26 th & 28 th ) Individual consultations – Return homework from chpts. 5 & 6 – Feedback on memorable case-related assignments (Local) intervention theory figure -- – see Caring Instructor slides from this wk Graph(s) Lit review article citation(s) and abstract(s) Revisions to Case Description and C-CEP/Logic Model Elements figure (as- needed) – Wed eve In-person: Alisha (4:50); Aleta (5:00); Marissa (5:10); Eva (5:20); Chelsea (5:30); Muriel (5:40); Krystle (5:50); Nicole (6:00); Jane (6:10); Brittany (6:20) Phone (Alvin @ 740-804-6275) : Jon (6:50); Kim (7:00); Shannon (7:10); Julie (7:20); Emily (7:30) – Fri am 40-minute general Q&A/discussion on memorable clients (9:10-9:50) Break (9:50) Individual consultations (10:00-11:55)

88 Week 6 (Sept. 26 th & 28 th ) Collect this weeks homework (chpts. 7 & 8) Homework due Oct 3 rd & 5 th – Mid-term Quiz (CarmenQuizzes; between 9/29 @ 10a and 10/2 @ 11:59) – Discussion Questions for Chpts. 9 & 10

89 Caring Instructor Intervention Lit Review General theory article on social support – Cohen, S., & Wills, T.A. (1985). Stress, social support, and the buffering hypothesis. Psychological Bulletin, 98(2):310-357. Cited by 6,571 Examines whether the positive association between social support and well-being is attributable more to an overall beneficial effect of support (main- or direct-effect model) or to a process of support protecting persons from potentially adverse effects of stressful events (buffering model)…It is concluded that there is evidence consistent with both models…Implications for theories of social support processes and for the design of preventive interventions are discussed.

90 Caring Instructor Intervention Lit Review Local intervention theory article: social support among graduate students – Goplerud, E.N. (1980). Social support and stress during the first year of graduate school. Professional Psychology, 11(2):283-290. Cited by 64 Describes the effects of varying levels of social interaction on 22 1st-yr graduate students' reports of stressful events and on their health and emotional problems during the 1st 6 mo of graduate study…An inverse relationship was found between frequency of social interactions with peers and faculty during the 1st 10 wks of study and the incidence of stressful life events and the number of reported physical and psychological disturbances throughout the next 6 mo. The quality of faculty–student interactions also emerged as an important factor that moderated the negative consequences of the major life changes associated with beginning graduate school. Conclusion Social support appears to be a crucial variable that moderates the negative consequences of the unavoidable life changes that occur during students' first year of graduate work. Frequency of social interactions with other students and with faculty was found to be inversely related to the intensity and duration of life disruptions and also to susceptibility to psychological and physical disturbances. Developing faculty awareness of their critical influence on graduate students' health and emotional well-being and helping new students expand their socially supportive contacts appear to be important primary and secondary prevention strategies to reduce graduate students' risk for stress-related problems.

91 (Local) Intervention Theory Diagram for Caring Instructor Intervention Increased [faculty] social support Increased well-being Key intervention elements 1.Express interest 2.Validate feelings 3.Share thoughts, suggestions 4.Encourage contact with field liaison 5.Improve teaching, class as-needed Decreased stress 1-5 rating of overall well- being

92 Week 7 (Oct. 3 rd & 5 th ) Return Chpts. 7 & 8 homework; discuss quiz Case management intervention elements Chpt. 9: Qualitative Data: Self-Monitoring Chpt. 10: Measurement Cautions --- (end Part II. Measurement of book) --- --- (begin final Part III. Evaluation designs) --- --- (begin transition memorable to current clients) --- Upcoming… Next wk: Chpt. 11: Baselining Week after next: Chpts. 12 & 13: Basic & Advanced Evaluation Designs

93 Mid-term Quiz Results & Feedback Scores …range: 15-20; mean: 18.0 (both sections) Overall, fine demonstration of ability to apply key practice evaluation concepts discussed in class to memorable client Essay questions subjective, with multiple reasonable applications of concepts to practice, yet some responses incorrect or incomplete Reminder to… – Use complete sentences when answering final quiz questions – Review slides pertaining to concept being asked about (e.g., pros & cons of individualized vs. standardized rating scales) Questions? Comments? Suggestions?

94 Case Management: General Practice Source Rothman, J. (1991). A model of case management: Toward empirically based practice. Social Work 36(6):520-528.

95 Case Management: Inpatient Health Source Taylor (1999). Comprehensive nursing case management: An advanced practice model. Nursing Case Management 4(1):2-10

96 Case Management: Mental Health Full Service attempted to provide all the clinical and support services needed by the client through the in vivo efforts of a specially trained interdisciplinary team. Brokering of services was minimized. Training in Community Living, Assertive Community Treatment, Assertive Outreach and The Bridge program (Test, 1992; Bond, McGrew & Fekete, 1995) are examples. Broker provided very little direct service to patients. Rather, services needed by the patient were arranged from among those available in the community mental health system. These programs included Broker, Specialist, Generalist and Supportive (Rubin, 1992; Solomon, 1992; Chamberlain & Rapp, 1991). Hybrid provided some services while other services were brokered. The distinction between full service and hybrid was a matter of degree. For example, programs characterized as PACT provided virtually all services, and were clearly full service. Other programs such as Intensive Case Management and Strengths expected staff to provide most services but would also broker some services. For the present review, Intensive Case Management, Expanded Broker, Family Case Management, Personal Strengths, Rehabilitation, and Clinical Case Management were considered hybrid models (Rubin, 1992; Solomon, 1992; Chamberlain & Rapp, 1991). Source Bedell et al. (2000). Case management: The current best practices and the next generation of innovation. Community Mental Health Journal 36(2):179-194.

97 CHAPTER 9: Qualitative Data in Single-System Designs: Self-Monitoring

98 Self-Monitoring (SM) Definition: client involved in observing & measuring some aspects about him/herself – Applicable when using structured logs and individualized rating scales (Chpt. 6) – Less/not applicable when using observation (Chpt. 7) or standardized rating scales (Chpt. 8) Maximal use of SM – and ideal for C-CEP – client… – Defines goals of service – Participates in constructing targets of intervention – Involved in data collection – Participates in interpretation of graphs – Informs helping professional when results have attained his/her goal(s) (client validity)

99 Self-Monitoring (cont) Qualitative form (Fig. 9.1, p.85) capturing data gathered using/following: – Narrative approach – Positive psychology / Strengths Perspective – Global assessment – Useful with memorable and/or current clients? Clients know themselves far better than the practitioner ever can. The role of the practitioner is essentially to bring out other positive alternatives for the client to use in resolving the presenting problem and move ahead in life goals. (p.83) Agree or disagree?

100 Fig. 9.1: Illustrative (Qual) SM Data Form Asset/strength #1 Asset/strength #2 Global: Overall situation

101 CHAPTER 10: Measurement Cautions

102 Issues Group practice/intervention measures – Observed by practitioner Participation >> attendance - # and/or % Engagement >> 1-5 rating by practitioner for each participant, averaged to compute single score for group – Self-monitored/reported by clients Perceived satisfaction, helpfulness or other domain >> mean 1-5 rating among participants using individualized rating scale Whenever we ask questions (or a person knows they are being observed), there is always some effect on the client.

103 Reactivity, Ethics & Homework Reactivity and obtrusiveness – Whenever we ask questions (or a person knows they are being observed), there is always some effect on the client. Clinical vs. research vs. practice evaluation pros & cons? – Minimize using archival records and observation Ethics – Clients define own goals – Clients understand & agree on interventions – Clients involved in choosing measures & eval designs – Clients determine if changes reflect real goals – Clients determine whether changes are sustainable Hmwk: Chpts. 9 & 10 due today; Chpt. 11 due next wk.

104 Week 8 (Oct. 10 th & 12 th ) Return assigns; review updated Carmen Grades --- (begin final Part III. Evaluation designs) --- --- (begin transition memorable to current clients) Chpt. 11: Baselining Briefing field instructor(s) on capstone project Next week – DUE: Chpts. 12 & 13 Discussion Questions – Peer evaluation of teaching (Dr. Balaswamy via Adobe Connect)

105 Grades Updated in Carmen Attendance – Wks 1-7 (through last week); out of 7 pts. Discussion – Chpts. 1-10 (received in class and via Carmen Dropbox, as of 10/9/Tues. @ 2p); out of 10 pts. – Zero pts. placeholder for chpts. not yet recvd. Midterm quiz – Chpts. 1-8; out of 20 pts. – Note mean 18.0 overall (17.9 Wed.; 18.0 Fri) Thus, as of last Fri. (10/5): 7 + 10 + 20 = 37 possible pts. (or 37% of grade) completed Remaining: 8 attendance + 5 discussion + 20 final quiz + 30 evaluation proposal paper = 63 pts.

106 CHAPTER 11: Baselining and the Beginning of Evaluated Practice

107 Chpt. 11: Baselining Critical assessment, presenting problem clarifying, starting point in C-CEP Case example: 7 y/o Clare victim of bullying at school – Local theory, baseline assessment questions What forms does bullying take, whos doing it, and to whom is it directed? What are teachers & staff doing about it? Where in school does the bullying occur? Does it also occur beyond the school day? When does it occur? How often does bullying occur? Are others present when the bullying occurs?

108 Chpt. 11: Baselining This is what the problem looked like when I entered the case, and this how it looked at several points during the intervention (or at the end of the intervention) (p.98). – Key problem targets (i.e., y-axis measures) in Clares case Graph 1, solid line: Extortion of lunch money (# times occurrence per school day; 0=no, 1=yes) Graph 1, dashed line: Social exclusion instances (# times per school day) Graph 2: Clares feelings of oppression (1-5 IRS; 1=very good … 3=mixed … 5=very sad)

109 Chpt. 11: Baselining Application to first client in field this year… – 11-9. Baselining is conducted to find clear patterns of the presenting concerns and the contexts in which they take place. How can we be sure we are collecting: a) the right kind of information; b) in the right amount; and, c) in the right way? Illustrate these points with a client situation. How many graphs needed for your client? Data collection method(s) required? – Observation, IRS and/or SRS How frequently, by whom, and by what means (e.g., logs) problem data will be collected?

110 Chpt. 11: Baselining Ideally concurrent baselining used – Both (A) reconstructed, in conjunction with, A current baseline (except when emergency intervention required, then (A) -> B, without A) – Purpose to establish an accurate picture of presenting problem(s) pattern(s) before intervention – Rough rule of thumb: 5-7 time (x-axis) units – Possibly more…however many time points needed to feel confident in accurately depicting problem pattern graphically during your initial baseline problem assessment stage of working with client

111 Chpt. 11: Baselining Application to first client in field this year… – 11-7. Why would we ever conduct reconstructed baselines, when we are beginning work with a client seated in front of us? – 11-8. How long should you collect baseline information? – 11-10. What patterns are you looking for when you collect baseline data? That is, what changes in these data are of main concern to the helping professional? – Openness to begin baselining with your current and/or new clients starting now? Possibly helpful in practice…definitely helpful in writing evaluation proposal (due 11/16)

112 Chpt. 11: Baselining 11-1. Which of the following is true of baselines? A. Intervention may take place during baselining, as needed and depending on the clients readiness. B. Systematic measures and/or observations are used on a regular and repeated basis throughout the baseline phase and whenever you return of a non-intervention phase. C. Baselining criteria should be selected by the helping professional. 11-2. If you had good archival data on a clients concern, you could use the archival information as a kind of reconstructed baseline. A. TrueB. False 11-3. If collecting baseline information from the client is having a negative impact on him or her, you should stop collecting baseline data and move into intervention, using the best available information and making suitable corrections as you go along. A. TrueB. False

113 Chpt. 11: Baselining 11-4. Baselining is an absolute essential in single-system evaluation, and must be performed at the beginning of every case. A. TrueB. False 11-5. Clients have to be directly involved in collecting baseline data. A. TrueB. False 11-6. We should always share baseline data with clients so that they can see the scope of their concerns and what is to be changed. A. TrueB. False

114 BREAK

115 Field Instructor Briefing & Next Week Email and/or print capstone project briefing packet and discuss with field instructor during next supervision meeting – Carmen – Content – Other – Field instructor capstone packet – Cover letter; 4 overview slides; proposal rubric; syllabus; and, sample clinical practice evaluation report (progressive relaxation and bulimia article) Emphasize NOT research, rather clinical practice evaluation/field education project Next week: Chpts. 12 & 13 (Basic & Advanced Evaluation Designs); peer eval teaching (Wed eve)

116 Week 9 (Oct. 17 th & 19 th ) Preview of next semester (SWK 7402) Field instructor feedback on capstone project? Evaluation designs (Chpts. 12 & 13) Coming up…

117 Preview of Next Semester Common elements across capstone sections, tracks – 9 meetings in class throughout term 3 at beginning; 3 near middle; and 3 at end – Grading elements Attendance (10-15%); Assignment(s) (10-15%); Presentation (10- 15%); and Final Report (60%) Student presentations (using PowerPoint) most meetings, based on assignments due that week of class Final report – Background & Methods sections revised from evaluation proposal, possibly throughout term if/as-needed – Results & Discussion sections (including dissemination or decision-making component) – Comparable rubrics – Final report due Apr. 8 th Unique elements across sections, tracks – Topics & assignments for each meeting dates

118 Mares Sections Schedule (Tentative) Wk (Date)LocationTopic & Assignment 1 (Jan. 8 th )Class Overview of online revision & resubmission, data collection monitoring system 2 (Jan. 15 th )ClassRevised proposals (RP) DUE [10 pts.]; RP presentations 3 (Jan. 22 nd )ClassRP presentations (cont) 4 (Jan. 29 th )---Begin data collection (most students) 5 & 6 (Feb. 5 th & 12 th )---Continued collection of baseline & early intervention data 7 & 8 (Feb. 19 th & 26 th )ClassBaselining and design presentations 9 & 10 (Mar. 5 th & 12 th )--- 11 (Mar. 19 th )Class Draft Results & Discussion (R&D) sections DUE; R&D presentations 12 (Mar. 26 th )--- 13 (Apr. 2 nd )Class End data collection (most students); Data Analysis (DA) presentations 14 (Apr. 9 th )ClassFinal report (FR) DUE (Apr. 8 th ) [60 pts.]; FR presentations 15 (Apr. 16 th )Class Reflections on continued C-CEP; feedback on capstone project, 7401/02 Plus: Attendance (18 pts.; 9 classes @ 2 pts.); Presentation (12 pts.) = 30 pts.

119 Field Instructor Feedback on Capstone Project?

120 Evaluation Designs Basic Designs: client outcomes only; no causal inference re/ effectiveness of intervention 1.B Only (my own term; variation of authors BABM Emergency Design from Chpt. 13) 2.AB*M** (Chpt. 12) 3.Multiple Baselines (from Chpt. 13) Advanced Designs: both client outcomes & causal inference 4.ABAM 5.ABABM 6.Changing Intensity

121 B-Only Design More realistic (in my view) version of authors BABM Emergency Design from Chpt. 13) No practical opportunity for baseline or maintenance period due to client risk and brief nature of intervention Applicable with single-session therapy and limited-time crisis intervention work – Healthcare: emergency departments; medical/surgical inpatient units & outpatient clinics – Child welfare: intake & investigation units – Poverty and homelessness: emergency food, clothing & shelter programs; public assistance & benefits application – Education & training: single-session classes, seminars, groups Compare group mean (y-axis) to threshold for each day of practice (x-axis), rather than comparing individual client or group cohort change from A (baseline) to B (intervention) periods…and ideally M (maintenance) period

122 AB*M** Design Builds on AB design. Adds maintenance phase M, in which practitioner teaches the client to take full control over a successful intervention B change since baseline A in the target. Asks client about goal attainment (indicated by *) after B and again (indicated by **) after M. This is the fundamental design for client-centered evaluation of practice, new because: – The client is directly asked about how well her or his goals are being achieved, to complement any objective measurement. – The clients takes charge of the measurement and the maintenance of the intervention.

123 AB*M** Example: Case Study (Jesse, runner; Figure 12.1)

124 AB*M** Steps 1.Identify the client and the clients goals. 2.Break down goals into specific objectives and measurable targets. Plot the data on a graph through baseline A and intervention B phases. 3.When the helping professional and client agree that positive change has occurred, go to a maintenance M phase, led by the client. 4.Collect M phase data to see if positive results persist. If not, reconsider B or training.

125 AB*M** Questions Definition – 12-1. An evaluation design is a formal plan in which something is done (the intervention) to something else (the target) with the objective-to-be tested that some change will occur approximating what the client is seeking through the service. A. TrueB. False Key elements 12-3. The meaning of the symbols A-B design is that an Action or intervention is introduced at A, and is then compared to the Baseline indicated by B. A. TrueB. False

126 AB*M** Questions (cont) Objective vs. subjective (client-validity) outcomes – 12-6. What does objective evaluation of services mean to you (in your own words)? – 12-8. What do you see as the main advantage of using the * in the basic AB*M** design? And what is the distinctive advantage of the ** in the basic AB*M** design? – Why dont we just accept what the client says about his or her goal attainment at the end of the B phase, and terminate the case? – Why do we accept what the client says when he or she is in charge of the intervention in the M phase?

127 Multiple Baselines (Figure 13.15)

128 Multiple Baselines Design Authors description – Ideal Advanced Design for practitioners reluctant to withdrawal a successful intervention from clients – 3 scenarios 2 persons with same problem in same setting 2 different problems with same person in same setting 2 different settings with same person and same problem My view – Falls under Basic Designs due to lack of causality (i.e., no quasi-experimental withdrawal of intervention) – 4 th scenario: 2 problems with differing amounts of baseline data available (most likely reconstructed from existing clinical records or other sources)

129 BREAK

130 Advanced Evaluation Designs Whos Happy? – Client and Practitioner If the client target behavior changes If the change is maintained by the client – Researcher If we can show that the intervention caused the change. – Cant everyone be happy? Yes, with some advanced designs: – ABA and ABAM – ABAB and ABABM – Multiple Baseline Designs (and those with M)

131 Advanced Evaluation Designs Some evaluation designs remove an intervention to see whether the target returns to a prior (undesired) state, as part of the process of investigating causality. To distinguish pre-intervention baselines from those times when intervention is intentionally removed, we will call the latter Observation- only Phases. The exact same data collection methods are being applied to the same target.

132 Advanced Evaluation Designs Adding an M phase and * and ** – Conventional designs, such as the AB, the ABA, the ABAB, and the multiple baseline design, all may likewise benefit from a maintenance M phase in which the client interprets results from his or her point of view (that is, * and **). On some occasions, the changing intensity designs and alternating intervention designs may likewise benefit.

133 ABAM Design

134 ABABM Design

135 Advanced Designs Question 13-9. When you compare results from an ABAB design, an ABAM design, and an ABABM design, you may see quite different results. Explain what differences might occur, and why one of these designs -- identify it -- is the strongest among the three patterns.

136 Changing Intensity Design

137 Authors case example – Developmentally delayed woman, Frida, in sheltered workshop employment program – Behavioral intervention: increasing periods of work (10 mins -> 15 mins -> 25 mins) rewarded with 1 min break talking with you (practitioner) – Aside: chronic disease/problem (i.e., developmental disability) reduces M (and F) period outcomes expectations Applies also to other evaluation designs (basic & advanced) – Nearly all practice may be considered changing intensity, given that most work winds down (M period) and ends (F period) at some point, leaving client to manage on his/her own Ethical practice within constraints of limited time and resources

138 Advanced Design Question 13-7. What do you see as the ethical issues in using advanced [quasi-experimental] designs? Consider how much stress one can impose on clients by making changes in the situation vs. our interest in scientifically evaluating the effectiveness of our practice. Why do we not just use the simplest scientific design, the AB*M** design, that provides some evidence of change due to the intervention?

139 Advanced Design Question 13-8. Alternative explanations are other possible reasons why the results came out as they did. The intervention is only one possible explanation for any change in the target. We argue that advanced designs reduce the impact of these alternative explanations by controlling the logic of the situation. Illustrate this with reference to either your memorable client or your first client, where you describe the… (i)Situation/presenting problem(s), (ii)Intervention, and (iii)Most appropriate advanced evaluation design to explain why chance alone would be a less likely explanation than the intervention

140 Coming up… Next week (Oct. 24 th & 26 th ) – DUE: Chpts. 14 & 15 Discussion Questions … last ones!! – Final Quiz on Carmen Chpts. 11-14 (excluding Chpts. 9, 10 & 15) – Topics: Baselining; Evaluation Designs; and, Data Analysis Opens 10/27/Sat @ 10a … closes 10/30/Tues @ 11:59p Week after next (Oct. 31 st & Nov. 1 st ) >> NO CLASS – DUE: Draft Background section of Eval Proposal to Carmen Dropbox by 11/3/Sat @ noon See 7401 Practice Evaluation Proposal Rubric on Carmen Practice setting; problem statement; intervention; theory of change Draft Figures 1 & 2 (C-CEP/Logic Model & Local Intervention Theory)

141 Week 10 (Oct. 24 th & 26 th ) Return homework; CarmenGrades updated Analysis of Data (Chpt. 14) Decision Making (Chpt. 15) >> to be discussed mid-Spring semester (ie, Wk 11/Mar. 20 th ; 8 th wk of 10-wk practice eval capstone project data collection period) Coming up…

142 CHAPTER 14: Analysis of Data: A Systematic and Holistic Approach

143 Chpt. 14: Data Analysis Methods Four methods of analysis (all valid, all limited) – Some or all should all be performed to obtain a systemic or holistic perspective on the situation 1.Trend analysis of objective data (Methods 1 & 3 in chapter) 2.Goal attainment for subjective client validity data (Method 2) (ie, * and ** in AB*M** design) 3.Non-overlapping data: baseline vs. intervention periods (Method 4) 4.Sustained time analysis: Maintenance period (Method 6) 5.Statistical analysis (Method 5): not discussed; beyond scope of book/class

144 Trend Analysis of Objective Data

145

146 Goal Attainment for Subjective Data Global assessment (from Fig. 9.1) Looking at your whole situation (problems, strengths, resources), how would you judge the changes in the complex whole since… – …we began working together? (* or change from A -> B), AND – …you began doing what we discussed on your own (** or change from B -> M) Objective rating (1-5 Likert scale) – 1=Great Deterioration – 2=Some Deterioration – 3=About the Same as Before – 4=Some Improvement – 5=Great Improvement Subjective, narrative explanation for objective rating (Why?) Domain-specific assessment (OPTIONAL)

147 Non-Overlapping Data: A vs. B Periods Effect Size, or Improvement Rate Difference (IRD)

148 Chapter 14 Discussion Question 14-2. Effect size is one way to put into numbers the extent of improvement that has occurred, if any, between one phase and another phase. A. TrueB. False

149 Sustained Time Analysis: B vs. M Periods Sustained time of successful (self-help) intervention = M period Recommended length of M period (ideal for use in post-MSW practice … vs. possibly contrived 10-wk capstone project data collection period) Low-severity target … equal to B period (most common) Medium-severity target … twice as long as B period High-severity target … three times longer than B period

150 Chapter 14 Discussion Question 14-1. Which of the following are correct with regard to a desirable intervention pattern? A. There should be real improvement in intervention, compared to the baseline phase. B. The client will feel like the intervention should end. C. Once the desired level of outcome has been attained, the helping professional can begin termination. D. The desired condition should be sustained for an indefinite period of time before termination of the case begins.

151 Chapter 14-Related Question What is the (a) primary strength and (b) limitation for each of the data analysis methods discussed?

152 Coming up… This weekend: Final Quiz (Chpts. 11-14): 10/27/Sat @ 10a – 10/30/Tues @ 11:59p Wk 11 (Oct. 31 st & Nov. 2 nd ): NO CLASS; Draft Background section of paper to Carmen Dropbox by 11/3/Sat @ noon Wk 12 (Nov. 7 th & 9 th ): Draft Methods section of paper to Carmen Dropbox by 11/10/Sat @ noon; review final quiz results & final paper rubric Wks 13 & 14 (Nov. 14 th /16 th & 21 st /23 rd ): NO CLASS; Individ consults as-needed (in-person, by phone and/or by Adobe Connect) – Final paper deadline extended from 11/16/Fri to 11/18/Sun at 11:59p Wk 15 (Nov. 28 th & 30 th ): Individ meetings during class to discuss final paper score and comments/feedback

153 Week 12 (Nov. 7 th & 9 th ) Final paper rubric Caring instructor intervention review Final quiz results - - - - - Upcoming – NO CLASS MEETING next 2 wks (during submission & grading of complete final paper/evaluation proposal) – Wk. 15 (Nov. 28 th & 30 th ) 15-minute consultations re/ feedback on paper (for revision after break) and final grade for 7401 Feedback on course Last date to submit any assignment: Nov. 25 th by 11:59 PM

154 Final Paper Rubric: Two sections; nine sub-sections Checklist: p.4 of Rubric Submission: Deadline extended to 11/18, 11:59 PM Revision: First 2 weeks of spring semester, responding to written comments/feedback received from me and field instructor; 10% of grade in 7402 Feedback: Provided by me during last class

155 Final Paper Fig. 1 C-CEP/Logic Model Example: Slide #32

156 Final Paper Fig. 2 (Local) Intervention Theory Ex: Slide #70

157 Caring Instructor Outcomes Excellent Good Average Fair Poor

158 Caring Instructor Review Having concluded the (lone vs. initial?) B (intervention) phase, how helpful in life overall? – 5=Very helpful – 4=Somewhat helpful – 3=Mixed/unsure – 2=Minimally helpful – 1=Not at all helpful Example of the initial asterisk (*) following B phase Shared decision-making; co-construction of next step… – End intervention >> B-only design – Continue, resuming after break >> custom B*MB design No change Change (Specify…)

159 Final Quiz Results Mean scores – Wed. eve.: 18.2 – Fri. a.m.: 18.4 – Overall: 18.3 Range (same for both sections): 15-20 Discussion of specific quiz items

160 Wk. 15 (Nov. 28 th & 30 th ) Consultations 15 minutes per student to… – Deliver written comments on paper – Discuss assignment scores & final grade in 7401 15 minute phone calls or in-persons meetings in my office (Wed. eve. section on 11/28 & 11/30) 15 minutes x 10 students (Fri. a.m.) = 2.5 hrs. (9:10-11:40 AM) – In-class – Random order – Bring homework to work on while waiting


Download ppt "SWK 7401. Evaluating Social Work Practice Alvin Mares, PhD, LSW Autumn 2012."

Similar presentations


Ads by Google