Presentation on theme: "It is impossible to present the philosophy of scientific realism and its application in social science in a 30 minute presentation It is possible to identify."— Presentation transcript:
It is impossible to present the philosophy of scientific realism and its application in social science in a 30 minute presentation It is possible to identify crucial problems in using evidence to inform policy recommendations And to explain how a realist perspective stresses the need to understand mechanisms and contexts in order to explain outcomes. The presentation will present examples of research that emphasise the importance of mechanisms.
EMPIRICISM description and measurement give us the facts about the world. the world exists independently of us. But knowledge is reduced to the level of events. theoretical descriptions must be defined in terms of observables (operationalisation) relies mainly on quantitative methods REALISM objects in the physical, social & psychological world exist & have properties independently of our theoretical concepts and discourse about them. maintains the linguistic objectivity of scientific theories. layered ontology Makes use of qualitative & quantitative methods. accepts that theories should be subject to empirical validation. INTERPRETIVISM hermeneutic social constructionist relativist everything we know is socially constructed (including science) tends reduce existence to texts, discourses and forms of consciousness relies mainly on qualitative methods.
From: BYGSTAD, B. & MUNKVOLD, B. E. (2011) In Search of Mechanisms. Conducting a Critical Realist Data Analysis. 32 nd International Conference on Information Systems, Shanghai 2011.
Related in various ways in Social Science (e.g. Giddens Structuration theory) Conflation and Reduction abound! (e.g. Brainism, Rational Choice Theory, class reductionism of (some) Marxism) Critical Realism attempts to engage with the complexity, openness, and ambiguity of social action.
Are they technologies like drug treatments or surgical interventions? Are they more akin to social or educational programmes which attempt to persuade people to change their behaviour? They consist for the most part of conversations – perhaps they are more like rhetoric (the art of persuasion).
The question what works? should be replaced by what works for whom in what context? Social programmes (such as programmes to reduce crime, increase health and well-being, or to promote recovery) are complex, often with multiple outcome goals. Notoriously, such programmes are evaluated (including with RCTs) as efficacious in one setting, but disappoint when transferred to another. The outcome of a mechanism is contextual, i.e. dependent on other mechanisms. Thus, a mechanism may produce an outcome in one context, and another in a different context. This contingent causality (Smith 2010) is inherent in all open systems. (Bygstad, B., & Munkvold, B. E. (2011). In Search of Mechanisms. Conducting a Critical Realist Data Analysis.)
Key Principles (from Nick Tilleys presentation Conducting Realist Evaluations) Programs are theories incarnate Evaluations are tests of theories Theories need to comprise context-mechanism-outcome conjectures Mechanisms refer to the ways in which effects are brought about Contexts refer to the conditions for the operation of mechanisms. They are seldom closed. Outcomes refer to the effects of mechanisms activated in context Mechanisms generally (though not always) involve reasoning and resources Programs work differently amongst different subgroups
Social programmes, as discussed in detail in Chapter 2, offer resources (material, social, cognitive) to subjects, and whether they work depends on the reasoning of these individuals. Subjects may seek out programmes (or not), volunteer for them (or not), find meaning in them (or not), develop positive feelings about them (or not), learn lessons from them (or not), apply the lessons (or not), talk to others about them (or not). It is within this interpretative process - or mechanism - that the causal powers of programmes reside. Any method of systematic review that omits such a vital agent from its core hypotheses automatically sets up a depleted inquiry. Pawson R. (2006) Evidence-Based Policy: A Realist Perspective (Chapter 3)
A realist framework: Outcome: Transformative change (provisionally defined as durable change in a clients frame of reference /emotional commentary, evidenced by their metacognitive accounts of attitude and relationship changes) Mechanisms: causal processes generating change - powers emergent from relations (for example modification of emotional commentary resulting from interactions in a therapeutic relationship) Contexts: structural powers bearing on the actualisation of mechanisms (including countervailing mechanisms, and powers of institutional and other social structures)
From: Pawson R. (1996) Theorizing the Interview. British Journal of Sociology 47/2, p.300
Koski-Jännes, A. (2005) On the Problems of Randomized Clinical Trials as Means of Advancing Clinical Practice. Evidence Based Practice: Challenges in Substance Abuse Treatment. Helsinki: Nordic Council for Alcohol and Drug Research (NAD). Morgenstern, J. & Mckay, J. R. (2007) Rethinking the paradigms that inform behavioral treatment research for substance use disorders. Addiction, 102/9, 1377-1389. Orford, J. (2008) Asking the right questions in the right way: the need for a shift in research on psychological treatments for addiction. Addiction, 103/5. Tucker, J. A. & Roth, D. L. (2006) Extending the evidence hierarchy to enhance evidence-based practice for substance use disorders. Addiction, 101/7, 918-32. Tudor Hart, J. (1997) What evidence do we need for evidence based medicine? Journal of Epidemiology and Community Health (1979-), 51/6, 62.
Weak theories: If interventions work through specific effects then some should be superior due to the efficacy of their ingredients. Theory-based matching effects should appear. Very little evidence for these. Markedly inconsistent findings in efficacy, moderator and mediator effects with all the evidence-supported interventions. Important differences between the outpatient and aftercare arms of Project MATCH. (Morgenstern, J., & McKay, J. R. (2007). Rethinking the paradigms that inform behavioral treatment research for substance use disorders. Addiction, 102(9), 1377-1389.)
Many trials Various meta-analytic reviews Some recent investigations into mechanisms Apodaca, T. R., & Longabaugh, R. (2009). Mechanisms of change in motivational interviewing: a review and preliminary evaluation of the evidence. Addiction, 104(5), 705-715 Morgenstern, J., Kuerbis, A., Amrhein, P., Hail, L., Lynch, K., & McKay, J. R. (2012). Motivational interviewing: A pilot test of active ingredients and mechanisms of change. Psychology of Addictive Behaviors. 26(4), 859-869.
Results show significant main effects for student residence, with more drinking and more problems reported by students living in fraternity and sorority houses. For students who live in the Greek system, it is possible that a house-based intervention (rather than the individual interviews conducted in this study) would be an effective prevention strategy. Current research conducted by our group is investigating the effectiveness of such a group-based intervention for these students.
What is the mechanism here? And what context facilitated its activation?
This is a eld where there is no lack of randomised controlled trials. Perhaps it is time to move from only studying whether MI works to also studying how it works, that is to study the mechanisms behind MI. (Smedslund et al 2011 p.28, emphasis in original).
Morgenstern, J., Labouvie, E., McCrady, B. S., Kahler, C. W., & Frey, R. M. (1997). Affiliation with Alcoholics Anonymous after treatment: A study of its therapeutic effects and mechanisms of action. Journal of consulting and clinical psychology, 65(5), 768. Morgenstern, J., Bux, D., Labouvie, E., Blanchard, K. A., & Morgan, T. J. (2002). Examining mechanisms of action in 12-step treatment: The role of 12-step cognitions. Journal of Studies on Alcohol and Drugs, 63(6), 665. Morgenstern, J., & Longabaugh, R. (2002). Cognitive–behavioral treatment for alcohol dependence: A review of evidence for its hypothesized mechanisms of action. Addiction, 95(10), 1475-1490. Morgenstern, J., Bux, D. A., Labouvie, E., Morgan, T., Blanchard, K. A., & Muench, F. (2003). Examining mechanisms of action in 12-Step community outpatient treatment. Drug and alcohol dependence, 72(3), 237-247. Longabaugh, R., Donovan, D. M., Karno, M. P., McCrady, B. S., Morgenstern, J., & Tonigan, J. S. (2005). Active Ingredients: How and Why Evidence Based Alcohol Behavioral Treatment Interventions Work. Alcoholism: Clinical and Experimental Research, 29(2), 235-247. Longabaugh, R. (2007). The search for mechanisms of change in behavioral treatments for alcohol use disorders: a commentary. Alcoholism: Clinical and Experimental Research, 31(s3), 21s-32s. Huebner, R. B., & Tonigan, J. S. (2007). The Search for Mechanisms of Behavior Change in Evidence Based Behavioral Treatments for Alcohol Use Disorders: Overview. Alcoholism: Clinical and Experimental Research, 31, 1s- 3s. Apodaca, T. R., & Longabaugh, R. (2009). Mechanisms of change in motivational interviewing: A review and preliminary evaluation of the evidence. Addiction, 104(5), 705-715. Longabaugh, R., & Magill, M. (2011). Recent advances in behavioral addiction treatments: focusing on mechanisms of change. Current psychiatry reports,13(5), 382-389.
Longabaugh, R., & Magill, M. (2012). Commentary on Michie et al.(2012): The lid is off the black box. Addiction, 107(8), 1441-1442. Morgenstern, J., Kuerbis, A., Amrhein, P., Hail, L., Lynch, K., & McKay, J. R. (2012). Motivational Interviewing: A Pilot Test of Active Ingredients and Mechanisms of Change. Psychology of Addictive Behaviors, 26(4), 859-869. In the last of these, the authors distinguish between active ingredients, which are things the programme or therapist does, and mechanisms of change which are processes that go on with the client, e.g. changes in reasoning or take-up of resources. The Palo Alto group (Moos, Finney etc) were historically interested in contexts and have attempted more recently to characterise treatment programmes according to theory-based identification of active ingredients: Bromet, E., & Moos, R. H. (1977). Environmental resources and the post-treatment functioning of alcoholic patients. Journal of Health and Social Behavior, 326-338. Cronkite, R. C., & Moos, R. H. (1980). Determinants of the post-treatment functioning of alcoholic patients: A conceptual framework. Journal of Consulting and Clinical Psychology, 48(3), 305. Holahan, C. J., & Moos, R. H. (1987). Personal and contextual determinants of coping strategies. Journal of personality and social psychology, 52(5), 946. Moos, R. H. (2007). Theory-based active ingredients of effective treatments for substance use disorders. Drug and alcohol dependence, 88(2), 109-121. Moos, R. H. (2007). Theory-based processes that promote the remission of substance use disorders. Clinical Psychology Review, 27(5), 537-551.
Problems in relying solely or mainly on an experimental research design to provide evidence about how alcohol or drug dependence treatment works (let alone recovery- promoting systems of care) have been recognised for 35 years. The EBM movement encouraged an almost exclusive emphasis on RCTs as the gold standard to try to identify efficacious treatments along the drug testing model. Longitudinal outcome studies have been sometimes grudgingly included but they have problems of their own. In the view of many (e.g Morgenstern & McKay 2007, Orford 2008,) the results have been disappointing or have reached a dead end. Morgenstern, J., & McKay, J. R. (2007). Rethinking the paradigms that inform behavioral treatment research for substance use disorders. Addiction, 102(9), 1377-1389. Orford, J. (2008). Asking the right questions in the right way: the need for a shift in research on psychological treatments for addiction. Addiction, 103(6), 875-885. *according to Prof. Nick Tilley, realist evaluation refuses to provide (meaningless) simple (minded) answers that are sometimes wanted but this of course does not mean that critical realist researchers are against evidence- based practice or reject RCTs as valueless.
Timko C., Moos R.H., Finney J.W., Lesar M.D. (2000) Long-term outcomes of alcohol use disorders: Comparing untreated individuals with those in Alcoholics Anonymous and formal treatment. Journal of Studies on Alcohol. 6(4):529–540. Kaskutas L. A., Bond J., Humphreys K. (2002) Social networks as mediators of the effect of Alcoholics Anonymous. Addiction 97: 891–900. Timko, C., Billow, R., & DeBenedetti, A. (2006). Determinants of 12-step group affiliation and moderators of the affiliation–abstinence relationship. Drug and alcohol dependence, 83(2), 111-121. Kelly, J. F., Magill, M., & Stout, R. L. (2009). How do people recover from alcohol dependence? A systematic review of the research on mechanisms of behavior change in Alcoholics Anonymous. Addiction Research & Theory,17(3), 236-259. Kelly, J. F., Stout, R. L., Magill, M., Tonigan, J. S., & Pagano, M. E. (2011). Spirituality in recovery: a lagged mediational analysis of Alcoholics Anonymous principal theoretical mechanism of behavior change. Alcoholism: Clinical and Experimental Research, 35(3), 454-463. Kelly, J. F., Stout, R. L., Magill, M., & Tonigan, J. S. (2011). The role of Alcoholics Anonymous in mobilizing adaptive social network changes: A prospective lagged mediational analysis. Drug and alcohol dependence, 114(2), 119-126. Kelly, J. F., Hoeppner, B., Stout, R. L., & Pagano, M. (2011). Determining the relative importance of the mechanisms of behavior change within Alcoholics Anonymous: a multiple mediator analysis. Addiction, 107(2), 289-299. Kelly, J. F., & Hoeppner, B. B. (2012). Does Alcoholics Anonymous work differently for men and women? A moderated multiple-mediation analysis in a large clinical sample. Drug and Alcohol Dependence.
Mediators and moderators do not quite get us to mechanisms but they are a good start. (this is beginning to be explicitly recognised in the discussion sections of some of the papers cited) Emerging clues: Changes in social network Self-efficacy (social for men, negative affect for women) Spiritual practices, reduction in negative affect for the most impaired (apparently less important for the less impaired) The mechanisms in realist terms are the way in which an individual uses the above to change their reasoning (including emotional reasons) and make use of resources. The contexts are the contingencies which inhibit or facilitate the activation of mechanisms (e.g. stigma > reduced occupational opportunities > reduced confidence and sense of purpose)
Our results suggest that the process of recruiting young drug users into a study and focusing their attention on substances before a brief intervention may be, in itself, sufficient to induce positive reactive effects on behaviour. (p.1024)
Cognitive/experiential 1.Consciousness raising: Gaining information that increases awareness about the current behavior pattern or the potential new behavior 2.Emotional arousal: Experiencing emotional reactions about the status quo and/or the new behavior 3.Self-reevaluation: Seeing and evaluating how the status quo or the new behavior fits in with or conflicts with personal values 4.Environmental reevaluation: Recognizing the positive and negative effects the status quo or new behavior have upon others and the environment 5.Social liberation: Noticing and increasing social alternatives and norms that help support the status quo and/or change and initiation of the new behavior Behavioral 1.Self-liberation: Making choices, taking responsibility for, and making commitments to engaging in a new behavior or a behavior change 2.Stimulus generalization or control: Creating, altering, or avoiding cues/stimuli that trigger or encourage a particular behavior 3.Conditioning or counterconditioning: Making new connections between cues and a behavior or substituting new, competing behaviors and activities in response to cues for the "old" behaviors 4.Rcinforcement management: Identi fying and manipulating the positive and negative reinforcers for current or new behaviors. Creating rewards for new behaviors while extinguishi ng (eliminating reinforcements) for current behavior. 5.Helping relationships: Seeking and receiving support from others (family, friends, peers) for current or new behaviors