Presentation on theme: "Unpacking the notion of program theory from a complexity lense: Can indigenous and cross-cultural perspectives help? Sanjeev Sridharan and Janet Smylie."— Presentation transcript:
Unpacking the notion of program theory from a complexity lense: Can indigenous and cross-cultural perspectives help? Sanjeev Sridharan and Janet Smylie Centre for Research on Inner City Health University of Toronto Benita van Wyk, Feedback Research & Analytics
Outline 1. Why is program theory important and useful? 2. What are the limitations of the standard approach to program theory? The view from evaluation 3. Ideas for a potential research project
Conceptualizing and Critiquing Programs: Program Planning and Evaluation Evaluation Literature Indigenous/ Cross-Cultural Literature Identify concepts to enhance program planning and evaluation Implement Concepts in Two Programs Evaluate Programs
Basis for dialogue Sustainability Connectivity Dynamics Heterogeneity
The important questions What is a program? Does the nature of the causal assumptions make a difference in the way we plan programs and evaluations? Dealing with complexity
An Example of a complex intervenion Dealing with complexity
An Example: Primary Prevention Have a Heart Paisley
The nature of the causal problem Note that the causal assumptions are not clearly specified Most assumptions are implied and of the form: If (A and B and C and D and E and at time T) then good outcomes might result The problem: – Causal contingencies not clearly specified or understood – Dynamic complexity not well understood
The incompleteness of program theories The logic model while complex is still incomplete A policy such as the smoking ban might lead to proximal impacts through perhaps an uncomplicated pathway; however the logic of the causal chain for intermediate and long-term outcomes might be incomplete Additional “programmatic” inputs might be needed. Information on the nature of such additional interventions can perhaps come from a synthesis of literature plus additional dialogue with stakeholders with an intimate knowledge of the processes by which programmes works.
Defining a complex intervention (MRC, 2000) “The greater the difficulty in defining precisely what exactly are the ‘active ingredients’ of an intervention and how they relate to each other, the greater the likelihood that you are dealing with a complex intervention.”
System Dynamic Approaches (Sterman, 2006) Constantly changing; Governed by feedback; Non-linear, History-dependent; Adaptive and evolving; Characterized by trade-offs; Policy resistance: “The result is policy resistance, the tendency for interventions to be defeated by the system’s response to the intervention itself.”
Features of complex health interventions (Pawson et al., 2004) The intervention is a theory or theories The intervention involves the actions of people. The intervention consists of a chain of steps These chains of steps or processes are often not linear, and involve negotiation and feedback at each stage. Interventions are embedded in social systems and how they work is shaped by this context. Interventions are prone to modification as they are implemented. Interventions are open systems and change through learning as stakeholders come to understand them.
An additional purpose of evaluation for complex interventions Explication Why? Multiple components– be explicit about the “active ingredients” Timeline of impact unclear Non-uniformity– Contextualizing Stakeholder involvement: Co-construction of intervention
On causality and complexity Does a cause operate the same way in simple systems as compared to complex systems? Are there different kinds of causes? E.g. Famines Precipitating causes Amplifying causes Causes of vulnerability Innovations in evaluation design and approaches
So what do programmes do? A need for a richer vocabulary on programmes as causes– can we learn from other sciences such as evolution or chemical kinetics? Different metaphors: Programmes as catalysts, programmes as foundations, programmes as operating systems Understanding complexity in multiple light – The system-as-cause – Understanding dynamic complexity
System-as-Cause Forrester JW. Counterintuitive behavior of social systems. Technology Review 1971;73(3):53-68. Meadows DH. Leverage points: places to intervene in a system. Sustainability Institute, 1999. Available at. Richardson GP. Feedback thought in social science and systems theory. Philadelphia, PA: University of Pennsylvania Press, 1991. Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.
“Solutions” Can Also Create New Problems Meadows DH, Richardson J, Bruckmann G. Groping in the dark: the first decade of global modelling. New York, NY: Wiley, 1982. Merton RK. The unanticipated consequences of purposive social action. American Sociological Review 1936;1936:894-904. Forrester JW. Counterintuitive behavior of social systems. Technology Review 1971;73(3):53-68. Policy resistance is the tendency for interventions to be delayed, diluted, or defeated by the response of the system to the intervention itself. -- Meadows, Richardson, Bruckman
Unpacking the causal contingencies of change If (A AND B AND C AND D AND E and at time T) then good outcomes might result What does this really mean? Unpacking the complexity of the change process What is missing in our understanding of how programs work?
Seeing and understanding connection “ You think you understand two because you understand one and one. You think you understand two because you understand one and one. But you must also understand ‘and’.” You think you understand two because you understand one and one. But you must also understand ‘and’.” -- Sufi Saying
1. The role of sustainability Why sustaining programs might be key to success Going further – sustainability of programs at the individual level – the critical importance of Continuity of Care
2. The limited understanding of heterogeneity Heterogeneity of needs Heterogeneity of mechanisms of change Heterogeneity of interventions
3. The role of connectivity between interventions Far too much focus of most evaluations on a single intervention as the unit of analysis Understanding connectivity between programs
Better understanding of ecology of interventions
4. Understanding the relationship between multiple levels of interventions Many complex interventions require programming at the community, neighborhood, school and individual level Multilevel alignment across the interventions
7. Structural complexity Policy resistance Translating programs to fit within the structures The system-as-cause
Next Steps Identify partners Identify concepts from Evaluation and Indigenous/Cross-Cultural Field/Literature that can help enhance the concepts Translate concepts into practical testable experiments
Other concepts and ideas? firstname.lastname@example.org