Presentation is loading. Please wait.

Presentation is loading. Please wait.

Logic models and qualitative synthesis Ideas and issues.

Similar presentations


Presentation on theme: "Logic models and qualitative synthesis Ideas and issues."— Presentation transcript:

1 Logic models and qualitative synthesis Ideas and issues

2 09/05/2015© The University of Sheffield 2 Why logic models? What are logic models? How do I construct a logic model? Qualitative synthesis Where does and does it not get you? Outline

3 09/05/2015© The University of Sheffield 3 Link practice to outcomes Explore complex relationships Examine processes and systems Include multiple sources of evidence Identify further research needs Healthcare research

4 Logic models/conceptual models/impact models A systematic and visual way to present relationships and outcomes Read from left to right following a chain of reasoning 09/05/2015© The University of Sheffield 4

5 Health promotion literature 09/05/2015© The University of Sheffield 5 Conveying connections between a programme activities and outcomes (the logic of the programme) Lando et al., 2006

6 09/05/2015© The University of Sheffield 6 Inputs Short- term outcomes Mid-term outcomes Long-term outcomes Activities Lando et al., 2006

7 Programme evaluation 09/05/2015© The University of Sheffield 7 Relationships among programme resources planned activities, and anticipated changes or results The Kellog Foundation, 2004

8 09/05/2015© The University of Sheffield 8 Resources /inputs ActivitiesOutputsOutcomes Impact Planned workIntended results W.K. Kellog Foundation, 2004

9 Public health 09/05/2015© The University of Sheffield 9 Explore ways in which intervention impacts on different levels of outcomes Joly et al., 2007

10 09/05/2015© The University of Sheffield 10 Inputs Outputs Long-term outcomes Short- term out- comes StrategiesInter- mediate outcomes Joly et al., 2007

11 Worked example We are examining interventions to improve the happiness of systematic reviewers We can find little published evidence reporting happiness outcomes We are wanting to model how an intervention might help reviewers 09/05/2015© The University of Sheffield 11

12 09/05/2015© The University of Sheffield 12 InterventionShort term outcomesMedium termLonger term impacts Social support Increased happiness Reduce adverse stress Improved mental health Prog- ramme of lunch time group walks Improved physical health Improved fitness Reduction of pain Improved diet Heart rate Biscuit intake Caffeine Headaches/ back/neck Cortisol Personal contact

13 Building on the method 09/05/2015© The University of Sheffield 13

14 NICE Centre for Public Health Excellence What interventions are effective in enhancing mental wellbeing in the workplace? 09/05/2015© The University of Sheffield 14

15 Previous systematic review (Graveling et al. 2008) lack of evidence for effectiveness of interventions BUT was this lack of evidence or lack of study designs meeting inclusion criteria? 09/05/2015© The University of Sheffield 15

16 09/05/2015© The University of Sheffield 16 2883 de-duplicated retrieved citations Full paper reviewed 299 Reject title/abstract 2584 Reject 297 Language 2, population 20, not relevant 43, Not peer reviewed 4, discussion/review 102, study design 101 Included papers 2 Are we missing something?

17 09/05/2015© The University of Sheffield 17 Expert reference group identified literature in addition to papers from systematic review Proposal to use logic modelling to explore relationships

18 09/05/2015© The University of Sheffield 18 Review paper45 Discussion paper35 Survey reporting associations31 Survey reporting prevalence30 Policy document/report29 Cohort study19 Book10 Meta analysis8 Cluster randomised controlled trial5 Case study4 Book chapter3 Qualitative study2 Randomised controlled trial1 Controlled before and after study1 Case control study1

19 09/05/2015© The University of Sheffield 19

20 Logic model creation Little detail regarding the method “was drafted by the authors” “was developed in conjunction with an expert panel” “A group with a facilitator identified activities, outcomes and resources needed, initial model circulated, then refined and clarified” A group reviewed existing models, identified principles of new model, the model was refined following feedback 09/05/2015© The University of Sheffield 20

21 A systematic way to build a model? 09/05/2015© The University of Sheffield 21

22 09/05/2015© The University of Sheffield 22 Logic model Primary qual data analysis Systematic review methods Qualitative synthesis

23 Method Data extraction Analysis 09/05/2015© The University of Sheffield 23 Synthesis

24 Method Extraction Description of elements impacting on wellbeing and associations between elements extracted on developed form Analysis 09/05/2015© The University of Sheffield 24 Synthesis

25 09/05/2015© The University of Sheffield 25 WARNING Reviewers of a nervous disposition look away now Treating all data equally Treating all data as qualitative

26 Method Extraction Analysis Line by line reading of extractions, coding of data, use of CAQDA software (Nvivo) for storage and retrieval 09/05/2015© The University of Sheffield 26 Synthesis

27 Qualitative synthesis Combining of primary qualitative data Emerging approaches (Meta-synthesis, thematic synthesis, meta-ethnography etc) Seeking and developing recurring patterns or themes across the data 09/05/2015© The University of Sheffield 27

28 Thematic synthesis (Thomas & Harden 2008), examine themes across set, and by process of comparison and contrast establish meta-themes (integrate and/or create) 09/05/2015© The University of Sheffield 28

29 For example Primary analysis – a paper describes outcomes following a buddy system intervention on employee wellbeing, elements of data coded as peer interaction. Another paper describes the link between working at home and reduced wellbeing, elements of data coded as social interaction. Secondary analysis – peer interaction and social interaction data combined into social support theme, later when other data added a category of social resources is created. 09/05/2015© The University of Sheffield 29

30 09/05/2015© The University of Sheffield 30 Health Well being Job satisfaction Work context Health and safety Management priorities/values Supervisor behaviour Feedback & appraisal Organisational climate Organisational justice Work-place support Employee participation Communication systems Industry type Individual factors Individual response to work Personality traits Self esteem Self efficacy Psychological flexibility Person-environment fit Values Social resources Individual responses to management style or working practices Optimal workload threshold Coping response Economic And social trends Fixed term contracts Flexible employment Health inequalities Cultural diversity Demographic changes New technology ICT Globalisation Change from production to service economy Individual characteristics Gender Age Social circumstances Education Ethnicity Marital status Predispositions National polices and practices Equal opportunities Anti-discrimination policies Family-friendly policies Maximum working hours legislation Behavioural outcomes Health-related behaviour Sickness absence Staff turnover Presenteeism Burnout Citizenship Attitudinal outcomes Commitment Motivation Engagement Employee expectations Perceptions of fair treatment Business outcomes Absence/turnover costs Performance Productivity Customer satisfaction Profitability Optimal staffing Satisfy statutory regulations Safety/avoidance of litigation Corporate social responsibility Corporate image Recruitment cost savings Attracting and retaining skilled workforce Competitiveness Insurance premiums/healthcare costs Work content Work demands Job control/decision latitude Effort required Rewards Role Working schedules Opportunity for learning/development Monotony Skills utilisation Worthwhile/fulfilling Stability/job future Occupational group Intervention pointsOutcomesWider influences

31 Where could logic modelling get you? May act as a common frame of reference, defining poorly developed concepts Used as a participatory and communication tool, user involvement? 09/05/2015© The University of Sheffield 31

32 Provides a systems perspective Increasing recognition of systems perspective in healthcare practice eg pt safety Recognition of complexity eg MRC Developing and evaluating complex interventions 09/05/2015© The University of Sheffield 32

33 Implementation Research eg Eccles et al. 2009, Cooksey Report Transfer of research findings into practice Complexity of context Fidelity of interventions 09/05/2015© The University of Sheffield 33

34 Used as a research development tool, consider issues that a research agenda needs to address Takes an inclusive approach to evidence, areas where little trial work Pilot work, Mapping review? 09/05/2015© The University of Sheffield 34

35 Links between advice services for patients, and health outcomes 09/05/2015© The University of Sheffield 35 Financial outcomes Health outcomes Advice

36 Where does it not get you? Effectiveness Issues with identification of material – selection bias Issues with hierarchy of evidence – representation of strength? Hierarchy of qual designs? Frequency? Shows where research has been done rather than evidence? 09/05/2015© The University of Sheffield 36

37 More issues Critical appraisal especially qualitative sources Issues with integration of diff types of data Removal of context Use of primary material in secondary synthesis Integration versus interpretation 09/05/2015© The University of Sheffield 37

38 Logic is the beginning of wisdom, not the end. Leonard Nimoy Logic will get you from A to B. Imagination will take you everywhere. Albert Einstein 09/05/2015© The University of Sheffield 38

39 09/05/2015© The University of Sheffield 39 Promise improvement, Exact retribution Check with Clients Logical Model for Creating Achievable and Sustainable Change Modalities over Time Put paper in printer Engage 1 st gear Realize that you look important if it’s really really confusing. Start thinking about creating a really comfusing chart. Realize that even you don’t understand any of this. Roll down hill Capture Vision Engage bullshit generator Report Outcomes Wax Eloquent The world is a better place because they actually paid you to do this. Analyze everything you can think of in depth Detailed cause and effect follows Build conceptual engine Engage graphics generator Realize that you look important if you’re the only one who understands it. Internalize Conceptualize Target audienceCreate messageEngage audience Realize/EngageActualize/EnergizePromote/EnhanceExpose/Repose Hire chimp Let Vision Go Re-capture Vision Jump start engine Achieve Goals Finalize Results Wax Car Fire chimp Put orange box here. Keep close watch on vision. Unknown author

40 References Baxter R, Killoran, A, Kelly MP, Goyder E. Synthesizing diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews. Public Health, 2010; 124: 99-106. Barbour R, Barbour M. Evaluating and synthesising qualitative research: the need to develop a distinctive approach. J Eval Clin Prac 2003;9:179–86. Bunton R, Macdonald G, editors. Health promotion: disciplines, diversity and developments. 2nd edn. London: Routledge; 2002 Cooksey D, A Review of UK health research funding. Norwich: HMSO; 2006. Eccles M, Armstrong D, Baker R et al. An Implementation Research Agenda. Implementation Science, 2009: 4: 18 doi: 10.1186/1748-590804018. Graveling R, Crawford J, Cowie H, Amati C, Vohra S. A review of workplace interventions that promote mental wellbeing in the workplace. Edinburgh: Institute of Occupational Medicine; 2008. Joly B, Polyak G, Davis M, Brewster J, Tremain B, Raevsky C, et al. Linking accreditation and public health outcomes: a logic model approach. J Public Health Manag Prac 2007;13:349–56. Kelly M, Stewart E, Morgan A, Killoran A, Fischer A, Threlfall A, Bonnefoy J. A conceptual framework for public health: NICE’s emerging approach. Public Health 2009;123:14–20. Kellog. Foundation, The logic model development guide. Michigan: Kellog Foundation; 2004. Lando J, Williams S, Williams B, Sturgis S. A logic model for the integration of mental health into chronic disease prevention and health promotion. Prev Chron Disease Pub Health Res Prac t Policy 2006: 3; 1-4. Schmitz C. Everything you wanted to know about logic models but were afraid to ask. Michigan: Kellog Foundation; 1999. 09/05/2015© The University of Sheffield 40

41 09/05/2015© The University of Sheffield 41 Tannihill A. Beyond evidence to ethics: a decision-making framework for health promotion, public health and health improvement. Health Promot Int 2008;23:380–90. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth 2008;8:45. Tones K, Green J. Health promotion: planning and strategies. London: Sage; 2004. Further reference lists/resources: http://www.cdc.gov/eval/resources.htm#logic model


Download ppt "Logic models and qualitative synthesis Ideas and issues."

Similar presentations


Ads by Google