We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byHerbert Baldwin
Modified over 2 years ago
Logic models and qualitative synthesis Ideas and issues
09/05/2015© The University of Sheffield 2 Why logic models? What are logic models? How do I construct a logic model? Qualitative synthesis Where does and does it not get you? Outline
09/05/2015© The University of Sheffield 3 Link practice to outcomes Explore complex relationships Examine processes and systems Include multiple sources of evidence Identify further research needs Healthcare research
Logic models/conceptual models/impact models A systematic and visual way to present relationships and outcomes Read from left to right following a chain of reasoning 09/05/2015© The University of Sheffield 4
Health promotion literature 09/05/2015© The University of Sheffield 5 Conveying connections between a programme activities and outcomes (the logic of the programme) Lando et al., 2006
09/05/2015© The University of Sheffield 6 Inputs Short- term outcomes Mid-term outcomes Long-term outcomes Activities Lando et al., 2006
Programme evaluation 09/05/2015© The University of Sheffield 7 Relationships among programme resources planned activities, and anticipated changes or results The Kellog Foundation, 2004
09/05/2015© The University of Sheffield 8 Resources /inputs ActivitiesOutputsOutcomes Impact Planned workIntended results W.K. Kellog Foundation, 2004
Public health 09/05/2015© The University of Sheffield 9 Explore ways in which intervention impacts on different levels of outcomes Joly et al., 2007
09/05/2015© The University of Sheffield 10 Inputs Outputs Long-term outcomes Short- term out- comes StrategiesInter- mediate outcomes Joly et al., 2007
Worked example We are examining interventions to improve the happiness of systematic reviewers We can find little published evidence reporting happiness outcomes We are wanting to model how an intervention might help reviewers 09/05/2015© The University of Sheffield 11
09/05/2015© The University of Sheffield 12 InterventionShort term outcomesMedium termLonger term impacts Social support Increased happiness Reduce adverse stress Improved mental health Prog- ramme of lunch time group walks Improved physical health Improved fitness Reduction of pain Improved diet Heart rate Biscuit intake Caffeine Headaches/ back/neck Cortisol Personal contact
Building on the method 09/05/2015© The University of Sheffield 13
NICE Centre for Public Health Excellence What interventions are effective in enhancing mental wellbeing in the workplace? 09/05/2015© The University of Sheffield 14
Previous systematic review (Graveling et al. 2008) lack of evidence for effectiveness of interventions BUT was this lack of evidence or lack of study designs meeting inclusion criteria? 09/05/2015© The University of Sheffield 15
09/05/2015© The University of Sheffield 16 2883 de-duplicated retrieved citations Full paper reviewed 299 Reject title/abstract 2584 Reject 297 Language 2, population 20, not relevant 43, Not peer reviewed 4, discussion/review 102, study design 101 Included papers 2 Are we missing something?
09/05/2015© The University of Sheffield 17 Expert reference group identified literature in addition to papers from systematic review Proposal to use logic modelling to explore relationships
09/05/2015© The University of Sheffield 18 Review paper45 Discussion paper35 Survey reporting associations31 Survey reporting prevalence30 Policy document/report29 Cohort study19 Book10 Meta analysis8 Cluster randomised controlled trial5 Case study4 Book chapter3 Qualitative study2 Randomised controlled trial1 Controlled before and after study1 Case control study1
09/05/2015© The University of Sheffield 19
Logic model creation Little detail regarding the method “was drafted by the authors” “was developed in conjunction with an expert panel” “A group with a facilitator identified activities, outcomes and resources needed, initial model circulated, then refined and clarified” A group reviewed existing models, identified principles of new model, the model was refined following feedback 09/05/2015© The University of Sheffield 20
A systematic way to build a model? 09/05/2015© The University of Sheffield 21
09/05/2015© The University of Sheffield 22 Logic model Primary qual data analysis Systematic review methods Qualitative synthesis
Method Data extraction Analysis 09/05/2015© The University of Sheffield 23 Synthesis
Method Extraction Description of elements impacting on wellbeing and associations between elements extracted on developed form Analysis 09/05/2015© The University of Sheffield 24 Synthesis
09/05/2015© The University of Sheffield 25 WARNING Reviewers of a nervous disposition look away now Treating all data equally Treating all data as qualitative
Method Extraction Analysis Line by line reading of extractions, coding of data, use of CAQDA software (Nvivo) for storage and retrieval 09/05/2015© The University of Sheffield 26 Synthesis
Qualitative synthesis Combining of primary qualitative data Emerging approaches (Meta-synthesis, thematic synthesis, meta-ethnography etc) Seeking and developing recurring patterns or themes across the data 09/05/2015© The University of Sheffield 27
Thematic synthesis (Thomas & Harden 2008), examine themes across set, and by process of comparison and contrast establish meta-themes (integrate and/or create) 09/05/2015© The University of Sheffield 28
For example Primary analysis – a paper describes outcomes following a buddy system intervention on employee wellbeing, elements of data coded as peer interaction. Another paper describes the link between working at home and reduced wellbeing, elements of data coded as social interaction. Secondary analysis – peer interaction and social interaction data combined into social support theme, later when other data added a category of social resources is created. 09/05/2015© The University of Sheffield 29
09/05/2015© The University of Sheffield 30 Health Well being Job satisfaction Work context Health and safety Management priorities/values Supervisor behaviour Feedback & appraisal Organisational climate Organisational justice Work-place support Employee participation Communication systems Industry type Individual factors Individual response to work Personality traits Self esteem Self efficacy Psychological flexibility Person-environment fit Values Social resources Individual responses to management style or working practices Optimal workload threshold Coping response Economic And social trends Fixed term contracts Flexible employment Health inequalities Cultural diversity Demographic changes New technology ICT Globalisation Change from production to service economy Individual characteristics Gender Age Social circumstances Education Ethnicity Marital status Predispositions National polices and practices Equal opportunities Anti-discrimination policies Family-friendly policies Maximum working hours legislation Behavioural outcomes Health-related behaviour Sickness absence Staff turnover Presenteeism Burnout Citizenship Attitudinal outcomes Commitment Motivation Engagement Employee expectations Perceptions of fair treatment Business outcomes Absence/turnover costs Performance Productivity Customer satisfaction Profitability Optimal staffing Satisfy statutory regulations Safety/avoidance of litigation Corporate social responsibility Corporate image Recruitment cost savings Attracting and retaining skilled workforce Competitiveness Insurance premiums/healthcare costs Work content Work demands Job control/decision latitude Effort required Rewards Role Working schedules Opportunity for learning/development Monotony Skills utilisation Worthwhile/fulfilling Stability/job future Occupational group Intervention pointsOutcomesWider influences
Where could logic modelling get you? May act as a common frame of reference, defining poorly developed concepts Used as a participatory and communication tool, user involvement? 09/05/2015© The University of Sheffield 31
Provides a systems perspective Increasing recognition of systems perspective in healthcare practice eg pt safety Recognition of complexity eg MRC Developing and evaluating complex interventions 09/05/2015© The University of Sheffield 32
Implementation Research eg Eccles et al. 2009, Cooksey Report Transfer of research findings into practice Complexity of context Fidelity of interventions 09/05/2015© The University of Sheffield 33
Used as a research development tool, consider issues that a research agenda needs to address Takes an inclusive approach to evidence, areas where little trial work Pilot work, Mapping review? 09/05/2015© The University of Sheffield 34
Links between advice services for patients, and health outcomes 09/05/2015© The University of Sheffield 35 Financial outcomes Health outcomes Advice
Where does it not get you? Effectiveness Issues with identification of material – selection bias Issues with hierarchy of evidence – representation of strength? Hierarchy of qual designs? Frequency? Shows where research has been done rather than evidence? 09/05/2015© The University of Sheffield 36
More issues Critical appraisal especially qualitative sources Issues with integration of diff types of data Removal of context Use of primary material in secondary synthesis Integration versus interpretation 09/05/2015© The University of Sheffield 37
Logic is the beginning of wisdom, not the end. Leonard Nimoy Logic will get you from A to B. Imagination will take you everywhere. Albert Einstein 09/05/2015© The University of Sheffield 38
09/05/2015© The University of Sheffield 39 Promise improvement, Exact retribution Check with Clients Logical Model for Creating Achievable and Sustainable Change Modalities over Time Put paper in printer Engage 1 st gear Realize that you look important if it’s really really confusing. Start thinking about creating a really comfusing chart. Realize that even you don’t understand any of this. Roll down hill Capture Vision Engage bullshit generator Report Outcomes Wax Eloquent The world is a better place because they actually paid you to do this. Analyze everything you can think of in depth Detailed cause and effect follows Build conceptual engine Engage graphics generator Realize that you look important if you’re the only one who understands it. Internalize Conceptualize Target audienceCreate messageEngage audience Realize/EngageActualize/EnergizePromote/EnhanceExpose/Repose Hire chimp Let Vision Go Re-capture Vision Jump start engine Achieve Goals Finalize Results Wax Car Fire chimp Put orange box here. Keep close watch on vision. Unknown author
References Baxter R, Killoran, A, Kelly MP, Goyder E. Synthesizing diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews. Public Health, 2010; 124: 99-106. Barbour R, Barbour M. Evaluating and synthesising qualitative research: the need to develop a distinctive approach. J Eval Clin Prac 2003;9:179–86. Bunton R, Macdonald G, editors. Health promotion: disciplines, diversity and developments. 2nd edn. London: Routledge; 2002 Cooksey D, A Review of UK health research funding. Norwich: HMSO; 2006. Eccles M, Armstrong D, Baker R et al. An Implementation Research Agenda. Implementation Science, 2009: 4: 18 doi: 10.1186/1748-590804018. Graveling R, Crawford J, Cowie H, Amati C, Vohra S. A review of workplace interventions that promote mental wellbeing in the workplace. Edinburgh: Institute of Occupational Medicine; 2008. Joly B, Polyak G, Davis M, Brewster J, Tremain B, Raevsky C, et al. Linking accreditation and public health outcomes: a logic model approach. J Public Health Manag Prac 2007;13:349–56. Kelly M, Stewart E, Morgan A, Killoran A, Fischer A, Threlfall A, Bonnefoy J. A conceptual framework for public health: NICE’s emerging approach. Public Health 2009;123:14–20. Kellog. Foundation, The logic model development guide. Michigan: Kellog Foundation; 2004. Lando J, Williams S, Williams B, Sturgis S. A logic model for the integration of mental health into chronic disease prevention and health promotion. Prev Chron Disease Pub Health Res Prac t Policy 2006: 3; 1-4. Schmitz C. Everything you wanted to know about logic models but were afraid to ask. Michigan: Kellog Foundation; 1999. 09/05/2015© The University of Sheffield 40
09/05/2015© The University of Sheffield 41 Tannihill A. Beyond evidence to ethics: a decision-making framework for health promotion, public health and health improvement. Health Promot Int 2008;23:380–90. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth 2008;8:45. Tones K, Green J. Health promotion: planning and strategies. London: Sage; 2004. Further reference lists/resources: http://www.cdc.gov/eval/resources.htm#logic model
Strategic Human Resource Management – Current Developments and the HE Context David Guest Professor of Organizational Psychology & Human Resource Management.
Helping people with mental health problems gain and retain employment – what works? Dr Bob Grove Director, Employment Programme.
PEOPLE STRATEGY People Strategy Developing our People Strategy 27th January 2015.
EHFG 7 October 2005 – Patient Safety & Research UK Presidency Workshop on Patient Safety, EHFG, Bad Hofgastein,7 September 2005 Kevin McCarthy, European.
Implementing NICE guidance
Focusing the question Janet Harris
Assessing student learning from Public Engagement David Owen National Co-ordinating Centre for Public Engagement Funded by the UK Funding Councils, Research.
Implementation and process evaluation: developing our approach Ann Lendrum University of Manchester Neil Humphrey University of Manchester Gemma Moss Institute.
Commissioning for Wellbeing Time banking and other initiatives in Plymouth Rachel Silcock.
Health and Safety Executive Health and Work Please feel free to use the information provided in this presentation in your own presentations. Where appropriate,
Integrated Framework for Reducing Racial and Ethnic Disparities in the Quality of Health Care Marshall H. Chin, MD, MPH, and Don Goldmann, MD University.
Mixed methods synthesis ESRC Methods Festival 2006 James Thomas Institute of Education, University of London.
Health Promotion as a Quality issue
A Sustainable Wales Better Choices for a Better Future Sustainable Development White Paper A Sustainable Wales - Better Choices for a Better Future.
Outcome mapping in child rights-based programming
Creating a service Idea. Creating a service Networking / consultation Identify the need Find funding Create a project plan Business Plan.
School of Population Health University of Melbourne Global systematic review initiatives: moving forward in partnership Elizabeth Waters.
© 2005 Prentice-Hall, Inc. 9-1 Chapter 9 Organizational Commitment, Organizational Justice, and Work- Family Interface.
WIN-WIN COMBINATION IN WORK PLACE BY:- DEBENDRA HANSDAH SENIOR MANAGER (MECH.)-SER.
Engage 2 Perform Mark Horton MBA How managers should engage with their staff.
© 2017 SlidePlayer.com Inc. All rights reserved.