Presentation on theme: "Mixed Methods Synthesis Research Kathleen Knafl, PhD, FAAN Frances Hill Fox Distinguished Professor University of North Carolina at Chapel Hill School."— Presentation transcript:
Mixed Methods Synthesis Research Kathleen Knafl, PhD, FAAN Frances Hill Fox Distinguished Professor University of North Carolina at Chapel Hill School of Nursing
Greetings from Our Research Team at University of North Carolina at Chapel Hill 2 National Institutes of Health 1R01NR012445-01A1
Overview of Presentation Systematic reviews as a form of research Designing a synthesis study Statement of research question; study aims Preliminary (scoping) work Methods Approach Search & sampling strategies Data extraction & appraisal Analysis & synthesis strategies Example of results from a mixed methods synthesis
Systematic Reviews as Research Synthesis research – summary of all pertinent studies on a specific question or topic Based on a sound research design (e.g., systematic review, meta-analysis, mixed-methods) Contributes to evidence-based practice & knowledge development Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (4 th ed.). Thousand Oaks, CA: Sage. Whittemore, R., et al. (2014). Methods for knowledge synthesis: An overview. Heart & Lung, 43, 453-461.
Examples from Family Synthesis Study Mixed Methods Synthesis of Research on Childhood Chronic Conditions and Family (1R01NR012445, 09/01/11 - 06/30/16) Map the relationships found among family system, family member, condition management, demographic, & healthcare system variables Describe the nature of interventions involving families of children with a chronic condition and their effects on child and family outcomes.
Preliminary Work: Assessing Feasibility & Breadth of Interest Is there a sufficient body of research for undertaking the proposed synthesis? What is the conceptual domain of interest? What will “count” as a relevant study for the synthesis? Are there limits on the conceptual domain of interest?
Scoping Study – feasibility Method Guided by definition family used in proposal Published 2000-2010 10 data bases searched Comparison of yields using different search terms (general vs. condition-specific search terms) Outcome Identified ≈ 900 English language research reports Evidence of a sufficient body of evidence to address research aims Further refinement of search strategies & targets Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8, 19-32. Levac, D., et al. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5:69.
Scoping Study – conceptual domain of interest Method Sub-sample of scoping study ≈300 reports of qualitative & quantitative research ≈35 reports of intervention studies Extraction of information using a structured template Qualitative/quantitative reports: Aims, sample, design, measures/themes Intervention reports: Also included intervention profile Outcome Topical map of content relevant to proposed aims Delimitation of initial conceptual boundaries of synthesis
Topical CategoryExample Areas of Inquiry Family system functioning Family relationships (e.g., satisfaction, attachment) Family structure (e.g., division of labor, routines) Family processes (e.g., communication, conflict) Affected child functioning Health status (e.g., general health, condition control) Wellbeing and functioning (e.g., quality of life) Family member functioning (parents, siblings) Wellbeing and functioning (e.g. quality of life) Performance of parenting role (e.g., parenting stress) Condition management & control Symptom management Regimen management (e.g., adherence, monitoring) I Initial Conceptualization of Study Variables (excerpt from larger table Included in proposal)
Reviewer Response to Addition of Scoping Study in Resubmission First submission “The proposed research questions and guiding framework contain a plethora or variable/constructs. Although a “scoping study was performed, it only addressed the number of reports that included family as a variable or topic” (reviewer #3) Resubmission “The initial search provides data that confirms the study data base is sufficiently large to move forward with the project”(reviewer #1). “Preliminary investigations have been utilized to design the study and provide key feasibility data” (reviewer #3).
Search Criteria & Strategies in FaSP Study What – Research on intersection of family life & childhood chronic conditions, excluding studies of end-of life & of children with a non-physical chronic condition or intellectual disability. Where – Published, English language reports When – Published between 1995 & “present”; updates through 2014 How – Electronic data bases, reference lists of retrieved reports Replicable – Adherence to PRISMA guidelines Moher, D., et al. (2009). Preferred reporting items for systematic reviews and meta- analyses: The PRISMA Statement. Journal of Clinical Epidemiology, 62, 1006-1012.
Screening Reports & Revising Search Criteria Screening Titles & abstracts (42,972 reviewed; 39,355 eliminated) Full report (3,617 reviewed; 2124 eliminated) Final sample – 1493 reports; budgeted for sample of 750 - PANIC!!! Revising search criteria Excluded reports published before 2000 Retained only RCT intervention studies Eliminated cerebral palsy & traumatic brain injury Further delimited what “counted” as family research – levels of relevance Knafl, K. et al. (2015). Delimiting Family in Syntheses of Research on Childhood Chronic Conditions and Family Life. Family Process,54, 173- 184.
Data Extraction & Appraisal Guided by purpose of synthesis Included: Key features of report & study methods (e.g. sample, conceptual underpinnings) Relevant content & findings for addressing research question Quality assessment
Formats for Extracting Research Results Construction of stand-alone results statements Identification & extraction of results related to specific variables, relationships, or effects Identification & extraction of results using an organizing structure guided by study question Identification & extraction of results using a theoretically driven organizing framework
Extraction - Stand-alone Results Statements (FaSP Study) In families of adolescents with type 1 diabetes, mothers’ & fathers’ depressive mood & anxiety (after controlling dependencies between mothers’ & fathers’ data) were significant predictors of fathers’ perceptions of adolescent efficacy & mothers’ perceptions of adolescent efficacy. In families of adolescents with type 1 diabetes, HbA1c was significantly & negatively associated with maternal & paternal perceptions of adolescent efficacy.
Extraction - Results about Selected Variables Vermaes, I., et al. (2012). Psychological functioning of siblings in families of children with chronic health conditions: A meta-analysis. Journal of Pediatric Psychology, 37(2), 166-184. Extraction & categorization of results that were indicators of internalizing behavior problems, externalizing behavior problems, & positive self-attributes based on study measures
Extraction – Results Related to Study Questions Swallow, V., et al., (2011). Fathers' contributions to the management of their child's long-term medical condition; A narrative review of the literature. Health Expectations, 15, 157-175. Results related to each of the following questions extracted: What is the impact of the condition on fathers’ ability to promote child wellbeing? What influences fathers’ involvement in management? What evidence is there of personal growth/benefit for the father? What is the impact of fathers’ involvement on family functioning?
Extraction - Results Related to Conceptual Framework Report ID Child IdentityIllness ViewManagement Mindset Parental Mutuality Results related to current definition of theme: Support: Delete: Refine: Results related to current definition of theme: Support: Delete: Refine: Results related to current definition of theme: Support: Delete: Refine Results related to current definition of theme: Support: Delete: Refine: Knafl, K., Deatrick, J., & Havill, N. (2012). Continued development of the Family Management Style Framework. Journal of Family Nursing, 18, 11-34.
Quality Appraisal – Qualitative Studies Overall appraisal: Indicate only those study features undermining credibility of findings. Findings are demonstrably plausible &/or sufficiently substantiated with data. Sample size & configuration are sufficient to support the findings. Features of the sample critical to the understanding of findings are described. Variations in findings by relevant sample & event characteristics are addressed. Sandelowski, M., Barroso, J., & Voils, C. (2007). Using qualitative metasummary to synthesize qualitative and quantitative descriptive studies. Research in Nursing & Health, 30, 99-111.
Quality Appraisal – Quantitative Overall appraisal: Indicate only those study features undermining credibility of findings. Internal validity threats Confounding: differences between comparison groups exist that could bias results (or differences not assessed) Reporting bias: selective reporting of results Missing data: significant amounts of missing data External validity threats Low participation rate Participants not representative of target population Von Elm, E. et al., (2008). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Journal of Clinical Epidemiology, 61, 344-349.
Example Outcome of Synthesis Aims Assess the strength of the relationship between family & parent functioning & child wellbeing Identify aspects of parents’ experiences that may enhance or impede their capacity to parent a child with arthrtis. Mixed-methods Approach Includes reports of qualitative, quantitative, & mixed methods research in sample Uses qualitative & quantitative analytic approaches
Mixed-methods Approach Question addressed with quantitative data What is the strength of the relationship between children’s physical & psychosocial wellbeing & parent & family system functioning? Question addressed with qualitative data What are aspects of parenting experience that may enhance or impede parenting capacity? Intent of mixed-method analysis was to identify key family variables related to child adaptation & use data about parenting experiences to provide explanatory insights about these relationships.
Data Source Sample criteria for searching the literature English language research reports published between 1/1/00 to 3/31/14 addressing families of children (<19 y/o) with arthritis Subsample of reports from larger synthesis project
Arthritis PRISMA Flow Chart Records identified through searching (n=1935) Additional records identified through ancestry approach (n=0) Records after duplicates removed (n=1451) Records screened (n=1451) Full-text articles assessed for eligibility (n=91) Studies included in dataset (n=35) Records excluded (n=1360) Full-text articles excluded, with reasons (n=56) Low-relevance family study (n=41) Not chronic physical condition (n=1) Not minor child (n=2) Not empirical research (n=1) Other (n=11) Qualitative studies included in sample (n=7) Quantitative studies included in sample (n=28)
Extracting Relevant Results Stand alone statements In children with JRA, higher levels of parental depression & anxiety were significantly related to lower child psychosocial functioning (Quantitative result - Effect size calculated) In families of children with arthritis, mothers & fathers reported that becoming a caregiver was entirely different from just being a parent because it included vigilant monitoring, responsibility for difficult daily treatments, & dealing with professionals in health, social service, & education. (Qualitative result)
Sample 35 reports from 24 studies 29 reports from 19 studies contributed to current analysis 22 reports (14 studies) quantitative 7 reports (5 studies) qualitative 3 studies longitudinal Reports from 21 journals (psychology, medicine, nursing, child development, family science, occupational therapy) Three reports judged to be low quality
Sample Parents Total across studies: N=1209 Range of parent participants across studies: 0 -180 Predominantly mothers; 8 studies reported including fathers; 7 studies did not specify which parent Children Total number of children across studies - 1143 Age: Mean age across studies 8.7 to 14.5 y/o 50% or more female in all but 3 studies *Numbers based on 29 reports used in present analysis
Coding to Access Relevant Data PARENT & CHILD CODES 1. Child Wellbeing 2. Parent Experience 3. Parent Role Performance 4. Parent View of Child Identity 5. Parent Wellbeing 6. Sibling Wellbeing CONDITION & DEMOGRAPHIC CODES 1.Condition Characteristics 2.Condition Management 3.Condition Perceptions 4.Demographics 5.Diagnosis FAMILY - SYSTEM CODES 1.Family Functioning 2.Family Relationships 3.Family Resources 4.Family Structure
Codes Used in this Analysis Perceptions of Parenting Experience Parenting experience Relationship Family Factors & Child Wellbeing Family function Family relations Family resources Family structure Child wellbeing Parent role performance Parent wellbeing
Accessing Relevant Data Query Report (Atlas.ti Software) - 29 quotation(s) (Parent Wellbeing & "Child Wellbeing - Psychological“ P103: AndrewsChaney2009ext.doc In parents of children with juvenile arthritis there was no significant main or interaction effect between child rated depression, parent report of child functioning, duration of condition, race, gender, child’ age, race and distress of parents. P104: AnthonyBromberg2011ext.doc For parents of children with mild to moderate juvenile arthritis, their depressive symptom ratings were not significantly correlated with child depressive symptoms, child anxiety, or pain. P111: cavallofeldman2009ext.doc In families with children with arthritis, child age- and sex-adjusted results indicate that higher parent-reported child psychosocial difficulties (but not difficulties related to gross or fine motor or somatic symptom domains of quality of life) were significantly associated with greater parent-reported parent psychological distress.
Accessing Relevant Data Query Report - 41 Quotations ("33 Parent Experience" & NOT "32 Child Wellbeing) P107: brittonmoore2002aext.doc Families of children with arthritis felt that many professionals failed to understand the families’ caregiving experience and often offered little meaningful help to families in great need of affirming support, encouragement, and practical information. P120: jonesrodger2009ext.doc Many parents of children with JIA felt that their home life revolved around managing the arthritis; in order to lead a ‘normal’ life outside the home, the parents needed to plan and organize to enable things to run smoothly and for the arthritis- related treatments to be confined to the home environment. P123: mcneill2004ext.doc For fathers of children with, when there was a sudden onset requiring hospitalization and if there was uncertainty about the diagnosis, fathers were very upset, often in a state of crisis; if the onset was more gradual, did not require hospitalization, and there was no fear for the child’s survival, the emotional impact of diagnosis was less intense.
Pooling Results – Linking Family & Child Variables through Meta-analysis Review all results/correlations about a relationship between parent/family variable & child wellbeing variable Determine what to aggregate Results about conceptually similar variables (e.g. parental depression & child psychosocial functioning) One result per variable per study – statistical requirement (e.g. if depression measured in both parents can only use one result in pooling) Ellis, P. (2010). Essential Guide to Effect Sizes: Statistical Power, Meta Analysis, and the Interpretation of Research Results. Cambridge University Press Cambridge, UK.
Results of Analysis of Quantitative Data 155 results reporting a relationship between a family factor & child wellbeing 65% of these addressed a relationship between parental characteristics & child wellbeing; remainder addressed family characteristics or parent-child relationship Meta-analysis identified three significant pooled correlations
Strength of Relationship between Family Factors & Child Wellbeing Seven reports addressed the relationship of parental depression/anxiety to child’s psychological status; pooled correlation.35 (95% CI.26 to.43, p <.001) Four reports addressed relationship of family conflict to child’s psychological functioning; pooled correlation -.29 (95% CI -.40 to -.18, p<.001) Three reports addressed the relationship between parental depression or anxiety and the child’s physical functioning, yielding a significant medium pooled correlation of.23 (95% CI.09 to.35, p<.001) Correlation effect size – small ≤.10; medium =.25; large ≥.40
Working with the Coded data – Parent Experience Code DocParent-Child Relations Impact on Parent Impact on Family 1201. Because of the demands of caregiving parents had an increased level of involvement in their child’s life. 2. Parent believed that in some ways the relationship to their child was closer, but in other ways the usual parent-child bond was lost. Despite the profound emotional experience JRA was for fathers they were reluctant to reveal their feelings to other family members. The time spent carrying out the treatment regimen excluded other children and limited time for other activities
Themes Characterizing the Parenting Experience – Parent-child relationship Caregiving activities are the focus of time with child and viewed as distinct from parenting (6 reports) Inability to control child’s pain and caregiving activities that inflict pain are especially difficult aspects of parenting experience (5 reports) Relationship with child is viewed as unlike usual parent-child relationships (4 reports)
Themes Characterizing the Parenting Experience – Impact on parents Having a child with JIA and its associated caregiving demands is a highly emotional, distressful experience for parents, which can interfere with other aspects of their lives (4 reports) JIA is an unpredictable, poorly understood condition (4 reports)
Themes Characterizing the Parenting Experience – Impact on family Condition management limits time for other activities and results in altered family roles and relationships (6 reports) Management of JIA is the focus of family life and impacts the entire family system (5 reports)
Discussion Reflections on the current state of research on children with arthritis & their families Most (75%) reports included data gathered from parents & the child Focus on relationship between characteristics of parents & children; fewer studies addressing family system variables Evidence that interventions to support parents & families will improve child wellbeing Evidence for targeting parents, the couple, &/or family system when developing interventions Need to address parent & family strengths as well as deficits
Discussion Advantages of mixed methods synthesis Identified key parent/family variables & provided insights into their contribution to child wellbeing 29/35 reports in sample contributed to analysis Synthesis limited by diversity of family & child variables across studies (23 measures of family/parent characteristics; 32 measures of child wellbeing)
“Like the artisans who construct a building from blueprints, bricks, and mortar, scientists contribute to a common edifice called knowledge. Theorists provide our blueprints and researchers collect the data that are our bricks. Research synthesists are the bricklayers and hodcarriers. It is their job to stack the bricks according to plan and apply the mortar that makes the whole thing stick” (Cooper, et al., 2009).