1. The good, the bad, and the ugly of multiple data sources Barbara J. McMorris, PhD ONRS Seminar May 19, 2011 UMN School of Nursing

2. Types of Triangulation Types of Triangulation MethodologicalAnalysisInterdisciplinaryInvestigatorData

3. Why is Data Triangulation Important? Better research study & results Reviewers getting savvier; asking for multiple sources of data More “tools” for your research tool box

4. Study 1: Administrative Claims Data + Self Reports

5. Rhinitis Study Goals Assess burden of allergic and non allergic rhinitis –What is the cost and utilization impact of treatment for rhinitis? Evaluate providers’ perceptions of rhinitis burden on patients and severity of symptoms

6. Rhinitis Survey Data Analysis % Agree/Strongly Agree

7. Rhinitis Claims Data Analysis Count of and Costs of Health Care Services Utilized during 1 Yr Follow-up Mean (SD) Mild Patients (N = 5,670) Moderate Patients (N = 22,216) Severe Patients (N = 7,217) Total (N=35,103) Number of Rhinitis-Related ER Visits 0.03 (0.75)0.05 (0.89)0.04 (0.69)0.05 (0.83) Number of Rhinitis-Related Office Visits 5.61 (10.1)8.12( 11.69)9.39 (12.85)7.98 (11.75) Number of Rhinitis-Related Specialist Office Visits 3.08 (8.50)5.65 (10.43)7.22 (11.79)5.56 (10.52) Total Rhinitis-Related Medical Cost $358 (725)$514 (919)$556 (952)$497 (900) Total Rhinitis-Related Pharmacy Costs $364 (417)$435 (474)$440 (478)$424 (467)

8. Reflection: “good-bad” Known sampling frame “Clean” processed data Timing of data collection different –Lag in health care claims being loaded –Physical report of severity Low response rate

9. Results used in marketing http://www.gsk.com/media/pressreleases/ 2007/2007_10_19_GSK1134.htm http://www.gsk.com/media/pressreleases/ 2007/2007_10_19_GSK1134.htm

10. Study 2: Self Reports + Provider Care Flow Sheets

11. RA Infusion Study Goals Describe patient-reported outcomes due to bDMARD infusions: –satisfaction –lost productivity –pain/discomfort Quantify the amount of time providers spend for initiating infusions (3 types) Describe provider satisfaction with infusions

12. Data Sources Health care claims database Provider survey Patient Survey Case Report Form Infusion Care Flow Sheet

13. RA Infusion Results Patient and Provider time associated with infusions Means & %’s Infusion 1 (N = 102) Infusion 2 (N = 74) Infusion 3 (N = 29) Total patient time away from work/home (min.) 172.6267.9446.9 % patients able to return to work45%17%29% % had friend/family member miss work 31%24%43% Total provider labor time (min.)86.7128.2237.9

14. Reflection: “good” Good example of triangulation Comprehensive picture of “burden” Small sample size

15. Results used in marketing http://www.orencia.com/rheumatoid- arthritis-medication/orencia-infusion.aspx http://www.orencia.com/rheumatoid- arthritis-medication/orencia-infusion.aspx

16. Study 3: Self Reports + Process Data + Official Records

17. Broadway HS Study Goals Provide a descriptive profile of pregnant and parenting teen girls Compare outcomes for Broadway and comparison girls:  progress toward graduation  delaying a repeat birth  keeping children fully immunized

18. Data Sources Girls’ self reports on pre/post surveys Case manager reports of time Official Records –Program –School district –County

19. Evaluation Results %

20. County Data MFIP and Other Assistance Broadway Students (N = 158) Comparison Group (N = 97) Ave counted months of MFIP9.710.76 Ave used months of MFIP25.826.9 2 or more children27% Used emergency assistance47%41% Case closed due to sanction limit9%13%

21. Reflection: kinda “ugly” Using survey created/revised by others Linking datasets – what is the link? Consent to access official records? What were the data collected for? –Who entered the data? Budget enough time & money for both data management & analysis?

22. Final Cautions about Data Triangulation Need well-defined goals (no “fishing”) Costly both in time and $$$ May not solve bias/error issues Replication is difficult Know your “official” records sources and their limitations Do you have “permission” to link data sets?

23. “triangulation state of mind”* “If you self consciously set out to collect and double-check findings using multiple sources and modes of evidence, the verification process will largely be built into the data-gathering process, and little more need be done than to report on one’s procedures.” *Miles & Huberman (1984: p.235)

24. Thank you! Useful citations: Begley, CM. (1996). Using triangulation in nursing research. Journal of Advanced Nursing 24:122-128. Thurmond, VA. (2001). The Point of Triangulation. Journal of Nursing Scholarship 22(3): 253-258.