1. Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua Li 1, David Hanauer 3 1 Department of Biomedical Informatics, Columbia University, New York, NY 2 School of Nursing, Columbia University, New York, NY 3 Department of Pediatrics, University of Michigan, Ann Arbor, MI

2. Big Health Data 2

3. Data Access for Life Scientists 3 Data Patient Privacy Data Governance Knowledge Gaps

4. Example Data Request Forms 4 Form TitleInstitution DISCOVERY Committee Data/Report RequestColumbia University PRIMARY CARE INFORMATION PROJECT DATA REQUEST FORMNYC-Department of Health DATA Request Statement of WorkNYC-Department of Health Internal Hub Data Request FormNYC-Department of Health Enterprise Data WarehouseNorthwestern University Honest Broker OfficeMichigan University CTRI Service Request FormUCSD UCSF MEDICAL CENTER CANCER REGISTRYUCSF Clinical Data Access Request Online FormBoston University Data Request FormKansas Univ. Medical Center Enterprise Data WarehouseVanderbuilt Metting Request with ICTR BMIUniversity of Wisconsin UCH RSS New Procotol ApplicationUniversity of Colorado ……others

5. Data Access for Life Scientists…Hard! 5 Vocabulary Differences Conceptualization Discrepancies Tacit Knowledge Gaps

6. Related Work Hruby GW, Cimino JJ, Patel V, Weng C, Toward a Cognitive Task Analysis for Biomedical Query Mediation, Proc of 2014 AMIA Joint Summits on Translational Science, 7-11 April 2014, San Francisco, CA, in press. Hruby GW, Boland MR, Cimino JJ, Gao J, Wilcox AB, Hirschberg J, Weng C, Characterization of the Biomedical Query Mediation Process, Proc of 2013 AMIA Joint Summits on Translational Science, 18-22 March 2013, San Francisco, CA, 89-93. 6

7. How do researchers and query analysts negotiate data needs via email? 7

8. Methods Participants: query analysts from CUMC – A: > 10 years of experience, no MD – B: novice analyst with a non-US MD – C: 2 years of experience, no MD Eight user-reported critical incidents, including 150 email messages Message de-identification Thematic analysis of dialog acts 8

9. Results 9

10. Themes in Dialogue Acts 10 Row LabelsM->QM->RQ->MQ->RR->MR->QR->R Total Need Clarification1 15333 52 Data transfer 17 Set up meetings 1 825 16 Follow-up1 117 10 Confirmation 5 218 Initial contact 2 6 8 Progress Reporting 21 115 Request problem specification11 12 5 Middle-point data verification 4 4 Request sample patient list 3 3 Revise query2 1 3 Send a starting patient list 3 3 Guide data interpretation 2 2 Request help with data interpretation 2 2 Request help with query formulation 1 12 Results Transfering 2 2 Data reformatting request 1 1 Guide data need clarification 1 1 Help with data interpretation 1 1 Progress Inquiry 1 1 Request help with data interpretation 1 1 Send a starting patient list; Setting up meetings 1 1 Send related information 1 1 Suggest Query Revision 1 1 Grand Total522658 3150

11. Top Themes 11 Row LabelsQ->RR->Qother Total Need Clarification1533…52 Data transfer17 … Set up meetings85…16 Follow-up17…10 Confirmation52…8 Initial contact26…8 Q: Query Analyst; R: Researcher; M: Manager

12. Example Communication Patterns 12

13. Aggregated Activity Transition Graph 13

14. Need Clarification Example 1 14 Hi Alice, Thank you very much for your help. I just want to make sure I get it right. About the code 531.9 from your list - there are fifth digits with 531.9 code (0 and 1) and non of them match your description in the list. Please look at the icd9 codes below and let me know the right ones. 531.90 GASTRIC ULCER 531.30 ACUTE GASTRIC ULCER 531.00 ACUTE GASTRIC ULCER WITH HEMORRHAGE, WITHOUT MENTION OF OBSTRUCTION 531.10 ACUTE GASTRIC ULCER WITH PERFORATION, WITHOUT MENTION OF OBSTRUCTION 531.20 ACUTE GASTRIC ULCER WITH HEMORRHAGE AND PERFORATION 531.70 CHRONIC GASTRIC ULCER 531.40 CHRONIC GASTRIC ULCER WITH HEMORRHAGE 531.50 CHRONIC GASTRIC ULCER WITH PERFORATION 531.60 CHRONIC GASTRIC ULCER WITH HEMORRHAGE AND PERFORATION 531.91 GASTRIC ULCER WITH OBSTRUCTION 531.31 ACUTE GASTRIC ULCER WITH OBSTRUCTION 531.01 AC STOMAC ULC W HEM-OBST 531.11 AC STOM ULC W PERF-OBST 531.21 AC STOM ULC HEM/PERF-OBS 531.71 CHRONIC GASTRIC ULCER WITH OBSTRUCTION 531.41 CHRONIC OR UNSPECIFIED GASTRIC ULCER WITH HEMORRHAGE, WITH OBSTRUCTION 531.51 CHRONIC OR UNSPECIFIED GASTRIC ULCER WITH PERFORATION, WITH OBSTRUCTION 531.61 CHRONIC GASTRIC ULCER WITH HEMORRHAGE, PERFORATION, AND OBSTRUCTION Thank you again. A Hi Alice, Thank you very much for your help. I just want to make sure I get it right. About the code 531.9 from your list - there are fifth digits with 531.9 code (0 and 1) and non of them match your description in the list. Please look at the icd9 codes below and let me know the right ones. 531.90 GASTRIC ULCER 531.30 ACUTE GASTRIC ULCER 531.00 ACUTE GASTRIC ULCER WITH HEMORRHAGE, WITHOUT MENTION OF OBSTRUCTION 531.10 ACUTE GASTRIC ULCER WITH PERFORATION, WITHOUT MENTION OF OBSTRUCTION 531.20 ACUTE GASTRIC ULCER WITH HEMORRHAGE AND PERFORATION 531.70 CHRONIC GASTRIC ULCER 531.40 CHRONIC GASTRIC ULCER WITH HEMORRHAGE 531.50 CHRONIC GASTRIC ULCER WITH PERFORATION 531.60 CHRONIC GASTRIC ULCER WITH HEMORRHAGE AND PERFORATION 531.91 GASTRIC ULCER WITH OBSTRUCTION 531.31 ACUTE GASTRIC ULCER WITH OBSTRUCTION 531.01 AC STOMAC ULC W HEM-OBST 531.11 AC STOM ULC W PERF-OBST 531.21 AC STOM ULC HEM/PERF-OBS 531.71 CHRONIC GASTRIC ULCER WITH OBSTRUCTION 531.41 CHRONIC OR UNSPECIFIED GASTRIC ULCER WITH HEMORRHAGE, WITH OBSTRUCTION 531.51 CHRONIC OR UNSPECIFIED GASTRIC ULCER WITH PERFORATION, WITH OBSTRUCTION 531.61 CHRONIC GASTRIC ULCER WITH HEMORRHAGE, PERFORATION, AND OBSTRUCTION Thank you again. A Hi A…………. ………………. ……………….. Hi A, These are relevant codes pasted below: ICD9 codes 041.86 Helicobacter pylori 151.0-151.9 Malignant neoplasm of stomach 209.23 Malignant Carcinoid Tumor of the Stomach 211.1 Benign neoplasm of stomach 230.2 Carcinoma in situ of stomach 235.2 Neoplasm of uncertain behavior of stomach, intestines, and rectum 531.9 Gastric ulcer unspecified 536.8 Unspecified functional disorder of stomach 537.9 Unspecified disorder of stomach and duodenum These are the CPT codes we need: 43620 43621 43622 43631 43632 43633 43634 43999 44238 49329 Hi A, These are relevant codes pasted below: ICD9 codes 041.86 Helicobacter pylori 151.0-151.9 Malignant neoplasm of stomach 209.23 Malignant Carcinoid Tumor of the Stomach 211.1 Benign neoplasm of stomach 230.2 Carcinoma in situ of stomach 235.2 Neoplasm of uncertain behavior of stomach, intestines, and rectum 531.9 Gastric ulcer unspecified 536.8 Unspecified functional disorder of stomach 537.9 Unspecified disorder of stomach and duodenum These are the CPT codes we need: 43620 43621 43622 43631 43632 43633 43634 43999 44238 49329

15. Need Clarification Example 2 15 Hi, >> Age 18 is the current age or diagnosis age? >> Do you care about which diagnosis is made first? Hi, >> Age 18 is the current age or diagnosis age? >> Do you care about which diagnosis is made first? Hi A…………. ………………. ……………….. I filed the discovery request today. I am requesting 2 lists of MRNs that meet the criteria below. LIST 1 Population: adults aged 18 years or older Date range: 1992-present Inclusion criteria: ICD 079.5, OR 079.53, OR 795.71, OR 042, OR 043, OR 044, AND ICD 155.0, OR 230 Data elements requested: Date of birth Age sex race/ethnicity I filed the discovery request today. I am requesting 2 lists of MRNs that meet the criteria below. LIST 1 Population: adults aged 18 years or older Date range: 1992-present Inclusion criteria: ICD 079.5, OR 079.53, OR 795.71, OR 042, OR 043, OR 044, AND ICD 155.0, OR 230 Data elements requested: Date of birth Age sex race/ethnicity

16. Need Clarification Example 3 16 I will double check. It is not 'primary', I will correct it. Then exclude V24.6 and <15. Then exclude ICU only patients, if I can find a way. By the way, I used 'age<18' according to your request doc. You prefer 15 instead? I will double check. It is not 'primary', I will correct it. Then exclude V24.6 and <15. Then exclude ICU only patients, if I can find a way. By the way, I used 'age<18' according to your request doc. You prefer 15 instead? Hi A…………. ………………. ……………….. Hi A, Thanks for the data. It does seem strange to me that the numbers increase so much in 2013. Can you confirm that the numbers are indeed correct? Also, can you tell me if the diagnosis is the "primary” diagnosis or not? We are most interested in those with a primary admitting diagnosis that is one of these variables. Please also confirm that those with concomitant V24.6 are excluded and those age <15 are excluded. Thanks, Researcher Hi A, Thanks for the data. It does seem strange to me that the numbers increase so much in 2013. Can you confirm that the numbers are indeed correct? Also, can you tell me if the diagnosis is the "primary” diagnosis or not? We are most interested in those with a primary admitting diagnosis that is one of these variables. Please also confirm that those with concomitant V24.6 are excluded and those age <15 are excluded. Thanks, Researcher

17. Iterative Need Clarification 17 Sample patients Data sample Data Requests …. ….. …. ….. Codes Concept Clarification Data Verification

18. Length of Query Processing Time 18 AnalystMD Experienc e CaseStartEndLength Avg ANo> 10 Y 110/8/1310/25/1317 days 26d 211/28/131/6/1440 days 312/28/131/17/1420 days BYes< 0.1 Y 410/28/1311/1/133 days 15d 510/22/1311/2/1310 days 610/1/1310/22/1321 days 71/6/141/23/1417 days CNo~ 2 Y 810/6/111/9/13>400 days >400d Medical knowledge may be associated with more efficient query processing

19. Discussion 19

20. Conclusions Decision support should not be just about user augmentation, but enable user engagement and shared decision making Besides alerts, reminders, information filters or summarizers, we also need better dialog- based decision support to bridge the gap between clinical researchers and Big Data 20

21. Towards Dialog-based Query Support

22. Limitations One institution Small sample Communication outside emails 22

23. Thank you NLM R01LM009886 (Bridging the semantic gap between clinical research eligibility criteria and clinical data, PI: Weng) 23 cw2384@columbia.edu http://people.dbmi.columbia.edu/~chw7007/