1. Patients’ Access to Medical Records: Experience with PatCIS James J. Cimino Division of General Internal Medicine & Department of Medical Informatics

2. Overview Motivation Architecture Functionality Recruitment Evaluation Techniques Results Discussion

3. Motivation Understanding on-line health information Access to personal health records Regulatory requirements are coming Commercial sites give patients access to their data What will happen to the patient? What will happen to the patient-provider relationship?

4. The Patient Clinical Information System (PatCIS) New York Presbyterian Hospital clinical data repository Web-based Clinical Information System (WebCIS) National Information Infrastructure contract from NLM: –give patients WebCIS –see what happens Pilot study conducted

5. Data Entry Review Advice Education Comments Help Logout Vital SignsBlood Sugar Data Entry patcis.cgi Web Server Web Browser Session Registry Usage Log Internet 2 3 6 PatCIS Architecture 1 CGI 4 5

6. Functionality Sign on Data entry Data review Education Advice Infobuttons Customization

33. Recruitment Mail physician consent forms to physicians Wait for physicians to suggest subjects Mail URL for consent form to subjects On-line enrollment Patient prints, signs and mails consent form Physician provides function-specific consent Mail user name, password and SecurID card to patients

34. Evaluation Issues Theories and methods from –cognitive and social sciences –usability engineering –studies of human-computer interaction Complexity and diversity of users leads to need for multi-method approach to evaluation –formative - feedback into design –summative - evaluate final product

35. Evaluation Challenges Varied users who interact from various locations –Less able to conduct controlled studies Current state-of-the-art –track user actions (e.g. clicks) - tells what they do, but not why –on-line questionnaires/feedback forms - often not filled in, limited questions –interviews - users often do not know what they do

36. Evaluation of Patient Systems What type of information do patients want? What is the effect on doctor-patient interaction? How does system affect patient understanding and communication with their providers? –understanding their own condition –decision making and management of their condition How to collect useful data from limited number of subjects?

37. Patient Variability Differences in prior knowledge Health differences Cognitive differences Cultural diversity Motivational differences Varied educational background

38. PatCIS Evaluation Questions What features of PatCIS are most used by patients? What features are least used and why? Are there usability issues that need to be resolved? How does use of PatCIS affect the doctor-patient interaction? Can patients comprehend information presented by PatCIS? Does use of PatCIS affect decision making and disease management?

39. 1. Video Based Usability Testing - in laboratory 2. Telephone Interviews (audiotape) Patient Interact via WWW 3. E-mail (to evaluators) Patient Information System (PatCIS) 4. Tracking User Actions - System Usage Database (log files) 5. On-line Questionnaire Data (sequenced forms) “Televaluation” of PatCIS (Kushniruk, Patel, Patel, Cimino, 2001)

40. Log File Analysis sandcar!Fri Oct 27 11:32:22 2000!cim.cpmc.columbia.edu!|patcis^login sandcar!Fri Oct 27 11:32:24 2000!cim.cpmc.columbia.edu!|patcis^Data Review sandcar!Fri Oct 27 11:32:28 2000!cim.cpmc.columbia.edu!|patcis^Data Review^Laboratory Detail^lab_detail.cgi sandcar!Fri Oct 27 11:32:30 2000!cim.cpmc.columbia.edu!|patcis^Data Review^Laboratory Detail^labSum.cgi sandcar!Fri Oct 27 11:32:35 2000!cim.cpmc.columbia.edu!|patcis^logout

41. Results Functions Enrollment System usage Function usage Adverse events

42. Enrollment Mailing to >200 physicians 13 physicians returned signed consent forms 19 subjects suggested 13 enrolled 12 used the system over 19 months 1 non-CPMC subject enrolled

43. Baseline Questionnaire data Initial patient expectations “will keep me informed of latest developments” “will enable me to better understand treatment” “possible information my doctor may forget” “timely availability of test results” Prior subject education –Majority of subjects (90 %) had some post-secondary education –Conditions patients were interested included cholesterol management, diabetes and chronic conditions –All were prior computer users

44. System Usage 131log-on failures 22wrong user name 51wrong password 58wrong Secure ID 33log-ons without any activity 466active sessions (261 logged out) ----- 630log-ons

45. Log-Ons Failures by User

46. Active Log-Ons by User

47. Average Monthly Log-Ons

48. Average Session Time by User

49. Minutes per Month

50. Function Usage I Data review: 1831 total –1518 laboratory 737 “Laboratory” button  1083 specific reports 186 “Laboratory Details” button 249 summaries –36 vital signs –35 diabetes flow sheets –212 reports (81 radiology, 35 pathology) –30 Microbiology

51. Function Usage I Data review: 1831 total I Data entry: 73 total –34 vital signs –39 diabetes flow sheets Education: 53 total Advice: 6 total –5 cholesterol guideline –1 mammography guideline Other: –10 newsgroups –83 infobuttons –2 comments –10 e-mail to physician –17 disclaimers –13 help

52. Adverse Events None reported

53. Interviews Conducted (via telephone) summer/fall 2000 Both patients and their physicians interviewed Audio recorded, transcribed and qualitatively analyzed Examples of interview probes (for patient interviews) –General usage What do you use PatCIS for, why? What features have you found most (least) useful? –Usability Have you had any problems using PatCIS? –Effects on health care interaction and decision making Do you feel PatCIS has changed the way you view health or illness? Has use of PatCIS changed in any way your interaction with your health care providers? Explain

54. Analyses of Interviews with Patients and Physicians Subjects felt that use of PatCIS had a direct impact on doctor-patient communication For patients –Improved communication with physician resulting from reviewing their own lab results prior to meeting with their physician – 85% For physicians –Allows for focus on issues of decision making and patient compliance during during doctor-patient interviews (rather than review of data) – 100% –Decreased workload in reviewing essential data

55. Excerpts from Patient Interviews “Communication is less in the way of getting information now, and more in the way of discussing treatment options and agreeing on a course of action, so to me its more efficient than the old way” “I look for trends in my medical data and if I see something I can contact the doctor to see what’s going on, what we can do, change meds or whatever”

56. Excerpts from Physician Interviews “Right now most of the communication takes place during the ten or fifteen minute visit and if I throw a lot of information at the patient about their condition or what I want them to do, its very hard for them to absorb all that.” “It (PatCIS) gives them a chance to go back and look at things about their health record that they can then ask better questions about in the limited time that we have during the visit. Its another channel of communication”

57. Additional Findings Low usage of certain functions (e.g. advice, educational) may have been due to lack of user knowledge of full system capability –Implications for providing further training Usability –Occasional difficulties in viewing graphs –Problems with linkages to outside resources Subjects did not have problems in learning how to use PatCIS for reviewing data No adverse effects of use of PatCIS were indicated by patients or physicians

58. Summary PatCIS was found to be usable by patients studied Patients indicated they understood the information accessed Review of data (Lab and reports) was the most important application of PatCIS by patients Patients and physicians indicated that PatCIS had improved their level of communication Low usage of other functions may have resulted from limited patient knowledge of system capabilities

59. Discussion Architecture supports integration, security and tracking Enrollment was disappointing Population was highly selected: by MD, by self, by Web Two patterns: monthly and daily Log-on difficulties overcome Laboratories are the most popular

60. Next Directions Diabetes mellitus patients –Data entry –Coordination with clinicians –Targeted educational materials

63. Conclusion Enthusiasm is not universal Technical issues were not a problem for our patients Privacy is achievable Patient understanding of their records was good Other features were of less interest Patient-physician impact was positive –better D-P communication –education of both patient and doctor –treatment negotiations with common understanding

64. Conclusions (cont.) Our Analyses indicate that use of systems like PatCIS can serve to facilitate processes involved in improved health care delivery through shared workload between doctor and patient, resulting in better communication and negotiation –Reducing physician workload – patients better informed of day-to-day variations in their own condition –Making patients more accountable for chronic diseases and management