1 Assessing non-English Language Proficiency of Clinicians who Bypass Interpreters Lisa Diamond, MD, MPH, Assistant AttendingMemorial Sloan-Kettering Cancer CenterImmigrant Health and Cancer Disparities ServiceDepartment of Psychiatry & Behavioral Sciences/Department of MedicineOfficeFunding: The California Endowment and NCI R21 CA168489
2 Acknowledgements Coauthors: Sukyung Chung, Palo Alto Medical Foundation Research Institute (AHRQ K01 HS and The California Endowment )Warren Ferguson, University of Massachusetts Medical CenterElizabeth Jacobs, University of Wisconsin – MadisonFrancesca Gany, MSKCC
3 BackgroundClear communication between clinicians and patients is essentialPatient-clinician communication associated with patient satisfaction, adherence to physician recommendations, and health outcomesPatients with LEP often experience poor patient-clinician communication
4 BackgroundLanguage concordance generally leads to better outcomes for LEP patientsFew studies of cancer screening show lower rates for LEP patients with language concordant providersFew studies have systematically measured clinician language proficiencyLanguage concordance associated with:better patient satisfaction with caremedication adherenceunderstanding of diagnoses and treatmentpatient centerednesshealth education.Having a language concordant provider leads tolower emergency room uselikelihood of missing medicationsand costBullet 2: but these were self report using non-validated tools so it is not clear if the “language concordant” clinicians in these studies were truly fluent or had adequate language skills to be able to communicate in a language other than English.
5 Study ObjectiveTo evaluate the accuracy of a structured self-assessment of non-English language proficiency compared to a validated oral proficiency interview for clinicians.we hypothesized that the oral proficiency interview results by clinicians on the low and high ends of the self-assessment scale would be more accurate than those of the clinicians whose self-assessments fell in the middle of the scale
6 Project Setting Palo Alto Medical Foundation (PAMF) >10% pts preferred language other than English (mainly Spanish, Mandarin, Cantonese)Massachusetts Community Health Centers (MA CHC)>30% pts preferred language other than English (mainly Spanish, Portuguese, Vietnamese, French)PAMF multi-specialty, not-for-profit with ~350 Primary Care Providers (PCPs) in 2010PCP: physicians, physician assistants, and nurse practitionersSelf-assessed fluency most common in Spanish, Mandarin, Cantonese at PAMFThe settings were chosen because both have large LEP patient populations with PCPs using non-English skills with patients but represent different practice organizations, geographic regions and patient socioeconomic backgrounds.
7 RecruitmentClinicians with any level of proficiency in Spanish, Mandarin, Cantonese, French, Portuguese, Vietnamese16 PAMF, 51 MA CHCSurvey - self-reported language proficiency, demographicsOral proficiency interviewGift card
8 Self-Assessment Scale Interagency Language Roundtable (ILR) ScaleScale consists of 5 main levels with descriptive explanations of eachAdapted for the study to be used as a self-reporting tool to reflect language proficiency in medical situationsThe ILR is an organization comprised of representatives from academia, government, and non-government organizationsThe foundations of the ILR assessment were developed by the US Foreign Service Institute after it determined in the 1950s that most Foreign Service officers had inadequate fluency in their work-related languages. It consists of a standard scale for language skills in speaking, listening, writing, and translating.Other organizations, such as the ACTFL, have adapted the ILR scale for their own proficiency guidelines but it has not been widely adopted within health care.It takes less than 5 minutes to complete. We elected to use the ILR due to its long history of use and rigorous development.
9 Adapted ILR Scale Excellent Speaks proficiently, equivalent to that of an educated speaker, and is skilled at incorporating appropriate medical terminology and concepts into communication. Has complete fluency in the language such that speech in all levels is fully accepted by educated native speakers in all its features, including breadth of vocabulary and idioms, colloquialisms, and pertinent cultural references.Very GoodAble to use the language fluently and accurately on all levels related to work needs in a healthcare setting. Can understand and participate in any conversation within the range of his/her experience with a high degree of fluency and precision of vocabulary. Unaffected by rate of speech. Language ability only rarely hinders him/her in performing any task requiring language; yet, the individual would seldom be perceived as a native.GoodAble to speak the language with sufficient accuracy and vocabulary to have effective formal and informal conversations on most familiar topics. Although cultural references, proverbs and the implications of nuances and idiom may not be fully understood, the individual can easily repair the conversation. May have some difficulty communicating necessary health concepts.FairMeets basic conversational needs. Able to understand and respond to simple questions. Can handle casual conversation about work, school, and family. Has difficulty with vocabulary and grammar. The individual can get the gist of most everyday conversations but has difficulty communicating about healthcare concepts.PoorSatisfies elementary needs and minimum courtesy requirements. Able to understand and respond to 2-3 word entry level questions. May require slow speech and repetition to understand. Unable to understand or communicate most healthcare concepts.
10 Oral Proficiency Interview Clinician Cultural and Linguistic Assessment (CCLA)Validated in 17 languagesAdministered by telephone, available 24/7$100/test30-40 minPassing score 80Developed by academic researchers and validated in 17 languages (including the languages we tested in this study).Test fee paid by The California Endowment (the funding organization).Prompts and instructions during the test are pre-recorded by native speakers to ensure that all subjects are given an identical testing experience.There are 5 sections that simulate medical encounters, demonstrating skills which could include explaining diagnoses, lifestyle modification, or the content of an English language form to a patient.Assessments are scored separately by two professional ratersThe CCLA passing score of 80% was established by test development experts.
11 AnalysisWilcoxon-Mann-Whitney test to assess equality in CCLA test score by languageSpearman test to assess correlation between CCLA scores and the ILR scale for overall sample and by languageKruskal-Wallis squared rank test to assess equality of variance in CCLA scores across ILR categoriesKW: because means and variances in test scores in each ILR group were correlated.We used non-parametric approaches in drawing statistical inferences because(1) sample size in some groups (e.g., ILR=“poor”) was too small(2) the test scores in the cells were not normally distributedStata version 11.2 (StataCorp, College Station, TX)
13 Summary Statistics of Sample Freq (%) or Mean [SD]LanguageSpanish53 (79.1%)Chinese9 (13.4%)Portuguese3 (4.5%)Vietnamese1 (1.5%)FrenchFemale52 (77.6%)Provider titleMD or DO59 (88.1%)PA, NP or CNM16 (23.9%)ILRExcellent6 (9.0%)Very GoodGood31 (46.3%)Fair12 (17.9%)Poor2 (3.0%)Test score75.9 [15.7]Spanish was the most common language testedfollowed by Chinese (Mandarin and Cantonese combinedOther languages tested included Portuguese, Vietnamese, and French.Most PCPs (88%) were physicians with a DO or MD degreeThe majority of the tested PCPs (78%) were female.The respondents rated their proficiency levels on the ILR as“Good” most frequently (46%)followed by “Very Good” (24%)“Fair” (18%)“Excellent” (9%)Only two (3%) rated their language proficiency levels as “Poor”The average CCLA test score was 76 out of a possible 100
14 ILR Scale vs. CCLA Test Score, by Language - There was a positive correlation between self-reported proficiency on the ILR scale and CCLA score (rho=0.49, p<0.001).Respondents who self-reported “Excellent” on the ILR scored 87 on average on the CCLA.There were only 2 points difference in the average score between those who reported “Very good” (80.8) and those who reported “Good” (78.5) on the ILR scale.Variance in CCLA scores were wider in the middle ILR categories, i.e., “Good” or “Very good”, than in the extreme categories, i.e., “Poor”, “Fair” or “Excellent” (p<0.01).
15 Language DifferencesSignificant correlation between ILR scale and CCLA scores:Spanish (n=53, rho=0.45, p<0.001)Other languages combined (n=5, rho=0.95, p<0.05)No significant correlation for those tested in Chinese (n=9, rho=0.42, p=0.25)Spanish respondents scored higher (77.9) than Chinese respondents (60.8) (p<0.05).
16 Limitations Small sample Focused on language proficiency only Two settings with different populationsILR scale not usually self-administeredILR adapted for this study to address clinician-patient interactionsUnable to assess relationship between clinician non-English language proficiency and quality of careWe are currently conducting additional research to better understand how clinician demographics, non-English language acquisition, and interpreter use vary by non-English language proficiency.Two settings which have different patient populations, clinician habits, and access to interpreter servicesThe ILR scale, while a valid scale, is not usually self-administered although it has been used this way in non-medical settingsILR adapted but no existing, validated self-assessment tools for clinician non-English language proficiency that could be substitutedunable to assess the impact of over- or underestimation of non-English language proficiency by clinicians
17 ConclusionsClinicians who self-assess on low and high ends of the ILR scale are accurateClinicians in the middle range may require verification of self-assessmentsHealth care organizations and providers need to understand limitations of self-assessmentResearch needed to understand level of language proficiency needed to provide safe and effective careClinicians who self-assess “Poor,” “Fair,” or “Excellent” non-English language proficiency on the ILR scale are accurate in their self-assessments
18 ReferencesInstitute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. The National Academies Press, Committee on Quality of Health Care in America; 2001.Stewart M, Brown JB, Boon H, Galajda J, Meredith L, Sangster M. Evidence on patient-doctor communication. Cancer Prevention & Control. Feb 1999;3(1):25-30.Institute of Medicine. Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academies Press; 2003.Lauderdale DS, Wen M, Jacobs EA, Kandula NR. Immigrant perceptions of discrimination in health care: the California Health Interview Survey Medical care. Oct 2006;44(10):Wisnivesky JP, Kattan M, Evans D, et al. Assessing the relationship between language proficiency and asthma morbidity among inner-city asthmatics. Medical care. Feb 2009;47(2):Green AR, Ngo-Metzger Q, Legedza AT, Massagli MP, Phillips RS, Iezzoni LI. Interpreter services, language concordance, and health care quality. Experiences of Asian Americans with limited English proficiency. Journal of general internal medicine. 2005;20(11):Ngo-Metzger Q, Sorkin D, Phillips R, et al. Providing High-Quality Care for Limited English Proficient Patients: The Importance of Language Concordance and Interpreter Use. Journal of general internal medicine. 2007;22(Suppl 2):Manson A. Language Concordance as a Determinant of Patient Compliance and Emergency Room Use in Patients with Asthma. Medical care. 1988;26(12):Baker DW, Parker RM, Williams MV, Coates WC, Pitkin K. Use and effectiveness of interpreters in an emergency department. JAMA : the journal of the American Medical Association. Mar ;275(10):Fernandez A, Schillinger D, Grumbach K, et al. Physician language ability and cultural competence. An exploratory study of communication with Spanish-speaking patients. Journal of general internal medicine. Feb 2004;19(2):Eamranond PP, Davis RB, Phillips RS, Wee CC. Patient-Physician Language Concordance and Lifestyle Counseling Among Spanish-Speaking Patients Journal of Immigrant and Minority Health 2009;11(6):Carter-Pokras O, O'Neill MJ, Cheanvechai V, et al. Providing linguistically appropriate services to persons with limited English proficiency: a needs and resources investigation. American Journal of Managed Care. Sep 2004;10 Spec No:SP29-36.Jacobs E, Sadowski L, Rathouz P. The impact of an enhanced interpreter service intervention on hospital costs and patient satisfaction. Journal of general internal medicine. November 2007;22(Suppl 2):Eamranond PP. Patient-physician language concordance and primary care screening among Spanish-speaking patients. Medical care. 2011;49(7):
19 ReferencesJo A, Maxwell A, Wong W, Bastani R. Colorectal cancer screening among underserved Korean Americans in Los Angeles County. Journal of Immigrant and Minority Health. 2008;10(2):Linsky A. Patient-provider language concordance and colorectal cancer screening. Journal of general internal medicine. 2011;26(2):Diamond LC, Luft HS, Chung S, Jacobs EA. “Does this Doctor Speak My Language?” Improving the Characterization of Physician non-English Language Skills. Health services research. 2011;In press.Diamond LC, Reuland DS. Describing Physician Language Fluency: Deconstructing Medical Spanish. JAMA : the journal of the American Medical Association. 2009;301(4):Diamond LC, Tuot DS, Karliner LS. The Use of Spanish Language Skills by Physicians and Nurses: Policy Implications for Teaching and Testing. Journal of general internal medicine. 2011;ePub ahead of print, DOI /sTuot DS, Lopez M, Miller C, Karliner LS. Impact of an easy-access telephonic interpreter program in the acute care setting: an evaluation of a quality improvement intervention. Joint Commission journal on quality and patient safety / Joint Commission Resources. Feb 2012;38(2):81-88.Better Communication, Better Care: Provider Tools to Care for Diverse Populations. 2010; Accessed January 5, 2012.Reuland D, Frasier P, Olson M, Slatt L, Aleman M, Fernandez A. Accuracy of Self-assessed Spanish Fluency in Medical Students. Teaching and Learning in Medicine. 2009;21(4):Tidwell L. Kaiser Permanente-Southern California Physicians Language Concordance Program: Meeting the Needs of LEP Patients. Health Care Interpreter Network: From Ad-Hoc to Best Practices in Healthcare Interpreting; July 16-17, 2009; Oakland, CAMoreno M, Otero-Sabogal R, Newman J. Assessing dual-role staff-interpreter linguistic competency in an integrated healthcare system. Journal of general internal medicine. 2007;22(Suppl 2):Language Testing Options. 2008; Accessed November 19, 2012.Tang G, Lanza O, Rodriguez F, Chang A. The Kaiser Permanente Clinician Cultural and Linguistic Assessment Initiative: research and development in patient-provider language concordance. American Journal of Public Health. 2011;101(2):US Census Bureau: Profiles of General Demographic Characteristics, 2000 Census of Population and Housing. 2001; Accessed May 20, 2008.Employment Tests and Selection Procedures. 2008; Accessed July 15, 2009.Conover WJ. Practical Nonparametric Statistics. 3rd ed: Wiley; 1999.