1. Assessing non-English Language Proficiency of Clinicians who Bypass Interpreters
Lisa Diamond, MD, MPH, Assistant Attending
Memorial Sloan-Kettering Cancer Center
Immigrant Health and Cancer Disparities Service
Department of Psychiatry & Behavioral Sciences/Department of Medicine
Office
Funding: The California Endowment and NCI R21 CA168489

2. Acknowledgements Coauthors:
Sukyung Chung, Palo Alto Medical Foundation Research Institute (AHRQ K01 HS and The California Endowment )
Warren Ferguson, University of Massachusetts Medical Center
Elizabeth Jacobs, University of Wisconsin – Madison
Francesca Gany, MSKCC

3. Background
Clear communication between clinicians and patients is essential
Patient-clinician communication associated with patient satisfaction, adherence to physician recommendations, and health outcomes
Patients with LEP often experience poor patient-clinician communication

4. Background
Language concordance generally leads to better outcomes for LEP patients
Few studies of cancer screening show lower rates for LEP patients with language concordant providers
Few studies have systematically measured clinician language proficiency
Language concordance associated with:
better patient satisfaction with care
medication adherence
understanding of diagnoses and treatment
patient centeredness
health education.
Having a language concordant provider leads to
lower emergency room use
likelihood of missing medications
and cost
Bullet 2: but these were self report using non-validated tools so it is not clear if the “language concordant” clinicians in these studies were truly fluent or had adequate language skills to be able to communicate in a language other than English.

5. Study Objective
To evaluate the accuracy of a structured self-assessment of non-English language proficiency compared to a validated oral proficiency interview for clinicians.
we hypothesized that the oral proficiency interview results by clinicians on the low and high ends of the self-assessment scale would be more accurate than those of the clinicians whose self-assessments fell in the middle of the scale

6. Project Setting Palo Alto Medical Foundation (PAMF)
>10% pts preferred language other than English (mainly Spanish, Mandarin, Cantonese)
Massachusetts Community Health Centers (MA CHC)
>30% pts preferred language other than English (mainly Spanish, Portuguese, Vietnamese, French)
PAMF multi-specialty, not-for-profit with ~350 Primary Care Providers (PCPs) in 2010
PCP: physicians, physician assistants, and nurse practitioners
Self-assessed fluency most common in Spanish, Mandarin, Cantonese at PAMF
The settings were chosen because both have large LEP patient populations with PCPs using non-English skills with patients but represent different practice organizations, geographic regions and patient socioeconomic backgrounds.

7. Recruitment
Clinicians with any level of proficiency in Spanish, Mandarin, Cantonese, French, Portuguese, Vietnamese
16 PAMF, 51 MA CHC
Survey - self-reported language proficiency, demographics
Oral proficiency interview
Gift card

8. Self-Assessment Scale
Interagency Language Roundtable (ILR) Scale
Scale consists of 5 main levels with descriptive explanations of each
Adapted for the study to be used as a self-reporting tool to reflect language proficiency in medical situations
The ILR is an organization comprised of representatives from academia, government, and non-government organizations
The foundations of the ILR assessment were developed by the US Foreign Service Institute after it determined in the 1950s that most Foreign Service officers had inadequate fluency in their work-related languages. It consists of a standard scale for language skills in speaking, listening, writing, and translating.
Other organizations, such as the ACTFL, have adapted the ILR scale for their own proficiency guidelines but it has not been widely adopted within health care.
It takes less than 5 minutes to complete. We elected to use the ILR due to its long history of use and rigorous development.

9. Adapted ILR Scale Excellent
Speaks proficiently, equivalent to that of an educated speaker, and is skilled at incorporating appropriate medical terminology and concepts into communication. Has complete fluency in the language such that speech in all levels is fully accepted by educated native speakers in all its features, including breadth of vocabulary and idioms, colloquialisms, and pertinent cultural references.
Very Good
Able to use the language fluently and accurately on all levels related to work needs in a healthcare setting. Can understand and participate in any conversation within the range of his/her experience with a high degree of fluency and precision of vocabulary. Unaffected by rate of speech. Language ability only rarely hinders him/her in performing any task requiring language; yet, the individual would seldom be perceived as a native.
Good
Able to speak the language with sufficient accuracy and vocabulary to have effective formal and informal conversations on most familiar topics. Although cultural references, proverbs and the implications of nuances and idiom may not be fully understood, the individual can easily repair the conversation. May have some difficulty communicating necessary health concepts.
Fair
Meets basic conversational needs. Able to understand and respond to simple questions. Can handle casual conversation about work, school, and family. Has difficulty with vocabulary and grammar. The individual can get the gist of most everyday conversations but has difficulty communicating about healthcare concepts.
Poor
Satisfies elementary needs and minimum courtesy requirements. Able to understand and respond to 2-3 word entry level questions. May require slow speech and repetition to understand. Unable to understand or communicate most healthcare concepts.

10. Oral Proficiency Interview
Clinician Cultural and Linguistic Assessment (CCLA)
Validated in 17 languages
Administered by telephone, available 24/7
$100/test
30-40 min
Passing score 80
Developed by academic researchers and validated in 17 languages (including the languages we tested in this study).
Test fee paid by The California Endowment (the funding organization).
Prompts and instructions during the test are pre-recorded by native speakers to ensure that all subjects are given an identical testing experience.
There are 5 sections that simulate medical encounters, demonstrating skills which could include explaining diagnoses, lifestyle modification, or the content of an English language form to a patient.
Assessments are scored separately by two professional raters
The CCLA passing score of 80% was established by test development experts.

11. Analysis
Wilcoxon-Mann-Whitney test to assess equality in CCLA test score by language
Spearman test to assess correlation between CCLA scores and the ILR scale for overall sample and by language
Kruskal-Wallis squared rank test to assess equality of variance in CCLA scores across ILR categories
KW: because means and variances in test scores in each ILR group were correlated.
We used non-parametric approaches in drawing statistical inferences because
(1) sample size in some groups (e.g., ILR=“poor”) was too small
(2) the test scores in the cells were not normally distributed
Stata version 11.2 (StataCorp, College Station, TX)

12. Results

13. Summary Statistics of Sample
Freq (%) or Mean [SD]
Language
Spanish
53 (79.1%)
Chinese
9 (13.4%)
Portuguese
3 (4.5%)
Vietnamese
1 (1.5%)
French
Female
52 (77.6%)
Provider title
MD or DO
59 (88.1%)
PA, NP or CNM
16 (23.9%)
ILR
Excellent
6 (9.0%)
Very Good
Good
31 (46.3%)
Fair
12 (17.9%)
Poor
2 (3.0%)
Test score
75.9 [15.7]
Spanish was the most common language tested
followed by Chinese (Mandarin and Cantonese combined
Other languages tested included Portuguese, Vietnamese, and French.
Most PCPs (88%) were physicians with a DO or MD degree
The majority of the tested PCPs (78%) were female.
The respondents rated their proficiency levels on the ILR as
“Good” most frequently (46%)
followed by “Very Good” (24%)
“Fair” (18%)
“Excellent” (9%)
Only two (3%) rated their language proficiency levels as “Poor”
The average CCLA test score was 76 out of a possible 100

14. ILR Scale vs. CCLA Test Score, by Language
 - There was a positive correlation between self-reported proficiency on the ILR scale and CCLA score (rho=0.49, p<0.001).
Respondents who self-reported “Excellent” on the ILR scored 87 on average on the CCLA.
There were only 2 points difference in the average score between those who reported “Very good” (80.8) and those who reported “Good” (78.5) on the ILR scale.
Variance in CCLA scores were wider in the middle ILR categories, i.e., “Good” or “Very good”, than in the extreme categories, i.e., “Poor”, “Fair” or “Excellent” (p<0.01).

15. Language Differences
Significant correlation between ILR scale and CCLA scores:
Spanish (n=53, rho=0.45, p<0.001)
Other languages combined (n=5, rho=0.95, p<0.05)
No significant correlation for those tested in Chinese (n=9, rho=0.42, p=0.25)
Spanish respondents scored higher (77.9) than Chinese respondents (60.8) (p<0.05).

16. Limitations Small sample Focused on language proficiency only
Two settings with different populations
ILR scale not usually self-administered
ILR adapted for this study to address clinician-patient interactions
Unable to assess relationship between clinician non-English language proficiency and quality of care
We are currently conducting additional research to better understand how clinician demographics, non-English language acquisition, and interpreter use vary by non-English language proficiency.
Two settings which have different patient populations, clinician habits, and access to interpreter services
The ILR scale, while a valid scale, is not usually self-administered although it has been used this way in non-medical settings
ILR adapted but no existing, validated self-assessment tools for clinician non-English language proficiency that could be substituted
unable to assess the impact of over- or underestimation of non-English language proficiency by clinicians

17. Conclusions
Clinicians who self-assess on low and high ends of the ILR scale are accurate
Clinicians in the middle range may require verification of self-assessments
Health care organizations and providers need to understand limitations of self-assessment
Research needed to understand level of language proficiency needed to provide safe and effective care
Clinicians who self-assess “Poor,” “Fair,” or “Excellent” non-English language proficiency on the ILR scale are accurate in their self-assessments

18. References
Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. The National Academies Press, Committee on Quality of Health Care in America; 2001.
Stewart M, Brown JB, Boon H, Galajda J, Meredith L, Sangster M. Evidence on patient-doctor communication. Cancer Prevention & Control. Feb 1999;3(1):25-30.
Institute of Medicine. Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academies Press; 2003.
Lauderdale DS, Wen M, Jacobs EA, Kandula NR. Immigrant perceptions of discrimination in health care: the California Health Interview Survey Medical care. Oct 2006;44(10):
Wisnivesky JP, Kattan M, Evans D, et al. Assessing the relationship between language proficiency and asthma morbidity among inner-city asthmatics. Medical care. Feb 2009;47(2):
Green AR, Ngo-Metzger Q, Legedza AT, Massagli MP, Phillips RS, Iezzoni LI. Interpreter services, language concordance, and health care quality. Experiences of Asian Americans with limited English proficiency. Journal of general internal medicine. 2005;20(11):
Ngo-Metzger Q, Sorkin D, Phillips R, et al. Providing High-Quality Care for Limited English Proficient Patients: The Importance of Language Concordance and Interpreter Use. Journal of general internal medicine. 2007;22(Suppl 2):
Manson A. Language Concordance as a Determinant of Patient Compliance and Emergency Room Use in Patients with Asthma. Medical care. 1988;26(12):
Baker DW, Parker RM, Williams MV, Coates WC, Pitkin K. Use and effectiveness of interpreters in an emergency department. JAMA : the journal of the American Medical Association. Mar ;275(10):
Fernandez A, Schillinger D, Grumbach K, et al. Physician language ability and cultural competence. An exploratory study of communication with Spanish-speaking patients. Journal of general internal medicine. Feb 2004;19(2):
Eamranond PP, Davis RB, Phillips RS, Wee CC. Patient-Physician Language Concordance and Lifestyle Counseling Among Spanish-Speaking Patients Journal of Immigrant and Minority Health 2009;11(6):
Carter-Pokras O, O'Neill MJ, Cheanvechai V, et al. Providing linguistically appropriate services to persons with limited English proficiency: a needs and resources investigation. American Journal of Managed Care. Sep 2004;10 Spec No:SP29-36.
Jacobs E, Sadowski L, Rathouz P. The impact of an enhanced interpreter service intervention on hospital costs and patient satisfaction. Journal of general internal medicine. November 2007;22(Suppl 2):
Eamranond PP. Patient-physician language concordance and primary care screening among Spanish-speaking patients. Medical care. 2011;49(7):

19. References
Jo A, Maxwell A, Wong W, Bastani R. Colorectal cancer screening among underserved Korean Americans in Los Angeles County. Journal of Immigrant and Minority Health. 2008;10(2):
Linsky A. Patient-provider language concordance and colorectal cancer screening. Journal of general internal medicine. 2011;26(2):
Diamond LC, Luft HS, Chung S, Jacobs EA. “Does this Doctor Speak My Language?” Improving the Characterization of Physician non-English Language Skills. Health services research. 2011;In press.
Diamond LC, Reuland DS. Describing Physician Language Fluency: Deconstructing Medical Spanish. JAMA : the journal of the American Medical Association. 2009;301(4):
Diamond LC, Tuot DS, Karliner LS. The Use of Spanish Language Skills by Physicians and Nurses: Policy Implications for Teaching and Testing. Journal of general internal medicine. 2011;ePub ahead of print, DOI /s
Tuot DS, Lopez M, Miller C, Karliner LS. Impact of an easy-access telephonic interpreter program in the acute care setting: an evaluation of a quality improvement intervention. Joint Commission journal on quality and patient safety / Joint Commission Resources. Feb 2012;38(2):81-88.
Better Communication, Better Care: Provider Tools to Care for Diverse Populations. 2010; Accessed January 5, 2012.
Reuland D, Frasier P, Olson M, Slatt L, Aleman M, Fernandez A. Accuracy of Self-assessed Spanish Fluency in Medical Students. Teaching and Learning in Medicine. 2009;21(4):
Tidwell L. Kaiser Permanente-Southern California Physicians Language Concordance Program: Meeting the Needs of LEP Patients. Health Care Interpreter Network: From Ad-Hoc to Best Practices in Healthcare Interpreting; July 16-17, 2009; Oakland, CA
Moreno M, Otero-Sabogal R, Newman J. Assessing dual-role staff-interpreter linguistic competency in an integrated healthcare system. Journal of general internal medicine. 2007;22(Suppl 2):
Language Testing Options. 2008; Accessed November 19, 2012.
Tang G, Lanza O, Rodriguez F, Chang A. The Kaiser Permanente Clinician Cultural and Linguistic Assessment Initiative: research and development in patient-provider language concordance. American Journal of Public Health. 2011;101(2):
US Census Bureau: Profiles of General Demographic Characteristics, 2000 Census of Population and Housing. 2001; Accessed May 20, 2008.
Employment Tests and Selection Procedures. 2008; Accessed July 15, 2009.
Conover WJ. Practical Nonparametric Statistics. 3rd ed: Wiley; 1999.