Presentation is loading. Please wait.

Presentation is loading. Please wait.

Standardized Letters of Recommendation

Similar presentations


Presentation on theme: "Standardized Letters of Recommendation"— Presentation transcript:

1 Standardized Letters of Recommendation
Anna H. Messner, MD Professor & Vice Chair Program Director Otolaryngology/Head & Neck Surgery Stanford University

2 Application season

3 Why have letters? Review of 966 LOR to General surgery program
24% were helpful in evaluation process Fortune JB. The content and value of letters of recommendation in the resident candidate evaluative process. Curr Surg 2002

4 Why have letters? The theory.
Information on past performance which may be a good predictor of future success Unique features Insight into typical performance (as opposed to maximal performance) Opportunity to explain a less than perfect application Opportunity to promote students he/she envisions as becoming future leaders in field.

5 Medicine “seems to have taken up permanent residence along the shores of Lake Wobegon. All of the applicants are above average—way above.” All are outstanding Schneider A. Why you can’t trust letters of recommendation. Chron High Educ 2000

6 Writing letters- The basics
1. Introduction 2. Relationship to applicant Length of time he/she has known subject Context of relationship Right to view waived 3. Describe subject’s “record,” Brief summary of subject’s academic performance Summary of clinical performance 4. Subject’s traits and performance are evaluated and compared to peers (+ unique features). 5. Summary sentence

7 OHNS Words/letter 763 letters Mean = 329 (82-1297)
Chair of dept (290) vs others (350) (p<.001) Letters Messner A, Shimahara E Laryngoscope 2009

8 What we currently do… Look for Standout adjectives (average 2.6 in OHNS letters) star, excellent, superb, gifted, extraordinary, outstanding, unique, exceptional, unparalleled, superstar, finest, exquisite, stellar, sterling, phenomenal, ideal, exemplary Grindstone adjectives: Hardworking, resolve, conscientious, dependable, meticulous, thorough, diligent, dedicated, careful, responsible, reliable, tireless, solid “Gets along with others”, “Team player” “Compassionate, Caring, Kind, Empathy” “Intelligent”

9 Features of letters Leadership potential High match rank
Sense of humor Final sentence: Intentionally ambiguous Recommend him without hesitation vs recommend highly vs highest recommendation vs absolute highest recommendation Strong recommendation vs very strong recommendation vs strongest recommendation Compare letters from same letter writer

10 Watch for doubt raisers
“Technical skills were average.” “I have discussed alternative career options” “Made an effort to be an effective team member.” “Average fund of knowledge” “His score of 210 accurately reflects his knowledge level.”

11 Doubt raisers Doubt Raisers
“It absolutely amazes me how he can work with others.” “He’s always trying.” “I assure you that no person would be better for the job.”

12 Typical Letter analysis: beginning & end
Dear Program Director: I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology. Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology. Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck. She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read. In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program. As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me. Sincerely,

13 Typical Letter- ignore duplication
Dear Program Director: I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology. Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology. Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck. She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read. In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program. As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me. Sincerely,

14 Typical Letter analysis: standouts, grindstone, team player
Dear Program Director: I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology. Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology. Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck. She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read. In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program. As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me. Sincerely,

15 Typical Letter analysis: doubt raisers
Dear Program Director: I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology. Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology. Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck. She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read. In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program. As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me. Sincerely,

16 Typical Letter Dear Program Director: I have been asked to submit a letter of recommendation on behalf of Jane Doe who has waived her right to see this letter. I have known Jane for the past 2 years since she first became interested in Otolaryngology. Jane is a fourth year medical student at X School of Medicine. She graduated from Y University in 2002 with a Bachelor of Arts and History. She has achieved several awards during her medical school tenure including “Outstanding Service” in April This is an award elected annually by current X medical students to the one third year student who contributes significantly to improving the lives of fellow students. Additionally, Jane obtained honors in pediatrics, a high pass in surgery, a high pass in obstetrics and gynecology. Jane rotated on the Head and Neck Surgery Service in the Department of Otolaryngology Head and Neck Surgery at S School of Medicine. During her rotation Jane performed in an outstanding manner. She is a bright and intelligent student who has an excellent knowledge of the head and neck anatomy and diseases that afflict the head and neck. She was actively engaged throughout her rotation in caring for the head and neck cancer patients in the ward. Additionally, she played an active role in the operating room where she clearly demonstrated an affinity for surgery. She has excellent hand-eye coordination and an excellent sense of the anatomy. Jane was well liked by her peers, the residents and the attending staff. She is a pleasant person to work with. She is hard working and while somewhat shy Jane is clearly well informed and well read. In general I was extremely impressed with this young student who I feel will make an outstanding contribution to any otolaryngology head and neck surgery training program. As a result, I would like to submit a very strong letter of recommendation on behalf of Jane Doe. If I can be of further assistance please feel free to contact me. Sincerely, NOTE: nothing unique, interesting, no comparison to peers, no mention leadership potential, no humor, no match potential

17 Letter of Minimal Assurance
“I believe that Joe is a strong candidate for a residency position and that he will ultimately be an excellent otolaryngologist and contributor to medicine. He is passionately committed to this goal and he is driven by that commitment. He is a unique person, and one whom you will enjoy meeting and interviewing. I would of course welcome a direct phone call regarding this letter. Thank you for considering Joe for an interview position.”

18 Current Status Average program: 250 applications x 4 letters = 1000 letters Average letter = 329 words Time = 1 min/letter (without analysis) 1000 min = 17 hours. Result: Letters often not used to make 1st cut Increased reliance on USMLE scores And… many not read at all.

19 Standardized letter of recommendation (SLOR)
Council of Emergency Medicine Residency Directors created SLOR Successfully used since 1996

20 Comparison of SLOR & NLOR
Part I: Compared 20 SLOR and 20 NLOR 7- point Likert-type scale 4 raters (2 experienced, 2 inexperienced) Results Interrater reliability of SLOR = 0.97, NLOR 0.78 Average time to interpret SLOR = 16 sec, NLOR = 90 sec Girzadas DV Jr, Harwood RC, Dearie J, Garrett S. Acad Emerg Med Nov;5(11):1101-4

21 Comparison of SLOR & NLOR
Part I: Compared 20 SLOR and 20 NLOR 7- point Likert-type scale 4 raters (2 experienced, 2 inexperienced) Results Interrater reliability of SLOR = 0.97, NLOR 0.78 Average time to interpret SLOR = 16 sec, NLOR = 90 sec Part II: 207 SLOR/NLOR pairs 54% given same numeric value, 39% differed by one point, 6% differed by 2 points. Overall correlation was 0.58. Girzadas DV Jr, Harwood RC, Dearie J, Garrett S. Acad Emerg Med Nov;5(11):1101-4

22 EM SLOR 173 surveys of PD organization (70% response rate)
“Compared with the narrative LOR, please rate the SLOR in terms of:” Ability to discriminate differences between candidates Better 75%, Same 18%, Worse 7% Ease of reading and incorporating into ranking scheme Better 84%, Same 12% Worse 4% Ease of completion Better 83%, Same 9%, Worse 4% Would you like to continue using the SLOR? Yes 90%, No 7% Keim SM, et al. Acad Emerg Med Nov;6(11):1141-6

23 Proposed Standardized Letter of Recommendation

24 Standardized LOR

25 Standardized LOR

26 Compare to other medical students

27 Standardized LOR

28 Standardized LOR

29 Standardized LOR

30 Standardized LOR

31 Standardized LOR

32 Standardized LOR

33 Advantages of SLOR More consistent information
Less duplication- of contents of application and letters Less time to complete & interpret Information less dependent on letter writer (more fair) (A superb letter writer may portray a “good” candidate in stratospheric terms, whereas a mediocre letter writer may portray a “superb” candidate less positively) Still have opportunity to tell about unique features of applicant

34 Disadvantages of SLOR Potential loss of nuances
Much harder to make the average student look stupendous Potential attending MD guilt Don’t get to show off fancy Department stationary But OK to add Logo, or put form on stationary

35 Next Steps Open for comments until February 1, 2012
Send to OPDO council will review all comments and revise proposed SLOR Assuming response is positive: Will post SLOR on SUO-AADO website distribution to PDs and Chairs Need Program director assistance to get the word out to faculty and applicants Investigate information re SLOR in Oto journal Start using in spring/summer 2012 Survey SUO/AADO/PD re use, then revise.

36 Conclusions amessner@ohns.stanford.edu
Traditional letters have the potential to help with identifying top residency candidates– but often do not. A SLOR has the potential to increase the value of LOR while saving time. Please send all comments to:


Download ppt "Standardized Letters of Recommendation"

Similar presentations


Ads by Google