1. © 2011 AMERICAN ACADEMY OF NEUROLOGY American Academy of Neurology Clinical Skills Examination Training Program (date) (speaker) (association)

2. © 2011 AMERICAN ACADEMY OF NEUROLOGY Goals of this session Understand the American Board of Psychiatry and Neurology (ABPN) performance requirements for the five Clinical Skills Evaluations (NEX) neurology residents must pass to achieve board certification. Appreciate the difference between a grade of ACCEPTABLE and BORDERLINE BUT UNACCEPTABLE. Improve the inter-rater reliability of faculty evaluators for acceptable versus unacceptable distinctions.

3. © 2011 AMERICAN ACADEMY OF NEUROLOGY What is your role within your department? 1.Department chair 2.Residency program director 3.Adult neurology faculty 4.Child neurology faculty 5.Resident 6.Fellow 7.Other

4. © 2011 AMERICAN ACADEMY OF NEUROLOGY For how many years have you been practicing as a neurologist? (Only faculty with a clinical appointment should answer.) 1.0-4 2.5-9 3.10-14 4.15-19 5.20-24 6.25 or more

5. © 2011 AMERICAN ACADEMY OF NEUROLOGY Have you ever served as an ABPN oral board examiner? (Only faculty with a clinical appointment should answer.) 1.Yes 2.No

6. © 2011 AMERICAN ACADEMY OF NEUROLOGY For how many NEX examinations have you served as a faculty evaluator? (Only faculty with a clinical appointment should answer.) 1.0 2.1-4 3.5-9 4.10-14 5.15-19 6.20 or more

7. © 2011 AMERICAN ACADEMY OF NEUROLOGY How confident are you that you understand the ABPN criteria for determining whether a resident's performance on an NEX merits a passing grade? 1.1 – Not at all confident 2.2 3.3 4.4 5.5 6.6 7.7 8.8 9.9 10.10 - Extremely confident

8. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neurology Clinical Evaluation Exercise (NEX) Every resident must pass an examination in each of 5 categories during residency to sit for the neurology written board exam.  Ambulatory  Critical care  Neuromuscular  Neurodegenerative  Child neurology (for adult neuro residents) or adult neurology (for child neuro residents)

9. © 2011 AMERICAN ACADEMY OF NEUROLOGY Why train faculty evaluators? ABPN oral board examiners underwent training Poor inter-rater reliability between former ABPN oral board examiners and local faculty. (Schuh, et al 2009)

10. © 2011 AMERICAN ACADEMY OF NEUROLOGY

11. What constitutes a pass or fail? 5 (marginal pass) Deficiencies or errors in history or exam, but enough information is obtained to formulate the case. 4 (marginal fail) Deficiencies or errors in history or exam, which result in the resident not being able to successfully formulate the case.

12. © 2011 AMERICAN ACADEMY OF NEUROLOGY Clinical Skills Examination Training (CSET) Program Designed by a workgroup through the Consortium of Neurology Program Directors. Objectives: Create a simple NEX training program that can be utilized by faculty at neurology training programs Focus on pass-fail distinction

13. © 2011 AMERICAN ACADEMY OF NEUROLOGY Clinical Skills Examination Training Program - Instructions We will review a series of 9 brief vignettes. For each, there will be an example of an omission on the part of the resident. Assume that the resident has performed a basic history and physical examination that is otherwise adequate Select PASS or FAIL based on whether the omission is egregious enough to warrant an overall failing grade. Review audience votes, as well as the voting results and talking points from previous participants. Open discussion (Up to 1 minute for each side.) Re-vote

14. © 2011 AMERICAN ACADEMY OF NEUROLOGY Objectives Distinguish between those mistakes and omissions that interfere with the formulation of the case. Training through reflection and discussion Data gathering

15. © 2011 AMERICAN ACADEMY OF NEUROLOGY Criteria for assigning a failing grade Deficiencies or errors in history or exam which result in the resident not being able to successfully formulate the case.

16. © 2011 AMERICAN ACADEMY OF NEUROLOGY

17. Critical Care Scenario: (PRE) A 55 year old woman with known myasthenia gravis is admitted to the intensive care unit with a chief complaint of shortness of breath and generalized weakness. Omission: The resident does not check neck flexion strength. 1.Pass 2.Fail Historical Pass Rate: 76.2% (n=101) Talking points: Pass: Helpful, but not crucial. Respiratory parameters are crucial. Fail: Might help me determine whether or not to intubate.

18. © 2011 AMERICAN ACADEMY OF NEUROLOGY Critical Care Scenario: (POST) A 55 year old woman with known myasthenia gravis is admitted to the intensive care unit with a chief complaint of shortness of breath and generalized weakness. Omission: The resident does not check neck flexion strength. 1.Pass 2.Fail Historical Pass Rate: 76.2% (n=101) Talking points: Pass: Helpful, but not crucial. Respiratory parameters are crucial. Fail: Might help me determine whether or not to intubate.

19. © 2011 AMERICAN ACADEMY OF NEUROLOGY Pediatrics Scenario: (PRE) A 6 year old boy with normal birth and development develops progressive difficulty walking and climbing stairs. His calves are large. Omission: The resident does not ask about family history. 1.Pass 2.Fail Historical Pass Rate: 7.7% (n=104) Talking points: Pass: Won’t affect workup. Fail: Family history can direct DNA testing

20. © 2011 AMERICAN ACADEMY OF NEUROLOGY Pediatrics Scenario: (POST) A 6 year old boy with normal birth and development develops progressive difficulty walking and climbing stairs. His calves are large. Omission: The resident does not ask about family history. 1.Pass 2.Fail Historical Pass Rate: 7.7% (n=104) Talking points: Pass: Won’t affect workup. Fail: Family history can direct DNA testing

21. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neuromuscular Scenario: (PRE) A 37 year old woman, previously healthy, presents to your office complaining of progressive muscle weakness and intermittent double vision. Omission: The resident does not stabilize the elbow joint when checking triceps strength. 1.Pass 2.Fail Historical Pass Rate: 96.2% (n=104) Talking points: Pass: This is a good teaching point, but doesn’t affect formulation. Fail: Poor exam suggests resident would not be able to properly assess in the future.

22. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neuromuscular Scenario: (POST) A 37 year old woman, previously healthy, presents to your office complaining of progressive muscle weakness and intermittent double vision. Omission: The resident does not stabilize the elbow joint when checking triceps strength. 1.Pass 2.Fail Historical Pass Rate: 96.2% (n=104) Talking points: Pass: This is a good teaching point, but doesn’t affect formulation. Fail: Poor exam suggests resident would not be able to properly assess in the future.

23. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neurodegenerative Scenario: (PRE) A 75 year old woman is brought to your office by her daughter for evaluation of mild memory loss. Omission: The resident does not ask about educational background. 1.Pass 2.Fail Historical Pass Rate: 75.7% (n=103) Talking points: Pass: If cognitive function is declining, level of education doesn’t matter. Fail: This is important to give the physician a frame of reference.

24. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neurodegenerative Scenario: (POST) A 75 year old woman is brought to your office by her daughter for evaluation of mild memory loss. Omission: The resident does not ask about educational background. 1.Pass 2.Fail Historical Pass Rate: 75.7% (n=103) Talking points: Pass: If cognitive function is declining, level of education doesn’t matter. Fail: This is important to give the physician a frame of reference.

25. © 2011 AMERICAN ACADEMY OF NEUROLOGY Ambulatory Scenario: (PRE) A 26 year old pregnant woman presents with a 3 week history of bilateral hand paresthesias. Omission: The resident does not check lower extremity reflexes. 1.Pass 2.Fail Historical Pass Rate: 15.8% (n=101) Talking points: Pass: This is probably carpal tunnel syndrome. Fail: Although unlikely, a cervical myelopathy needs to be excluded.

26. © 2011 AMERICAN ACADEMY OF NEUROLOGY Ambulatory Scenario: (POST) A 26 year old pregnant woman presents with a 3 week history of bilateral hand paresthesias. Omission: The resident does not check lower extremity reflexes. 1.Pass 2.Fail Historical Pass Rate: 15.8% (n=101) Talking points: Pass: This is probably carpal tunnel syndrome. Fail: Although unlikely, a cervical myelopathy needs to be excluded.

27. © 2011 AMERICAN ACADEMY OF NEUROLOGY Critical Care Scenario: (PRE) A 75 year old woman presents to the emergency department with sudden onset of right sided weakness and slurred speech. Omission: The resident does not check for a Babinski sign. 1.Pass 2.Fail Historical Pass Rate: 85.9% (n=78) Talking points: Pass: Not part of the NIH stroke scale. Fail: This is necessary to localize the lesion.

28. © 2011 AMERICAN ACADEMY OF NEUROLOGY Critical Care Scenario: (POST) A 75 year old woman presents to the emergency department with sudden onset of right sided weakness and slurred speech. Omission: The resident does not check for a Babinski sign. 1.Pass 2.Fail Historical Pass Rate: 85.9% (n=78) Talking points: Pass: Not part of the NIH stroke scale. Fail: This is necessary to localize the lesion.

29. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neuromuscular Scenario: (PRE) Hospital consultation to see a 21 year old man with a history of progressive ascending weakness. Omission: The resident does not ask about recent vaccinations. 1.Pass 2.Fail Historical Pass Rate: 87.2% (n=78) Talking points: Pass: Wouldn’t affect diagnosis or management. Fail: A recent vaccine could be an immunologic trigger for this.

30. © 2011 AMERICAN ACADEMY OF NEUROLOGY Neuromuscular Scenario: (POST) Hospital consultation to see a 21 year old man with a history of progressive ascending weakness. Omission: The resident does not ask about recent vaccinations. 1.Pass 2.Fail Historical Pass Rate: 87.2% (n=78) Talking points: Pass: Wouldn’t affect diagnosis or management. Fail: A recent vaccine could be an immunologic trigger for this.

31. © 2011 AMERICAN ACADEMY OF NEUROLOGY Pediatrics Scenario: (PRE) A 12 year old boy presents with repetitive movements of one hand. Omission: The resident does not take a medication history. 1.Pass 2.Fail Historical Pass Rate: 8.1% (n=74) Talking points: Pass: Not likely to be relevant in a child. Fail: Medication-induced movement disorder is on the differential.

32. © 2011 AMERICAN ACADEMY OF NEUROLOGY Pediatrics Scenario: (POST) A 12 year old boy presents with repetitive movements of one hand. Omission: The resident does not take a medication history. 1.Pass 2.Fail Historical Pass Rate: 8.1% (n=74) Talking points: Pass: Not likely to be relevant in a child. Fail: Medication-induced movement disorder is on the differential.

33. © 2011 AMERICAN ACADEMY OF NEUROLOGY Ambulatory Scenario: (PRE) A patient presents with chronic, episodic vertigo. Omission: The resident does not ask about hearing loss. 1.Pass 2.Fail Historical Pass Rate: 18.2% (n=77) Talking points: Pass: Wouldn’t affect work up or treatment. Fail: Vestibular tumor or Ménière’s disease are considerations

34. © 2011 AMERICAN ACADEMY OF NEUROLOGY Ambulatory Scenario: (POST) A patient presents with chronic, episodic vertigo. Omission: The resident does not ask about hearing loss. 1.Pass 2.Fail Historical Pass Rate: 18.2% (n=77) Talking points: Pass: Wouldn’t affect work up or treatment. Fail: Vestibular tumor or Ménière’s disease are considerations

35. © 2011 AMERICAN ACADEMY OF NEUROLOGY Session complete

36. © 2011 AMERICAN ACADEMY OF NEUROLOGY How confident are you that you understand the ABPN criteria for determining whether a resident's performance on an NEX merits a passing grade? 1.1 – not at all confident 2.2 3.3 4.4 5.5 6.6 7.7 8.8 9.9 10.10 – Extremely confident

37. © 2011 AMERICAN ACADEMY OF NEUROLOGY Developed by the AAN Clinical Skills Examination Training Workgroup Zach London, MD Joe Kass, MD Vicki Shanker, MD Sonja Potrebic, MD, PhD Yousef Mohammed, MD Gauri Pawar, MD Rachel Ditrapani, MD Ryan Walsh, MD