1. TRUST ME, I’M A (CERTIFIED) DOCTOR
INNOVATIONS AND IMPROVEMENTS IN MAINTENANCE OF CERTIFICATION
MARSHALL L. LAND, JR., MD
R.J. MCKAY, JR., MD GREEN & GOLD PROFESSOR OF PEDIATRICS
UNIVERSITY OF VERMONT COLLEGE OF MEDICINE
AMERICAN BOARD OF PEDIATRICS, ADVISOR FOR DIPLOMATE OUTREACH AND STRATEGIC PLANNING
May 3, 2019

2. Disclosures
I have no relevant financial relationships with the manufacturer(s) of any commercial product(s) and/or provider(s) of commercial services discussed in this CME activity.
I do not intend to discuss an unapproved/investigative use of a commercial product/device in my presentation.

3. Objectives Brief history of “Why” MOC
Overview of recent changes in MOC
Part 2: Lifelong Learning and Self Assessment
Part 3: Cognitive Expertise
Part 4: Improvement In Practice
Fees
Listen for your ideas!

4. Why do I have to do MOC?

5. The ABP Mission Began 1933 Mission to the public
To certify pediatricians based on standards of excellence that lead to high quality care. The ABP certification provides assurance to the public that a pediatrician fulfills the continuous evaluation requirements that encompass the six core competencies.

6. The Evolution of Board Certification
1934
Permanent
certification

7. The Evolution of Board Certification
Re-test
every 7 years
Point system,
test every 10 years
1934
1988
2003
2010
2019
4 part program,
test every 10 years
Permanent
certification
MOCA-Peds

8. Why these changes?

9. Explosion of knowledge
PEDIATRICS
1000 MORE ARTICLES PER YEAR BEING SUBMITTED THAN 7 YEARS AGO

10. Explosion of knowledge: weekly/daily

11. Recognition of Medical Errors
IOM: “To Err is Human” & “Crossing the Quality Chasm”
Medical errors  100,000 deaths per year (now est. 400,000)
Hospitalized children medical error rate 1.81–2.96/100 discharges
25% incidence of medication errors

12. Improvements and innovations
Continuing to try to make this process better
LESS BURDENSOME
LESS TIME CONSUMING
MORE MEANINGFUL
MORE RELEVANT

13. Evolution of Board Certification
Single test of knowledge 
Periodic test of knowledge 
More continuous measurement of knowledge
acquisition and quality improvement

14. Evolution of Board Certification
MOC is about “staying current”, but it
is also about attempting to improve the
care we are giving, even if that care is
at a high level

15. 4 Part MOC Professional Standing (Part 1)
Lifelong Learning and Self Assessment (Part 2)
Cognitive Expertise (Part 3)
Improvement In Practice (Part 4)

16. Part 2 WHAT’S NEW? “I already keep up……
Lifelong Learning and Self Assessment
“I already keep up……
Why can’t CME count for Part 2?”
WHAT’S NEW?

17. Part 2 Changes MOC points for CME! ACCME Collaboration
AAP Chapter meetings, NCE, Grand Rounds, etc
As of 12/11/18:
Total number MOC Part 2 activities registered: 4,153
2018 number of MOC points awarded: 123,944.25

18. Part 2 Changes New Self Assessments: Question of the Week Enhancements
Professionalism in Global Health
Care of a Transgender Child
Immigration Health Care
Updating Patient Safety
A Journey of Improvement: The Basics of QI
Question of the Week Enhancements

19. Question of the Week Painless and Fun
How good is the power of suggestion to manage a child with functional abdominal pain?

20. To use or not to use ibuprofen post tonsillectomy:
Question of the Week
Painless and Fun
To use or not to use ibuprofen post tonsillectomy:
That is the question

21. Question of the Week Painless and Fun
What’s the best way to fight off a recurrent case of
the “runs” after a bout of c. diff.?

22. Part 3
Cognitive expertise
“That exam every 10 years is not a learning experience……
I hate the Testing Center”
WHAT’S NEW?

23. Maintenance of Certification Assessment for Pediatrics
Assessment FOR Learning

24. MOCA-Peds Overview Continuous assessment tool
Questions sent quarterly; flexibility in answering
Delivered via web or mobile device at home or office
Focuses on assessment and learning
Fulfill the MOC Part 3 requirement/ also Part 2 points

25. Question Overview
Application of fundamental knowledge of everyday pediatrics Short clinical vignettes, mimic real world situations Multiple-choice, single-best answer 5 minutes per question, may use resources (eg, books, internet) Instant feedback, clinical rationale, and references

30. General Phase-in Plan Pilot 2017 and 2018 Now live 2019!
With changes guided by diplomate input – clinical practice guidelines
Now live 2019!
Subspecialties will phase in over time ( )
Enter MOCA-Peds at start of the 5-year MOC
cycle during which next exam due

31. Pediatric Subspecialties
2019
Child Abuse Pediatrics
Pediatric Gastroenterology
Pediatric Infectious Diseases
2020
Developmental-Behavioral Pediatrics
Neonatal-Perinatal Medicine
Pediatric Nephrology
Pediatric Pulmonology
2021
Pediatric Critical Care Medicine
Pediatric Endocrinology
Pediatric Hospital Medicine
Pediatric Rheumatology
2022
Adolescent Medicine
Pediatric Cardiology
Pediatric Emergency Medicine
Pediatric Hematology-Oncology

32. Pilot Participation 2017: 5,081 of 6,814 (74.6%)
Subspecialists: 81.1% registered

33. Participant Experience: Using The Platform
Any technical issues with the MOCA-Peds system?
Quarter 1
Quarter 2
Quarter 3
Quarter 4
Yes
16.1%
10.8%
7.5%
8.6% No
83.9%
89.2%
92.5%
91.4%
Total
100%
100.0%
Sample Size: Quarter 1 = 4,181, Quarter 2 = 3,772; Quarter 3 = 3,761; Quarter 4 = 4,016

34. Overall Acceptability: General Pediatrics
Which Part 3 method do you expect to use to maintain your General Pediatric (GP) certification?
Frequency
Percent
MOCA-Peds GP (the new online format)
2,748
96.2%
The GP proctored exam at the testing center 40
1.4%
I plan to stop maintaining my GP certification (for any reason) 68
2.4%
Total
2,856
100.0%

35. Overall Acceptability: Subspecialists
Which Part 3 method do you expect to use to maintain your subspecialty certification(s)?
Frequency
Percent
MOCA-Peds (the new online format), once available for my subspecialty
686
95.3%
My subspecialty proctored exam at the testing center 15
2.1%
I plan to stop maintaining my subspecialty certification (for any reason) 19
2.6%
Total
720
100.0%

36. Agreement about the MOCA-Peds model
Overall Satisfaction
Agreement about the MOCA-Peds model
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Both SA/A
20 questions per quarter will be feasible for my schedule
3.1%
6.7%
7.2%
49.9%
33.0%
82.9%
A feasible method for subspecialists to keep up-to-date with general pediatric knowledge 
2.0%
2.9%
22.3%
48.1%
24.6%
72.7%

37. Agreement about the MOCA-Peds model
Overall Satisfaction
Comments
“THANK YOU for listening to pediatricians…”
Agreement about the MOCA-Peds model
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Both SA/A
20 questions per quarter will be feasible for my schedule
3.1%
6.7%
7.2%
49.9%
33.0%
82.9%
A feasible method for subspecialists to keep up-to-date with general pediatric knowledge 
2.0%
2.9%
22.3%
48.1%
24.6%
72.7%
“The MOCA-Peds pilot is a great improvement over the secure exam! Thank you so much for creating this new form of evaluation for pediatricians…so much less stressful, I am learning a lot while I take the exam and find the exam useful for my daily pediatric practice.”

38. Overall Satisfaction Agreement about the MOCA-Peds model.
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Both SA/A
Overall satisfied with the MOCA-Peds model as a replacement for the previous Part 3 testing
2.1%
2.9%
6.8%
43.7%
44.5%
88.2%
Less anxious about taking an exam using the online MOCA-Peds system compared to a proctored exam at a secure testing center
0.9%
2.7%
7.0%
32.6%
56.9%
89.5%

39. Overall Satisfaction
What one enhancement would you recommend that the ABP make to improve MOCA-Peds?
Agreement about the MOCA-Peds model.
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Both SA/A
Overall satisfied with the MOCA-Peds model as a replacement for the previous Part 3 testing
2.1%
2.9%
6.8%
43.7%
44.5%
88.2%
Less anxious about taking an exam using the online MOCA-Peds system compared to a proctored exam at a secure testing center
0.9%
2.7%
7.0%
32.6%
56.9%
89.5%
“Would eliminate the 5 minute limit per question or lengthen significantly”
“A continuous exam lasting for 4 out of 5 years is excessive”
______
New knowledge: articles and guidelines

40. Participant Experience: Time Spent
For the entire MOCA-Peds pilot in 2017 how much time spent answering questions (including time spent studying, looking at reference material, taking questions, and reviewing feedback)?
Frequency
Percent
< 2 hours
144
5.0%
2 to < 5 hours
618
21.6%
5 to < 10 hours
756
26.5%
10 to < 20 hours
648
22.7%
20 to < 40 hours
450
15.8%
> 40 hours
240
8.4%
Total
2,856
100.0%
< 2 minutes/question
38 mins for 20 questions

41. Participant Experience: Resource Use
Most common resources:
UpToDate: 57.1%
Government websites (eg, CDC, NIH, NICHD): 33.7%
Search engines (eg Google, Yahoo): 58.6%
Professional sites (eg, AAP) 26.7%
Textbook:18.8%
PREP: 8.3%
How often did you use external resources?
Quarter 1
Quarter 2
Quarter 3
Quarter 4
No Resource Use
19.2%
17.4%
15.8%
14.7%
With 1 to 5 questions
42.6%
41.3%
39.9%
39.2%
With 6 to 10 questions
22.2%
25.0%
24.8%
With more than 10 questions
16.0%
19.0%
19.4%
21.4%
Total
100%

42. Participant Experience: Resource Use
Most common resources:
UpToDate: 57.1%
Government websites (eg, CDC, NIH, NICHD): 33.7%
Search engines (eg Google, Yahoo): 58.6%
Professional sites (eg, AAP) 26.7%
Textbook:18.8%
PREP: 8.3%
How often did you use external resources?
Quarter 1
Quarter 2
Quarter 3
Quarter 4
No Resource Use
19.2%
17.4%
15.8%
14.7%
With 1 to 5 questions
42.6%
41.3%
39.9%
39.2%
With 6 to 10 questions
22.2%
25.0%
24.8%
With more than 10 questions
16.0%
19.0%
19.4%
21.4%
Total
100%

43. Did learning occur? Yes 97.6%
Most significant practice changes you made as a result of
participation in the 2017 pilot?
“More diligent with antibiotic mgx in otitis media. In general am practicing more evidence based medical care with confidence.”
Learn, refresh, or enhance medical knowledge based on using MOCA-Peds in the 2017 pilot?
Yes
97.6%
“Started to pay more attention to features of autism.”
Able to apply any of what you learned to your clinical practice?
“I identified a Kawasaki patient based on review; ..that is just one instance “
Yes, I have already
1,727
62.0%
No, but I plan to moving forward 
468
16.8%
No, because my practice area is not general pediatrics focused 
451
16.2%
No, for another reason (please explain)
141
5.0%
Total
2,787
100.0%
“Follow up on High BP”
“Became aware of my deficiency in
acute drug intoxication”

45. Comments
“I'm a neonatologist with a follow up clinic for babies discharged from the NICU. In many areas I realized that some of my practice in the clinic might have been dated. I now have far more frequent discussions with both my general and subspecialty peds colleagues regarding the outpatient care of my former patients seen in their clinics.
Having to go read up on the topics I got wrong in my answers was also enlightening. I became more facile in this process as the year went by and in fact found myself reading far more general pediatrics than I had anticipated at the beginning of the year. I truly believe that this should be the way of the future to ensure practitioners keep up to date.”

46. “I have really enjoyed this, something I would never have thought I would say about the certification exam. I am both learning and meeting my requirement for certification, which is reflective of real practice!”
“It is AMAZING. I love it. Honestly, I am learning a lot by doing it, and it’s very easy to make time to learn from it. This is a far better learning tool than a proctored exam.”
“It is a great way for a subspecialist to keep up to date on general pediatrics. I would do it again. I think I will retain the general pediatrics information and actually be able to improve the clinical care I provide my subspecialty patients better…”

47. + MOCA-Peds 2019 CORE KNOWLEDGE NEW KNOWLEDGE 45 learning objectives
1 question per learning objective
45 total questions +
15 repeat questions
Based on confidence/relevance ratings and questions missed
NEW KNOWLEDGE
Up to 4 articles/guidelines
Up to 2 questions per article
Questions appear after Q2 begins
Up to 4 “time-sensitive” questions
Quickly developed and delivered questions based on current events
Total Number of Questions
Annually
Quarterly

48. Learning objectives 2019
Diagnose and manage a neonate with an abnormal head size or shape.
Diagnose and manage neonatal abstinence syndrome.
Evaluate a child with hemoptysis.
Evaluate a hypotonic infant.
Evaluate a patient with leukocoria.
Evaluate an adolescent with a testicular mass.
Evaluate and manage a child at risk for type 2 diabetes.
Evaluate and manage a child with gross hematuria.
Evaluate and manage a prepubertal girl with vaginal discharge.
Evaluate and manage back pain in children.
Evaluate and manage neonatal jaundice.
Evaluate suicide risk in an adolescent.

49. Learning objectives 2019
Diagnose and manage a neonate with an abnormal head size or shape.
Diagnose and manage neonatal abstinence syndrome.
Evaluate a child with hemoptysis.
Evaluate a hypotonic infant.
Evaluate a patient with leukocoria.
Evaluate an adolescent with a testicular mass.
Evaluate and manage a child at risk for type 2 diabetes.
Evaluate and manage a child with gross hematuria.
Evaluate and manage a prepubertal girl with vaginal discharge.
Evaluate and manage back pain in children.
Evaluate and manage neonatal jaundice.
Evaluate suicide risk in an adolescent.

50. Learning objectives 2019
Diagnose and manage a neonate with an abnormal head size or shape.
Diagnose and manage neonatal abstinence syndrome.
Evaluate a child with hemoptysis.
Evaluate a hypotonic infant.
Evaluate a patient with leukocoria.
Evaluate an adolescent with a testicular mass.
Evaluate and manage a child at risk for type 2 diabetes.
Evaluate and manage a child with gross hematuria.
Evaluate and manage a prepubertal girl with vaginal discharge.
Evaluate and manage back pain in children.
Evaluate and manage neonatal jaundice.
Evaluate suicide risk in an adolescent.

51. Learning objectives 2019
Diagnose and manage a neonate with an abnormal head size or shape.
Diagnose and manage neonatal abstinence syndrome.
Evaluate a child with hemoptysis.
Evaluate a hypotonic infant.
Evaluate a patient with leukocoria.
Evaluate an adolescent with a testicular mass.
Evaluate and manage a child at risk for type 2 diabetes.
Evaluate and manage a child with gross hematuria.
Evaluate and manage a prepubertal girl with vaginal discharge.
Evaluate and manage back pain in children.
Evaluate and manage neonatal jaundice.
Evaluate suicide risk in an adolescent.

52. New knowledge 2019 For General Pediatrics
Cheung AH, Zuckerbrot RA, Jensen PS, et al. Guidelines for adolescent depression in primary care
(GLAD-PC): Part II. Treatment and ongoing management. Pediatrics. 2018; 141 (3): e
Flynn JT, Kaelber DC, Baker-Smith CM, et al. Clinical practice guideline for screening and
management of high blood pressure in children and adolescents. Pediatrics. 2017; 140(3): e
Greenhawt M, Turner PJ, Kelso JM. Administration of influenza vaccines to egg allergic recipients:
A practice parameter update Ann Allergy Asthma Immunol ; 120(1):

53. New knowledge 2019 For General Pediatrics
Cheung AH, Zuckerbrot RA, Jensen PS, et al. Guidelines for adolescent depression in primary care
(GLAD-PC): Part II. Treatment and ongoing management. Pediatrics. 2018; 141 (3): e
Flynn JT, Kaelber DC, Baker-Smith CM, et al. Clinical practice guideline for screening and
management of high blood pressure in children and adolescents. Pediatrics. 2017; 140(3): e
Greenhawt M, Turner PJ, Kelso JM. Administration of influenza vaccines to egg allergic recipients:
A practice parameter update Ann Allergy Asthma Immunol ; 120(1):

54. New knowledge 2019 For General Pediatrics
Cheung AH, Zuckerbrot RA, Jensen PS, et al. Guidelines for adolescent depression in primary care
(GLAD-PC): Part II. Treatment and ongoing management. Pediatrics. 2018; 141 (3): e
Flynn JT, Kaelber DC, Baker-Smith CM, et al. Clinical practice guideline for screening and
management of high blood pressure in children and adolescents. Pediatrics. 2017; 140(3): e
Greenhawt M, Turner PJ, Kelso JM. Administration of influenza vaccines to egg allergic recipients:
A practice parameter update Ann Allergy Asthma Immunol ; 120(1):

55. New knowledge 2019 For General Pediatrics
Cheung AH, Zuckerbrot RA, Jensen PS, et al. Guidelines for adolescent depression in primary care
(GLAD-PC): Part II. Treatment and ongoing management. Pediatrics. 2018; 141 (3): e
Flynn JT, Kaelber DC, Baker-Smith CM, et al. Clinical practice guideline for screening and
management of high blood pressure in children and adolescents. Pediatrics. 2017; 140(3): e
Greenhawt M, Turner PJ, Kelso JM. Administration of influenza vaccines to egg allergic recipients:
A practice parameter update Ann Allergy Asthma Immunol ; 120(1):

56. Scoring
Life happens……… Drop lowest 4 quarters of performance at end of year 4

57. Take the proctored exam
Five-Year MOC Cycle
What if I did not pass at the end of Year 4, or if I chose not to participate in MOCA-Peds?
Year 1
Year 2
Year 3
Year 4
≤ 72 questions
≤ 72 questions
≤ 72 questions
≤ 72 questions
Year 5
Take the proctored exam

58. What if I am meeting the passing standard at the end of Year 4?
Five-Year MOC Cycle
What if I am meeting the passing standard at the end of Year 4?
Year 1
Year 2
Year 3
Year 4
≤ 72 questions
≤ 72 questions
≤ 72 questions
≤ 72 questions
Because of the 4 lowest quarters rule, those performing well enough may be able to stop at end of Year 3*
Year 5
Options:
Take a break until the next 5-year MOC cycle
Continue learning and receive MOC Part 2 credit
*Diplomates may choose to continue participating for MOC Part 2 credit

59. Passing Rate
Based on 2017 pilot, passing rate will be over 95%

60. Part 3 Options MOCA-Peds Proctored Exam Default with enrollment
No additional fees
Earn Part 2 points
Proctored Exam
Every 5-years at proctored site
Additional fee to cover cost of seat fee and processing
No Part 2 credit

61. Ongoing Improvement
Continue to obtain feedback from participants about MOCA-Peds
What works? What does not?
What new issues/concerns are identified by diplomates?
Annual surveys, focus groups, user groups, interviews
Evaluate subspecialty-specific issues as each is implemented
Information to the ABP to support continuous improvement of MOCA-Peds

65. Part 4 WHAT’S NEW? “There’s no evidence this stuff improves care…
Improvement in practice
“There’s no evidence this stuff improves care…
It’s a waste of time”
WHAT’S NEW?

66. Improvement in practice
The ABP is making every attempt to encourage people to engage in quality improvement activities which will give credit for work already being done in their offices
How am I doing?
What might I do even better?
How might I do that?
Check it out.

67. Expanded Part 4 Pathways
Significant increase in those getting credit for QI projects initiated by:
Individual, team, network, or institution (not online modules)
PCMH, small or large group QI projects, institutional QI leadership
Portfolios
Streamlined application process
New: Multi-activity PIMs

68. “QI shown to be an
effective tool…to
bridge gaps in care.”
“ensure…every child
gets the right care
every time.”

69. AAP Making MOC Better AAP has portfolio sponsorship status
Allows Academy to evaluate QI activities and approve
projects internally for MOC Part 4 credit
Collaborative learning
Multiple practices/sites work together (e.g., CQN, VIPN)
***AAP Chapter meetings***

70. AAP Making MOC Better PediaLink QI Web-based tool for small groups
Projects proposed/led by volunteer pediatricians
Education in QI for pediatric practices (EQIPP)
Online courses for self-directed learning
Poster/platform presentations at NCE
Support sessions on MOC at NCE
Understanding MOC: Requirements/Resources
Pedialink QI

71. WHAT’S NEW? FEES $1304 is way too much to pay…
What if I stop doing this in 2 years?
WHAT’S NEW?

72. FEES
Annual payment option
$1304 for 5 year cycle
$275 annual fee

73. Summary We know you’re already doing great work as pediatricians.
However, as a profession, we can show that
you are staying current and making an effort to improve your (great) practice.
Maintaining our certification in a meaningful and relevant way - including
our collaborative efforts to improve that process -
can show that commitment to improving the health care of children.

74. What are YOUR creative ideas?