1. Standardization of Pharmacist Competency: The 2009 National Pharmacy Practice Survey and NAPLEX Standard Setting
NABP President, William T. Winsley, MS, RPh
NABP Competency Assessment Senior Manager Maria Boyle, MS, RPh

2. North American Pharmacist Licensure Examination® (NAPLEX®)
Purpose:
To assess a candidate’s competence, knowledge, skills, and abilities to practice pharmacy

3. Licensing Examinations
Measure skills directly related to the purpose of licensing (protection of the public)
Content domain should be clearly defined and justified in terms of its importance in a given profession
A survey of practitioners provides the basis for defining the balance of content within an examination
Serves as one criteria to make decisions regarding professional practice
Professional test developers follow standardized practices to ensure that tests measure what they purport to measure

4. 2009 NABP National Pharmacy Practice Survey

5. Methodology
Suggested updates to the competency statements are reviewed by the NABP Advisory Committee on Examinations and the Executive Committee
Once the updated competency statements are approved, they are used to construct a practice analysis survey
In the 2009 update, recommended changes included the addition of pharmacoeconomics to three competency statements as it relates to clinical care and pharmaceutical outcomes
The survey is piloted among a subpopulation of pharmacists before being launched as a large-scale survey

6. Methodology
NAPLEX competency statements are continually evaluated by subject matter experts (SMEs)
All SMEs are practicing pharmacists and/or academicians
An advisory committee of SMEs oversees the process of the competency statements review and the practice analysis
SMEs review appropriate literature and provide perspectives as incumbents to gather information to support the content domain description

7. 2009 Sampling Population NABP pharmacist database
National Association of Pharmacy Regulatory Authorities (NAPRA)
Pharmacy Practice faculty
APPE Preceptors
AACP assisted NABP:
by identifying a sample of pharmacy practice faculty to solicit as academic participants, and
by forwarding an invitation to participate to the Directors of experiential education at colleges of pharmacy to assist in targeting experiential preceptors
82.6% of US respondents indicated “preceptor experience” in the last five years

8. Survey Online survey was available from April to mid-July 2009
Respondents were asked to rate each competency statement on two scales: Criticality and Frequency
A five-point rating scale was used: 1 (low) to 5 (high)

9. Survey Rating Scales Criticality Frequency
1 Not serious (e.g. has no effect)
1 Very rarely (e.g. monthly or less)
2 Minimally serious (e.g. causes inconvenience)
2 Rarely (e.g. weekly)
3 Moderately serious (e.g. hinders therapeutic
progress or may endanger public health and
safety)
3 Occasionally (e.g. daily)
4 Highly serious (e.g. worsens the patient’s
condition or is likely to endanger public
health and safety)
4 Often (e.g. hourly)
5 Critically serious (e.g. is life threatening or
will definitely endanger public health and
5 Very often (e.g. many times per hour)

10. Examples of Competency Statements
Identify and assess patient information including medication, laboratory, and disease state histories
Identify specific uses and indications for drug products and recommend drugs of choice for specific diseases or medical conditions
See the NAPLEX/MPJE registration bulletin at

11. Demographics of Respondents

12. Total Respondents ,958
United States ,244
Canada

13. Year of Licensure Prior to 1980 20.8% 1980 to 1989 21.5% 1990 to 1999
23.6%
2000 to 2009
34.1%
Total
100%

14. Professional Degree 48% BS/BPharm 16% Earned both professional degrees
34% PharmD
48% BS/BPharm
16% Earned both professional degrees
2% “No response”
27% Reported completing residency

15. Primary Practice Setting
Academia
263
8.9%
Community
1,260
42.6%
HMO 50
1.7%
Home Health Care 19
0.6%
Hospital
1,211
40.9%
Mail Service 49
Other
106
3.6%
Total
2,958
100.0%

16. American Indian/Alaskan Native Asian/Pacific Islander
Ethnicity and Gender
American Indian/Alaskan Native
0.5%
Asian/Pacific Islander
9.1%
Black
3.1%
Hispanic
2.1%
White
77.4%
Other
2.4%
Missing
5.3%
Female
53%
Male
43.4%
Missing
3.6%

17. Analysis
Survey data were analyzed to determine measures on items and to translate those measures into content weights for the test blueprint
Proportion of weights based on criticality vs frequency were made
The NAPLEX Review Committee and the Advisory Committee on Examinations reviewed the outcomes and made recommendations to the Executive Committee

18. NAPLEX Blueprint Changes
Area 1: Assess Pharmacotherapy to Assure Safe and Effective Therapeutic Outcomes
Increased from 54% to 56% of examination (equates to three additional items delivered)
Competency 1.2 now includes pharmacoeconomic factors as they relate to the identification and selection of pharmacotherapeutic agents
Competency 1.3 now includes the evaluation and management of drug regimens to enhance safe, effective, and economic patient outcomes

19. NAPLEX Blueprint Changes, cont.
Area 2: Assess Safe and Accurate Preparation and Dispensing of Medications
Decreased from 35% to 33% of examination
Area 3: Assess, Recommend, and Provide Health Care Information that Promotes Public Health
Competency now includes the evaluation of pharmacoeconomic data as it applies to patient care

20. What’s New with the NAPLEX?
New competency statements and blueprint went into effect March 1, 2010
A new passing standard was set
Variety of item types which include selected-response and constructed response test items
Enforcement of score invalidation in cases regarding candidate misconduct

21. NAPLEX Standard Setting

22. Establishing a Cut-Score
Critical test development issue, particularly in high-stakes, criterion-based testing
Legitimacy of the standard setting process is a significant component of validity evidence
What degree of mastery over the content domain (knowledge, skills and abilities) is needed to establish competency?

23. Overview
Panelists (NAPLEX Review Committee, practitioners, academicians, preceptors) were recruited for a two day workshop
NABP provided training on test development and standard setting
Practice examples, group discussions, and opportunities to make practice ratings were provided

24. Standard Setting Session
Panelists were asked to conceptualize the knowledge, skills, and abilities necessary for entry-level practice
Key considerations focused on public health and safety

25. NAPLEX Standard Setting
Rater training
Standard setting process
Expectations
How data will be used
Defining a qualified candidate
Characteristics of a sufficiently knowledgeable yet minimally competent candidate (SKYMC) were discussed
Item ratings
Judges consider, answer, and rate real NAPLEX items to make judgments on what a SKYMC candidate would get correct

26. NAPLEX Standard Setting
Preliminary statistical analysis of all of the data collected was performed
Feedback and impact data were reported to the panelists
Rating process was repeated and resulting data analyzed
Considerations for inter-rater reliability
Considerations for subjectivity regarding pass rates
Discussion of outcomes among judges

27. Policy and Recommendations
Ranges of potential cut-scores were computed; an impact analysis was conducted
The NAPLEX Review Committee and the Advisory Committee on Examinations analyzed outcomes and impact data
A recommendation was made to the Executive Committee for a final decision

28. Program Development

29. The Pharmacist Assessment for Remediation and Evaluation (PARE)
NABP Executive Committee recognized the need for an objective assessment to address situations of questionable pharmacist competency and remediation
The purpose of the assessment is to provide a multidimensional assessment for the boards of pharmacy that can be utilized as a contributory factor in making decisions about conditional pharmacist practice issues

30. The Pharmacist Assessment for Remediation and Evaluation (PARE)
Intended for experienced practitioners
Content domains:
Clinical pharmacy
Medication safety management
Professional ethics
The same industry standards applied to all NABP examination and assessment programs shall be implemented
Expectation for a 2011 launch

31. Questions? Thank you!