2. Test Blueprinting for Standard and Concept-Based Curricula
ATI Education Summit 2017
Lois Churchill, MN, RN
Dianna Johnston, DNP, RN, NE-BC

3. Objectives Explain the benefits of using test blueprinting.
List the steps in test blueprinting development.
Explain how to set the specifications for the elements at the course and program levels.
Explain how to plan for remediation activities after conducting an item analysis.

4. Purpose of Test Blueprints
Tool used to determine parameters of a test prior to its development.
Program blueprints outline by semester the percent of each element on tests delivered during the semester.
Course blueprints outline the percent of each element on course tests.

5. Steps in Test Blueprint Development
Identify learning elements to be measured
Determine percent of items for each difficulty level (Bloom/Anderson & Krathwohl)
NCLEX® Content Area
Nursing Process
QSEN
BSN Essentials
Massachusetts Nurse of the Future
Determine the weight to be assigned to selected elements
Program
Course
Conduct item analysis 55

6. Percent of Items for Each Cognitive Level
Bloom’s Taxonomy - Knowledge, Comprehension,
Application, Analysis, Synthesis, Evaluation
(Bloom, 1956)
Anderson & Krathwohl’s Taxonomy - Remember, Understand, Apply, Analyze, Evaluate, Create (Anderson & Krathwohl, 2001)
Foundational Level
Remember
Understanding
Critical Thinking / Clinical Judgment
Apply
Analyze

7. RN NCLEX® Test Plan 2016
Updated every 3 years Based on practice analysis (First 12 months of practice)
Client Needs
Percentage of Items from Each Category/Subcategory
Safe and Effective Care Environment
Management of Care
17-23%
Safety and Infection Control
9-15%
Health Promotion and Maintenance
6-12%
Psychosocial Integrity
Physiological Integrity
Basic Care and Comfort
Pharmacological Therapies
12-18%
Reduction of Risk Potential
Physiological Adaptation
11-17%

8. PN NCLEX® Test Plan 2017
Updated every 3 years Based on practice analysis (First 12 months of practice)
Client Needs
Percentage of Items from Each Category/Subcategory
Safe and Effective Care Environment
Coordinated Care
18-24%
Safety and Infection Control
10-16%
Health Promotion and Maintenance
6-12%
Psychosocial Integrity
9-15%
Physiological Integrity
Basic Care and Comfort
7-13%
Pharmacological Therapies
Reduction of Risk Potential
Physiological Adaptation

9. Obtaining Information about NCLEX ® Test Plans
The detailed test plan for item writers/item reviewers and nurse educators can be accessed at the web sites below:
PN 2017 Detailed Test Plan

10. Level of Difficulty Bloom’s Taxonomy
Knowledge
Comprehension
Application
Analysis
Anderson & Krathwohl’s Taxonomy for Learning
Remember
Understand
Apply
Analyze
Evaluate
Create
Foundational Thinking vs. Critical Thinking

11. Nursing Process

12. QSEN Competencies Quality and Safety Education for Nurses Competencies
Program with goal of preparing future nurses KSA necessary to improve quality of care- 6 competencies
Massachusetts Nurse of the Future integrates the QSEN competencies and IOM recommendations
Quality and Safety Education for Nurses was an initiative to prepare nurses with the competencies needed to provide quality care. Robert Wood Johnson Foundation supported this nationwide project. This project incorporates the institute of medicine competencies. The AACN includes the IOM competencies of the BSN Essentials. Billings and Halstead p 28. IOM competencies: patient centered care, collaboration with interdisciplinary teams, awareness of quality standards, knowledge and use of evidence-based practice and informatics and technology. Billings & Halstead p 336
Nurses play key role in protecting client safety and proving quality care. AACN essential core competencies: critical thinking, health care systems and policies, communication, illness and disease management, ethics, information and health care technologies. Billings and Halstead p 99

13. AACN Essentials of Baccalaureate Education for Professional Nursing Practice
American Association of Colleges of Nursing An autonomous arm of AACN, the Commission on Collegiate Nursing Education (CCNE) ensures the quality and integrity of baccalaureate graduates
Quality standards

14. AACN BSN Essentials (continued)

15. Massachusetts Nurse of the Future Core Competencies©
Competency
Definition
Patient-Centered Care
The Nurse of the Future will provide individualized care that recognizes patient’s preferences, values and needs and respects the patient or designees as a full partner in providing compassionate, coordinated, appropriate, safe care.
Professionalism
The Nurse of the Future will demonstrate accountability for practicing nursing within established moral, legal, ethical, regulatory and humanistic principles.
Leadership
The Nurse of the Future will influence the behavior of individuals within their environment in a way that will facilitate the establishment and acquisition/achievement of shared goals.
Systems-Based Practice
The Nurse of the Future will demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal quality and value.
Informatics & Technology
The Nurse of the Future will use information and technology to communicate, manage knowledge, mitigate error and support decision-making.
Communication
The Nurse of the Future will interact effectively with patients, families, and colleagues, fostering mutual respect and shared decision-making, to enhance patient satisfaction and health outcomes.
Teamwork & Collaboration
The Nurse of the Future will function effectively within nursing and interdisciplinary teams, fostering open communication, mutual respect, shared decision-making, team learning and development.
Safety
The Nurse of the Future will minimize risk of harm to patients and providers through both individual performance and system effectiveness.
Quality Improvement
The Nurse of the Future will use data to monitor outcomes and care processes, and use improvement methods to design and test changes to continuously improve the quality and safety of healthcare.
Evidence-Based Practice
The Nurse of the Future will identify, evaluate and use the best current evidence coupled with clinical expertise and consideration of patient’ preferences, experience and values to make practice decisions.
the minimum expectations for all nurses as they complete their pre-licensure education
The ten essential competencies, which guide nursing curricula and practice
all the competencies are of equal importance.

16. Foundational Thinking
Ability to remember and understand basic information
Clinical manifestations
Expected reference range of laboratory results
Causes/risk factors for an alteration in health
Expected/side/adverse effects of medication or treatment
Medication purpose/mechanism of action

17. Critical Thinking/Clinical Judgment
Ability to apply knowledge and use higher level critical thinking skills (assess, analyze, and evaluate) when making decisions about clinical problems a client scenario poses.
Define the problem accurately
Make inferences based on data
Identify and examine alternative solutions
Recognize assumptions and faulty reasoning
Develop a plan of action
Prioritize nursing interventions
Perform dosage calculation
Analyze new information
Draw conclusions about a plan’s effectiveness
Revise a plan based on evaluation

18. Sample Program Test Blueprint
Sample Program Test Blueprint
Cognitive Level %
NCLEX © Category %
Nursing Process %
Year
Foundational
Critical Thinking / Clinical Judgment
Management of care
Safety and Infection Control
Health Promotion & Maintenance
Psychosocial Integrity
Basic Care & Comfort
Pharmacological & Parental Therapies
Reduction of Risk Potential
Physiological Adaptation
Assessment
Diagnosis
Planning
Implementation
Evaluation
Remember
Understand
Apply
Analyze
Evaluate
Create
Sem1 10 50 40 5 7 3 15 20 25
Sem2 60 12 11
Semester 1
(15 – 20%)
(45 – 50%)
(40 – 45%)
(17-23%)
(9-15%)
(6-12%)
(12-18%)
(11-17%)
Semester 2
(0 – 5%)
(15-25%)
(65 – 70%)
Avg. 6 13 18

19. Leveling Elements
Establish a range for the percent of items in each element
Avoid using exact numbers
Assign the range of percent based upon
Placement of course in the curriculum
Course objectives
Degree of target difficulty
Place increased weight on higher level processes as students progress through the curriculum
(Oermann and Gaberson, 2014)

20. Test Blueprinting- Traditional vs. Concept-based Curriculum
Traditional Curriculum
Leveled by course
Content is body system focused
Concept-based Curriculum
Leveled through the curriculum across the lifespan
Content is concept-focused
Leveling Elements are the same for either curriculum type

21. Example: Weight of Level of Difficulty/Program Level
Recommended range (weight) for the percent of items in each difficulty level by program level
Example:
(The above percents are for example only and are not recommendations.)

23. Linking Test Items to the Syllabus
Use one of the following to link items on test to syllabus
Course objectives
Unit objectives
Course content

24. Number of Alternate Items
NCSBN has been using alternate item format questions on NCLEX®-RN/PN since 2003.
Alternate items give students the opportunity to demonstrate competence in alternative ways such as calculations, identifying graphics.
Choice of item format should be based on content being tested.
Number of alternate items should depend on the length of the test.
Some experts recommend limiting type of alternate format items to no more that 3 per test.
(Oermann and Gaberson, 2014)

25. Key Components of Item Analysis
Item Difficulty: Is the item too hard or too easy?
Item Discrimination: Who is answering the item correctly?

26. Using Data from Item Analysis to Plan for Content Review- The p value
p value ( also known as difficulty level) is the per cent of students who chose the correct answer. The goal is a p value of 30 to 80% for majority of the items on the test.
For purposes of content review, any item with a p value of 50% or less should be reviewed. (A p value of 50% means that half of the students who took the exam chose the incorrect answer)

27. Using Data from Item Analysis to Plan for Content Review- The Point Biserial Index
The point biserial index is an indication of the quality of an item.
Can range from to It tells us whether the high-ability students or the low ability students choose the key. We want a positive point biserial for the key. That means high ability students chose the key.
Goal is a 0.30 for the key.
For purposes of content review, any item with negative point biserial for the key should be reviewed.

28. Hint Write items as you prepare for class.
Always write rationales for both the key and distractors before you administer the test.

29. Steps in Item Analysis for Content Review
Ask a peer for input about the quality of the item.
Two sets of eyes are better than one.
Peer Review
Can you link it to a course or unit objective? Is it entry-level practice? Can you link it to the NCLEX© test plan?
Can you identify the step of the nursing process?
For concept-based curricula, is the item at the proper level of difficulty inline with your curriculum?
If you have used the item with a prior group of cohorts, did the prior stats meet the acceptable ranges?
Relevance
Is the stem free of extraneous content?
Is the question in the stem clear?
Is there only one correct answer?
Item Quality

30. Content Remediation
Policies on content remediation should be incorporated into the program’s testing policies.
Explore with students either individually or in a group their understanding of the content/concept.
Assign activities for review.
Retest the content/concept if possible.

31. Summary and Final Thoughts
Questions and comments
Plans for moving forward
Aha moments

32. References
American Association of Colleges of Nursing (AACN), 2008 The essentials of baccalaureate education for professional nursing practice. Retrieved from
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Boston, MA: Pearson Education Group.
Billings, D., & Halstead, J. (2016). Teaching in nursing: A guide for faculty (5th ed.). St. Louis, MO: Saunders.
Bloom, B.S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. Handbook I: Cognitive domain. New York, NY: Longmans, Green.

33. References (Continued)
Massachusetts Nurse of the Future Nursing Core Competencies© Retrieved from
National Council of State Boards of Nursing. (2016). NCLEX-RN® Examination: Test plan for the National Council licensure examination for registered nurses. Retrieved from
National Council of State Boards of Nursing. (2017). NCLEX-PN® Examination: Test plan for the National Council licensure examination for nurses. Retrieved from
Oermann, M., & Gaberson, K. (2014). Evaluation and testing in nursing education. (5th ed.). New York, NY: Springer.
Quality and Safety Education for Nurses Institute (2013). Competencies. Retrieved from