High-Stakes Assessment: Using Evidence for Admission Policy Decisions Cheryl Shaffer PhD, RN Associate Dean School of Nursing

Background O First-time licensure pass rates have been used as a quality indicator for nursing program success. O The Accreditation Commission for Education in Nursing (ACEN) require evidence of achievement of the licensure examination pass rates being at or above the national mean.

Background O The National Council of State Boards of Nursing publishes a new NCLEX (Licensure Examination) test plan and passing standard every three years. O The latest version of the NCLEX-RN test plan (April 2013) will remain in effect until March 31, 2016.

The Problem O Nationwide, the competency level for safe and effective entry-level nursing practice has significantly and permanently changed. O As such, the April 2013 passing standard rise was the largest single increase ever in the history of the RN licensure exams.

First time NCLEX Pass Rates National Average School of NursingNew York Average %80.56%84.05% %82.96%87.15% 2013 * 83.04%73.02%78.14% 2014 ** % % 82.34% * 2010 and 2013 test plan ** Only January graduates

Previous Admission Criteria O 3.0 cumulative grade point average for: ENG101, PSY101, BIO130 and BIO132 (and SOC 101 for Advanced Placement Program) O A minimum grade of C must be earned for BIO130 and BIO132 O All prerequisite courses may be retaken only once to achieve a higher grade. The second grades will be used for admission consideration.

Previous Admission Criteria O Admission to the nursing programs has always been competitive O Based on the NCLEX results, it was hypothesized the current criteria for admission was no longer an accurate predictor of program success

Repeated Course Withdrawals O Approximately 25% of admitted students withdraw or repeat their science courses O Incentives to withdraw, opposed to course completion from such prerequisite courses may be attributed to their use as preadmission criteria. O Students understand that final averages are crucial in competitive selections.

Repeated Course Withdrawals O Through regression analysis, repeated attempts in preadmission science courses were found to be a significant predictor of NCLEX-RN outcome in SCCC graduates from 2010 and 2011 (N=335) O There was a significant, negative correlation between the number of science course attempts and passing the NCLEX-RN on the first attempt (r= , p=0.000). O Repeating science course grades were found to be a significant predictor of NCLEX success, accounting for 25 percent of the total variance (F 1, 164 = 5.732, R 2 = 0.254, p = 0.000). Shaffer, C., McCabe, S. (2013). Evaluating the predictive validity of preadmission academic criteria: high stakes assessment. Teaching and Learning in Nursing, 8 (4),

Repeated Course Withdrawals O 82 students did not graduate May 2014 due to withdrawal from a nursing course or achieving a grade less than a C O Of those 82 students, 42 % withdrew 1 or more times from a Science prerequisite

Review of the Literature O Nursing program admission criteria in the literature and other SUNY community colleges and universities on Long Island was reviewed. O The most common admission criterion was administration of the entrance exam, the Assessment Technologies Institute (ATI) Test of Essential Academic Skills (TEAS V). O This result was combined with pre-requisite course grades and the use of a scoring rubric.

The TEAS V O Computerized test, 209 minutes, 150 multiple choice items O Measures students basic or entry-level program readiness in the major content areas of Reading, Mathematics, Science and English/Language usage. O The TEAS V score report will identify students overall academic preparedness necessary to support learning of nursing course content.

The TEAS V O Academic Preparedness Level category of: O Exemplary- high level O Advanced-high level O Proficient-moderate level O Basic- low level O Developmental-very low level

Assessment Project O June 2014, all Fall 2014 (N=225) newly admitted students took the TEAS V post admission O Students individual performance profile can was used to guide additional preparation for the nursing program over the summer.

Results Academic Preparedness At-Risk Area Qualified for Admission Unqualified for Admission ReadingMathematicsScience English Language 72%32%11%20%30%12%

Results- Math Percent of Students Deficient Numbers and Operations Identify decimal placement 64% Compare rational numbers 61% Divide fractions 57% Solve word problems 55% Algebraic Applications Add, subtract, multiply polynomials solve equations or inequalities 62% Measurement Convert quantities from 1 scale to another 62%

Nursing Students Computation Expectations O Read a dosage calculation word problem O Set up dosage calculation in fractional form O Ratio and proportion method O Cross multiplying fractions O Substitute converted units into proportion O Metric system and weight measurement conversions O Round to nearest tenths and hundredths

Results- Reading Percent of Students Deficient Paragraph Comprehension Identify characteristics of different passages61 % Identify topic, main idea, supporting details, themes68 % Analyze and evaluate use of text structure55 %

Results- English/Language Percent of Students Deficient Grammar and Word Meanings in Context Apply subject-verb agreement rules65 % Use of grammar for stylistic effect and enhance clarity60% Use context clues to determine meaning of words61% Use dialogue correctly58% Spelling and Punctuation Capitalization80%

Nursing Students Writing Expectations O Using APA style for reference page for various sources O (textbook, periodicals, internet sources), O Proper use of punctuation or grammar, organization of ideas O Apply evidence in the literature to patient assignments

Results- Science Percent of Students Deficient Life Science Identify parts of a cell 67 % Explain changes in DNA 68 % Explain heredity information 62 % Compare and contrast chromosomes and genes 61 % Physical Science Explain physical and chemical properties 63 % Balance and identify chemical reactions 62 % Distinguish among energies 56 % Scientific Reasoning Recognize and analyze alternative explanations and models 70 %

Nursing Students Science Expectations O Build on prerequisite courses to apply knowledge of the human body anatomy and physiology, and principles of microbiology to learn concepts of pathophysiology O Development of critical thinking, problem solving, analysis and synthesis to provide patient and family nursing care

Common Themes O Students demonstrated ability of subject domain knowledge and comprehension O Students were deficient in ability to: O Apply O Convert O Evaluate O Explain O Solve O Distinguish O Analyze O Compare and contrast O Develop alternative explanations

Use of TEAS V Data for Decision Making O Development of a proposal to change admission policy O Prioritize curricular goals, identify and correct gaps O Improve teaching of higher level cognitive processes O Allocate resources O Interdisciplinary collaboration

New Admission Policy ( in addition to prerequisite course work) O All* RN program applicants are required to take the pre-admission Test for Essential Academic Skills (TEAS V) and achieve a minimum of Proficient Level O Within an academic year (September 1 through August 31), applicants may only take the TEAS V twice, and must register and pay all fees each time.

Answers to FAQ O TEAS V is now offered at the Sayville Center O Cost $56 O Directions on how to register is on the School of Nursing Website O Prospective students are advised use the TEAS study guide manual and take practice assessments online prior to taking the exam

Expected Outcomes O Admitted students will possess the academic preparedness for program success O Students with lower level of academic preparedness can focus on areas to be strengthened before applying to program O Take additional course O Academic labs O Review course