Closing the Loop on Student Learning Assessment Mohua Bose, Ph.D Mary Beth Hanner, Ph.D, RN, ANEF Director of Outcomes Assessment Provost and Chief Academic Officer Excelsior College, Albany, NY Presentation at MSCHE, December 12, 2011

Purpose  Describe a framework that addresses assessment of student learning in a non- traditional degree completion institution.  Discuss improvements based on assessment results at the program and institutional level.  Describe the process of facilitating an assessment culture.

Background  Excelsior College founded in 1971 (previously Regents College).  1998 Chartered as a private, independent college.  Large student body (approximately 33,000 students).  Distance education, flexible learning format. Philosophy: “What you know is more important that where or how you learned it”

Background  Valuing prior learning is a central premise for adult learners (Green, 1998).  Acceptance of a variety of credit sources: Examples:  Regionally accredited institutions  ACE/PONSI approved courses  Exams – CLEP, DANTE,EC  EC portfolio assessment for prior learning  Online courses at Excelsior and preferred providers  Residency requirement (presently minimal, historically none).

Background  Accreditation  Middle States (all programs)  NLNAC (specialized accreditation - Nursing)  ABET (specialized accreditation - Technology)  IACBE (specialized accreditation - Business)

Challenge: Who is a typical student? Career oriented working adult Average age: 39 year College 1 College 4 College 3 College 2 College 5 Excelsior College No first time, full time students. Credits transferred from other approved sources ranges from 25% to 75% of total credits required for the degree program. At a distance

Accountability and Continuous Improvement OAIR Steering Committee OAIR Office School of Business & Tech School of Liberal Arts School of Nursing School of Health Sciences Institutional Assessment Plan for Student Learning (IAPSL)

 Provides a framework for assessment of learning outcomes at course, program and institutional level.  Addresses the key question “How effectively do we deliver on our mission and make a distinctive impact relative to our resources?” (Collins, 2011).  IAPSL link: institution institution

Establish clear learning outcomes Use the Results Provide learning opportunities Assess Student Learning Assessment Process at Schools

Step 1: Establish Learning Outcomes  Ultimate responsibility to determine learning outcomes for a program rests with the faculty.  Instructional/Adjunct faculty receive an online faculty training that orients them to EC. The orientation includes an introduction to learning outcomes assessment.  Development of online courses requires completion of a structured template that involves program, course and module outcomes.

Step 1: Establish Learning Outcomes (continued)  A raised awareness of the importance of learning outcomes assessment at the College due to OAIRSC and OAIR offices. Senior level personnel significantly involved (eg. Deans).  In addition to OAIR, an assessment director within the schools work with the faculty.  Continuous and evolving process in defining learning outcomes.

Step 2: Create Learning Opportunities Monitoring the learning opportunities provided to students:  Course revisions based on: Centralized student course evaluations Faculty evaluation of courses Faculty curriculum committee  Curriculum reviews

Example: Curriculum Review  Committee of 3 reviewers (some members are completely external to the college).  2-3 days including two teleconferences.  Reviewers examine curricular structure, topical areas covered, course descriptions.  Provide feedback on curriculum rigor, curriculum currency, and program outcomes attainability and currency.  Feedback from the external curriculum review is taken back to program faculty.

Step 3: Assessing Student Learning  Direct assessments of program outcome measures for all programs using: o Capstones, Portfolios, or Exams (theory or performance) o State licensing exams (e.g. NCLEX)  Direct assessments of general education outcomes using: o ETS Proficiency Profile & VALUE Rubrics

Step 3: Assessing Student Learning  Also have indirect assessments:  Alumni surveys  Supervisor surveys Centralized for all degree programs

Discipline Specific Example on Closing the Loop

Associate Degree in Nursing NCLEX pass rates  Consistently at or above national average over a decade.  Dipped below for one quarter in national and state pass rates by approximately 1%.  Conducted analysis of subscales on the NCLEX exam.  Examined the CPNE exam.

Associate Degree in Nursing  Analysis of the data revealed that students scored lower on o Knowledge of adolescent growth and development o Care of the ill adolescent  After faculty review – new curriculum requirements.  Growth and Development across the Lifespan.

Associate Degree in Nursing  After approximately 2 years Scores exceeded both state and national levels NCLEX scores in overall exam and subscales  Remained up for a number of years.  Continuing to monitor.

General Education Example

General Education  Assess Writing  Outcome: Write with grammatical competence, clarity, and coherence  Administered standardized assessment: ETS Proficiency Profile  Reading  Writing  Mathematics  Critical thinking

General Education  ETS Proficiency Profile administered annually 2009, 2010, and  Results pertaining to Writing show that Excelsior students at the Associate’s and Bachelor’s level have performed at or above the ETS norms for writing.

General Education  College decided to use multiple measures to assess writing.  Use of VALUE rubrics in writing  Capstone papers from Bachelor’s degree programs across the College used.  Independent panel of reviewers.  Data collection completed, shows variations in student performance.  Closing the loop takes time.

Key Steps in Implementing Outcomes Assessment  Institutional support and commitment.  Creation of a college-wide oversight committee to set a relevant and strategic agenda for assessment of student learning and institutional performance.  A visionary and guiding document for implementation of outcomes assessment College-wide.

Key Steps in Implementing Outcomes Assessment  Training and implementation of an assessment management tool/database.  Centralized assessment model supported by a distributed model of assessment staff.  Showcase assessment results - Quarterly Dean’s presentations at Steering Committees.  Bi-annual assessment audits.  Continuous and evolving process!