Assessment of Academic Advising Charlie Nutt, Ed.D. Executive Director NACADA: The Global Community for Academic Advising NACADA Executive Office Kansas State University 2323 Anderson Ave, Suite 225 Manhattan, KS Phone: (785) Fax: (785) © 2007 National Academic Advising Association The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association.

Assessment “Assessment is a process that focuses on student learning, a process that involves reviewing and reflecting on practice as academics have always done, but in a more planned and careful way” (Ewell, 2000)

Assessment “Assessment is the systematic collection, review, and use of information about educational programs* undertaken for the purpose of improving student learning* and development*” (Marchese, 1993) * Advising is part of the educational process, not simply a “service”

Assessment “Assessment is the means used to measure the outcomes of education and the achievement of students with regard to important competencies” (Pellegrino, Chudowsky, and Glaser, 2001)

Assessment “Assessment reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time” (Banta, 1996)

For Academic Advising… “Assessment is the process through which we gather evidence about the claims we are making with regard to student learning and the process/delivery of academic advising in order to inform and support improvement” (Campbell, 2008) Assessment

What Is Assessment – The Intentions Assessment is intended to be a positive process, yet its connotations are often negative The focus has often been on activities that demonstrate accountability to the exclusion of those that are aimed at improvement

What Assessment is NOT Assessment is NOT episodic Assessment is NOT just about measurement Assessment is NOT about evaluating the performance of an individual staff / faculty / student Assessment is NOT solely an administrative process Assessment is NOT easy or quick

Assessment is An on-going cycle of activity A gathering of a variety of information and data Using this feedback for improvement of individual or program performance A team effort with faculty, staff, and administrators actively engaged A complex process of comparison

Goals of Assessment Improving academic advising process delivery programs Enhancing student success persistence retention

Assessment or Evaluation? What Distinguishes Assessment from Evaluation? evaluation usually measures advisor effectivenessevaluation usually measures advisor effectiveness assessment usually measures programmatic outcomesassessment usually measures programmatic outcomes evaluation of individual performance and evaluation of effectiveness of processes may be used as part of an overall assessment designed to measure program outcomesevaluation of individual performance and evaluation of effectiveness of processes may be used as part of an overall assessment designed to measure program outcomes

Assessment or Evaluation? What Distinguishes Assessment from Evaluation? assessment should be continuous and imbedded in the culture while evaluation is episodicassessment should be continuous and imbedded in the culture while evaluation is episodic assessment focuses on programmatic issues while evaluation focuses on individual performances of advisorsassessment focuses on programmatic issues while evaluation focuses on individual performances of advisors

The Assessment Cycle (Maki, 2002, 2004) Gather Evidence Interpret Evidence Identify Outcomes Implement Change Mission/Purposes Educational Objectives

The Assessment Flowchart (adapted from Darling, 2005) Student Learning Outcomes Cognitive, Psychomotor, AffectiveProcess/Delivery Outcomes Mapping the Experience What experiences? When or by when? Gathering Evidence When gathered? Where gathered? How often gathered? From whom gathered? How gathered? Minimum performance criteria for success? Values Vision Mission Goals Programmatic Outcomes Sharing and Acting Upon the Results Interpret how results inform practice How and with whom to share interpretation Follow up on implemented changes Start the process all over again!

Identifying Key Stakeholders: Who Should Be Involved? Colleagues, faculty, administrators, institutional researchers, staff, students, and institutional community Decide how the assessment team will interact, overlap, and/or support other institutional efforts Encourage stakeholders on and off campus Continuous communication and feedback is a must!

Values – What is considered important in regard to academic advising on the campus Vision – The aspirations of what academic advising can be on the campus Mission – The statement which reflects the purpose of academic advising on the campus that serves as the institution’s roadmap to reach its vision and affirm its values for academic advising Key Terms

Goal Statements – how the mission will be achieved – how values, visions, and missions will be enacted Programmatic Outcomes – what you expect to occur – what you expect students to know, do, and appreciate

Key Terms Process/Delivery Outcomes – Articulate the expectations for how academic advising is delivered and what information should be delivered through the academic advising experience Student Learning Outcomes (SLOs) – Articulate what students are expected to know (cognitive learning), do (behavioral learning), and appreciate (affective learning) as a result of involvement in the academic advising experience Mapping – The process of determining when, where and how the outcomes for advising will be accomplished over the students’ academic careers

Cognitive SLOs What do we want students to KNOW as a result of participating in academic advising? Know general education requirements Know about academic support services Know how to use the student information system to register Know how to use the catalog Etc…….

Behavioral/Psychomotor SLOs What do we want students to Doas a result of participating in academic advising? Generate their degree audit Make advising appointments Keep advising appointments Ask for help Access course descriptions and degree requirements using the online catalog Etc….

Affective SLOs What do we want students to Value or Appreciate as a result of participating in academic advising? Value/Appreciate general education Value/Appreciate the advising relationship Value/Appreciate the process of learning Etc….

Mapping the Learning Experience What should be learned: e.g., student will know the components of the institution’s General Education requirements Where it should be learned: e.g., orientation workshops, advising sessions, personal reading of catalog or curriculum guide When or By When it should be learned: e.g., prior to first year (orientation); by end of first year (via advising sessions); by end of first year (via personal reading)

Once the desired process/delivery and student learning outcomes have been identified, as well as when and where they will occur, the next step is to determine who or what will be measured and how the data will be gathered… …using multiple measures of varying types

Examples of Existing Tools To be used as just one measure among multiple measures – ACT Survey of Academic Advising – Noel-Levitz Student Satisfaction Inventory (SSI) – Winston and Sandor’s Academic Advising Inventory (AAI) – CAS Standards for Advising elated/CASStandardsForAdvising.pdf – NACADA Assessment of Advising Commission

Measures can (and should) include existing institutional data Information from Institutional Research, Admissions, Registrar, etc. can provide tracking data, GPAs, retention rates, and other information you can utilize as assessment data – this can be a source of some of the multiple measures utilized (in addition to formal instruments, satisfaction inventories, and others)

Dangers of Satisfaction Surveys there is often a difference between an advisee receiving good, effective academic advising and being satisfied with the advising process: –if any negative information is exchanged during the advising interaction, the student may respond negatively to the survey items even though the information provided was correct and the process of the interaction was appropriate –the student will likely rate the advising provided based on the type of interaction desired (e.g., informational, relational)

For both process/delivery and student learning outcomes, you need to identify the minimum criteria for success of the outcome measure, e. g., number of students exhibiting a specific learning performance percentage of students exhibiting a specific learning performance advisor rating of student performance student rating of specific aspect of advising process advisor rating of specific aspect of advising process etc.

At the end of the day, assessment of academic advising is all about… developing consensus around collective expectations about student learning gathering evidence in order to understand student learning using this evidence to support improvements in advising that will contribute to improvements in student learning

Thank You!