1. © 2015 NACADA The Global Community for Academic Advising The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. © 2015 NACADA The Global Community for Academic Advising The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. NACADA Executive Office at Kansas State University 2323 Anderson Ave, Suite 225, Manhattan, KS 66502-2912 P: 785-532-5717 | F: 785-532-7732 | E: nacada@ksu.edu Chrissy L. Davis Jones Administrators’ Institute Spokane Falls Community College February 2015 PROGRAM ASSESSMENT: The Administrator’s Role

2. Participants will… learn basic assessment terminology; learn basic assessment terminology; identify two (different) tools they can use to assess the advising program on their campuses; identify two (different) tools they can use to assess the advising program on their campuses; be active learners during this session. be active learners during this session. OUTCOMES

3. FOUNDATION: TERMINOLOGY Evaluation: a process of examining or reviewing individuals or programs to measure performance. Evidence: outcomes that establish support of behaviors, attitudes, or (external) attributes. Lagging indicators: Measure goal accomplishment (easy to measure but hard to influence) Leading indicators: Predict goal achievement (maybe more difficult to measure) Mapping: process to determine when, where, and how outcomes for advising will be attained Multiple measures: several measure of the same construct Process outcome: expectations about the process of delivery of academic advising across the institution; focus on service vs. advisors Programmatic objective: statements of what the program wants students and/or advisors to be able to do and know (tend to be more specific than goals) Student Learner Outcome (SLO): statements of what students can expect to learn (i.e., knowledge, skills, and/or value) from the advising experience

4. ASSESSMENT Assessment is an intentional, systematic research-based process of collecting, interpreting, and utilizing data from multiple sources and techniques (Upcraft & Schuh, 1996; Robbins, 2009) to focus on enhancing student learning and development (Palomba & Banta, 1999; McClenney & McClenney, 2004)

5. PURPOSES of Assessment “INSTITUTIONAL CURIOSITY”, I.E. STUDENT LEARNING AND STUDENT DEVELOPMENT (MAKI, 2002, 2004).

6. INTENTIONS of Assessment Assessment is intended to be a positive process, yet, its connotations are often negative (Nutt, 2013) Assessment is intended to be a positive process, yet, its connotations are often negative (Nutt, 2013) The focus has often been on activities that demonstrate accountability to the exclusion of those that are aimed at improvement (Nutt, 2013). The focus has often been on activities that demonstrate accountability to the exclusion of those that are aimed at improvement (Nutt, 2013). Our perspectives often influence receptiveness Our perspectives often influence receptiveness

7. EVALUATION Evaluation is part of the assessment process, in which the results are analyzed and transformed into a format that is useable (Upcraft & Schuh, 1996; Upcraft, 2003).

8. ELEMENTS of program assessment for Advising Administrators

9. ASPIRE VISION PURPOSE MISSION GENERIC ACTIONS GOALS SPECIFIC ACTIONS OBJECTIVES END RESULTS OUTCOMES

10. Mission Statement CHECKLIST What is the purpose of academic advising on your campus? What is the purpose of academic advising on your campus? What does your institution and advising unit(s) value about academic advising? What does your institution and advising unit(s) value about academic advising? How do these elements tie into the overall assessment plan for academic advising? How do these elements tie into the overall assessment plan for academic advising?

11. © 2015 NACADA The Global Community for Academic Advising The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. © 2015 NACADA The Global Community for Academic Advising The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. NACADA Executive Office at Kansas State University 2323 Anderson Ave, Suite 225, Manhattan, KS 66502-2912 P: 785-532-5717 | F: 785-532-7732 | E: nacada@ksu.edu In order to provide high-quality learning opportunities, the mission of academic consulting is to support students in achieving their goals through building an effective working relationship between student and consultant. An effective academic consulting working relationship provides a holistic approach: guiding students to relevant resources to foster their success; collaborating with others who have complementary expertise; and designing an educational, engagement, and academic success plan based on the students' values, attributes, learning styles, and academic/career/goals.

12. DEVELOPING Goals & Objectives Both goals and objectives MUST: support the mission statement & serve as a guide for learning outcomes

13. GoalObjective Meaning The purpose toward which an endeavor is directed. Something that one's efforts or actions are intended to attain or accomplish; purpose; target. Example I want to achieve success in the field of genetic research and do what no one has ever done. I want to complete this thesis on genetic research by the end of this month. Action Generic action, or better still, an outcome towards which we strive. Specific action - the objective supports attainment of the associated goal. Measure Goals may not be strictly measurable or tangible. Must be measurable and tangible. Time frame Longer termMid to short term

14. Learning OUTCOMES know, do, value, and appreciation Learning outcomes are what we expect student to know, do, value, and appreciation as a result of their academic advising experience.

15. Mapping: The process of determining when, where, and how the advising outcomes will be accomplished over the duration of students’ academic journeys.

16. Mapping Exercise (see handout)

17. After mapping out the learning experience using the mission statement, common goal, and objectives… What’s next?

18. Needs ASSESSMENT fixes, and situational crises. Needs ASSESSMENT Tools for constructive and positive change that is rational, logical, and functional transformation---not change driven by controversy, quick fixes, and situational crises.

19. Sample Questions How efficient is the program? How efficient is the program? How is effective practices measured? How is effective practices measured? What is the overall satisfaction rating? What is the overall satisfaction rating? Are student learning outcomes being met? Are student learning outcomes being met? Are advisors trained at the appropriate level of experience needed? Are advisors trained at the appropriate level of experience needed? What plan is in place to ensure continuous professional development for advisors? What plan is in place to ensure continuous professional development for advisors?

20. DATA Collection Attitudinal & Learning surveys/questionnaires Attitudinal & Learning surveys/questionnaires Student focus groups Student focus groups Student forums Student forums Existing data approach Existing data approach Key Informant Key Informant *Know how you will respond to the data before it arrives *No instrument is perfect, so plan for multiple measures

21. NOTE! At least one of your measures should include existing institutional data

22. CAUTION! Satisfaction Surveys

23. Lagging & Leading INDICATORS

24. Lagging indicators---measure goal accomplishment *Easy to measure but hard to influence *Tells you little to nothing about how the outcomes will be achieved *There is little early warning(s) about being on track to achieve the goal(s). LAGGING INDICATORS

25. Leading indicators---Predict goal achievement * Indicators can be influenced, but the indicators may be more difficult to measure * Tells you how the outcomes are achieved * There are early warning(s) about being on track to achieve the goal(s). LEADING INDICATORS LEADING INDICATORS DATA TO GUIDE FUTURE ACTIONS

26. © 2015 NACADA The Global Community for Academic Advising The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. © 2015 NACADA The Global Community for Academic Advising The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. NACADA Executive Office at Kansas State University 2323 Anderson Ave, Suite 225, Manhattan, KS 66502-2912 P: 785-532-5717 | F: 785-532-7732 | E: nacada@ksu.edu LAGGING & LEADING INDICATORS EXAMPLE

27. What are your institution’s goals? What are your institution’s goals? What processes support the achievement of the goals? What processes support the achievement of the goals? What are your institution's SLOs? What are your institution's SLOs? What individual actions comprise the processes? What individual actions comprise the processes? Which actions can be affected and will make an improvement? Which actions can be affected and will make an improvement? – What data do you need to know? – When and how often do you need it? DEVELOPING LEADING INDICATORS

28. CONTINUE… Mapping Exercise (refer to handout)

29. SLO #2: Students will know where and how to access appropriate academic support services to meet their academic goals. Questions to consider: What would be the model path to achieve this goal? What would be the model path to achieve this goal? What individual actions comprise this process? What individual actions comprise this process? Which actions can be affected and will make an improvement? Which actions can be affected and will make an improvement? What do you need to know (i.e. data) and when to affect the outcome? What do you need to know (i.e. data) and when to affect the outcome? SFCC’S ADVISING SLO CONTINUE…

30. Sharing the Results Follow-Up & Follow Thru Implications for Future Practices Culture of Evidence OVERVIEW OF THE PROCESS Interpreting the Results

31. TO CONCLUDE Assessment of academic advising is about… developing (realistic) expectations for learning; developing (realistic) expectations for learning; gathering evidence to understand students’ academic journeys; gathering evidence to understand students’ academic journeys; using evidence to support improvement of the advising system that will contribute to continuous quality improvement of the learning process and student achievement. using evidence to support improvement of the advising system that will contribute to continuous quality improvement of the learning process and student achievement. Talk to and work with the experts---IR staff Talk to and work with the experts---IR staff

32. THANK YOU!