1. STUDENT AFFAIRS ASSESSMENT A comprehensive, centralized approach Lori Varlotta, Vice President for Student Affairs Sacramento State WASC Conference on Assessment Irvine, CA October 16-18, 2008

2. Learning Outcomes for Today’s Participants This session will help participants: Differentiate outcome based assessment from satisfaction based assessment Articulate the potential benefits of assessment in student affairs Write direct learning outcomes that measure learning goals Identify useful types of assessment and evidence-gathering tools

3. What is Assessment? Sacramento State’s Division of Student Affairs sees assessment as an ongoing process primarily aimed at one of two things: 1) improving programs or services, and 2) understanding and eventually increasing student learning. To reach those goals, the Division must: Make program objectives and student learning outcomes explicit and public Set appropriate criteria and high expectations; systematically gather, analyze, and interpret evidence to determine how well programs and services matches those expectations and standards Use the resulting data to document, explain, and improve programs, services, and student learning outcomes

4. Expanding the Definition of Student Learning At Sacramento State University, the Division of Student Affairs “defines student learning as a comprehensive, holistic, transformative activity that integrates academic learning and student development, processes that have often been considered separate, and even independent of each other. When we say learning, then, we do not mean exclusively or primarily academic instruction, the acquisition of disciplinary content, or classroom learning.” (quoted directly from Learning Reconsidered: A Campus-wide Focus on the Student Experience)

5. Why Do It? Assessment will allow us to: Demonstrate how Student Affairs contributes to student learning and student success Showcase our strengths Identify areas where we can improve Strategically align ourselves with –Chancellors’ Office Graduation Initiative –CSU Access to Excellence –University planning – Destination 2010 –WASC requirements to demonstrate the impact of Student Affairs on student learning Respond to public calls for increased accountability –(e.g. VSA College Portrait)

6. Other Good Reasons to do Assessment The Vice President of Student Affairs holds each director responsible for conceptualizing and implementing an assessment plan The directors’ annual evaluation includes commentary on assessment Assessment data has supported requests for additional staff and/or funding

7. Role of Institutional Research To support the vision of the Vice President of Student Affairs and assist Student Affairs Directors (via on-going consultations, one- on-one meetings, survey development and data analyses) in developing departmental assessment plans that are aligned with Destination 2010 and the WASC re- accreditation. Utilizing On Campus Expertise Role of other Campus Experts

8. Why this Particular Assessment Approach is So Exciting A true collaboration with departments within and outside the Division of Student Affairs (Institutional Research, ASI, program units, Provost, academic departments) Comprehensive and centralized – a level of consistency throughout the Division and uniquely designed for each department Directly tied to the Chancellor’s Office Graduation Initiative, WASC requirements, Strategic Plan, etc. Access to Excellence Moves beyond student satisfaction to student learning

9. Six Step Assessment Model 1.Mission— 2-3 sentences that articulate office name, primary purpose, primary activities, audience. 2.Goals— top 3-5 “planning-type” statements. 3.Program Objectives and Learning Outcomes— 3-5 specific (i.e. measurable) statements. 4.Measures—Direct measures can verify learning outcomes, attitudinal change or behavior modification. Indirect measures typically reflect student satisfaction and/or self-reported understandings. 5.Results — Include a brief narrative of findings and/or essential tables or graphs. The results and subsequent analysis should indicate the extent to which the program objective or student learning outcome was met. 6.Conclusion —Summarizes the collection and analyses of data and then "closes the loop" by identifying what decisions or program modifications are or will be made on the basis of these analyses.

10. Step 1 – Mission Statement The departmental or program mission should be an expanded statement of either the institutional or divisional purpose. Extract from University Mission Statement: California State University, Sacramento is dedicated to the life- altering potential of learning that balances a liberal arts education with depth of knowledge in a discipline. We are committed to providing an excellent education to all eligible applicants who aspire to expand their knowledge and prepare themselves for meaningful lives, careers, and service to their community..… Academic Advising Center Mission The Academic Advising Center offers new student orientation, mandatory freshman advising, and advising on General Education and graduation requirements for all students. The Center engages students in a developmental process that helps clarify and implement individual educational plans consistent with their skills, interests, and values. Through individual appointments, group advising sessions, and presentations the professional staff, faculty advisors, and student interns help students understand the university’s academic requirements as well as its policies and procedures. As a result, students are better prepared to take responsibility for their education and persist towards a timely graduation. Primary Activities Primary Purpose Target Audiences Office Name

11. Step 2 – Goals Goals: Are broad statements that describe the overarching long-range intended outcomes of an administrative unit. Example: Help students clarify and implement individual educational plans which are consistent with their skills, interests, and values. Are usually not measurable and need to be further developed as separate distinguishable outcomes. Example: Prepare students to take responsibility for their education and persist towards a timely graduation.

12. Step 2 – Goals (cont’d) Goals : Are primarily used for general planning and are used as the starting point to the development and refinement of outcomes. Example: Help students gain an understanding of the university’s academic requirements as well as its policies and procedures.

13. Step 3 – Objectives Objectives : Are specific statements that describe desired outcomes derived from the goal statements of the unit. Are typically one of two types: program objectives or student learning outcomes. Example – Program Objective: All freshmen will participate in a three-phase comprehensive, proactive advising and major/career exploration program by Spring 2008. Example – Student Learning Outcome: 70% of selected students will be able to demonstrate knowledge of their college preparatory requirements and the resources available to assist them.

14. Step 3 – Objectives (cont’d) Objectives : Program objectives help staff identify how a program, workshop, or activity can be modified or improved; while student learning outcomes reveal the changes in attitudes or behaviors that a student user can describe or demonstrate after utilizing a service or program.

15. Components of Learning Outcomes Audience: For whom is the program aimed? Behavior: What do you expect the audience to know/be able to do? Conditions: Under what conditions or circumstances will the learning occur? Degree: How much will be accomplished, how well will the behavior need to be performed, and at what level?

16. Indirect Learning Outcomes Indirect Learning Outcomes - self-reported statements/ comments that reveal a perceived increase in understanding or appreciation. The perception is not verified through any demonstration of knowledge acquisition or observed behavioral / attitudinal change.

17. Methods to Assess Indirect Learning Outcomes Satisfaction surveys Program / evaluation surveys Questionnaires Inventories Face-book responses Informal peer-to-peer conversations (e.g. with RAs, orientation leaders)

18. Direct Student Learning Outcomes Direct Student Learning Outcomes – The abilities, information retention, knowledge acquisition, attitudinal or behavioral changes that students can demonstrate after participating in a program or utilizing a service

19. Methods to Assess Direct Learning Outcomes Portfolios Capstones Performances Common assignments Narratives with reflection Tests Competency observations Observations of employee or student behaviors Juried art exhibits National licensure examinations Standardized tests Work samples

20. Differentiating Direct and Indirect Learning Outcomes Direct or Indirect? 1)T / F This orientation session has helped me understand the foreign language requirement at Sacramento State. 2)Which of the following examples fulfill the foreign language requirement necessary to graduate from Sacramento State? A) Demonstrated fluency in a language other than English B) Passed the AP foreign language exam with a score of 3 or higher C) Successfully completed, with C- or better, 3 years of high school foreign language D) All of the above

21. Changing Indirect Learning Outcomes to Direct Learning Outcomes Distribute Orientation Survey Handout Initiate group activity

22. Direct Student Learning Instrument Distribute Post-Test Orientation Assessment Handout

23. Step 4 – Measures SLO – Orientation Participants: Students who complete the orientation post test that measures their short term retention of information related to GE Advising, Academic requirements and campus resources will score 85% or better. The post test will be administered at the end of the 2-day orientation program. SLO – Orientation Leader: Orientation leaders observed in a twenty-minute structured role play exercise will earn ‘4’ or better in each of competency areas described on the rubric developed for this training. Program Objective – Orientation: 95% or students and parents will indicate on their program evaluation that they were ‘satisfied’ or ‘very satisfied’ with all aspects of the orientation.

24. Step 5 – Results The results should highlight all significant findings and indicate the extent to which the program/service reached its intended outcomes. Example: Freshman Orientation pre/post tests showed that out of the 11 questions asked, 6 of the questions fell short of the 85 percent goal of knowledge acquisition. However, of those 6 questions not meeting the goal, there still was significant improvement between the pre-test and post-test responses. Step 6 – Conclusions The conclusions should explain how the findings from data will be used to improve the program and/or increase student learning. Example: The Orientation Coordinator will use the data collected to identify the areas that need to be emphasized in the student training sessions offered to orientation assistants.

25. Suggested Reading 1.The National Association of Student Personnel Administrators & The American College Personnel Association. (2004). Learning Reconsidered: A campus- wide focus on the student experience. 2.American College Personnel Association (1994). The student learning imperative: Implications for student affairs. Washington, DC. 3.American College Personnel Association. (1996) The student learning imperative: Implications for student affairs [on-line]. Available: www.acpa.nche.edu/sli/sli.htm 4.American College Personnel Association and National Association of Student Personnel Administrators (1997). Principles of good practice for student affairs [on-line]. Available: www.acpa.nche.edu/pgp/principle.htm 5.Joint task Force on Student Learning. (1998) Powerful partnerships: A shared responsibility for learning [on-line]. Available: www.aahe.org/teaching/tsk_frce.htm 6.National Association of Student Personnel Administrators (1987). A perspective on student affairs. Washington DC.

26. Other Resources WASC website http://www.wascweb.org CAS Standards http://www.cas.edu/gags.htmlhttp://www.cas.edu/gags.html Kansas Study Website http://Kansasstudy.orghttp://Kansasstudy.org National CC Benchmark Project http://www.NCCBP.orghttp://www.NCCBP.org Powerful Partnerships: A Shared Responsibility for Learning http://www.aahe.org/assessment/joint.htm http://www.aahe.org/assessment/joint.htm Student Learning Imperative http://www.acpa.nche.edu/sli/sli.htm http://www.acpa.nche.edu/sli/sli.htm Association of Theological Schools http://www.ats.eduhttp://www.ats.edu Sacramento State’s VSA http://www.csus.edu/checkusout/http://www.csus.edu/checkusout/ Sacramento State’s Student Affairs’ Assessment http://www.csus.edu/wascaccreditation/Assessment.stm http://www.csus.edu/wascaccreditation/Assessment.stm