UOFYE Assessment Retreat

Our Goals for the Day Understand a broad overview of the student affairs assessment cycle and assessment competencies Build skills in the areas of writing student learning outcomes and survey design Leave the retreat with outcomes articulated for our program and plans for how and when to assess those outcomes

Assessment is grounded in action Good assessment relates to the mission and goals of your office, and informs decision making to create change. You have to use the information, not just for the sake of knowing more, or putting on a shelf.

Assessment Skills and Knowledge (ASK) recommended by ACPA Assessment Design Articulating Learning and Development Outcomes Selection of Data Collection and Management Methods Assessment Instruments Surveys Used for Assessment Purposes Interviews and Focus Groups Used for Assessment Purposes

Assessment Skills and Knowledge (ASK) recommended by ACPA Assessment Methods: Analysis Benchmarking Program Review and Evaluation Assessment Ethics Effective Reporting and Use of Results Politics of Assessment Assessment Education

Assessment Cycle Today, we are going to focus on the first three steps

3 Stages of Assessment Assessment Planning Data Collection Assessment Reporting

“All decisions regarding research, evaluation, or assessment projects are strongly linked with the purpose for conducting the project, the research questions asked, and the objectives addressed. Therefore, the first step in any assessment project is to determine the purpose for the project.” - R. M. Cooper For the LC: Do we want to review the admissions standards into the program? Do we want to understand how students experience the program? Do we want to learn to what degree involvement from community partners builds affinity to our office?

Assessment Planning Grounded in goals, strategic priorities, CAS standards, etc. Determine what is important to our area and unit. Define the purpose of the assessment. What do you want to learn? Who should be studied? All students, a sample? How does current literature inform your project? Let questions guide methodology What data already exists? Establish a timeline, methodology, sample How will you use assessment to improve practice? How will you report results and to whom? What do you want to learn? Part of ongoing process, result of a special request, data suggest that further analysis is needed?

Key Questions What are we trying to do? Why? What is my program/office/course supposed to accomplish? What do I want my students to be able to do and/or know as a result of my course/program? How well are we doing what we are trying to do? How do we know? How do we use the information to improve or celebrate successes? Do the improvements we make contribute to our intended end result?

Data Collection Why are you collecting data? Who will be collecting data? What data will be collected? When will the data be collected? Where is the data collected? How is the data collected?

Data Collection Qualitative Quantitative Mixed Methods

Qualitative Interviews Focus Groups Document Review (pictures with captions, one sentence summary, muddiest point, journals) Case Studies Observation

Quantitative Pre-existing data (demographics, admissions characteristics, class standing) Commercial instruments Surveys Descriptive statistics (participation data)

Pre Existing Data IPEDS (degree completion enrollment, graduation rates, crime) Common Data Set (enrollment, persistence, admissions, student life) NSSE CIRP

Mixed Methods Triangulation Explanatory Exploratory Use both qualitative and quantitative to interpret data. Explanatory Conduct a follow-up qualitative study after a quantitative study. Exploratory Build a quantitative study based on the results from a qualitative study

Advantages Disadvantages Using a Survey Advantages Easy to administer Inexpensive Large audience in short amount of time Yield data that is easy to analyze Cover variety of topics Candid responses Disadvantages Options might not capture participants true opinions Cannot provide clarification Low response rates Limited sample (online – limited to computer and internet access)

Advantages Disadvantages Using a Focus Group Advantages Quick and inexpensive Good for lower literacy levels (ESL, children) Rich data in participants own language Group dynamic allows responses to build Can ask follow up questions Disadvantages Requires a trained moderator Quality and usefulness depends on moderator Data difficult to analyze Sample is not representative Lack of anonymity

Reporting Go back to your assessment purpose, and answer those questions Include other interesting or relevant data A report should have clarity of purpose Like a good resume, may be tailored differently to different audiences Focus on what is most useful, not most interesting; avoid “shiny object” syndrome Use visuals strategically PCs review every single comment in open ended for program development. Summarize or aggregate for Directors/VPs Detailed methodology should be used when reporting to faculty or assessment professionals, but for others, a summary is just enough Remember, your purpose is to use the data in some way, so need to make it persuasive

Reporting Formats Comprehensive Report (UOFYE Survey Report) Briefs Infographic Executive Summary Posters Marketing Materials Social media Oral Presentation

Assessment Ethics Coordinate with IR and CSSL to see if someone is asking similar questions If planning to present the data outside of the institution, get IRB approval. (Another training for another day…) Remove personal bias. You shouldn’t design an assessment to prove you have a great program, you should be asking, “What is the student experience in my program” or “What is effective and what can I improve.” Beware of the power and politics of information