1. UOFYE Assessment Retreat

2. Our Goals for the Day
Understand a broad overview of the student affairs assessment cycle and assessment competencies
Build skills in the areas of writing student learning outcomes and survey design
Leave the retreat with outcomes articulated for our program and plans for how and when to assess those outcomes

3. Assessment is grounded in action
Good assessment relates to the mission and goals of your office, and informs decision making to create change.
You have to use the information, not just for the sake of knowing more, or putting on a shelf.

4. Assessment Skills and Knowledge (ASK)
recommended by ACPA
Assessment Design
Articulating Learning and Development Outcomes
Selection of Data Collection and Management Methods
Assessment Instruments
Surveys Used for Assessment Purposes
Interviews and Focus Groups Used for Assessment Purposes

5. Assessment Skills and Knowledge (ASK)
recommended by ACPA
Assessment Methods: Analysis
Benchmarking
Program Review and Evaluation
Assessment Ethics
Effective Reporting and Use of Results
Politics of Assessment
Assessment Education

6. Assessment Cycle
Today, we are going to focus on the first three steps

7. 3 Stages of Assessment Assessment Planning Data Collection
Assessment Reporting

8. “All decisions regarding research, evaluation, or assessment projects are strongly linked with the purpose for conducting the project, the research questions asked, and the objectives addressed.
Therefore, the first step in any assessment project is to determine the purpose for the project.”
- R. M. Cooper
For the LC:
Do we want to review the admissions standards into the program?
Do we want to understand how students experience the program?
Do we want to learn to what degree involvement from community partners builds affinity to our office?

9. Assessment Planning
Grounded in goals, strategic priorities, CAS standards, etc. Determine what is important to our area and unit.
Define the purpose of the assessment. What do you want to learn?
Who should be studied? All students, a sample?
How does current literature inform your project?
Let questions guide methodology
What data already exists?
Establish a timeline, methodology, sample
How will you use assessment to improve practice? How will you report results and to whom?
What do you want to learn? Part of ongoing process, result of a special request, data suggest that further analysis is needed?

10. Key Questions What are we trying to do? Why?
What is my program/office/course supposed to accomplish?
What do I want my students to be able to do and/or know as a result of my course/program?
How well are we doing what we are trying to do? How do we know?
How do we use the information to improve or celebrate successes?
Do the improvements we make contribute to our intended end result?

11. Data Collection Why are you collecting data?
Who will be collecting data?
What data will be collected?
When will the data be collected?
Where is the data collected?
How is the data collected?

12. Data Collection
Qualitative
Quantitative
Mixed Methods

13. Qualitative Interviews Focus Groups
Document Review (pictures with captions, one sentence summary, muddiest point, journals)
Case Studies
Observation

14. Quantitative
Pre-existing data (demographics, admissions characteristics, class standing)
Commercial instruments
Surveys
Descriptive statistics (participation data)

15. Pre Existing Data
IPEDS (degree completion enrollment, graduation rates, crime)
Common Data Set (enrollment, persistence, admissions, student life)
NSSE
CIRP

16. Mixed Methods Triangulation Explanatory Exploratory
Use both qualitative and quantitative to interpret data.
Explanatory
Conduct a follow-up qualitative study after a quantitative study.
Exploratory
Build a quantitative study based on the results from a qualitative study

17. Advantages Disadvantages
Using a Survey
Advantages
Easy to administer
Inexpensive
Large audience in short amount of time
Yield data that is easy to analyze
Cover variety of topics
Candid responses
Disadvantages
Options might not capture participants true opinions
Cannot provide clarification
Low response rates
Limited sample (online – limited to computer and internet access)

18. Advantages Disadvantages
Using a Focus Group
Advantages
Quick and inexpensive
Good for lower literacy levels (ESL, children)
Rich data in participants own language
Group dynamic allows responses to build
Can ask follow up questions
Disadvantages
Requires a trained moderator
Quality and usefulness depends on moderator
Data difficult to analyze
Sample is not representative
Lack of anonymity

19. Reporting
Go back to your assessment purpose, and answer those questions
Include other interesting or relevant data
A report should have clarity of purpose
Like a good resume, may be tailored differently to different audiences
Focus on what is most useful, not most interesting; avoid “shiny object” syndrome
Use visuals strategically
PCs review every single comment in open ended for program development. Summarize or aggregate for Directors/VPs
Detailed methodology should be used when reporting to faculty or assessment professionals, but for others, a summary is just enough
Remember, your purpose is to use the data in some way, so need to make it persuasive

20. Reporting Formats Comprehensive Report (UOFYE Survey Report) Briefs
Infographic
Executive Summary
Posters
Marketing Materials
Social media
Oral Presentation

21. Assessment Ethics
Coordinate with IR and CSSL to see if someone is asking similar questions
If planning to present the data outside of the institution, get IRB approval. (Another training for another day…)
Remove personal bias. You shouldn’t design an assessment to prove you have a great program, you should be asking, “What is the student experience in my program” or “What is effective and what can I improve.”
Beware of the power and politics of information