1. Using Outcomes Assessment Data to Change and Improve
Introduction
An example from Highline College
Reflective practice using data
Break
Using data to direct and amplify change
Connecting assessment data and your action plans
Reflective practice – what is your data story?

2. Outcomes assessment: Why it matters
Participation
Geoscience course enrollment
Broadening participation in geoscience courses
Course Success Outcomes
Success in geoscience courses
Geoscience course success by student sub-group
Pathways Progression Outcomes
Enrollment in multiple geoscience courses
Major in geoscience program
Equity- and outcomes assessment, link to action plan
Starting an OA project is always challenging – new and complex. You’ve been remarkably patient and responsive. As few hiccups along the way, but has gone remarkably smoothly. We consider this a pilot and we’re learning too.
Connection to project goals – improve instruction, broaden participation, and enhance pathways
What we do as scientists: Research, hypothesis testing, and your action plan
Value to institutions and to geoscience community
Value to YOU

3. Questions to consider
Considering the outcomes measures we used, what did you (do you) expect to see?
What surprised you in your data? Did you have an “aha” moment?
Write
Share
Short report-outs

4. An Example from Highline College
What I thought I knew…
Geoscience has less ethnic diversity than the College as a whole
African American students have lower success rates than other groups
We have more women in our classes than the College as a whole
International students struggle in our classes

5. Enrollment demographics Geoscience Vs Highline as a whole
Students
Asian
African American
Native American
Latino/
Hispanic
White
International
Other/Mixed
Women
Highline
10,765 (credit)
20%
17%
<1%
25%
59%
Geoscience
213
11% 2%
15%
40% 6% 7%
Environmental Science 83
12%
13% 1% 8%
42%
63%
Geology
109
22% 9% 3%
19%
38%
56%
Oceanography 21
10% 0%
24%
43% 5%
57%

6. Enrollment demographics Geoscience Vs Highline as a whole
Students
Asian
African American
Native American
Latino/
Hispanic
White
International
Other/Mixed
Women
Highline
10,765 (credit)
20%
17%
<1%
25%
59%
Geoscience
213
11% 2%
15%
40% 6% 7%
Environmental Science 83
12%
13% 1% 8%
42%
63%
Geology
109
22% 9% 3%
19%
38%
56%
Oceanography 21
10% 0%
24%
43% 5%
57%

7. Completion (C or Higher) By Ethnicity
Course (Students)
Asian
African American
Native American
Latino/Hisp
White
International
All
Environmental Issues (27) 0%
75%
50%
92%
78%
ENVS 301 (9)
100%
67%
89%
Intro to Env Science (22)
80%
88%
86%
Intro to Env Science (25)
91%
SHAPING THE EARTH (24)
40%
60%
63%
GEOLOGIC CATASTROPHES (29)
62%
GEOLOGIC CATASTROPHES (23)
25%
73%
61%
Puget Sound Landslides (9)
Intro to Oceanography (21)
56%
57%
ALL (213)
65%
58%
77%
N (Total Students) 37 23 4 33 85 13
213

8. Completion (C or Higher) By Ethnicity
Course (Students)
Asian
African American
Native American
Latino/Hisp
White
International
All
Environmental Issues (27) 0%
75%
50%
92%
78%
ENVS 301 (9)
100%
67%
89%
Intro to Env Science (22)
80%
88%
86%
Intro to Env Science (25)
91%
SHAPING THE EARTH (24)
40%
60%
63%
GEOLOGIC CATASTROPHES (29)
62%
GEOLOGIC CATASTROPHES (23)
25%
73%
61%
Puget Sound Landslides (9)
Intro to Oceanography (21)
56%
57%
ALL (213)
65%
58%
77%
N (Total Students) 37 23 4 33 85 13
213

9. Completion (C or Higher) by Ethnicity Courses with Changes Made
Course (Students)
Asian
African American
Native American
Latino/Hisp
White
International
All
Environmental Issues (27) 0%
75%
50%
92%
78%
ENVS 301 (9)
100%
67%
89%
Intro to Env Science (22)
80%
88%
86%
Intro to Env Science (25)
91%
SHAPING THE EARTH (24)
40%
60%
63%
GEOLOGIC CATASTROPHES (29)
62%
GEOLOGIC CATASTROPHES (23)
25%
73%
61%
Puget Sound Landslides (9)
Intro to Oceanography (21)
56%
57%
ALL (213)
65%
58%
77%
N (Total Students) 37 23 4 33 85 13
213

10. Completion (C or Higher) rates by Ethnicity and Subject
Asian
African American
Native American
Latino/Hisp
White
International
All
Environmental Science
78%
82%
100%
86%
91%
70%
Geology
83%
50%
67%
48%
76%
71%
Oceanography
60%
56%
57%

11. Evening/Weekend vs. Daytime Classes
Enrollment
Asian
African American
Native American
Latino/Hisp
White
International
Other/2 Race
All
Evening+Weekend
5 (11%)
8 (18%)
3 (7%)
19 (42%)
2 (4%) 45
Day
32 (19%)
15 (9%)
4 (2%)
30 (18%)
66 (39%)
11 (7%)
10 (6%)
168
Completion (C or Higher)
Asian
African American
Native American
Latino/Hisp
White
International
All
Evening+Weekend
80%
75%
67%
95%
50%
84%
Day
78%
60%
57%
76%
82%
73%
Finish with “I know the information from my program much better and I can come up with ideas for changes that will make the most significant improvements.”

12. Reflective Practice Using Data
Debra Bragg & Heather McCambly

13. SAGE 2YC Outcomes Assessment Template

14. Reflective Practice Gibbs’ (1998) Reflective Cycle
Description: What happened?
Feelings: What were you thinking and feeling?
Evaluation: What was good and bad about the experience?
Analysis: What sense can you make of the situation?
Conclusion: What else could you have done?
Action Plan: If it arose again what would you do?
Reflective Practice

15. Equity-Minded Practice
Color-blindness
Diversity
Equity
Equity- Mindedness

16. What is Your Data Story?
What do your data tell you about your geoscience program, courses and students?
What data points are most compelling and most challenging?
What type of story is emerging?
Growth or decline
Universal or singular
Inclusive or exclusive
Equity or inequity
What story do you want to tell?

17. Discussion Questions
What story do your data tell? What else would you like to know?
What have you done to reflect on your data?
What will you do to engage others in equity-minded reflection in the future?

18. Outcomes Assessment: Using Data to Direct and Amplify Change
Outcomes Assessment: Using Data to Direct and Amplify Change

19. Step 1: What are we changing?
Past
What has changed?
Future
What will change?
Self
Others
Step 2: What could these changes… change?
Reiterating a change hypothesis: Based on the past and future strategies, described above, we could expect to see a change in X measure over time in Y classes/groups.

20. Gallery walk Walk the room and review other change agent’s posters
Use post-its to note where you see strong alignments between hypotheses and actions, or where you have questions
Note if you think a different type of data would be needed to align with the change work
Gallery walk

21. Debrief Changes Outcomes Collected Data
Are your actions or plans, your hypothesis, and the data you’re collecting in alignment?
If not, what needs rethinking?:
The data collection to capture meaningful outcomes?
The action plan to amplify potential strategies for impact?
Changes
Outcomes
Collected Data

22. Connecting assessment data and your action plans If-then statements
If I change the introductory course to make it more relevant, then I will see more students, including more women, taking a second environmental science course
If I change the experience in oceanography so that it is more supportive of students, then I will see more students, including more students of color, asking about oceanography careers and 4YCU oceanography programs
If I include more exposure to career possibilities, workplaces, and professionals, then more of our students, including first-generation students, will transfer to geoscience programs at our neighboring 4YCU.
A hypothesis that you will test, backwards design and alignment

23. Possible approaches Practices Events Programs Policies
Complementary, collectively contribute to a comprehensive strategyPractices include what you do in your teaching and advising

24. Observations
Students of color are underrepresented in geoscience courses, across the board
While 20% of the students in my courses identify as Hispanic or Latino, only 50% of those students earn a C or higher
Of the students who take more than one geoscience course, 80% are male

25. What might you choose to do in each case?
Students of color are underrepresented in geoscience courses, across the board
Think about what you could do to attract more students of color into geoscience courses.
Survey current geoscience students about the departmental climate?
Add science spotlights to your courses, to try to improve the experience of students of color who do take geoscience courses?
Organize and promote participate in a career fair?
Work with your institution’s advising staff for incoming students?
While 20% of the students in my courses identify as Hispanic or Latino, only 50% of those students earn a C or higher
Think about how you can support Hispanic/Latino students in geoscience courses.
Add societally-relevant course content?
Teach metacognitive strategies?
Add academic support for all students?
Ask your institution’s Multi-Cultural Affairs office for advice?
Add a values-affirming writing assignment to all of your courses?
Of the students who take more than one geoscience course, 80% are male
Think about what you could do to interest more women in the geosciences.
Actively recruit all female students who earn a grade of C or higher in any geoscience course to take additional geoscience courses? (What percentage of female students earn a grade of C or higher in their first geoscience courses?)
Add science spotlights that focus on female geoscientists?
Expand on measures of success. What change do you want to see.

26. Connecting assessment data and your action plans: How will you know if you are successful?
Now that you’ve reflected on your data and your work in this session, what are some ideas for what you might do differently in the next year in your teaching? In your program?
And how will you know the impact? What would you consider to be a measure of success?
Your task: Generate some ideas for your action plan based on your outcomes assessment data, including the data related to equity. Make a direct link between each strategy and the outcomes.
Write for 10 minutes
Share
Report out
backwards design and alignment, outcomes (measures of success)