1. Assessment Basics PNAIRP Conference Thursday October 6, 2011
Anne Marie Karlberg
Director of Institutional Research and Assessment
Whatcom Community College
(360)

2. Today’s Outcomes Define assessment.
Identify three levels of assessment.
Articulate three types of assessment information or data and provide examples of each.
Identify two phases in creating an effective outcomes assessment process.
Other items:
Assessment plan and report
Accreditation
Website
Outcomes: By the end of this session, participants will be able to…

3. Today’s Outcomes
Define assessment.

4. What is assessment?
“The systematic collection of information about student learning…to inform decisions about how to improve learning.”
(Walvoord, 2004, p. 2)
This will be a review for some of you.
But first, we need to step back and make sure we’re all operating with a similar definition of assessment.
This is the most succinct and clearest definition I know of assessment. Assessment is...

5. Purposes improvement (formative) accountability (summative)
2 purposes of assessment are
1. improve student learning and performance (internal improvement)
2. demonstrate to external accreditation bodies that relationships exist between the college’s mission and learning outcomes.

6. Today’s Outcomes Define assessment.
Identify three levels of assessment.

7. 3 Levels of Assessment
Level
College

8. 3 Levels of Assessment
Level
College
Program

9. 3 Levels of Assessment
Level
College
Program
Course

10. Today’s Outcomes Define assessment.
Identify three levels of assessment.
Articulate three types of assessment information or data and provide examples of each.

11. 1. Direct Indicators of Student Learning
Level
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Require students to demonstrate their learning through essays, demonstrations, presentations, etc.
There are 3 types of data that are used to evaluate student learning…
Direct indicators of student learning require students to demonstrate their learning through, for example, essays, capstone projects, demonstrations, and presentations.
Sometimes people refer to “Direct indicators” as “outcomes assessment”. Outcomes are things students are able to do at the end of their college experience, their program, or a course.

12. College Outcomes Level Type of data College Program Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Require students to demonstrate their learning through essays, demonstrations, presentations, etc.
…overarching skills that are emphasized throughout many courses in all programs at WCC.
CLAs define the skills we would like all students to develop by the time they graduate (with a certificate or a degree).

13. Program Outcomes Level Type of data College Program Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Require students to demonstrate their learning through essays, demonstrations, presentations, etc.
…overarching skills that are emphasized and reinforced throughout several courses in a specific program.
They are measurable statements that define what students will be able to do – over and above the college outcomes – by the end of a certificate or degree.
Note: A program is all or substantially all of a body of coursework leading to a degree or certificate (e.g., AAS degree, massage practitioner AS degree, accounting certificate, etc.)

14. Course Outcomes Level Type of data College Program Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Require students to demonstrate their learning through essays, demonstrations, presentations, etc.
…the most important skills students should develop by the end of a course.
They are unique to a specific course.

15. Examples of Outcomes (ECE)
College outcome: communication
Program outcome: Create and modify environments and experiences to meet the individual needs of all children, including children with developmental delays and special abilities.
Course outcome: Recognize 6 strategies for dealing with children’s behavior that the students find challenging.

16. Indirect indicators of student learning
Level
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Indirect indicators (surveys)
CCSSE
Graduate / employer surveys
Course evaluations
Ask students to reflect on their learning through surveys, focus groups, interviews (students’ perceptions of their learning)
2. Indirect indicators ask students to reflect on their learning thru for example surveys, focus groups, and interviews (so these are students perceptions of their learning).

17. Institutional Data Level Type of data College Program Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Indirect indicators (surveys)
CCSSE
Graduate / employer surveys
Course evaluations
Institutional data (numbers and rates)
Diversity comparisons
Graduation and retention rates
Course completion rates
3. Institutional data such as retention and graduation rates, do not necessarily indicate student learning but do reflect the overall condition and effectiveness of the college.
Colleges need to collect this data at the college, program, and course levels in order to have a complete assessment and institutional research program.
Reflect the overall condition and effectiveness of the College

18. Today’s Outcomes Define assessment.
Identify three levels of assessment.
Articulate three types of assessment information or data and provide examples of each.
Identify two phases in creating an effective outcomes assessment process.

19. 1. Direct Indicators of Student Learning
Level
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Require students to demonstrate their learning through essays, demonstrations, presentations, etc.

20. Two phases of outcomes process
(1)
Development
(2) Implementation

21. Development of outcomes process Educate faculty / staff / students
State learning outcomes
Develop rubrics to measure outcomes
Determine courses that will introduce / reinforce / assess outcomes (e.g., curriculum map)
Include outcomes on syllabi
Develop activities in required courses that will teach outcomes
Develop activities in required courses that will assess outcomes
State level of expected performance
Establish a schedule for assessment
Determine who will interpret results
This example is at the college level…
educate faculty / staff / students about assessment
revise outcomes
develop assessment tools (e.g., rubrics) to measure outcomes
determine which courses or experiences will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map)
include outcomes on syllabi
collect instructional assignments, activities, projects, or experiences, in required courses that will be used to teach outcomes at entry, midway, and exit
collect activities, experiences, projects, essays, or assignments in required courses that will be used to assess outcomes at entry and exit
attach anchor papers (i.e., examples) for each level of the scoring guide/rubric scale
Usually takes years to develop well.

22. (2) Implementation of outcomes process
Assess students at (entry and) exit for outcomes
Analyze (entry and) exit assessment data
Present analysis to faculty/students and consult on the results
Use the data to improve and revise curriculum
Document the process and how the data were used to improve learning

23. Today’s Outcomes Define assessment.
Identify three levels of assessment.
Articulate three types of assessment information or data and provide examples of each.
Identify two phases in creating an effective outcomes assessment process.
Other items:
Assessment plan and report

25. Within WCC’s Assessment Plan, we’ve created a subplan for each of the 9 quadrants of this table. I am going to provide excerpts of examples of each under only the direct indicator row.

26. Today’s Outcomes Define assessment.
Identify three levels of assessment.
Articulate three types of assessment information or data and provide examples of each.
Identify two phases in creating an effective outcomes assessment process.
Other items:
Assessment plan and report
Accreditation

27. Summary of NWCUU Recommendations
Implement program (academic) and service review processes
Identify and assess student learning outcomes (at the course, program, and college levels)
Identify, collect, analyze, and use appropriate and accurate data to determine the effectiveness of our accomplishments

28. Recommendation 1
Implement program (academic) and service review processes
Programs: all or substantially all of a body of coursework leading to a degree or certificate (e.g., massage therapy certificate, associate of arts and science degree, etc.)
Services: activities that support programs (e.g., bookstore, information technology, entry and advising, conference and events services, etc.)
In the next short sequence of slides, for each of the R’s I will outline our plan for this academic year.

29. Direct indicators (learning outcomes)
Services Review
Recommendation #1
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Indirect indicators (surveys)
CCSSE
Graduate / employer surveys
Course evaluation
Institutional data (numbers and rates)
Achieving the Dream
Graduation & retention rates
Course completion rates
R#1 contains 2 components. The program review involves the program column and services review involves a similar process with our services.

30. Recommendation 2
Implement program (academic) and service review processes
Identify and assess student learning outcomes (at the course, program, and college levels)

31. Direct indicators (learning outcomes)
Level
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Indirect indicators (surveys)
CCSSE
Graduate / employer surveys
Course evaluation
Institutional data (numbers and rates)
Achieving the Dream
Graduation & retention rates
Course completion rates
Recommendation #2

32. Recommendation 3
Implement program (academic) and service review processes
Identify and assess student learning outcomes (at the course, program, and college levels)
Identify, collect, analyze, and use appropriate and accurate data to determine the effectiveness of our accomplishments
R#3 involves generating accurate data and using it to improve our college.

33. Direct indicators (learning outcomes)
Level
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Indirect indicators (surveys)
CCSSE
Employer surveys
Course evaluation
Institutional data (numbers and rates)
Achieving the Dream
Graduation & retention rates
Course completion rates
R#3 covers the last row of this assessment grid.
Recommendation #3

34. Direct indicators (learning outcomes)
Services Review
Recommendation #1
Type of data
College
Program
Course
Direct indicators (learning outcomes)
Core learning abilities
Program outcomes
Course outcomes
Indirect indicators (surveys)
CCSSE
Employer surveys
Course evaluation
Institutional data (numbers and rates)
Achieving the Dream
Graduation & retention rates
Course completion rates
Recommendation #2
This is a good summary diagram of how the 3 R’s relate to one another and the assessment work. As you can see, R#1 overlaps with Rs #2 and 3.
Recommendation #3

35. Today’s Outcomes Define assessment.
Identify three levels of assessment.
Articulate three types of assessment information or data and provide examples of each.
Identify two phases in creating an effective outcomes assessment process.
Other items:
Assessment plan and report
Accreditation
Website

36. Website http://www.faculty.whatcom.ctc.edu/InstResearch/
We have an Assessment website.
We are constantly adding new data and information to the website and we hope people will use this information to make decisions.
This and other presentations are posted on the website under “Teaching, Learning, and Assessment Resources”.

37. Comments and Questions?