Assessment Basics PNAIRP Conference Thursday October 6, 2011 Anne Marie Karlberg Director of Institutional Research and Assessment Whatcom Community College (360) 383-3302 amkarlberg@whatcom.ctc.edu http://faculty.whatcom.ctc.edu/InstResearch/index.htm

Today’s Outcomes Define assessment. Identify three levels of assessment. Articulate three types of assessment information or data and provide examples of each. Identify two phases in creating an effective outcomes assessment process. Other items: Assessment plan and report Accreditation Website Outcomes: By the end of this session, participants will be able to…

Today’s Outcomes Define assessment.

What is assessment? “The systematic collection of information about student learning…to inform decisions about how to improve learning.” (Walvoord, 2004, p. 2) This will be a review for some of you. But first, we need to step back and make sure we’re all operating with a similar definition of assessment. This is the most succinct and clearest definition I know of assessment. Assessment is...

Purposes improvement (formative) accountability (summative) 2 purposes of assessment are 1. improve student learning and performance (internal improvement) 2. demonstrate to external accreditation bodies that relationships exist between the college’s mission and learning outcomes.

Today’s Outcomes Define assessment. Identify three levels of assessment.

3 Levels of Assessment Level College

3 Levels of Assessment Level College Program

3 Levels of Assessment Level College Program Course

Today’s Outcomes Define assessment. Identify three levels of assessment. Articulate three types of assessment information or data and provide examples of each.

1. Direct Indicators of Student Learning Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Require students to demonstrate their learning through essays, demonstrations, presentations, etc. There are 3 types of data that are used to evaluate student learning… Direct indicators of student learning require students to demonstrate their learning through, for example, essays, capstone projects, demonstrations, and presentations. Sometimes people refer to “Direct indicators” as “outcomes assessment”. Outcomes are things students are able to do at the end of their college experience, their program, or a course.

College Outcomes Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Require students to demonstrate their learning through essays, demonstrations, presentations, etc. …overarching skills that are emphasized throughout many courses in all programs at WCC. CLAs define the skills we would like all students to develop by the time they graduate (with a certificate or a degree).

Program Outcomes Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Require students to demonstrate their learning through essays, demonstrations, presentations, etc. …overarching skills that are emphasized and reinforced throughout several courses in a specific program. They are measurable statements that define what students will be able to do – over and above the college outcomes – by the end of a certificate or degree. Note: A program is all or substantially all of a body of coursework leading to a degree or certificate (e.g., AAS degree, massage practitioner AS degree, accounting certificate, etc.)

Course Outcomes Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Require students to demonstrate their learning through essays, demonstrations, presentations, etc. …the most important skills students should develop by the end of a course. They are unique to a specific course.

Examples of Outcomes (ECE) College outcome: communication Program outcome: Create and modify environments and experiences to meet the individual needs of all children, including children with developmental delays and special abilities. Course outcome: Recognize 6 strategies for dealing with children’s behavior that the students find challenging.

Indirect indicators of student learning Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Indirect indicators (surveys) CCSSE Graduate / employer surveys Course evaluations Ask students to reflect on their learning through surveys, focus groups, interviews (students’ perceptions of their learning) 2. Indirect indicators ask students to reflect on their learning thru for example surveys, focus groups, and interviews (so these are students perceptions of their learning).

Institutional Data Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Indirect indicators (surveys) CCSSE Graduate / employer surveys Course evaluations Institutional data (numbers and rates) Diversity comparisons Graduation and retention rates Course completion rates 3. Institutional data such as retention and graduation rates, do not necessarily indicate student learning but do reflect the overall condition and effectiveness of the college. Colleges need to collect this data at the college, program, and course levels in order to have a complete assessment and institutional research program. Reflect the overall condition and effectiveness of the College

Today’s Outcomes Define assessment. Identify three levels of assessment. Articulate three types of assessment information or data and provide examples of each. Identify two phases in creating an effective outcomes assessment process.

1. Direct Indicators of Student Learning Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Require students to demonstrate their learning through essays, demonstrations, presentations, etc.

Two phases of outcomes process (1) Development (2) Implementation

Development of outcomes process Educate faculty / staff / students State learning outcomes Develop rubrics to measure outcomes Determine courses that will introduce / reinforce / assess outcomes (e.g., curriculum map) Include outcomes on syllabi Develop activities in required courses that will teach outcomes Develop activities in required courses that will assess outcomes State level of expected performance Establish a schedule for assessment Determine who will interpret results This example is at the college level… educate faculty / staff / students about assessment revise outcomes develop assessment tools (e.g., rubrics) to measure outcomes determine which courses or experiences will be used to introduce, reinforce, and/or assess outcomes at entry, midway, and exit (e.g., curriculum map) include outcomes on syllabi collect instructional assignments, activities, projects, or experiences, in required courses that will be used to teach outcomes at entry, midway, and exit collect activities, experiences, projects, essays, or assignments in required courses that will be used to assess outcomes at entry and exit attach anchor papers (i.e., examples) for each level of the scoring guide/rubric scale Usually takes years to develop well.

(2) Implementation of outcomes process Assess students at (entry and) exit for outcomes Analyze (entry and) exit assessment data Present analysis to faculty/students and consult on the results Use the data to improve and revise curriculum Document the process and how the data were used to improve learning

Today’s Outcomes Define assessment. Identify three levels of assessment. Articulate three types of assessment information or data and provide examples of each. Identify two phases in creating an effective outcomes assessment process. Other items: Assessment plan and report

Within WCC’s Assessment Plan, we’ve created a subplan for each of the 9 quadrants of this table. I am going to provide excerpts of examples of each under only the direct indicator row.

Today’s Outcomes Define assessment. Identify three levels of assessment. Articulate three types of assessment information or data and provide examples of each. Identify two phases in creating an effective outcomes assessment process. Other items: Assessment plan and report Accreditation

Summary of NWCUU Recommendations Implement program (academic) and service review processes Identify and assess student learning outcomes (at the course, program, and college levels) Identify, collect, analyze, and use appropriate and accurate data to determine the effectiveness of our accomplishments

Recommendation 1 Implement program (academic) and service review processes Programs: all or substantially all of a body of coursework leading to a degree or certificate (e.g., massage therapy certificate, associate of arts and science degree, etc.) Services: activities that support programs (e.g., bookstore, information technology, entry and advising, conference and events services, etc.) In the next short sequence of slides, for each of the R’s I will outline our plan for this academic year.

Direct indicators (learning outcomes) Services Review Recommendation #1 Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Indirect indicators (surveys) CCSSE Graduate / employer surveys Course evaluation Institutional data (numbers and rates) Achieving the Dream Graduation & retention rates Course completion rates R#1 contains 2 components. The program review involves the program column and services review involves a similar process with our services.

Recommendation 2 Implement program (academic) and service review processes Identify and assess student learning outcomes (at the course, program, and college levels)

Direct indicators (learning outcomes) Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Indirect indicators (surveys) CCSSE Graduate / employer surveys Course evaluation Institutional data (numbers and rates) Achieving the Dream Graduation & retention rates Course completion rates Recommendation #2

Recommendation 3 Implement program (academic) and service review processes Identify and assess student learning outcomes (at the course, program, and college levels) Identify, collect, analyze, and use appropriate and accurate data to determine the effectiveness of our accomplishments R#3 involves generating accurate data and using it to improve our college.

Direct indicators (learning outcomes) Level Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Indirect indicators (surveys) CCSSE Employer surveys Course evaluation Institutional data (numbers and rates) Achieving the Dream Graduation & retention rates Course completion rates R#3 covers the last row of this assessment grid. Recommendation #3

Direct indicators (learning outcomes) Services Review Recommendation #1 Type of data College Program Course Direct indicators (learning outcomes) Core learning abilities Program outcomes Course outcomes Indirect indicators (surveys) CCSSE Employer surveys Course evaluation Institutional data (numbers and rates) Achieving the Dream Graduation & retention rates Course completion rates Recommendation #2 This is a good summary diagram of how the 3 R’s relate to one another and the assessment work. As you can see, R#1 overlaps with Rs #2 and 3. Recommendation #3

Today’s Outcomes Define assessment. Identify three levels of assessment. Articulate three types of assessment information or data and provide examples of each. Identify two phases in creating an effective outcomes assessment process. Other items: Assessment plan and report Accreditation Website

Website http://www.faculty.whatcom.ctc.edu/InstResearch/ We have an Assessment website. We are constantly adding new data and information to the website and we hope people will use this information to make decisions. This and other presentations are posted on the website under “Teaching, Learning, and Assessment Resources”. http://www.faculty.whatcom.ctc.edu/InstResearch/

Comments and Questions?