1. Developing an Assessment Plan Owens Community College Assessment Day Workshops November 13-14, 2009 Anne Fulkerson, Ph.D. Institutional Research

2. Agenda What is assessment? Why assess? Types of assessment Basic steps in the assessment planning process Identifying outcomes Defining measures Evaluating measures Other important considerations OCC assessment plan template & rubric Sample assessment plan How IR can help

3. What is Assessment? “Assessment is the ongoing process of: Establishing clear, measurable expected outcomes of student learning. Ensuring that students have sufficient opportunities to achieve those outcomes. Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations. Using the resulting information to understand and improve student learning.” Suskie (2004), pg. 3

4. Why Assess? Demonstrate effectiveness Improve student learning Measure Learning Measure Learning Adapted from Suskie (2004) and Maki (2004) Provide Learning Opportunities Establish Learning Outcomes Implement Change Implement Change

5. Types of Assessment Formative— assessment that takes place while learning is in progress (or while a new program is being developed) in order to provide feedback for improvement Summative— assessment that takes place at the end of a course or program to document that learning has occurred or to make judgments about the efficacy of a program

6. Establish Mission Statement

7. Identify Student Learning Outcomes Student Learning Outcomes : Specifiable activities, products, skills, abilities, behaviors, attitudes, or pieces of knowledge that students attain as a result of their involvement in a particular set of educational experiences. Represent the most important competencies that all students should possess Do not reflect ALL learning that might occur Often aligned with accrediting body standards No magic number, but keep them manageable

8. Expressing Student Learning Outcomes Too vague: Students will demonstrate information literacy skills Too specific: Students will be able to use institutional online services to retrieve information Better: Students will locate information and evaluate it critically for its validity and appropriateness

9. Expressing Student Learning Outcomes Fuzzy Terms:Action Words: Know Understand Become aware of Appreciate Think critically Demonstrate knowledge Learn Write proficiently Remember Identify Perform Create Define Summarize Explain Discuss Describe Solve Find/Locate Analyze Evaluate Apply Debate/Argue Synthesize Integrate Research Choose Construct Design Develop Organize Use

10. Bloom’s Taxonomy (Revised) Based on an adaptation of Anderson & Krathwohl (2001) http://uwf.edu/cutla/assessstudent.cfm Thinking Skills

11. Not Measurable: Recognize a need for lifelong learning and plan for personal and professional growth Demonstrate an historical knowledge of the symphonic, string orchestra, and chamber ensemble repertoire Sample Student Learning Outcomes Measurable: Describe and adopt a plan for ongoing professional development and lifelong learning Describe the historical development of the symphonic, string orchestra, and chamber ensemble repertoire http://uwf.edu/cutla/assessstudent.cfm

12. Identify Opportunities for Students to Achieve Each Learning Outcome Course / Activity / Experience Learning Outcome 1. 2. 3. 4. 5.

13. Define Measures for Each Outcome Types of Measures: Quantitative —numeric (e.g., test scores, placement rates, GPA, structured surveys) Qualitative —described by words rather than numbers (e.g., interviews, focus groups, observations, rubrics) Direct —require students to display knowledge or skill (e.g., tests, performances, assignments) Indirect —second-hand evidence (e.g., surveys, course evaluations, journal reflections)

14. Word of Caution Don’t measure everything, just because you can Pick the best measures Keep it manageable

15. Determine How Measures will be Evaluated Criterion-based benchmarks— compares student performance with a pre-established standard. Norm-referenced benchmarks— compares student performance with a standardized norm or a group of peers. Value-added approach— compares student performance at two points in time to see if they have improved. Longitudinal/historical approach— compares current students with prior students.

16. Other Things to Consider What resources are needed? Who’s responsible for collecting and analyzing the data? Build support through participation in the planning process Assessment plans are perpetual drafts

17. OCC Assessment Plan Template https://intranet.owens.edu/committees/outcomes/index.htmlintranet.owens.edu/committees/outcomes/index.html

18. OCC Assessment Plan & Report Rubric https://intranet.owens.edu/committees/outcomes/index.htmlintranet.owens.edu/committees/outcomes/index.html

19. Sample Assessment Plans Owens— https://www.owens.edu/portrait/index.html https://www.owens.edu/portrait/index.html College of Charleston, SC— http://spinner.cofc.edu/~oap/docs.html http://spinner.cofc.edu/~oap/docs.html Supplemental Instruction

20. How IR Can Help Data support and analysis Assessment consulting Survey development Rubric development Identifying appropriate metrics Identifying existing data sources

21. References Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing: Herndon, VA. Palomba, C. & Banta, T.W. (1999). Assessment Essentials. Planning, Implementing, and Improving Assessment in Higher Education. John Wiley & Sons: San Francisco, CA. Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Jossey-Bass: San Francisco, CA. Owens Community College, Student Learning Assessment Committee (2009). Building your SLAC assessment plan: Instructions for the SLAC assessment plan template version #1: intranet.owens.edu/committees/outcomes/index.html. intranet.owens.edu/committees/outcomes/index.html University of West Florida, Center for University Teaching, Learning, & Assessment. http://uwf.edu/cutla/Assessres.cfm.http://uwf.edu/cutla/Assessres.cfm