1. 2006 Fall Workshop PLANNING and ASSESSMENT: A TUTORIAL FOR NEW DEPARTMENT CHAIRS – A REFRESHER COURSE FOR OTHERS

2. Planning and Assessment Workshop Session 2 Assessment of Student Learning Outcomes and Departmental Goals T. Gilmour Reeve Director of Strategic Planning Kari Wood Coordinator, Teaching, Learning, & Technology Center 2:00 – 3:45 November 10, 2006

3. OUTCOMES ASSESSMENTS OF STUDENTS’ LEARNING Two boys are walking down the street. The first boy says, “I’ve been really busy this summer. I’ve been teaching my dog to talk.” His friend responds, “Wow! I can’t wait to have a conversation with your dog.” The first boy shakes his head. “I said I’ve been teaching him. I didn’t say he learned anything.” M.J. Allen Assessing Academic Programs in Higher Education

4. Planning and Assessment Cycle Planning (purpose & goals) Implementation courses, programs, strategies Assessment (measurement & evaluation)

5. Planning and Assessment Possible Responses to Assessment: 1. No changes needed; 2. Change Purpose and Goals; 3. Change Programs, research, service; 4. Change Assessments. Planning purpose & goals Implementation Courses, programs, strategies Assessment measurement & evaluation

6. Levels of Educational Assessment o Institutional SACSCOC – “ Institutional Effectiveness” Addresses all areas and units in delivering all programs and services o Academic departments and programs Departmental goals and expected student outcomes for degree programs o Individual courses Student assessment in specific courses & teacher evaluations

7. Planning and Assessment Workshop – Session 2 There are three basic aspects to these SACSCOC standards: oThe institution DOES planning and assessment; oThe institution USES planning and assessment to improve programs and services; oThe institution DOCUMENTS the planning and assessment activities and the improvements made.

8. Planning and Assessment Workshop – Session 2 Texas Tech University: Planning and Assessment University Policies:  OP 10.13 – Strategic Planning and Assessment “ The university, including all academic programs and support operations, is engaged in an ongoing and comprehensive process of planning and assessment. All areas (divisions and colleges) and units (departments, centers, and institutes within areas) must conform to the university policies as specified in this OP.”

9. Planning and Assessment Workshop – Session 2 Texas Tech University: Planning and Assessment University Policies: OP 10.13 (cont.) “Academic programs, including program content, quality, and assessment, are the primary responsibility of the faculty within the academic disciplines. Learning outcomes assessments of academic programs are included in the departments’ annual assessment reports.”

10. Planning and Assessment Workshop – Session 2 OP 32.06 – Faculty Responsibility Specifically, within the first week of the semester, faculty members must provide a course syllabus that includes: a. The course outline; b. The expected learning outcomes from the course; c. The methods of assessing those outcomes; and d. The criteria for grade determination.

11. Assessment of Student Learning Outcomes in the Courses

12. Related Terms (1)Testing Associated primarily with traditional methods of assessment (2) Evaluation Requires the interpretation and use of information to make judgments about student learning and/or pedagogy (3) Grading May provide insufficient evidence of student learning…Why?

13. Assessment is an ongoing process of: 1)Establishing clear, measurable outcomes of student learning; 2)Ensuring that students have sufficient opportunities to achieve those outcomes; 3)Systematically gathering, analyzing, and interpreting evidence of student learning; and 4)Using the resulting information to understand and improve the learning process.

14. Definition A learning outcome is a specific statement that describes: (1) knowledge, (2) skills, and (3) attitudes that students are expected to learn upon successful completion of a course.

15. Examples After completing this course, students will be able to: Art Appreciation  Differentiate major works of art based on the period they represent. Introduction to Research Methods  Compare and contrast quantitative and qualitative research methodology.

16. Guidelines Effective learning outcome statements: A = Use concrete, action verbs C = Are specific to the course E = Focus on the end, not the means S = Are student-centered + A = Are assessable ACES+A

17. Action Verbs: Cognitive Domain RememberUnderstandApplyAnalyzeEvaluateCreate Count Define Describe Identify Label List Outline Quote Recite Select Compute Convert Defend Discuss Estimate Explain Infer Paraphrase Predict Summarize Calculate Classify Demonstrate Graph Modify Prepare Produce Solve Translate Use Analyze Arrange Combine Develop Diagram Differentiate Discriminate Illustrate Relate Separate Appraise Assess Compare Contrast Critique Interpret Judge Rate Support Test Compose Compile Create Design Devise Generate Integrate Plan Propose Reconstruct Adapted from Anderson and Krathwohl’s (2001) revised version of Bloom’s taxonomy

18. Action Verbs: Other Domains Affective: Argue, convince, debate, express, internalize, justify, support, uphold Psychomotor: Assemble, build, dissect, dribble, sculpt, repair, sketch, suture

19. Outcomes are course-specific.  Limit your learning outcomes to 5-10 statements for each course.  These statements will focus on the most important outcomes for the course, not the program or an individual unit. A C ES+A

20. Focus on the end, not the means. AC E S+A  What should students know or be able to do upon successful completion of the course? NOT  In what activities will students participate throughout the course?

21. Outcomes are student-centered. ACE S +A  What should students know or be able to do upon successful completion of the course? NOT  What will instructors do throughout the course?

22. Outcomes are assessable. ACES+ A  How can the outcome be measured?  How can we use the information to improve the learning process?

23. Assessment is an ongoing process of: 1)Establishing clear, measurable outcomes of student learning; 2)Ensuring that students have sufficient opportunities to achieve those outcomes; 3)Systematically gathering, analyzing, and interpreting evidence of student learning; and 4)Using the resulting information to understand and improve the learning process.

24. Assessment is an ongoing process of: 1)Establishing clear, measurable outcomes of student learning; 2)Ensuring that students have sufficient opportunities to achieve those outcomes; 3)Systematically gathering, analyzing, and interpreting evidence of student learning; and 4)Using the resulting information to understand and improve the learning process.

25. Best Assessment Practices 1)Assess before, during, and after student learning has occurred. 2)Align assessment methods with the stated learning outcomes. 3)Use a variety of assessment methods.

26. Framework of Assessment Methods TraditionalAlternative Selected- Response Constructed- Response ProductPerformanceProcess- Focused Multiple-choice True-False Matching Fill-in-the-blank Short answer Label a diagram “Show your work” Essay Research paper Lab report Story/play Poem Portfolio Art exhibit Science project Model Video/audiotape Poster presentation Powerpoint Dance Recital Demonstration Athletic skill Dramatic reading Enactment Debate Teach a lesson Observation Interview Presentation Double-entry journal http://newton.nap.edu/html/classroom_assessment/ch4_t1.html

27. Classroom Assessment Technique (CAT) Short-Writes: oOne Minute Paper: What was the most important thing you learned in class? What question remains unanswered? oMuddiest Point: What was most confusing about your reading assignment last night? oApplication Cards: Describe one practical application of what we’ve discussed today.

28. Assessment is an ongoing process of: 1)Establishing clear, measurable outcomes of student learning; 2)Ensuring that students have sufficient opportunities to achieve those outcomes; 3)Systematically gathering, analyzing, and interpreting evidence of student learning; and 4)Using the resulting information to understand and improve the learning process.

29. Closing the Loop  Are the learning outcomes appropriate?  Is the curriculum aligned with the stated learning outcomes?  Are your instructional methods effective?

30. Closing the Loop (cont.) “There is no such thing as effective teaching in the absence of learning. Teaching without learning is just talking.” -Angelo & Cross (1993)

31. Planning and Assessment Workshop – Session 2 ASSESSMENT OF STUDENT LEARNING IN THE COURSE QUESTIONS AND COMMENTS?

32. Planning and Assessment Workshop – Session 2 ASSESSING STUDENT LEARNING IN THE ACADEMIC PROGRAMS (PROGRAM-LEVEL ASSESSMENT OF STUDENT LEARNING OUTCOMES)

33. Planning and Assessment Workshop – Session 2 The term “program-level” refers to learning outcomes and assessment methods related to a degree program Program-level assessment investigates what the students are learning in their degree programs Program-level assessment is not the same as the course grades for students.

34. Planning and Assessment Workshop – Session 2 “Seventy-five percent of students will earn a grade of X [“C” or “B”] or better in Course 1234 ” Concern for using course grades as measures in program level assessment: o A grade is an aggregate of the student’s performances on class assignments (not directly tied to a specific expected outcome) o Different students will earn the same course grades but will perform differently on course assignments o Focusing on grades for assessing programs may encourage grade inflation o Using a minimal course grade of “C” only states what is required for graduation

35. Planning and Assessment Workshop – Session 2 Program-level learning outcomes and assessment are reflected in the Strategic Plan – on the goals related to undergraduate and graduate education But the expected outcomes for each degree program do not have to be stated in the STRATEGIC PLANS but are reported in the ANNUAL ASSESSMENT REPORTS

36. Planning and Assessment Workshop – Session 2 o The first step in writing and assessing program-level learning outcomes is determining the purpose of the degree program o It is important to engage faculty in this (and all subsequent) steps in the process

37. Planning and Assessment Workshop – Session 2 Questions to ask to help determine the course purpose include: o What role does this program play in the department or college? o What is unique or special about this academic area/degree program? o What will the students “do” with their degrees?

38. Planning and Assessment Workshop – Session 2 Once the program’s purpose has been established, specific learning outcomes can be developed A program-level expected learning outcome is a formal statement of what the program’s faculty expects students to learn in a particular degree program

39. Planning and Assessment Workshop – Session 2 Expected learning outcomes define: o What faculty expect students to know at the end of the degree program AND o What faculty expect students to be able to do at the end of the degree program

40. Planning and Assessment Workshop – Session 2 Once the learning outcomes for a degree have been established, a variety of methods are used to assess if the students’ learning matches those expected outcomes.

41. Planning and Assessment Workshop – Session 2 Once the learning outcomes for a degree have been established, a variety of methods are used to assess if the students’ learning matches those expected outcomes.

42. Planning and Assessment Workshop – Session 2 Departments should develop assessment plans that outline how and when they will be investigating how well both the current students and students who have graduated have learned/mastered the program-level learning outcomes

43. Planning and Assessment Workshop – Session 2 Methods to Assess Current Students’ Learning o Capstone courses o Review of senior projects o Focus groups o Students’ performance on comprehensive exams

44. Planning and Assessment Workshop – Session 2 Methods to Assess Graduated Students’ Learning o Graduate follow-up surveys o Performance on licensure exams o Job placement data o Exit interview/surveys

45. Planning and Assessment Workshop – Session 2 Strategies for using outcomes assessments of student learning: oEngage the faculty o Begin with discussions at first faculty meetings of the year o Use available resources (TLTC and Office of Strategic Planning – see websites) o End the year with a faculty review – make improvements based on assessments. CLOSE THE LOOP!

46. Planning and Assessment Workshop – Session 2 See examples in handouts for incorporating outcomes assessments in the Annual Assessment Reports Also additional information on outcomes assessment is provided in the handouts Questions and comments?