1. Utilization-focused Assessment in Foundations Curriculum: Examining RCLS 2601: Leisure in Society Clifton E. Watts, PhD Dept. of Recreation & Leisure Studies East Carolina University wattsc@ecu.edu Abstract: Utilization-focused assessment is increasingly important when considering the format and delivery of instruction. Leisure in Society (RCLS 2601) is an ECU Social Science Foundations Curriculum course. Course formats are implemented as lecture or distance education. Multiple class sections require several faculty instructors, who bring varying levels of teaching experience and strengths in content knowledge. The purpose of this poster is to demonstrate how focused instructional training and curriculum refinement occurs through assessment utilization and planning. RCLS 2601 Overview: Assessment: Learning outcomes are assessed in face-to-face lectures, as well as distance education courses developed for the world wide-web. Specifically, RCLS 2601 focuses on three outcomes for Foundations Curriculum: Outcome 1: Understanding of Leisure Studies and Social Science: Students will learn the subject matter of Leisure Studies, a discipline in the Social Sciences Assessment Method: Direct assessment through embedded test questions on Leisure Meanings and Theory. These questions are assessed on the final exam. Criterion for Success: Answers reported to reflect the percent of students who answer each question correctly. The target is to have 70% of students enrolled in RCLS 2601 course identify the correct answer for each question. Outcome 2: Understanding Research Methodology in Leisure Studies Students will learn the research methodology, principles, and concepts required to understand and conduct undergraduate-level research in Leisure Studies Assessment Method: Direct assessment through embedded test questions on Research Methods and Techniques in Leisure Studies. These questions are assessed on the final exam. Criterion for Success: Answers are reported to reflect the percent of students who answer each question correctly. The target is to have 70% of students enrolled in RCLS 2601 course identify the correct answer for each question. Outcome 3: Demonstrating a Discipline Understanding Students will learn about the discipline’s contribution to general knowledge Assessment Method: Direct assessment through embedded test questions on Research Methods Leisure Studies. These questions are assessed on the final exam for RCLS 2601 students. Assessment of the third outcome also utilizes an indirect measure of student opinions related to how RCLS 2601 contributes to fundamental knowledge and abilities essential enhancing quality of life. Criterion for Success: Direct assessment counts correct answers and summarizes counts to reflect the percent of students who answer each question correctly. The target is to have 70% of students enrolled in RCLS 2601 course identify the correct answer for each question. Indirect assessment examines the mean values of the student opinion items with the standard mean value of five on a seven point (seven indicating strongly agree). Likert scales assess the extent to which students agree with how this course develops awareness and competence on how leisure is experienced, and the ability of how cultural and contextual variations impact leisure experiences. Conclusion: Foundations Curriculum outcomes are developed to support students’ acquisition of fundamental knowledge and enhance their abilities to learn and live successfully. Utilization-focused assessment is increasingly important when considering the format and delivery of instruction. This is especially relevant to RCLS 2601, a course taught as face-to-face lecture or through distance education. Multiple sections of the class require several faculty instructors, who often bring varying levels of teaching experience and strengths in content knowledge. Assessment was used to pinpoint where inconsistencies in outcome achievement existed, and develop appropriate instructional training and curriculum refinement to address these inconsistencies in order to enhance student learning. Over the three year period, assessment results were utilized to: -Assure consistency between distance education and lecture-based instructional formats -Develop orientation and training meetings for incoming faculty -Establish a monitoring system for distance education and lecture-based formats -Foster peer mentoring from experienced faculty in content areas where inconsistent achievement was identified -Revise and strengthen course content in distance education and lecture-based formats As a result of these actions, we observed: -Consistent performance on outcomes between distance education and lecture- based instructional formats -Better performance in outcome areas where achievement was lacking or where performance was inconsistent In the case of RCLS 2601, the utilization-focused framework has spurred action and helped close gaps related to instructional format, while setting standards for all faculty members to achieve. Assessment thrives when monitoring and program refinement processes function to support utilization plans. 2009-10 Assessment Outcomes 1 and 3 fell below standard Many items used in direct assessment were below 70% target Inconsistent performance on outcome assessment results were observed when comparing Distance Education (DE) and Lecture formats (DE sections performing better). 2010-11 Approach 1. Prioritize graduate assistant and adjunct training 2. Identify DE and Lecture course monitors 3. Strengthen Lecture-based units and implement DE developed units into lecture-based formats 2010-11 Assessment Improved scores, some individual items remained below target % for Outcomes 1 and 3 Mean scores for indirect assessment of Outcome 3 above target means for all items 2011-12 Approach 1. Continue graduate assistant and adjunct training 2. Continue utilizing DE and Lecture course monitors 3. Target curriculum refinement based on low performing items in assessment 4. Implement revised curricula in DE and Lecture units 2011-12 Assessment Continued improvement for Outcomes 1 and 3 in Fall 2011. All items above target percentages. In Spring 2012 we observed differences on outcome measures between classes taught by full time and adjunct faculty. Measures for all three outcomes were lower for adjunct when compared to full time faculty. 2012-13 Approach 1. Continue graduate assistant and adjunct training 2. Continue utilizing DE and Lecture course monitors 3. For first-time lecturers, utilize course monitors to oversee or serve as guest lecturers in content areas where outcome performance is inconsistent Results: Leisure in Society (RCLS 2601) fulfills social science foundations requirements by exploring the concept of leisure from historical, socio- cultural, and individual perspectives. Consistent with the goals of a liberal education, this course is designed to encourage students to think critically, understand contexts, reflect, and take action. This course provides students with the fundamental knowledge and abilities essential to living worthwhile lives through developing awareness about how they utilize leisure, and examining how leisure supports human function and culture.