ROBBIE TEAHEN ROBERT VON DER OSTEN KIM WILBER TracDat Overview

One Approach to Outcomes

Purpose of Assessment Simply: To advance the quality of student learning through careful elaboration of intended learning, meaningful measures of students learning achievements, and systematic collection of data that informs instructional and other improvements – at the level of courses, programs, colleges, and institutions. Assessment involves going beyond the evaluation of individual student performance (Teahen, 2008).

The Learning Design Cycle Assess performance Determine learning Needs Analyze learner needs Specify learning outcomes Plan learning activities Curriculum Development

Session Outcomes As a result of participating in todays session, you will: Demonstrate your ability to log in to the system Change your password Describe the meaning of relevant terms, such as reporting or assessment units and learning outcomes Create an appropriate learning outcome Enter assessment unit (e.g. program) information (if unit admin) Create at least one learning outcome entry Know the tentative implementation plans/timelines – pending further refinement Be prepared to continue to enter target outcomes/goals, assessment plans, and your results on an ongoing basis

Goals of the Implementation Create a repository for conserving assessment information Stimulate cross-discipline sharing of learning and institutional effectiveness outcomes Focus unit-level efforts on specifying and monitoring assessment outcomes Encourage course-, program-, and institutional-level learning enhancements Streamline reporting requirements for academic units

Topics to be Included Today User roles Access Changing the password Entering data – outcomes, plans, and results Attaching documents Linking assessment unit outcomes to reporting unit standards or plans Producing reports Resources (Documents, power users, and Oversight team, Kim)

Roles and Security Roles: User Reports-Only Program-level admin Ferris Administrator (Kim Wilber) Access and Security levels and permissions: Form (see Academic Affairs assessment website) Approvals by Deans

Attempting to Allay Common Concerns Comparing instructor performance... Outcomes and measures are expected to be highly unique, so it will not be possible to compare performance across programs or disciplines Instructor Evaluation Instructors will continue to be evaluated by existing methods. If they want to use this information to demonstrate their effectiveness, its their option. What Information is Reported Inclusion is decided at the appropriate level... What the program personnel put in is what they decide to enter. Most reporting will be at the program level, and course-level reported is expected to evolve at the This Too Shall Pass... Assessment of student learning has been required since Expectations have heightened. I Dont Have Time (Its too much work)... Assessment is integral to the work of a professional educator. It is a part of the cycle of instructional design and the continuous improvement approach of plan, do, check, act

Assessment Unit Information Show/Tell/Do (using the Demo Site) Unit Name Mission Sites Offered Accreditation Body Date of next accreditation visit Advisory Board? Next program review data

Assessment Plan Information Show/Tell/Do (using Demo) Outcome name Outcome statement Assessment Method Criterion for Success Related Goals Additional assessment methods

Outcome Checklist describe one of the major skills that is an intended outcome for a course or program represent a skill that a competent individual would use outside the context of the course begin with an action verb describing what the learner will be able to do upon completion of this course/program are measurable and observable require application of skill, knowledge, or attitude/value – at a level of application or above on Blooms taxonomy present a clear, concise, and precise statement describing the action specify a single performance/outcome, not a combination describe learner performance, not the instructor's activities, learning plans, or instructional strategies

Sample Program-Level Outcome and Measure(s) Outcome: Technology Use Outcome Statement: Learners will demonstrate their use of common functions associated with software relevant to the discipline (e.g. MS Office, SPSS, CAD, etc.) Measure(s): 1. Capstone project assignments will incorporate the utilization of common software applications associated with the field. Rubrics will be provided for each that address learner performance in technology use. 2. Exams in the second-year major course will incorporate timed tests utilizing identified software to produce documents appropriate to meet external performance requirements. Standards of the profession will be utilized to assess learners performance. 3. Throughout the program, individual course requirements will incorporate and report on technology-use performance by students, as appropriate

Components to Enter in Plan (Program) Outcome Name Outcome Measures Outcome Type Status (active or not) Star

Dont Forget! SAVE CHANGES (button at bottom)

Means of Assessment Relating to a particular outcome, specify: Method Category list still being refined Offer your suggestions Method (Description) Criterion What will success look like for this program or course? Schedule When will assessment methods be implemented? Frequency?

Sample Results and Action Plan Learners will demonstrate their use of common functions associated with software relevant to the discipline (e.g. MS Office, SPSS, CAD, etc.) Measure(s): 1. Capstone project assignments will incorporate the utilization of common software applications associated with the field. Rubrics will be provided for each that address learner performance in technology use. Results: Review of 32 capstone projects for students in the X program during the spring of 2008 revealed that 95% of the learners were able to perform all specified functions within MS Word and Powerpoint, but just 62% could demonstrate their abilities to perform specified functions within Access. Further, AutoCAD design capabilities were rated to be at an average level of 3 on a scale of 1 to 5, with 10% of the soon-to-graduate students not meeting minimum standards for the profession. Action Plan: Faculty within the major will meet in August 2008 to examine the curriculum to determine where and how Access and CAD are introduced and reinforced and development supplemental units to assist students to more adequately achieve intended outcomes. Faculty meetings will address this performance concern and monitor curricular changes and end-of-year performance of students in spring Results from the Spring 2009 will be reported and determination made about whether additional curricular reform is required.

Linking Documents and to Goals Attach documents... Examples: Rubrics Standards Assignments Link to standards or criteria Such as accreditation standards Program outcomes General education outcomes

Tentative Timelines Enter Unit information immediately (mission, accreditor, etc.) Enter Program Outcomes and Measures before end of semester Enter at least one program outcome by end of spring Enter multi-section and general education course assessment plans during spring 2009 Enter outcomes as available for course-level assessments

Regarding the Learning Paradigm: "The result of this paradigm shift is a college where faculty are the designers of powerful learning environments, where curriculum design is based on an analysis of what a student needs to know to function in a complex world rather than on what the teacher knows how to teach, where the college is judged, not on the quality of the entering class, but on the quality of aggregate learning growth possessed by its graduates, where compartmentalized departments are replaced by cross-disciplinary cooperatives, and where every employee has a role to play and a contribution to make in maintaining a learner-centered environment (p. 5). -Bill Flynn, Palomar College, 1998