Effective Outcomes Assessment Program Directors 101 Pando ™ Workshop Effective Outcomes Assessment Meredith Davison John Burns

Objectives By the end of this session, learners will be able to: Foster student success without lowering the bar Distinguish between student outcomes assessment and program evaluation Describe the key components of an effective assessment or evaluation plan Develop and implement an effective student outcomes assessment plan Develop and implement an effective program evaluation plan

Outcomes The key to good outcomes assessment is to Begin with the end in mind! If you don’t have a map you cannot get to where you want to go. Having the wrong map can lead to the same thing.

Types of Outcomes Program Outcomes Student Outcomes

Definitions Outcomes Assessment Answers the question – what (learning) has changed about the student between matriculation and graduation (and who or what is responsible)? Program outcomes specify the measurable knowledge, skill, or behavior you expect students to be able to demonstrate as a result of their participation in the program. Assessment uses a feedback loop that involves identifying desired results; collecting and analyzing relevant information (data); and then using the findings to make changes to your program.

What is the purpose of outcomes assessment? C1.01 Evaluation “The Program must implement an ongoing program self-assessment process that is designed to document program effectiveness and foster program improvement.”

Why Is Outcomes Assessment Important? Evaluate effectiveness and alignment of curricula Improve the program and its standing Inform planning and decision making Understand the impact of program changes Highlight program successes Provide evidence of need when requesting resources Advise students with a set of learning outcomes

C1.01 Annotation Self-assessment process Collection of data and interpreting evidence Student learning Program functions and outcomes Qualitative and quantitative data Careful thought to data: Collection Management Interpretation

Outcome Assessment: Student and Program Should lead to: - Identification of strengths and opportunities for improvement - Planning and modification of curriculum for enhancement - Reassessment of measurement effectiveness

Value of Monitoring Tracks progress easily! # of Cs in pre-clinical coursework Attrition rates Deceleration rates PACKRAT EOR Exams PANCE Graduation rates

System of Outcomes Assessment Systematized Organized Comprehensive Cost Effective Useful

Infrastructure: Culture of Assessment Buy in from faculty Transparency Follows institutional procedures Resources Linked to faculty evaluation, budget planning Integrated in syllabi Dissemination!

Assessment Cycle* These are given to you! List in course syllabi Develop clearly articulated written statements of learning outcomes Design experiences that provide opportunities to achieve outcomes Document, document! Use existing exams Use results of assessments to improve teaching and learning Assess student achievement of outcomes *Figure modified from the Middles States Commission on Higher Education

5 Stages 2. Planning and implementation of the curriculum 1. Establish clearly defined expected outcomes 2. Planning and implementation of the curriculum 3. Assess the program and students to collect data on the results of implementation. 4. Analyze the data and evaluate it relative to the expected program and student learning outcomes 5. Make program changes as needed and return to # 2. (#1 if a new outcome needs to be added.)

Components Measureable Outcomes Curricular MAPS Assessment instruments (direct and indirect) Exams Rubrics Schedule Activity and Results (data collection) Example: teach reading of ECG, get the test results Evidence of Analysis of Results (analyze) Example: figure out which portions of ECG students did well/poorly on; ask if this is associated with other results, i.e. cardiac physiology scores Evidence of change based on assessment results Documentation

Does the curriculum cover everything students need to know? As a Program Director… Does the curriculum cover everything students need to know?

Group Exercise 5-7 minutes Take a moment to review one of your Program level assessment goals Review your assigned goals; discuss what experience you have had as it pertains to this goal and what data you used to achieve this particular goal. If you do not yet have a personal experience to draw from, be creative!

Developing a Plan Determine WHAT to be measured Base on Mission and Goals Examples of Graduate Success PANCE performance (actual score, pass/not pass) Employment (yes/no, type, geographic) Examples of Didactic Success Preceptor satisfaction [knowledge, skills, professionalism (aka attitudes)] PAEA Copyright 2016

Final Words Appreciate the culture of assessment we have in accredited professional programs. Use the ARC-PA standards as the framework of your assessment plan. Don’t be afraid of assessment, it’s actually easier than it sounds.

Resources Accreditation Review Commission on Education for the Physician Assistant (2010). Accreditation Standards for Physician Assistant Education, 4th edition. Commission on Colleges of the Southern Association of Colleges and Schools, (2010). Principles of Accreditation: Foundations for Quality Enhancement. Author. Available: http://sacscoc.org/principles.asp Suskie, L.(2009). Assessing Student Learning: A Common Sense Guide, 2nd ed. San Francisco: Jossey-Bass. Berk R. (2006). Thirteen Strategies to Measure College Teaching, 1st ed. Sterling, Va: Stylus Publishing National Institute for Learning Outcomes Assessment http://www.learningoutcomeassessment.org