1. Effective Outcomes Assessment
Program Directors 101 Pando ™ Workshop
Effective Outcomes Assessment
Meredith Davison
John Burns

2. Objectives By the end of this session, learners will be able to:
Foster student success without lowering the bar
Distinguish between student outcomes assessment and program evaluation
Describe the key components of an effective assessment or evaluation plan
Develop and implement an effective student outcomes assessment plan
Develop and implement an effective program evaluation plan

3. Outcomes The key to good outcomes assessment is to
Begin with the end in mind!
If you don’t have a map you cannot get to where you want to go.
Having the wrong map can lead to the same thing.

4. Types of Outcomes
Program Outcomes
Student Outcomes

5. Definitions Outcomes Assessment
Answers the question – what (learning) has changed about the student between matriculation and graduation (and who or what is responsible)?
Program outcomes specify the measurable knowledge, skill, or behavior you expect students to be able to demonstrate as a result of their participation in the program.
Assessment uses a feedback loop that involves identifying desired results; collecting and analyzing relevant information (data); and then using the findings to make changes to your program.

6. What is the purpose of outcomes assessment?
C1.01 Evaluation
“The Program must implement an ongoing program self-assessment process that is designed to document program effectiveness and foster program improvement.”

7. Why Is Outcomes Assessment Important?
Evaluate effectiveness and alignment of curricula
Improve the program and its standing
Inform planning and decision making
Understand the impact of program changes
Highlight program successes
Provide evidence of need when requesting resources
Advise students with a set of learning outcomes

8. C1.01 Annotation Self-assessment process
Collection of data and interpreting evidence
Student learning
Program functions and outcomes
Qualitative and quantitative data
Careful thought to data:
Collection
Management
Interpretation

9. Outcome Assessment: Student and Program
Should lead to:
- Identification of strengths and opportunities for improvement
- Planning and modification of curriculum for enhancement
- Reassessment of measurement effectiveness

10. Value of Monitoring Tracks progress easily!
# of Cs in pre-clinical coursework
Attrition rates
Deceleration rates
PACKRAT
EOR Exams
PANCE
Graduation rates

11. System of Outcomes Assessment
Systematized
Organized
Comprehensive
Cost Effective
Useful

12. Infrastructure: Culture of Assessment
Buy in from faculty
Transparency
Follows institutional procedures
Resources
Linked to faculty evaluation, budget planning
Integrated in syllabi
Dissemination!

13. Assessment Cycle*
These are given to you!
List in course syllabi
Develop clearly articulated written statements of learning outcomes
Design experiences that provide opportunities to achieve outcomes
Document, document!
Use existing exams
Use results of assessments to improve teaching and learning
Assess student achievement of outcomes
*Figure modified from the Middles States Commission on Higher Education

14. 5 Stages 2. Planning and implementation of the curriculum
1. Establish clearly defined expected outcomes
2. Planning and implementation of the curriculum
3. Assess the program and students to collect data on the results of implementation.
4. Analyze the data and evaluate it relative to the expected program and student learning outcomes
5. Make program changes as needed and return to # 2. (#1 if a new outcome needs to be added.)

15. Components Measureable Outcomes Curricular MAPS
Assessment instruments (direct and indirect)
Exams
Rubrics
Schedule
Activity and Results (data collection)
Example: teach reading of ECG, get the test results
Evidence of Analysis of Results (analyze)
Example: figure out which portions of ECG students did well/poorly on; ask if this is associated with other results, i.e. cardiac physiology scores
Evidence of change based on assessment results
Documentation

16. Does the curriculum cover everything students need to know?
As a Program Director…
Does the curriculum cover everything students need to know?

17. Group Exercise 5-7 minutes
Take a moment to review one of your Program level assessment goals
Review your assigned goals; discuss what experience you have had as it pertains to this goal and what data you used to achieve this particular goal.
If you do not yet have a personal experience to draw from, be creative!

18. Developing a Plan Determine WHAT to be measured
Base on Mission and Goals
Examples of Graduate Success
PANCE performance (actual score, pass/not pass)
Employment (yes/no, type, geographic)
Examples of Didactic Success
Preceptor satisfaction [knowledge, skills, professionalism (aka attitudes)]
PAEA Copyright 2016

20. Final Words
Appreciate the culture of assessment we have in accredited professional programs.
Use the ARC-PA standards as the framework of your assessment plan.
Don’t be afraid of assessment, it’s actually easier than it sounds.

21. Resources
Accreditation Review Commission on Education for the Physician Assistant (2010). Accreditation Standards for Physician Assistant Education, 4th edition.
Commission on Colleges of the Southern Association of Colleges and Schools, (2010). Principles of Accreditation: Foundations for Quality Enhancement. Author. Available:
Suskie, L.(2009). Assessing Student Learning: A Common Sense Guide, 2nd ed. San Francisco: Jossey-Bass.
Berk R. (2006). Thirteen Strategies to Measure College Teaching, 1st ed. Sterling, Va: Stylus Publishing
National Institute for Learning Outcomes Assessment