1. Implementing Good Practices in Outcomes- Based Program Review International Assessment & Retention Conference June 8, 2007 - St Louis, MO Marilee Bresciani, San Diego State University Cyd Jenefsky, John F. Kennedy University marilee.bresciani@mail.sdsu.edumarilee.bresciani@mail.sdsu.edu & jenefsky@jfku.edu jenefsky@jfku.edu marilee.bresciani@mail.sdsu.edujenefsky@jfku.edu

2. Outcomes of Session Evaluate good-practice principles in your institution’s program review processes Analyze good-practice principles applied to a case study Identify next steps for applying/ improving good-practices in your own institution

3. Inverted Funnel

4. Criteria for Good Practices for Outcomes-Based Assessment Program Review Criterion 1: Clear understanding of goals & expectations for program review Criterion 2: Collaboration Criterion 3: Use of results

5. Criterion 4: Awards and recognition Criterion 5: Resources to support program review Criterion 6: Coordination of the process

6. Criterion 7: Flexibility Criterion 8: Addressing barriers Criterion 9: Evaluation of the program review process

7. What good practice criteria do you already have in place at your institution? the sample criteria checklist to apply to your institution. Use the sample criteria checklist to apply to your institution.

8. Inverted Funnel Reminder

9. Case Study: JFK University Illustration of implementing criteria 1, 2, 5, 6, 7 & 8

10. JFKU: Process Design Groundwork: Univ-wide key stakeholder involvement High-credibility, active faculty Explicit leadership commitment Accreditation review as lever Point person in charge of development

11. Process Design Research-based –Good-practice research –Models from good-practice institutions Iterative Start with ‘what matters’ to stakeholders Train-the-trainer model Clear committee goals and timeline

12. Implementation Actively cultivate buy-in Start with “champions” Professional development at each step Address fears, anxieties, concerns –inc. one-on-one conversations Stakeholder representatives

13. Guiding Questions What do we want our students to learn by the time they complete our program? – –Intended student learning outcomes What are they actually learning? – –Actual student learning results

14. Guiding Questions (cont’d) How well are they learning these outcomes? – – Levels of achievement (based on explicit criteria) How do we know? – – Evidence of student learning

15. Guiding Questions (cont’d) Closing the loop: How are we using the evidence to guide decisions for improvement? Do the improvements we make work?

16. Capacity-Building Process builds capacity for assessment –does not presume capacity –workforce retraining Built into conceptual framework Train-the-trainer –expert –peer-to-peer training “Zone of proximal development”

17. Flexibility Constantly self-assessing & adapting Process serves “end-users” Disparate cultural groups Work with multiple accreditation standards Learning as we go

18. Apply JFK’s Process What criteria can you easily identify in their program manual that you could adapt and apply to your institution?

19. How do we move our institutions toward implementation of good practice criteria?

20. Suggestions for Implementation Create a well-represented, well-respected OBAPR committee Organize the committee’s role and responsibilities Articulate expectations for outcomes-based assessment program review Plan short-range and long-range goals

21. Suggestions (cont’d) Identify existing resources and processes and identify new resources Establish a communication plan Discuss implementation barriers and strategies to overcome them Move forward with flexibility

22. What are your organization’s next steps for implementing good practices in OBAPR?

23. What other questions do you need to ask yourselves?

24. The process is not a means unto its own end; it is a way to systematically engage in daily critical inquiry about discovering what works well and what needs to be improved. (Maki, 2004).

25. References Bresciani, M.J. (2006). Outcomes-Based Academic and Co-Curricular Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing

26. Criterion Sample Selection References American Association of Higher Education (1994). Nine principles of good practice for assessing student learning. Retrieved March 17, 2006, from http://www.aahe.org/assessment/principl.html. http://www.aahe.org/assessment/principl.html Eckel, P., Green, M., and Hill, B. (2001). On change V— Riding the waves of change: Insights from transforming institutions. Washington DC: American Council of Education. Lopez, C. (1997). Opportunities for improvement: Advice from consultant-evaluators on assessing student learning. Evidence of strong institutional support for assessing student learning. Retrieved March 16, 2006, from http://www.ncahigherlearningcommission.org/resources/ assessment/index.html.

27. One Minute Evaluation What is the most valuable lesson that you learned from this workshop? What is the most valuable lesson that you learned from this workshop? What is one question that you still have?

28. Contact info: marilee.bresciani@mail.sdsu.edu jenefsky@jfku.edu