2. THE DIVERGENT BUT COMPLEMENTARY PATHS OF COMPREHENSIVE PROGRAM REVIEW AND STUDENT LEARNING OUTCOMES ASSESSMENT A Case Study Presentation At the 2004 AAHE Assessment Conference On June 15, 2004 By Ed Rugg & Val Whittlesey Kennesaw State University

3. TWO MAJOR DOMAINS OF ASSESSMENT PROGRAM REVIEW LEARNING OUTCOMES

4. FOCUS OF PROGRAM REVIEW UNIT OF ANALYSIS A DEGREE PROGRAM AN ORGANIZATIONAL UNIT ASSESSMENT ISSUES INPUTS, PROCESSES, OUTPUTS QUALITY INDICATORS PRODUCTIVITY INDICATORS CENTRALITY & VIABILITY ROI & INVESTMENT POTENTIAL

5. FOCUS OF LEARNING OUTCOMES UNIT OF ANALYSIS THE GRADUATE OF THE PROGRAM STUDENT LEARNING ASSESSMENT ISSUES KNOWLEDGE OF GRADUATES SKILLS OF GRADUATES ATTITUDES/VALUES OF GRADUATES EDUCATIONAL GROWTH/GAINS

6. COMPLEMENTARY OVERLAP Learning Outcomes Constitute an Important, but Only One, Set of Outputs Evaluated in Program Review

7. ACCREDITATION’S QUEST FOR QUALITY ENHANCEMENT Southern Association’s (SACS/COC) PRINCIPLES OF ACCREDITATION is subtitled: FOUNDATIONS FOR QUALITY ENHANCEMENT “The concept of quality enhancement is at the heart of the Commission’s philosophy of accreditation….an institution is expected to document quality and effectiveness in all its major aspects.” (p. 5)

8. SACS CORE REQUIREMENTS FOR PROGRAM REVIEW 2.5 The institution engages in ongoing, integrated, and institution- wide research-based planning and evaluation processes that incorporate a systematic review of programs and services….(Institutional Effectiveness) 2.12 The institution has developed an acceptable Quality Enhancement Plan…that …is part of an ongoing planning and evaluation process. (QEP)

9. SACS COMPREHENSIVE STANDARDS FOR LEARNING OUTCOMES 3.4.1 …each educational program for which academic credit is awarded …establishes and evaluates program and learning outcomes. 3.5.1 The institution identifies college- level competencies within the general education core and provides evidence that graduates have attained those competencies.

10. SACS COMPREHENSIVE STANDARD ON INSTITUTIONAL EFFECTIVENESS 3.3 Institutional Effectiveness 3.3.1 The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.

11. IMPLEMENTING A DUAL FOCUS ON ASSESSMENT AT KSU  COMPREHENSIVE PROGRAM REVIEW  ASSURANCE OF LEARNING

12. IMPORTANCE OF A WEB-BASED ARCHIVE OF DOCUMENTATION Managing Complex Multi- Year CPR and AOL Processes Anytime Access, Referral & Communication on CPR & AOL Sharing Experiences & Good Practices Documenting Compliance & Improvement for Accreditation

13. ASSESSMENT REPORTING PRIOR TO 2003 From 1994 to 2003, academic departments at KSU sent their annual assessment reports to the Office of Institutional Planning. These nonstandard annual reports, ranging from 1 to 50+ pages, were simply filed away with no follow-up or feedback.

14. DISSATISFACTION WITH THE “A” WORD & REPORTS Unfocused Reports – assessments rarely tied to learning outcomes or program goals. Uneven Reports – from exemplary/sophisticated content to inadequate/unsophisticated. Unappreciated – widespread aversion to and avoidance of assessment.

15. ASSURANCE OF LEARNING INITIATIVE AT KSU Established an Assurance of Learning (AOL) Council in Spring of 2003 with High-Level Academic Leadership (AVPAA as Chair). Council Membership Includes Two Assessment-Oriented Faculty from each College, the Director of Excellence in Teaching & Learning, and the Director of Institutional Effectiveness.

16. AOL COUNCIL FUNCTIONS The AOL Council serves as a steering committee and advisory/consultative group to academic programs as they design, implement, and improve their assessments of student learning outcomes. The AOL Council provides common structures and good practices for learning outcomes assessment and collegial formative feedback on annual reports at the program level.

17. EIGHT ELEMENTS OF STUDENT LEARNING OUTCOMES ASSESSMENT Articulating Student Learning Outcomes Connecting Outcomes to the Curriculum Connecting Outcomes to Assessment Methods Articulating Expected Results with Respect to Outcomes Articulating the Assessment Plan for Collecting and Analyzing Data Collecting and Analyzing Data Comparing Actual and Expected Results Using Results for Improvement

18. BEGINNING AT THE BEGINNING IN 2003-2004 Articulating General and Specific Student Learning Outcomes for Each Degree Program and General Education Connecting Learning Outcomes to the Curriculum

19. IMPLEMENTATION OF A COLLABORATIVE MODEL Reporting and Evaluation Guidelines Formal Workshops Informal Brown Bag Consultations Council Review of and Formative Feedback on AOL Reports Assessment Resources Examples of Good Practice

20. AOL COUNCIL REVIEW PROCESS IN SPRING 2004 AOL Council used Evaluation Guidelines to provide formative feedback on the program reports. Used subcommittee assignments for Council’s reviews. 48 AOL program reports reviewed with their authors by the Council.

21. LESSONS LEARNED & NEXT STEPS

22. QUESTIONS?? PowerPoint & Handouts Available for Download at http://ie.kennesaw.edu