1. State/System Initiatives for Academic Quality: Reviewing the Options
Peter T. Ewell
National Center for Higher Education Management Systems (NCHEMS)
University of North Carolina Strategic Directions Initiative
December 12, 2012

2. Topics for Discussion What Others Are Doing (Approaches, Purposes)
Strengths and Drawbacks of Particular Approaches
Some Lessons from Experience 2

3. State Purposes for Quality Assessment
Regulating Student Flow (SD “Rising Junior”)
Assessing Institutional Performance (WV CLA)
Performance Funding (TN Schedule)
“Institution-Centered” Mandate (VA)
Technical Assistance (WV Accreditation Review) 3

4. How Many States are Doing What?
Common Standardized Test (5)
Common Student Surveys (6)
Academic Performance Indicators
Retention (44), Completion (44), Transfer (34)
Graduate Placement (17)
Policy Requiring Institutional Assessment (21) 4

5. What Are Institutions Doing?
Standardized Tests (39%)
National Surveys (76%)
Local Surveys (52%)
Alumni Surveys (43%)
Course Embedded Assessment (81%)
Portfolios (84%)
Employer Surveys (62%) 5

6. Standardized Tests Advantages: Drawbacks
Allows Comparison or Benchmarking
Professionally Developed
Drawbacks
Limited Guidance for Improvement
Uncertain Alignment with Taught Curriculum
Hard to Motivate Students to Take them Seriously 6

7. Student Surveys (National and Local)
Advantages:
Allows Comparison or Benchmarking (If National)
Professionally Developed (If National)
Can Point to Things to Improve
Drawbacks
Self-Reported Learning and Behaviors May be Unreliable
Hard to Get Good Response Rates 7

8. Academic Performance Indicators
Advantages:
Clear and Succinct Communication
Allows Comparison or Benchmarking
Drawbacks
Limited Guidance for Improvement
Susceptible to Misinterpretation or Manipulation
Can Induce Action to Fix the Number, Not the Problem 8

9. Alumni Surveys Advantages: Drawbacks
Targeted on Outcomes that Matter (Job and Graduate School Placement)
Credible to External Audiences (“Satisfied Customers”)
Can Point to Things to Improve
Drawbacks
Self-Reported Learning May be Unreliable
Hard to Get Good Response Rates 9

10. Institution-Centered Approaches
Advantages:
Institutional Buy-In and Support
Allows Tailoring Methods to Mission
Can Point to Things to Improve
Drawbacks
Can’t Compare Results Across Campuses
What to Report to Stakeholders and the Public? 10

11. Some Lessons from Experience
Balance Purposes: Accountability and Improvement
Multiple Approaches are Best (“Triangulation”)
Include a Common Core of Outcomes (e.g. DQP)
Think Carefully About Public Reporting (Unit of Analysis, Comparison)
Make the Consequences for Institutions Clear
Build in a Pilot and a Formal Periodic Review of the Approach After X Years 11