1. Improving Undergraduate Education through the Assessment of Student Learning
SACS-COC Institute on Quality Enhancement and Accreditation
July 30, 2006
Peter T. Ewell
National Center for Higher Education Management Systems

2. Looking Back: Origins of the Assessment Movement Two Decades Ago
Undergraduate Reform Reports of
Internal Stimulus: Call for More Coherent Teaching/Learning Approaches and Information for Improvement
External Stimulus: Stakeholder Demands for Information on “Return on Investment”
Tensions in Motive and Message Ever Since

3. Why Didn’t Assessment Go Away?
Pressure to Produce Evidence of Student Learning Outcomes Never Let Up
By Early 1990s, Accreditors Replace States as Primary External Stimulus to Get Started
Intermittent Federal Interest in Assessment as an Element of National Accountability
But Resulting Faculty Ambivalence About a Process Seen as “External” and “Administrative”

4. Looking Back: What’s Been Accomplished?
Assessment Is for the Most Part Perceived as Inevitable and Legitimate
Vast Majority of Institutions Have Statements of Learning Outcomes (General and Programmatic)
A “Semi-Profession” of Folks Involved in Assessment
Steadily Growing Sophistication with Respect to Methods of Gathering Evidence

5. Looking Back: What Hasn’t Happened?
Authentic Integration of Assessment into Faculty Cultures and Behaviors
Assessment Activities Still Largely “Added On” to the Curriculum Instead of Being Embedded In It
Systematic and Widespread Use of Assessment Results for Institutional and Curricular Improvement
Proactive and Sincere Institutional Engagement with Accrediting Organizations Around Topics of Assessment

6. Assessment as a “Perpetual Movement?”
Most Social Movements…
Fade as Fads, or
Go Away Because Core Ideas are Mainstreamed
The “Assessment Movement” Has Done Neither...
External Requirements Keep it Alive
But External Pressures also Constrain Faculty Buy-In and Meaningful Institutional Use
Moving Beyond a “Perpetual Movement” Will Demand...
Making Assessment Real to Faculty by Connecting It to the Actual Practice of Teaching and Learning
Re-Focusing Accountability on the Authentic Student Abilities that Society Says it Needs

7. Some Prominent Changes in Higher Education’s Operating Environment
Doing More with Less
Need for Curricula that are Effective and Efficient
Need Information About Curricular Functioning to Enable Effective Action
Changes in Instructional Delivery
Competencies and “Deep Learning”
Student Engagement and Role of Technology
New Views of Accountability
“Quality”— From Inputs/Processes to Results
Stakeholder Voices — Students and Employers

8. Challenges to the Academy: The Internal Dimension
Changing Paradigm of Teaching and Learning
Resulting Changes in Academic Roles, Behaviors and Structures
Nature and Role of “Faculty” in Instruction
Patterns of Student Enrollment
[Technology as a “Wild Card”]

9. A Changing Paradigm of Teaching and Learning
From “Faculty Teaching” to “Students Learning”
Students “Make Their Own Paths” through Multiple Learning Opportunities
Explicit Designs for Learning Based on Research

10. Changing Paradigm: Possible Lines of Response
Demonstrated Achievement Becomes Paramount, not “Seat Time”
Students “Teach” One Another
Individual (and Asynchronous) Paths
Technology Seen as Opportunity to Re- Think [not as a “Solution”]

11. Some Implications for Student Assessment Processes
Assessments Reinforce Common Standards for Learning Across Curricula and Classes
“Seamless” Assessments Become an Integral Part of Curriculum and Pedagogy
Assessments Emphasize Connections and Longitudinal Development, not Just Attainment

12. Assessment Approaches: A Resulting Shift in Emphasis
Accountability-Based: Assessments Added onto Instruction to “Check Up” on the System in the Aggregate
Scholarship and Improvement: Assessments Built into the System to Simultaneously Assure Standards and Provide Feedback on Collective Performance

13. Kinds of Information Needed
Alignment of Key Learning Outcomes Across Units, Sequences, and Courses
Match Between Curricular Design, Delivery, and Student Experience
Match Between Instruction and Needs of Diverse “Student Bodies”
Effectiveness of Particular Innovations and Interventions

14. Changing Nature of the Faculty Role
Dis-aggregation of Instructional Roles
More Things for Faculty to Do
New Potential Career Patterns and Paths

15. New Faculty Roles: Possible Lines of Response
Emphasize Peer Support and Collaboration in Faculty Development
Recognize and Regularize Alternative Career Paths
Recognize “Mentorship” as the One Thing You Can’t Responsibly Outsource

16. Kinds of Information Needed
Capturing and Re-Aggregating Data About Discrete Instructional Functions
Accounting Technology-Based Costs
Tracking Faculty “Assets”

17. Changing Patterns of Student Attendance
Increased Levels of Multi-Institutional Attendance
Increased Complexity in Course-Taking Behavior within Institutions
Greater (and Unpredictable) Time Lapses Between Instructional Encounters

18. Changing Patterns of Attendance: Possible Lines of Response
Coherence Based on Common Practices [and Outcomes], not Common Content
Stress Ways for Students to Exploit and Reflect on Their Own Experiences
Establish Clear Transition Points at which to Assess Student Mastery of Key Concepts

19. Kinds of Information Needed
Relationships Between Particular Institutional Experiences and Particular Outcomes
Tracking Student Learning Styles and Individual Paths of Development
“Episode-based” [as opposed to time-based] Data Structures

20. Challenges to the Academy: The External Dimension
Increased Accountability
Changing Expectations Regarding What Students Know and Can Do
The “New Competition”

21. What Are States Doing? Forces Influencing State Approaches
Decreased Agency Capacity Due to Funding Cuts
Momentum of K-12 Reform (NCLB)
Political Uncertainty and Instability
Types of State Policy Responses
P-16 Alignment
Performance Measures and “Report Cards”
Achievement Testing (Driven by K-12)

22. What’s Shaping Accreditation?
Forces Influencing Accreditors
Pressure for Specific Performance from Federal Government
Demands from Institutions to “Add Value”
New Models from Other Sectors and Abroad
“New Looks” in Accreditation
Focus on Outcomes and Effectiveness
Presenting Evidence [e.g. “Portfolios”]
Review Approaches [e.g. “Academic Audits”]
Connection to Institutional Planning [e.g. QEP]

23. Learning Outcomes: What Employers Expect
Higher-Order “Literacies” as Well as Specific Skills
Framed in Terms of “Practice” [not “Knowledge”]
As Much About Attitudes as Academics [“Soft Skills”]

24. External Forces: Some Resulting Influences on the Academy
Increased Emphasis on Credentialing
Modularity and Acceleration to Increase Accessibility
Pressure to Respond to “Students as Customers”
Accountability “Superstructures” that Divert Attention and Information Resources

25. Kinds of Information Needed
Outcomes and Performance Measures
Data on What Experiences/Services Students Can Expect
Peer Comparisons [Increasingly Outside the Academy] and Comparative Performance
Needs and Satisfaction of External Stakeholders

26. Attributes of a Meaningful “Culture of Evidence”
Shared Recognition That Many (But Not All) Things Are Knowable
An Accessible Store of Information About Organizational Condition and Performance
An Attitude Toward Problem-Solving that Minimizes “Finger-Pointing”
Clear Follow-Through On Decisions Made and Why They Were Taken

27. Cultures of Evidence: Success Factors
Visible Metaphor of Scholarship
Beginning with Real Problems and Processes, not with “Method”
Consistent Messages from Leadership
Periodically “Re-Socializing” the Community
“Closing the Loop” with Action
Turn now to what we think we have learned about some of the practicalities of making assessment processes work…”tactics” of assessment in any setting..
- Begin with factors inducing and inhibiting success..
- Then move to the particularly delicate topic of what the relationship ought to be between quality assurance agencies and institutions (or internally between assessment offices and units/faculties) in order to get real engagement and improvement..
[SLIDE]

28. Cultures of Evidence: Inhibiting Factors
Either Excessive or Non-Existent Consequences
Alien Language and “Management Culture”
Excessive Complexity
Burnout and “Committee Fatigue”
Changing the Rules
Noted lots of things wrong as well
…and note particularly that our experience suggests that institutions (like learners) will not always follow a linear path of development here--may fall back or forget things
…need to be constantly vigilant to be effective
[SLIDE]

29. Meaningful Assessment is More About Mindset than Method
Questions About Learning are not Just Matters of Opinion
What Information Might We Collect?
What Might We Expect to Find?
What Difference Would Finding Out Make?
Assessment is About Improving Practice
What are We Trying to Fix?
How Good is Good Enough?
What Changes are Implied?

30. The Bottom Line
For Internal Management, “Seat of the Pants” Decision-making is no Longer Sufficient
Information Used Openly, Consistently, and Continuously to Inform Academic Decisions
For External Constituencies, “Trust Me” is no Longer Sufficient
Need Clear, Understandable Evidence of Student Academic Attainment
Be Vigilant about the Information You Choose and the Signals it Sends
Make Sure that What You Measure is What You Value
Harness the Accreditation Process to Make it Happen!