1. Understanding Middle States Expectations for Assessment
MSCHE Annual Conference
December 2009
Understanding Middle States Expectations for Assessment
Basically explicates our brochure on our expectations for assessment, in the appendix C of the self-study handbook (p. 60)
Copies of these slides are on the goal handout you received when you checked in
Linda Suskie, Vice President
Middle States Commission on Higher Education
3624 Market Street, Philadelphia PA 19104
Web:

2. What the Heck is Going on with Accountability & Assessment?

3. The US Accreditation “System”
Regional accreditors
All require liberal arts foundation
Oldest, strongest reputation
Historically examined inputs, not outcomes
National accreditors
Mostly colleges without liberal arts foundation
Specialized accreditors
Mostly programs, not colleges
State licensure
All accreditors voluntary, membership-controlled
Regional accreditors:
6 regions, all independent but collaborate, and standards are similar
Oldest
Strongest reputation
Developed to facilitate transfer among colleges within a particular region
Often will not accept transfer credits from students from colleges without regional accreditation
National accreditors: career training, bible colleges
Specialized accreditors: business, teacher education, engineering, health professions, art, dance, theater
Optional – prestige
Some disciplines have multiple accrediting agencies (business, teacher ed)
State licensure: some require accreditation to be licensed, some don’t.
Some licensed institutions are not accredited – can operate, but students can’t get Federal grants & loans
Also, students often can’t transfer their credits to accredited institutions, sometimes can’t get jobs
Largely unregulated until mid-1960s….

4. 1965 Higher Education Act (HEA)
Title IV funds go only to colleges accredited by Federally recognized accreditors.
Pell, SEOG, Trio, Migrant grants
Federally-insured student loans
Accreditors must comply with HEA criteria to be recognized.
Explosion of institutions seeking accreditation
Community colleges
Specialized colleges

5. 1980s and 1990s HEA reauthorization
1986: First outcomes assessment language
1998: Assessment language strengthened
Regional accreditors rewrote standards to emphasize assessment of student learning outcomes
“Learning-centered” movement
1980s: Movement—and assessment movement—began
1995: Barr & Tagg’s seminal article in Change published
Research on what promotes student learning & success

6. Recent Decades: A Changing World
Shifting public policy
Higher education more private than public good
Students pay more and expect money’s worth
Broadening market for higher education
Most well-paying jobs require post-secondary education.
“Money’s worth” is better pay.
Not necessarily a richer education

7. 2000s: Calls for Accountability
2007 Spellings Commission
2008 Higher Education Act negotiations
Public information on quality & effectiveness
Transparent - easy to find & understand
Systematic information, not anecdotes
Comparable assessments
Value-added assessments
Judith Eaton (2007) has noted that government, charities, churches, and corporations are all being held increasingly accountable. She has argued that, in this climate, it is “more and more difficult for colleges and universities, which spend hundreds of billions of public and private dollars annually, to argue persuasively that they should not be more accountable for what they produce with those dollars”
We need systematic, compelling assessments, and rigorous peer review of them, to stave off further government intervention when HEA is next renegotiated.

8. Will Assessment Ever Go Away?
Federal regulations
Other calls & mandates for accountability
“Learning-centered” focus

9. Understanding Standards 7 & 14
St. Peter’s joke
Paul Pastor
Patrick Pilot
“when you preached, people slept. When he flew, people prayed.”

10. Assessment as Part of a Four-Step Cycle
1. Goals
2. Programs, Services & Initiatives
4. Using Results
New brochure page 3:
What is assessment of institutional effectiveness?
Like TQM: plan – do – check – act
Goals include:
Institutional goals (strategic plan)
Unit goals (divisions, schools, colleges)
Goals for administrative offices
Goals for academic & support programs
3. Assessment/evaluation used synonymously
3. Assessment/ Evaluation

11. 9. Student Support Services
1. Mission & Goals
2. Planning
8. Admissions
3. Resources
9. Student Support Services
4. Leadership/Governance
10. Faculty
5. Administration
11. Educational Offerings
6. Integrity
12. General Education
14 accreditation standards
Pink are Institutional Context
Yellow are Educational Effectiveness
Orange are in institutional context section but really address both
(red + yellow = orange)
Every standard has an assessment element, so 7 & 14 may seem redundant.
7 & 14 are where all the assessments in the other standards are considered holistically
Where are our overall S&W regarding assessment?
13. Related Educ. Activities
7. Institutional Assessment
14. Asmt. of Student Learning

12. Institutional Effectiveness: Are We Achieving…
7. Mission & Goals
Community
Service
Scholarship
14. Student Learning
Diversity
Productivity/
Efficiency
How 7 & 14 fit together
Access
Revenue
Generation

13. So What Does Middle States Want?

14. Have a goal for anything you do and assess how well you’re achieving it.
Institutional goals (mission & strategic plan)
Administrative goals
Division goals
Administrative unit goals
Student learning goals
Institutional
Gen Ed curriculum
Academic programs
Student development programs
Support programs
Goals should be related as appropriate
e.g., institutional goal to be “student-centered”
Financial aid ofc should have a related goal
Budget office maybe not

15. Make sure your students graduate with the learning you value.
What knowledge, skills, competencies, and attributes does a successful student have?
Why do you think these are important?

16. Make sure you achieve whatever else you want to achieve.
Mission
Strategic goals
Other important goals

17. Are you satisfied with your results?
Why or why not?
If not, what are you doing about it?
I like this question better than “Did you meet your goals?” because it doesn’t imply failure.
Remember—all of these approaches are imperfect
Choose the ones that are most appropriate for you.

18. Questions a Reviewer Might Ask
New brochure pages 6-7
HANDOUT: Rubric to help you judge compliance
Big overall question is, “Are the right things happening?”

19. For Each Goal… (Institutional, Gen Ed, Program)
How is the goal being assessed?
What are the results of those assessments?
How have those results been used for improvement?
Goals
Assessments
Improvements
Examples of top half:
e.g., institutional goal for civic engagement
Biology might have a goal that students understand environmental impact of human actions
This would be addressed in some bio courses but not all
e.g., institutional goal to be “student centered”
Registrar might have a goal to this end.
Budget office might not! 19 19

20. How Much Has Been Implemented?
i.e., for those goals and programs that don’t yet have active assessment processes
Appropriate timelines: not overly long until implementation, not overly long cycles
Some plans are overly ambitious – raises a red flag
“data lust”
Are there any significant missing pieces? 20 20

21. What Do Assessment Results Tell Us?
Do results demonstrate…
Achievement of mission and goals?
Sufficient academic rigor? 21 21

22. Do Institutional Leaders Support and Value a Culture of Assessment?
Is there adequate support for assessment?
Overall guidance, coordination, resources
Are assessment efforts recognized & valued?
Are efforts to improve teaching recognized & valued?
Do the institutional leaders and institutional community “get it” regarding assessment?
Is the institution moving from compliance mode to seeing the real value of assessment?
Is there a climate that fosters risk-taking and change? 22 22

23. Is the Process Sustainable?
Simple
Practical
Detailed
Ownership
Appropriate timelines
i.e., for those goals and programs that don’t yet have active assessment processes
Appropriate timelines: not overly long until implementation, not overly long cycles
Some plans are overly ambitious – raises a red flag
“data lust” 23 23

24. Where is the Institution Going with Assessment?
Pat McGuire: “Compassionate rigor”
EXERCISE: Bad asmt report
Will momentum slow after this review?
What Commission action will most help the institution keep moving? 24 24

25. What Should Institutions Document?
Clear statements of goals
Organized, sustained assessment process
Principles, guidelines, support
What assessments are already underway
What assessments are planned, when, & how
Assessment results documenting progress toward accomplishing goals
How results have been used for improvement
New brochure page 5: replaces Chapter 4 of orange book
Should an assessment “plan” be written?
Not 100% mandatory, but really hard to imagine an effective one that isn’t.
The important thing is that it doesn’t need to be ONE written document.

26. How Might Institutions Document This?
Need not be a fancy bound document!
An overview in the self study
A chart or “roadmap” for assessment documentation in the self study or as an appendix
More thorough information in an appendix, online, and/or burned onto CD
A few samples of student work
Exemplary, adequate, inadequate
We don’t prescribe how an institution will document this, so be open to anything! May be any or all of the above.

27. MSCHE’s Fundamental Expectations for Assessment
Read the directions.
Keep it useful…and used.
Tie assessments to important goals.
For student learning, include some “direct” evidence.
Use multiple measures.
Keep doing something everywhere, every year.
HANDOUT: Direct vs indirect evidence of student learning 27 27

28. Bottom Line on Moving Ahead
Keep assessment useful.
Keep things simple.
Especially in terms of time
Ask MSCHE about anything that doesn’t make sense.
Value assessment.
Just do it!
HANDOUTS
Bibliography
Info on assessment models & best practices

29. Volunteer for Middle States Evaluation Teams!
Go to
Click on “Evaluators”
Consider joining as an Evaluation Team Associate.