Understanding Middle States Expectations for Assessment MSCHE Annual Conference December 2009 Understanding Middle States Expectations for Assessment Basically explicates our brochure on our expectations for assessment, in the appendix C of the self-study handbook (p. 60) Copies of these slides are on the goal handout you received when you checked in Linda Suskie, Vice President Middle States Commission on Higher Education 3624 Market Street, Philadelphia PA 19104 Web: www.msche.org E-mail: LSuskie@msche.org

What the Heck is Going on with Accountability & Assessment?

The US Accreditation “System” Regional accreditors All require liberal arts foundation Oldest, strongest reputation Historically examined inputs, not outcomes National accreditors Mostly colleges without liberal arts foundation Specialized accreditors Mostly programs, not colleges State licensure All accreditors voluntary, membership-controlled Regional accreditors: 6 regions, all independent but collaborate, and standards are similar Oldest Strongest reputation Developed to facilitate transfer among colleges within a particular region Often will not accept transfer credits from students from colleges without regional accreditation National accreditors: career training, bible colleges Specialized accreditors: business, teacher education, engineering, health professions, art, dance, theater Optional – prestige Some disciplines have multiple accrediting agencies (business, teacher ed) State licensure: some require accreditation to be licensed, some don’t. Some licensed institutions are not accredited – can operate, but students can’t get Federal grants & loans Also, students often can’t transfer their credits to accredited institutions, sometimes can’t get jobs Largely unregulated until mid-1960s….

1965 Higher Education Act (HEA) Title IV funds go only to colleges accredited by Federally recognized accreditors. Pell, SEOG, Trio, Migrant grants Federally-insured student loans Accreditors must comply with HEA criteria to be recognized. Explosion of institutions seeking accreditation Community colleges Specialized colleges

1980s and 1990s HEA reauthorization 1986: First outcomes assessment language 1998: Assessment language strengthened Regional accreditors rewrote standards to emphasize assessment of student learning outcomes “Learning-centered” movement 1980s: Movement—and assessment movement—began 1995: Barr & Tagg’s seminal article in Change published Research on what promotes student learning & success

Recent Decades: A Changing World Shifting public policy Higher education more private than public good Students pay more and expect money’s worth Broadening market for higher education Most well-paying jobs require post-secondary education. “Money’s worth” is better pay. Not necessarily a richer education

2000s: Calls for Accountability 2007 Spellings Commission 2008 Higher Education Act negotiations Public information on quality & effectiveness Transparent - easy to find & understand Systematic information, not anecdotes Comparable assessments Value-added assessments Judith Eaton (2007) has noted that government, charities, churches, and corporations are all being held increasingly accountable. She has argued that, in this climate, it is “more and more difficult for colleges and universities, which spend hundreds of billions of public and private dollars annually, to argue persuasively that they should not be more accountable for what they produce with those dollars” We need systematic, compelling assessments, and rigorous peer review of them, to stave off further government intervention when HEA is next renegotiated.

Will Assessment Ever Go Away? Federal regulations Other calls & mandates for accountability “Learning-centered” focus

Understanding Standards 7 & 14 St. Peter’s joke Paul Pastor Patrick Pilot “when you preached, people slept. When he flew, people prayed.”

Assessment as Part of a Four-Step Cycle 1. Goals 2. Programs, Services & Initiatives 4. Using Results New brochure page 3: What is assessment of institutional effectiveness? Like TQM: plan – do – check – act Goals include: Institutional goals (strategic plan) Unit goals (divisions, schools, colleges) Goals for administrative offices Goals for academic & support programs 3. Assessment/evaluation used synonymously 3. Assessment/ Evaluation

9. Student Support Services 1. Mission & Goals 2. Planning 8. Admissions 3. Resources 9. Student Support Services 4. Leadership/Governance 10. Faculty 5. Administration 11. Educational Offerings 6. Integrity 12. General Education 14 accreditation standards Pink are Institutional Context Yellow are Educational Effectiveness Orange are in institutional context section but really address both (red + yellow = orange) Every standard has an assessment element, so 7 & 14 may seem redundant. 7 & 14 are where all the assessments in the other standards are considered holistically Where are our overall S&W regarding assessment? 13. Related Educ. Activities 7. Institutional Assessment 14. Asmt. of Student Learning

Institutional Effectiveness: Are We Achieving… 7. Mission & Goals Community Service Scholarship 14. Student Learning Diversity Productivity/ Efficiency How 7 & 14 fit together Access Revenue Generation

So What Does Middle States Want?

Have a goal for anything you do and assess how well you’re achieving it. Institutional goals (mission & strategic plan) Administrative goals Division goals Administrative unit goals Student learning goals Institutional Gen Ed curriculum Academic programs Student development programs Support programs Goals should be related as appropriate e.g., institutional goal to be “student-centered” Financial aid ofc should have a related goal Budget office maybe not

Make sure your students graduate with the learning you value. What knowledge, skills, competencies, and attributes does a successful student have? Why do you think these are important?

Make sure you achieve whatever else you want to achieve. Mission Strategic goals Other important goals

Are you satisfied with your results? Why or why not? If not, what are you doing about it? I like this question better than “Did you meet your goals?” because it doesn’t imply failure. Remember—all of these approaches are imperfect Choose the ones that are most appropriate for you.

Questions a Reviewer Might Ask New brochure pages 6-7 HANDOUT: Rubric to help you judge compliance Big overall question is, “Are the right things happening?”

For Each Goal… (Institutional, Gen Ed, Program) How is the goal being assessed? What are the results of those assessments? How have those results been used for improvement? Goals Assessments Improvements Examples of top half: e.g., institutional goal for civic engagement Biology might have a goal that students understand environmental impact of human actions This would be addressed in some bio courses but not all e.g., institutional goal to be “student centered” Registrar might have a goal to this end. Budget office might not! 19 19

How Much Has Been Implemented? i.e., for those goals and programs that don’t yet have active assessment processes Appropriate timelines: not overly long until implementation, not overly long cycles Some plans are overly ambitious – raises a red flag “data lust” Are there any significant missing pieces? 20 20

What Do Assessment Results Tell Us? Do results demonstrate… Achievement of mission and goals? Sufficient academic rigor? 21 21

Do Institutional Leaders Support and Value a Culture of Assessment? Is there adequate support for assessment? Overall guidance, coordination, resources Are assessment efforts recognized & valued? Are efforts to improve teaching recognized & valued? Do the institutional leaders and institutional community “get it” regarding assessment? Is the institution moving from compliance mode to seeing the real value of assessment? Is there a climate that fosters risk-taking and change? 22 22

Is the Process Sustainable? Simple Practical Detailed Ownership Appropriate timelines i.e., for those goals and programs that don’t yet have active assessment processes Appropriate timelines: not overly long until implementation, not overly long cycles Some plans are overly ambitious – raises a red flag “data lust” 23 23

Where is the Institution Going with Assessment? Pat McGuire: “Compassionate rigor” EXERCISE: Bad asmt report Will momentum slow after this review? What Commission action will most help the institution keep moving? 24 24

What Should Institutions Document? Clear statements of goals Organized, sustained assessment process Principles, guidelines, support What assessments are already underway What assessments are planned, when, & how Assessment results documenting progress toward accomplishing goals How results have been used for improvement New brochure page 5: replaces Chapter 4 of orange book Should an assessment “plan” be written? Not 100% mandatory, but really hard to imagine an effective one that isn’t. The important thing is that it doesn’t need to be ONE written document.

How Might Institutions Document This? Need not be a fancy bound document! An overview in the self study A chart or “roadmap” for assessment documentation in the self study or as an appendix More thorough information in an appendix, online, and/or burned onto CD A few samples of student work Exemplary, adequate, inadequate We don’t prescribe how an institution will document this, so be open to anything! May be any or all of the above.

MSCHE’s Fundamental Expectations for Assessment Read the directions. Keep it useful…and used. Tie assessments to important goals. For student learning, include some “direct” evidence. Use multiple measures. Keep doing something everywhere, every year. HANDOUT: Direct vs indirect evidence of student learning 27 27

Bottom Line on Moving Ahead Keep assessment useful. Keep things simple. Especially in terms of time Ask MSCHE about anything that doesn’t make sense. Value assessment. Just do it! HANDOUTS Bibliography Info on assessment models & best practices

Volunteer for Middle States Evaluation Teams! Go to www.msche.org Click on “Evaluators” Consider joining as an Evaluation Team Associate.