Presentation on theme: "Understanding Middle States Expectations for Assessment"— Presentation transcript:
1 Understanding Middle States Expectations for Assessment MSCHE Annual ConferenceDecember 2009Understanding Middle States Expectations for AssessmentBasically explicates our brochure on our expectations for assessment, in the appendix C of the self-study handbook (p. 60)Copies of these slides are on the goal handout you received when you checked inLinda Suskie, Vice PresidentMiddle States Commission on Higher Education3624 Market Street, Philadelphia PA 19104Web:
2 What the Heck is Going on with Accountability & Assessment?
3 The US Accreditation “System” Regional accreditorsAll require liberal arts foundationOldest, strongest reputationHistorically examined inputs, not outcomesNational accreditorsMostly colleges without liberal arts foundationSpecialized accreditorsMostly programs, not collegesState licensureAll accreditors voluntary, membership-controlledRegional accreditors:6 regions, all independent but collaborate, and standards are similarOldestStrongest reputationDeveloped to facilitate transfer among colleges within a particular regionOften will not accept transfer credits from students from colleges without regional accreditationNational accreditors: career training, bible collegesSpecialized accreditors: business, teacher education, engineering, health professions, art, dance, theaterOptional – prestigeSome disciplines have multiple accrediting agencies (business, teacher ed)State licensure: some require accreditation to be licensed, some don’t.Some licensed institutions are not accredited – can operate, but students can’t get Federal grants & loansAlso, students often can’t transfer their credits to accredited institutions, sometimes can’t get jobsLargely unregulated until mid-1960s….
4 1965 Higher Education Act (HEA) Title IV funds go only to colleges accredited by Federally recognized accreditors.Pell, SEOG, Trio, Migrant grantsFederally-insured student loansAccreditors must comply with HEA criteria to be recognized.Explosion of institutions seeking accreditationCommunity collegesSpecialized colleges
5 1980s and 1990s HEA reauthorization 1986: First outcomes assessment language1998: Assessment language strengthenedRegional accreditors rewrote standards to emphasize assessment of student learning outcomes“Learning-centered” movement1980s: Movement—and assessment movement—began1995: Barr & Tagg’s seminal article in Change publishedResearch on what promotes student learning & success
6 Recent Decades: A Changing World Shifting public policyHigher education more private than public goodStudents pay more and expect money’s worthBroadening market for higher educationMost well-paying jobs require post-secondary education.“Money’s worth” is better pay.Not necessarily a richer education
7 2000s: Calls for Accountability 2007 Spellings Commission2008 Higher Education Act negotiationsPublic information on quality & effectivenessTransparent - easy to find & understandSystematic information, not anecdotesComparable assessmentsValue-added assessmentsJudith Eaton (2007) has noted that government, charities, churches, and corporations are all being held increasingly accountable. She has argued that, in this climate, it is “more and more difficult for colleges and universities, which spend hundreds of billions of public and private dollars annually, to argue persuasively that they should not be more accountable for what they produce with those dollars”We need systematic, compelling assessments, and rigorous peer review of them, to stave off further government intervention when HEA is next renegotiated.
8 Will Assessment Ever Go Away? Federal regulationsOther calls & mandates for accountability“Learning-centered” focus
9 Understanding Standards 7 & 14 St. Peter’s jokePaul PastorPatrick Pilot“when you preached, people slept. When he flew, people prayed.”
10 Assessment as Part of a Four-Step Cycle 1. Goals2. Programs, Services & Initiatives4. Using ResultsNew brochure page 3:What is assessment of institutional effectiveness?Like TQM: plan – do – check – actGoals include:Institutional goals (strategic plan)Unit goals (divisions, schools, colleges)Goals for administrative officesGoals for academic & support programs3. Assessment/evaluation used synonymously3. Assessment/ Evaluation
11 9. Student Support Services 1. Mission & Goals2. Planning8. Admissions3. Resources9. Student Support Services4. Leadership/Governance10. Faculty5. Administration11. Educational Offerings6. Integrity12. General Education14 accreditation standardsPink are Institutional ContextYellow are Educational EffectivenessOrange are in institutional context section but really address both(red + yellow = orange)Every standard has an assessment element, so 7 & 14 may seem redundant.7 & 14 are where all the assessments in the other standards are considered holisticallyWhere are our overall S&W regarding assessment?13. Related Educ. Activities7. Institutional Assessment14. Asmt. of Student Learning
12 Institutional Effectiveness: Are We Achieving… 7. Mission & GoalsCommunityServiceScholarship14. Student LearningDiversityProductivity/EfficiencyHow 7 & 14 fit togetherAccessRevenueGeneration
14 Have a goal for anything you do and assess how well you’re achieving it. Institutional goals (mission & strategic plan)Administrative goalsDivision goalsAdministrative unit goalsStudent learning goalsInstitutionalGen Ed curriculumAcademic programsStudent development programsSupport programsGoals should be related as appropriatee.g., institutional goal to be “student-centered”Financial aid ofc should have a related goalBudget office maybe not
15 Make sure your students graduate with the learning you value. What knowledge, skills, competencies, and attributes does a successful student have?Why do you think these are important?
16 Make sure you achieve whatever else you want to achieve. MissionStrategic goalsOther important goals
17 Are you satisfied with your results? Why or why not?If not, what are you doing about it?I like this question better than “Did you meet your goals?” because it doesn’t imply failure.Remember—all of these approaches are imperfectChoose the ones that are most appropriate for you.
18 Questions a Reviewer Might Ask New brochure pages 6-7HANDOUT: Rubric to help you judge complianceBig overall question is, “Are the right things happening?”
19 For Each Goal… (Institutional, Gen Ed, Program) How is the goal being assessed?What are the results of those assessments?How have those results been used for improvement?GoalsAssessmentsImprovementsExamples of top half:e.g., institutional goal for civic engagementBiology might have a goal that students understand environmental impact of human actionsThis would be addressed in some bio courses but not alle.g., institutional goal to be “student centered”Registrar might have a goal to this end.Budget office might not!1919
20 How Much Has Been Implemented? i.e., for those goals and programs that don’t yet have active assessment processesAppropriate timelines: not overly long until implementation, not overly long cyclesSome plans are overly ambitious – raises a red flag“data lust”Are there any significant missing pieces?2020
21 What Do Assessment Results Tell Us? Do results demonstrate…Achievement of mission and goals?Sufficient academic rigor?2121
22 Do Institutional Leaders Support and Value a Culture of Assessment? Is there adequate support for assessment?Overall guidance, coordination, resourcesAre assessment efforts recognized & valued?Are efforts to improve teaching recognized & valued?Do the institutional leaders and institutional community “get it” regarding assessment?Is the institution moving from compliance mode to seeing the real value of assessment?Is there a climate that fosters risk-taking and change?2222
23 Is the Process Sustainable? SimplePracticalDetailedOwnershipAppropriate timelinesi.e., for those goals and programs that don’t yet have active assessment processesAppropriate timelines: not overly long until implementation, not overly long cyclesSome plans are overly ambitious – raises a red flag“data lust”2323
24 Where is the Institution Going with Assessment? Pat McGuire: “Compassionate rigor”EXERCISE: Bad asmt reportWill momentum slow after this review?What Commission action will most help the institution keep moving?2424
25 What Should Institutions Document? Clear statements of goalsOrganized, sustained assessment processPrinciples, guidelines, supportWhat assessments are already underwayWhat assessments are planned, when, & howAssessment results documenting progress toward accomplishing goalsHow results have been used for improvementNew brochure page 5: replaces Chapter 4 of orange bookShould an assessment “plan” be written?Not 100% mandatory, but really hard to imagine an effective one that isn’t.The important thing is that it doesn’t need to be ONE written document.
26 How Might Institutions Document This? Need not be a fancy bound document!An overview in the self studyA chart or “roadmap” for assessment documentation in the self study or as an appendixMore thorough information in an appendix, online, and/or burned onto CDA few samples of student workExemplary, adequate, inadequateWe don’t prescribe how an institution will document this, so be open to anything! May be any or all of the above.
27 MSCHE’s Fundamental Expectations for Assessment Read the directions.Keep it useful…and used.Tie assessments to important goals.For student learning, include some “direct” evidence.Use multiple measures.Keep doing something everywhere, every year.HANDOUT: Direct vs indirect evidence of student learning2727
28 Bottom Line on Moving Ahead Keep assessment useful.Keep things simple.Especially in terms of timeAsk MSCHE about anything that doesn’t make sense.Value assessment.Just do it!HANDOUTSBibliographyInfo on assessment models & best practices
29 Volunteer for Middle States Evaluation Teams! Go toClick on “Evaluators”Consider joining as an Evaluation Team Associate.