Presentation on theme: "Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and."— Presentation transcript:
1Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. MiddaughAssistant Vice President for Institutional Research and PlanningUniversity of DelawareCommissioner and Member of Executive CommitteeMiddle States Commission on Higher EducationAndrea A. LexDean of Planning and Institutional ResearchPrince George’s Community College
2“The nicest thing about not planning is that failure comes as a complete surprise and is not preceded by a period of worry and depression.” John Preston, Boston College “A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools.” Douglas Adams, Hitchhikers Guide to the Galaxy
4What is accreditation?A means of providing quality assurance in the areas ofStudent learning and achievementCurriculum development and supportFacultyFacilities and equipmentFinance, administration, and governanceIntegrity and compliance
5Why is it done? Voluntary self-regulation Voluntary self-examination Opportunity for peer reviewQuality assurancePublic accountabilityAnd, in the case of regional accreditationAbility to receive Federal financial aid dollars
6What do we use? Characteristics of Excellence Institutional MissionInstitutional EffectivenessPlanningResource AllocationLeadershipGovernanceAdministrationIntegrityInstitutional AssessmentEducational EffectivenessAdmissions & RetentionSupport ServicesFacultyOfferingsGeneral EducationRelated Education ActivitiesAssessment of Student Learning
7What have we learned?That the quality of an institution’s accreditation self study hinges on how it addresses the standards related toassessing institutional effectiveness (Standard 7)assessing student learning outcomes (Standard 14)implementing a vital and comprehensive strategic planning process (Standard 2).
8Commission-Directed Follow-up (2004-2009) 81 percent required some sort of follow-upStudent Learning Outcomes (S 14)Institutional Effectiveness (S 7)Strategic Planning (S 2)Sufficient Resources (S 3)Enrollment Management (S 8)
9Creating and Closing the Loop Institutional Mission is the Foundation of & Our Reason for Being
10Creating and Closing the Loop Characteristics of Excellence Institutional MissionEducational EffectivenessAdmissions & RetentionSupport ServicesFacultyOfferingsGeneral EducationRelated Education ActivitiesAssessment of Student LearningTo what extent have students achieved the educational and co-curricular goals set forth by an institution as it recruits and retains these individuals?-- Assessment of Student Learning (Standard 14)
11Creating and Closing the Loop Standard 14: Student Learning OutcomesAssessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals.
12Selected Fundamental Elements for MSCHE Standard 14 Articulated expectations for student learning (at institutional, degree/program, and course levels)Documented, organized, and sustained assessment processes (that may include a formal assessment plan)Evidence that student learning assessment information is shared and used to improve teaching and learningDocumented use of student learning assessment information as part of institutional assessment
13Creating and Closing the Loop Characteristics of Excellence Institutional MissionInstitutional EffectivenessPlanningResource AllocationLeadershipGovernanceAdministrationIntegrityInstitutional AssessmentTo what extent has the institution acquired and allocated human and fiscal resources in a manner that supports the achievement of its goals?-- Assessment of Institutional Effectiveness (Standard 7)
14Creating and Closing the Loop Standard 7: Institutional AssessmentThe institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards.
15Selected Fundamental Elements for MSCHE Standard 7 Documented, organized, and sustained assessment processes to evaluate the total range of programs and services, achievement of mission, and compliance with accreditation standardsEvidence that assessment results are shared and used in institutional planning, resource allocation and renewalWritten institutional strategic plan(s) that reflect(s) consideration of assessment resultsEvidence that institutional assessment findings are used to improve student success
16Creating and Closing the Loop Characteristics of Excellence Institutional MissionPlanningPlanningInstitutional EffectivenessPlanningResource AllocationLeadershipGovernanceAdministrationIntegrityInstitutional AssessmentEducational EffectivenessAdmissions & RetentionSupport ServicesFacultyOfferingsGeneral EducationRelated Education ActivitiesAssessment of Student LearningPlanningTo what extent are an institution’s policy, decision making and resource allocation processes shaped by a systematic, evidence-based, and mission-guided planning process? Assessment of Institutional Planning (Standard 2)
17Creating and Closing the Loop Standard 2: Planning, Resource Allocation and Institutional RenewalAn institution conducts ongoing planning and resource allocation based on its mission and goals, develops objectives to achieve them, and utilizes the results of its assessment activities for institutional renewal.Implementation and subsequent evaluation of the success of the strategic plan and resource allocation support the development and change necessary to improve and to maintain quality.
18Selected Fundamental Elements for MSCHE Standard 2 Clearly stated goals and objectives, at both the institutional and unit levels, that reflect conclusions drawn from assessments that are used for planning and resource allocation at those levelsPlanning and improvement processes that are clearly communicated, provide for constituent participation, and incorporate the use of assessment resultsWell-defined decision-making processes and authority that facilitates planning and renewal and the assignment of responsibility for improvement and assurance of accountabilityPeriodic assessment of the effectiveness of planning, resource allocation, and institutional renewal processes
19Central Threads Running Through All Accreditation Requirements: Planning must be systematicPlanning must be rooted in an institution’s missionPlanning must be predicated on analytical and evaluative informationPlanning must be used for institutional decisions, including resource allocation
20The Closed LoopInstitutions must plan effectively in order to be effective.Where that is the case, the accreditation process is an affirmation of the evidence of that effectiveness.
21Components of This Curriculum Best Practice in PlanningBest Practice in AssessmentClosing the Loop Between Planning and AssessmentReal Life Case Studies
22End ResultInstitutions must plan effectively in order to be effective.Where that is the case, the accreditation process is an affirmation of the evidence of that effectiveness.
23Components of This Curriculum Best Practice in PlanningBest Practice in AssessmentClosing the Loop Between Planning and AssessmentReal Life Case Studies
24In thinking about what good planning is, it is useful to begin the discussion by describing what it is not: George Keller, Academic Strategy: the Management Revolution in American Higher Education, Johns Hopkins University Press, 1983.Strategic Planning is not:The production of a blueprint.A set of platitudes.The personal vision of a president or board of trustees.A collection of departmental plans, compiled and edited.
25Strategic Planning is not: Done by planners.A substitution of numbers for important intangibles.A form of surrender to market conditions and trends.Something done on an annual retreat.A way of eliminating risks.An attempt to read tea leaves and outwit the future.
26Strategic Planning ISAcademic strategic decision-making means that a college or university and its leaders are active rather than passive about their position in history.Strategic planning looks outward and is focused on keeping the institution in step with the changing environment.Academic strategy making is competitive, recognizing that higher education is subject to economic market conditions and to incredibly strong competition.Strategic planning concentrates on decisions, not on documented plans, analyses, forecasts, and goals.
27Strategic Planning ISStrategy making is a blend of rational and economic analysis, political maneuvering, and psychological interplay. It is therefore participatory and highly tolerant of controversy.Strategic planning concentrates on the fate of the institution above everything else.
29Let’s begin at the beginning . . . Why are we here?
30Mission StatementA good mission statement is a carefully reasoned analysis of what an institution aspires to be, and the core values that it embraces.It avoids cliché language, e.g., “Students and faculty will interact in a rich intellectual environment in which each individual has the opportunity to achieve their full potential.” Noble sentiment, but says nothing about the institution’s purposes and priorities.Mission must speak to central institutional issues, e.g., desired balance between undergraduate and graduate education; relative emphasis on teaching, research, and service, respectively; and so on.
31Mission StatementMission statements are characterized by a sense of vision that, while not immutable, nonetheless represents a long-term statement of institutional values and direction around which human and fiscal resource allocation decisions can be made.While cognizant of the institution’s ever changing external environment, mission statements are not whimsical, morphing with each new market trend that emerges.The mission statement provides a clear sense of direction around which action-oriented goal statements and measurable planning objectives can be developed.
32Planning GoalsGoal statements are derived from the institutional mission, and help to define policy.For example, the mission statement might say that “The University affirms its historic mission of providing the highest quality education for its undergraduate students, while maintaining excellence in selected graduate programs.”The mission statement is underscoring the primacy of undergraduate instruction in the curriculum. The question for planners is how to provide that high quality undergraduate instruction.
34Planning GoalsThe “how” translates into specific, action-oriented planning goals aimed at moving the institution toward a fuller realization of its mission.Possible goal related to the undergraduate education mission statement: “The University will continue to attract and retain the most academically talented and diverse undergraduate students, and support their intellectual, cultural, and ethical development as citizens and scholars.”Action verbs such as “attract,” “retain,” and “support” elevate the goal statement to policy level. How do we know that policy is being carried out? Measurable planning objectives.
35Meeting Goals How will we know when we get there? What will we look like when we get there?
36Planning ObjectivesPlanning objectives provide empirical evidence of the extent to which planning goals are being achieved. Consider the following planning objectives as they relate to our goal to attract, retain, and support academically talented and diverse students.Retain a freshman admissions target of 3200 to 3400 students annually, with an admissions profile for academic year 2007 of 23,000 applications, a 40 percent admit rate, and a yield rate in excess of 35 percent.Improve the alignment of undergraduate enrollment distribution and instructional resource distribution across the disciplines, especially with respect to faculty.
37Planning Objectives Continued Maintain a freshman-to-sophomore retention rate above the national average for highly selective institutions, and seek to achieve a consistent rate of 90 percent or higher.Maintain a graduation rate above the national average for highly selective institutions, and seek to achieve a consistent six-year rate of 75 percent or higher.Increase minority and international enrollment, with retention and graduation rates for those populations consistent with the university-wide averages for all students.
38How are we doing? How close are we to getting there? How are we doing along the way?
39Planning ObjectivesThe defining characteristic for any good planning objective is that it must be measurable. In other words, assessable.Colleges or universities embarking on any planning process – long range, strategic, tactical – require a systematic institutional research capability.While smaller institutions may not have an office of institutional research, per se, they must nonetheless have the capability of quantitatively and qualitatively assessing the extent to which planning objectives are being implemented, planning goals are being achieved, and the institutions mission is being realized.
40Breakout SessionUsing your own Institutional Mission Statement, create at least five planning goals and associated measurable planning objectives that will help assess the extent to which that mission is being realized.
41Linking Planning and Assessment Two Case Studies at The Same Institution
43University of Delaware Strategic Planning: 1990 - 2000
44University of Delaware Mission Statement “The University reaffirms its historic mission to provide the highest quality education for its undergraduate students, while maintaining excellence in selected graduate programs. The faculty are responsible for helping students learn to reason critically and independently, gain knowledge of the diverse culture and environment in which they live, and develop into well-informed citizens and leaders. To accomplish these goals, the University provides a learning setting enriched by undergraduate student research, experiential learning, and study-abroad programs. The University places high priority on the education of qualified Delaware residents and provides opportunity for a diverse group of citizens to participate in postsecondary education. Since the University is located in a state with a small population, providing programs of quality and diversity requires a community of student-scholars that reaches beyond the boundaries of the state, one that reflects the nation’s racial and cultural diversity.“The University of Delaware also aspires to excellence in graduate education, the heart of which is scholarship and research. The creation, application, and communication of knowledge is a primary goal of the institution and of every faculty member, providing the substance for creative, informed teaching. Research is typically based on cooperation between faculty and students, whereby faculty mentors teach students to conduct independent research and to master problem-solving techniques. Through involvement of undergraduates in faculty research, the University creates a special bond between its undergraduate and graduate programs.“The University is also committed to providing service to society, especially in Delaware and the neighboring region. Public service is a responsibility of every academic unit. In addition, each faculty member is responsible for service to the University community and to his or her profession. The University emphasizes practical research, provides extension services, and works to solve problems confronting the community.(University of Delaware, Middle States Self-Study Report, April 2001)
45Case Study – University of Delaware In 1987, the University embarked on a comprehensive, long-range planning process, termed Project Vision. Over a period of 18 months, the campus developed a planning document with a broad spectrum of planning goals and measurable objectives embracing all aspects of University operations.In Fall of 1988, the President who initiated Project Vision suddenly resigned. At the same time, the Delaware economy along with that of the entire mid-Atlantic region was plunging into deep recession
46Case Study – University of Delaware Rather than let 18 months of planning activity go for naught – even though resources would be scarce for the foreseeable future – a panel of distinguished senior faculty was assembled to review the Project Vision planning document and to cull out those goals and objectives that were clearly consistent with, and essential to furthering the University mission.The resulting document, Focused Vision, was economical when compared with its progenitor, both in terms of words and resource requirements. However, the economy would still clearly preclude anything even remotely approaching implementation.In 1990, the University hired its 25th President, David P. Roselle.
47Case Study – University of Delaware In order to maintain planning momentum, the President consulted with his senior staff, the faculty, and appropriate constituencies across campus to determine those areas that required immediate attention.From these consultations, the President articulated four strategic initiatives that would constitute the focus of decision-making and resource allocation activity in the immediate future. Those initiatives were competitive compensation for faculty and staff; enhanced access to the University for undergraduates through expanded availability of financial aid; a more student-centered campus environment; and renovation and rehabilitation of campus facilities.
48Case Study – University of Delaware These priorities were not a “wish list.” They grew out of a careful examination of empirical data provided by the University’s Office of Institutional Research and Planning and other data sources. Consider the following:When compared with the 24 Category Doctoral I universities in the states contiguous to Delaware, and the District of Columbia, in 1991 the average salary for all three major faculty ranks at the University of Delaware ranked near the bottom of the list.The Student College Selection Survey indicated that students were receiving offers of more aid from admissions competitors, and that the aid packages had more grants and fewer loans than University aid packages. Not surprising, the University was at a competitive disadvantage for academically talented students.
49Case Study – University of Delaware University scores on the ACT Student Opinion Survey suggested that the institution had considerable room for improvement with respect to student satisfaction with programs and services, and with a number of areas in student life.The University was looking at in excess of $200 million in deferred maintenance to its buildings and grounds.
50Case Study – University of Delaware A critical factor in moving forward with these initiatives was getting the campus to understand that the economy was in recession and that there would be no immediate or massive infusions of new resources.Colleges and universities have multiple revenue streams – tuition, state appropriation in the case of public institutions, contracts and grants, gifts, etc. While growing revenue streams is an important strategic initiative, so too is the commitment to not balance budgets on the backs of students through inordinately large tuition increases.Resource reallocation would be the primary source of funding the four strategic initiatives, and it was critical that the campus understand from where funds were reallocated, and why.
51The University went on public record in 1991: Average total compensation for faculty at each academic rank would be at or above the median within five years for the 24 Category I Doctoral Universities identified as salary peers.Total undergraduate financial aid from all sources would increase by 100 percent within five years.Student satisfaction with programs and services at the University, as measured through the ACT Student Opinion Survey would demonstrate significant gains within five years.The University would commit itself to a policy of annually setting aside at least 2 percent of the replacement value of the physical plant, to be used for facilities renovation and rehabilitation.
55Results - FacilitiesBy 2000, the University had renovated every classroom in its entire building inventory, retrofitting most with state-of-the-art teaching technology.An aggressive program of fundraising enabled not only the aforementioned renovation and rehabilitation, but also the construction of several new classroom and student services buildings.The University is now on a cycle of planned maintenance, as opposed to deferred maintenance.
56Results – From an Accreditation Perspective “ The University of Delaware has every reason to take enormous pride in what it has accomplished over the past 10 years. A decade ago, it was coming out of a period of considerable turmoil. Today, the University is seen as a national model for the integration of information technology in every aspect of university life: teaching and learning, research and service, academic support, and campus administration. It has created a physical plant that has few, if any, peers among public universities and would be the envy of most private colleges. These substantial achievements could not have happened without extraordinary leadership from the senior administration.”“ Better than almost any university we are familiar with, Delaware has a clear sense of what it wants to be, namely, a university that offers high quality undergraduate education with targeted areas of excellence in graduate education and research.”" The review team was enormously impressed by the high level of morale that pervades the faculty, staff, and students. Almost without exception, the people we spoke to take great pride in being part of the University.”Middle States Evaluation Team, 2001
57The University submitted its Periodic Review Report to the Commission on Higher Education in May 2006, outlining its strategies for incorporating systematic assessment of student learning outcomes into all facets of campus planning.The Commission thanked the University for its Report, commended the University on its content, and reaffirmed accreditation.
58What Was Not Apparent to MSCHE… Despite an exemplary self study and PRR process, planning was not yet institutionalized into a formal, iterative process at the institution.The four strategic initiatives articulated in 1990 continued to be the benchmarks against which the University measured success well into the 2000 decade.The University needed to move forward with a new strategic planning effort that would more substantively incorporate planning into the organizational fiber of the institution.
60University of Delaware Strategic Planning: 2007 - ???
61The Path to ProminenceOn July 1, 2007, Patrick T. Harker succeeded David Roselle as the 26th President of the University.As former Dean of the Wharton School at the University of Pennsylvania, President Harker was intimately familiar with the concept of strategic planning.He immediately appointed a Strategic Planning Committee led by two distinguished senior faculty and comprised largely of individuals from the academic as opposed to administrative side of the institution.
62The Path to ProminenceDuring the academic year, the Strategic Planning Committee met with over 100 constituent groups from within, and external to the University. These information-gathering sessions were part of a comprehensive SWOT analysis.In April 2008, the Report of the Strategic Planning Committee was delivered to the President.In July, 2008, the President presented the Path to Prominence, the strategic planning document that would guide institutional activity for the next several years.
63The Path to Prominence: The Strategic Initiatives A Diverse and Stimulating Undergraduate Academic EnvironmentA Premier Research and Graduate UniversityExcellence in Professional EducationThe Initiative for the PlanetThe Global InitiativeAn Engaged University
64The Path to Prominence: The Strategic Initiatives Each of the six initiatives was operationalized with measurable goals and planning objectives.The Path to Prominence document and supporting materials were broadly circulated via the University website: for maximum transparency and accountability with respect to what was being accomplished.In Fall 2008, the U.S. and Delaware economies began to unravel, placing severe financial constraints on full implementation of the strategic initiatives.
65The Path to Prominence: The Strategic Initiatives While each of the six initiatives remain viable and will receive attention regardless of economic circumstances, hard decisions needed to be made with respect to where resources would be allocated.It was determined that three initiatives would be the primary (but not exclusive) focus of resource allocation:- A Diverse and Stimulating Undergraduate Academic Environment- A Premier Research and Graduate University- Excellence in Professional Education
66The Path to Prominence: Moving Forward The University decided to take advantage of the MSCHE decennial accreditation reaffirmation process, and utilize the “Special Topics Model” of self study as a means of documenting progress in implementing these three key strategic initiatives.The scheduled 2011 site visit by an MSCHE Evaluation Team affords the University not only the opportunity to demonstrate compliance with the 14 standards in Characteristics of Excellence; it provides us with a team of expert consultants in the topical areas on which we have elected to focus. Their feedback on the content of the Special Topics Self Study will prove invaluable in moving the institution forward.
67The Path to Prominence: Tactical Issues In any strategic planning process, there needs to be a clear linkage between human and fiscal resource allocation and the initiatives outlined in the strategic plan.The University of Delaware has adopted a responsibility centered budget management model that is designed to financially reward activity that supports full implementation of key strategic initiatives.The University has also designated an Associate Provost for Institutional Effectiveness, tasked with overseeing assessment activity that measures the effective implementation of The Path to Prominence.
68The Path to Prominence: Tactical Issues The Steering Committee for the 2011 MSCHE has tasked itself with responsibility for design of a formal, iterative institutional strategic planning process that will, on an ongoing basis, continuously evaluate the implementation of existing strategic initiatives, assess the continuing relevance of initiatives over time, and develop new initiatives, as appropriate, in the face of a changing external environment and/or advent of specific strategic opportunities.The intent is to institutionalize the strategic planning process so that it is truly an organizational function rather than the creation or vision of any particular individual.
69Considerations When Creating a Strategic Planning Process
70“Planning to Plan:” Organizing Your Institution for the Planning Process
71Organizational Structure The Planning Committee: Who Serves? How Representative? What is the Scope of Responsibilities:If the Planning Committee is to be effective, it must be “blessed” by the Chief Executive Officer and the Chief Academic Officer. The President and Provost need not be members of the Committee, but ideally will attend the initial meeting, and will formally receive the Committee Report.
72Organizational Structure Who staffs the Planning Committee?Institutional ResearchBudget OfficeOthersFacultyStaff
74Planning Tools Institutional Studies: Admissions Monitoring Analyses Faculty Workload AnalysesBudget Support DataPersistence/Graduation AnalysesCompensation Studies: Market/EquityFinancial Ratio AnalysesBenchmarking StudiesEconomic Impact Studies
75Planning Tools Academic Program Review Programmatic Self-Examination Internal and External Review TeamsFocus on Several Levels of QualityPrograms With National/International DistinctionPrograms of Regional Distinction (Meet Local Needs)Programs That, With an Infusion of Resources, Could Achieve DistinctionPrograms Targeted for Retrenchment or Elimination
76Planning Tools Environmental Scan: Demographic issues b. Market issues c. Political issuesd. Economic issuese. Technological issuesf. Natural disastersg. Other
77Planning Tools – SWOT Analysis Take an honest look at your institution within the context of a SWOT analysis (Strengths; Weaknesses; Opportunities; Threats)Leveraging Situation - Opportunity + StrengthConstraining Situation - Opportunity + WeaknessVulnerable Situation – Threat + StrengthProblematic Situation – Threat + Weakness
78In the Final Analysis, Planning is Directed at Answering Four Basic Questions: Who are the markets we are trying to serve?What services must be in place to fully serve those markets?What is the institutional “branding” that will enable our college or university to appeal to those markets?How will we know if we are successful in serving those markets?
79Who are the markets we are trying to serve? Largely Dictated by MissionBaccalaureate Colleges: Primarily undergraduate students (what kinds?)Master’s Institutions: Undergraduate students, selected graduate students, targeted state/community partnershipsDoctoral Universities: Undergraduate students, graduate students, research/public service contractors/grantors, broad range of regional, national, international partnershipsCommunity Colleges: Matriculated undergraduate students, occasional students with specific training needs, general interest students, local businesses, county government
80What services must be in place to fully serve those markets What services must be in place to fully serve those markets? Dictated by MarketsAcademic Support Services (Library, computing, advising, tutoring, etc.)Student Support Services (Residence life, counseling, health services, student center, recreation services, etc.)Institutional Support Services (General administrative support, sponsored research, extension office, alumni/governmental relations, etc.)
81What is the institutional “branding” that will enable our college or university to appeal to those markets? Market SpecificUniversity of Delaware: “A Teaching University”Trenton State College : “The College of New Jersey”University of Phoenix: “The University for Working Adults”
82How will we know if we are successful in serving those markets How will we know if we are successful in serving those markets? Middle States Assessment StandardsStandard 7: The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards.Standard14: Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals.
83Breakout SessionHow would you respond to the four key planning questions within the context of your institution’s mission?
84In the Final Analysis, Planning is Directed at Answering Four Basic Questions: Who are the markets we are trying to serve?What services must be in place to fully serve those markets?What is the institutional “branding” that will enable our college or university to appeal to those markets?How will we know if we are successful in serving those markets?
91Drilling DownWhy do some students to whom we extend an offer of admission choose to attend our institution?Why do other students to whom we extend an offer of admission choose to attend a different school?How is our institution perceived by prospective students within the admissions marketplace?What sources of information do students draw upon in shaping those perceptions?What is the role of financial aid in shaping the college selection decision?
92Continuing/Returning Students Student Satisfaction Research- ACT Survey of Student Opinions- Noel-Levitz Student Satisfaction InventoryACT Survey of Student OpinionsStudent use of, and satisfaction with 21 programs and services typically found at a college or university (e.g. academic advising, library, computing, residence life, food services, etc.)Student satisfaction with 43 dimensions of campus environment (e.g., out-of-classroom availability of faculty, availability of required courses, quality of advisement information, facilities, admissions and registration procedures, etc.)Self-estimated intellectual, personal, and social growth; Overall impressions of the college experienceNOTE: Survey is available in four-year and two-year college versions.
94What About Non-Returning Student Research? Drilling Deeper….. Commercial instruments exist, but response rates tend to be low, and reported reasons for leaving politically correct – personal or financial reasons.For the last several years, we have administered the Survey of Student Opinions during the Spring term to a robust sample of students across freshman, sophomore, junior, and senior classes.The following Fall, the respondent pool is disaggregated into those who took the Survey and returned in the Fall, and those who took the Survey, did not return in the Fall, and did not graduate.Test for statistically significant differences in response patterns between the two groups.
95Student Engagement College Student Expectations Questionnaire (CSXQ) College Student Experiences Questionnaire (CSEQ)National Survey of Student Engagement (NSSE)
97Alumni ResearchCommercially prepared instruments exist. Decide if they meet your needs or if you have to develop your own.Decide early on why you are doing this research: Are you assessing the continuing relevance of the college experience? Are you cultivating prospects for the Development Office? Both? Be up front if fund raising is a component.Decide the which classes you need to survey; don’t go after every living alumnus unless you are a very young institution.
98Non-Student Measures of Institutional Effectiveness: Teaching Productivity, Instructional Costs, and Externally Funded Scholarship
10110 Key Strategic Indicators for Institutional Health Revenue StructureExpenditure StructureExcess (Deficit) of Current Fund Revenues Over Current Fund ExpendituresPercentage of Freshman Applicants Accepted and Percentage of Applicants Who EnrollRatio of Full Time Students to Full Time Faculty
10210 Key Strategic Indicators for Institutional Health Institutional Scholarship and Fellowship Expenditures as a Percent of Total Tuition and Fees IncomeTenure Status of Full Time FacultyPercent of Total Full Time Equivalent Employees Who Are FacultyEstimated Maintenance Backlog as a Percentage of the Replacement Value of PlantPercent of Living Alumni Who Have Given At Any Time During The Past Five Years
103Think of the Budget as Buckets of Money Two Kinds of Buckets: RevenuesExpenditures
104Major Revenue Streams Tuition and Fees Governmental Appropriations Contracts and GrantsEndowment IncomeSales and Services of Educational ActivitiesAuxiliary OperationsOther
105Major Expenditure Categories InstructionResearchPublic serviceAcademic SupportStudent ServicesInstitutional SupportOperation and Maintenance of PlantStudent AidMandatory TransfersNon-Mandatory TransfersAuxiliary EnterprisesNote: The term “Education and General Expenditures” refers to the core costs of doing business related to the institution’s mission of teaching, research, and service. It does not include non-mission related activities which typically fall under the umbrella of auxiliary enterprises.
110Instructional Demand Ratio Instructional ExpendituresTotal Operating Income
111Institutional Support Demand Ratio Institutional Support ExpendituresTotal Operating Income
112IPEDS Peer Analysis System Useful ResourceIPEDS Peer Analysis System
113Institutional “Dashboards” that report on key success indicators can be an succinct means of reporting on basic measures of institutional effectiveness.
114Claims of institutional effectiveness are stronger when focal institution’s measures on important measures are compared with those of peer institutions.
115Choosing Peer Groupings Scientific – Cluster Analysis or Other Multivariate ToolPragmatic - e.g. Compensation Peers“Whatever!” - e.g. Admissions Peers
116We are extending the Dashboard concept to include key variables related to the University’s new Strategic Plan that enable us to compare our position vis-à-vis actual peers and aspirational peers.
117Dashboard Variables Being Collected STUDENT CHARACTERISTICS Percent of Accepted Applicants Matriculated Total Minority Percentage in Undergraduate Student Body Freshman to Sophomore Retention Rate Four Year Graduation Rate Six Year Graduation Rate RESEARCH ACTIVITY Total R&D Expenditures per Full Time Faculty Total Service Expenditures per Full Time Faculty FINANCE University Endowment as of June 30 Alumni Annual Giving Rate INSTRUCTION Full Time Students per Full Time Faculty Total Degrees Granted Total Doctoral Degrees Granted Percent of Faculty With Tenure Percent of Faculty That Are Full Time Percent of Women Among Full Time Faculty Percent of Minorities Among Full Time Faculty Percent of Minorities Among Full Time Staff Total Faculty Who Are National Academy Members
119Assessing Student Learning Outcomes I’ll provide only a brief overview, as there are others (Linda Suskie, Trudy Banta, Jeff Seybert) who are far better versed than I am.That said, understand that assessment of student learning is at the core of demonstrating overall institutional effectiveness.Assessment of student learning is a direct response to the inadequacy of student grades for describing general student learning outcomes.
120According to Paul Dressel of Michigan State University (1983), Grades Are: “ An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material. ”
121There is no “one size fits all” approach to assessment of learning across the disciplines None of these should be applied to evaluation of individual student performance for purposes of grading and completion/graduation status.1. Standardized TestsGeneral Education or Discipline SpecificState, Regional, or National Licensure Exams2. Locally Produced Tests/Items“Stand Alone” or Imbedded3. Portfolios/Student ArtifactsCollections of Students’ WorkCan Be Time Consuming, Labor Intensive, and Expensive4. Final ProjectsDemonstrate Mastery of Discipline and/or General Education5. Capstone Experiences/CoursesEntire Course, Portion of a Course, or a Related Experience (Internship, Work Placement, etc.)
123Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes Start from institutional Mission: The Mission Statement should define the institution and its core values, communicate what business(es) it will engage in, and convey a sense of its unique character.Understand the Market and create a “Brand”: Market and branding decisions are important in today’s competitive environment, and should naturally flow from the mission. Avoid rushing into new markets without careful research/planning.Long Range and Strategic Planning: these types of plans should set the stage, articulating directions chosen to help the institution achieve its mission. Focus here is on what to do.
124Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes Tactical and operational Planning: These plans have a shorter time horizon and focus on how to achieve the goals and purposes that have been set.Integrate Academic, Resource, and Facilities Elements: Academic decisions should be the driving force, but good planning is holistic in approach.Set Priorities and Establish a Timetable: Decisions about priorities and timelines should recognize the “art of the possible” for a unique institutional environment.
125Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes Plans as Guidelines, not Monuments: The process is much more important than the plan itself. Change is constant.Identify and Implement Appropriate Linkages: Results from assessments should flow to budget decisions and into new rounds of planning. Department/division work should link to institutional work.Leadership is Imperative: none of this will happen without direct support of campus executives.
126Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes This is About Change Management: Respect the institution’s culture and history; be realistic in expectations; work toward the sustainable.Encourage Broad Participation and Wide Communication: These characteristics provide a solid foundation for a collegial process, and improve prospects for a successful implementation.Student Learning Outcomes Assessments and assessment of Institutional Effectiveness: Both types of assessment are important in determining if the institution is achieving its goals.
127Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes Develop a Capacity for Institutional Research: Internal research and external benchmarking is essential in assessment, but the function can be institutionally organized in many ways.Judge Progress and Re-Calibrate the Plan: Consider how the institution is doing in reaching its goals and purposes, and whether processes are all they should be.Do Not Re-Invent the Wheel: Use external resources such as the Society for College and University Planning and other topical higher education organizations and data bases.