Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and.

Integrating Higher Education Planning and Assessment: Concepts and Strategies
Michael F. Middaugh Assistant Vice President for Institutional Research and Planning University of Delaware Commissioner and Member of Executive Committee Middle States Commission on Higher Education Andrea A. Lex Dean of Planning and Institutional Research Prince George’s Community College

“The nicest thing about not planning is that failure comes as a complete surprise and is not preceded by a period of worry and depression.” John Preston, Boston College “A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools.” Douglas Adams, Hitchhikers Guide to the Galaxy

A Brief Overview of Accreditation

What is accreditation? A means of providing quality assurance in the areas of Student learning and achievement Curriculum development and support Faculty Facilities and equipment Finance, administration, and governance Integrity and compliance

Why is it done? Voluntary self-regulation Voluntary self-examination
Opportunity for peer review Quality assurance Public accountability And, in the case of regional accreditation Ability to receive Federal financial aid dollars

What do we use? Characteristics of Excellence
Institutional Mission Institutional Effectiveness Planning Resource Allocation Leadership Governance Administration Integrity Institutional Assessment Educational Effectiveness Admissions & Retention Support Services Faculty Offerings General Education Related Education Activities Assessment of Student Learning

What have we learned? That the quality of an institution’s accreditation self study hinges on how it addresses the standards related to assessing institutional effectiveness (Standard 7) assessing student learning outcomes (Standard 14) implementing a vital and comprehensive strategic planning process (Standard 2).

Commission-Directed Follow-up (2004-2009)
81 percent required some sort of follow-up Student Learning Outcomes (S 14) Institutional Effectiveness (S 7) Strategic Planning (S 2) Sufficient Resources (S 3) Enrollment Management (S 8)

Creating and Closing the Loop
Institutional Mission is the Foundation of & Our Reason for Being

Creating and Closing the Loop Characteristics of Excellence
Institutional Mission Educational Effectiveness Admissions & Retention Support Services Faculty Offerings General Education Related Education Activities Assessment of Student Learning To what extent have students achieved the educational and co-curricular goals set forth by an institution as it recruits and retains these individuals? -- Assessment of Student Learning (Standard 14)

Standard 14: Student Learning Outcomes Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals.

Selected Fundamental Elements for MSCHE Standard 14
Articulated expectations for student learning (at institutional, degree/program, and course levels) Documented, organized, and sustained assessment processes (that may include a formal assessment plan) Evidence that student learning assessment information is shared and used to improve teaching and learning Documented use of student learning assessment information as part of institutional assessment

Institutional Mission Institutional Effectiveness Planning Resource Allocation Leadership Governance Administration Integrity Institutional Assessment To what extent has the institution acquired and allocated human and fiscal resources in a manner that supports the achievement of its goals? -- Assessment of Institutional Effectiveness (Standard 7)

Standard 7: Institutional Assessment The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards.

Documented, organized, and sustained assessment processes to evaluate the total range of programs and services, achievement of mission, and compliance with accreditation standards Evidence that assessment results are shared and used in institutional planning, resource allocation and renewal Written institutional strategic plan(s) that reflect(s) consideration of assessment results Evidence that institutional assessment findings are used to improve student success

Institutional Mission Planning Planning Institutional Effectiveness Planning Resource Allocation Leadership Governance Administration Integrity Institutional Assessment Educational Effectiveness Admissions & Retention Support Services Faculty Offerings General Education Related Education Activities Assessment of Student Learning Planning To what extent are an institution’s policy, decision making and resource allocation processes shaped by a systematic, evidence-based, and mission-guided planning process? Assessment of Institutional Planning (Standard 2)

Standard 2: Planning, Resource Allocation and Institutional Renewal An institution conducts ongoing planning and resource allocation based on its mission and goals, develops objectives to achieve them, and utilizes the results of its assessment activities for institutional renewal. Implementation and subsequent evaluation of the success of the strategic plan and resource allocation support the development and change necessary to improve and to maintain quality.

Clearly stated goals and objectives, at both the institutional and unit levels, that reflect conclusions drawn from assessments that are used for planning and resource allocation at those levels Planning and improvement processes that are clearly communicated, provide for constituent participation, and incorporate the use of assessment results Well-defined decision-making processes and authority that facilitates planning and renewal and the assignment of responsibility for improvement and assurance of accountability Periodic assessment of the effectiveness of planning, resource allocation, and institutional renewal processes

Central Threads Running Through All Accreditation Requirements:
Planning must be systematic Planning must be rooted in an institution’s mission Planning must be predicated on analytical and evaluative information Planning must be used for institutional decisions, including resource allocation

The Closed Loop Institutions must plan effectively in order to be effective. Where that is the case, the accreditation process is an affirmation of the evidence of that effectiveness.

Components of This Curriculum
Best Practice in Planning Best Practice in Assessment Closing the Loop Between Planning and Assessment Real Life Case Studies

End Result Institutions must plan effectively in order to be effective. Where that is the case, the accreditation process is an affirmation of the evidence of that effectiveness.

Components of This Curriculum
Best Practice in Planning Best Practice in Assessment Closing the Loop Between Planning and Assessment Real Life Case Studies

In thinking about what good planning is, it is useful to begin the discussion by describing what it is not: George Keller, Academic Strategy: the Management Revolution in American Higher Education, Johns Hopkins University Press, 1983. Strategic Planning is not: The production of a blueprint. A set of platitudes. The personal vision of a president or board of trustees. A collection of departmental plans, compiled and edited.

Strategic Planning is not:
Done by planners. A substitution of numbers for important intangibles. A form of surrender to market conditions and trends. Something done on an annual retreat. A way of eliminating risks. An attempt to read tea leaves and outwit the future.

Strategic Planning IS Academic strategic decision-making means that a college or university and its leaders are active rather than passive about their position in history. Strategic planning looks outward and is focused on keeping the institution in step with the changing environment. Academic strategy making is competitive, recognizing that higher education is subject to economic market conditions and to incredibly strong competition. Strategic planning concentrates on decisions, not on documented plans, analyses, forecasts, and goals.

Strategic Planning IS Strategy making is a blend of rational and economic analysis, political maneuvering, and psychological interplay. It is therefore participatory and highly tolerant of controversy. Strategic planning concentrates on the fate of the institution above everything else.

Let’s begin at the beginning . . .
Why are we here?

Mission Statement A good mission statement is a carefully reasoned analysis of what an institution aspires to be, and the core values that it embraces. It avoids cliché language, e.g., “Students and faculty will interact in a rich intellectual environment in which each individual has the opportunity to achieve their full potential.” Noble sentiment, but says nothing about the institution’s purposes and priorities. Mission must speak to central institutional issues, e.g., desired balance between undergraduate and graduate education; relative emphasis on teaching, research, and service, respectively; and so on.

Mission Statement Mission statements are characterized by a sense of vision that, while not immutable, nonetheless represents a long-term statement of institutional values and direction around which human and fiscal resource allocation decisions can be made. While cognizant of the institution’s ever changing external environment, mission statements are not whimsical, morphing with each new market trend that emerges. The mission statement provides a clear sense of direction around which action-oriented goal statements and measurable planning objectives can be developed.

Planning Goals Goal statements are derived from the institutional mission, and help to define policy. For example, the mission statement might say that “The University affirms its historic mission of providing the highest quality education for its undergraduate students, while maintaining excellence in selected graduate programs.” The mission statement is underscoring the primacy of undergraduate instruction in the curriculum. The question for planners is how to provide that high quality undergraduate instruction.

How will we achieve our mission?

Planning Goals The “how” translates into specific, action-oriented planning goals aimed at moving the institution toward a fuller realization of its mission. Possible goal related to the undergraduate education mission statement: “The University will continue to attract and retain the most academically talented and diverse undergraduate students, and support their intellectual, cultural, and ethical development as citizens and scholars.” Action verbs such as “attract,” “retain,” and “support” elevate the goal statement to policy level. How do we know that policy is being carried out? Measurable planning objectives.

Meeting Goals How will we know when we get there?
What will we look like when we get there?

Planning Objectives Planning objectives provide empirical evidence of the extent to which planning goals are being achieved. Consider the following planning objectives as they relate to our goal to attract, retain, and support academically talented and diverse students. Retain a freshman admissions target of 3200 to 3400 students annually, with an admissions profile for academic year 2007 of 23,000 applications, a 40 percent admit rate, and a yield rate in excess of 35 percent. Improve the alignment of undergraduate enrollment distribution and instructional resource distribution across the disciplines, especially with respect to faculty.

Planning Objectives Continued
Maintain a freshman-to-sophomore retention rate above the national average for highly selective institutions, and seek to achieve a consistent rate of 90 percent or higher. Maintain a graduation rate above the national average for highly selective institutions, and seek to achieve a consistent six-year rate of 75 percent or higher. Increase minority and international enrollment, with retention and graduation rates for those populations consistent with the university-wide averages for all students.

How are we doing? How close are we to getting there?
How are we doing along the way?

Planning Objectives The defining characteristic for any good planning objective is that it must be measurable. In other words, assessable. Colleges or universities embarking on any planning process – long range, strategic, tactical – require a systematic institutional research capability. While smaller institutions may not have an office of institutional research, per se, they must nonetheless have the capability of quantitatively and qualitatively assessing the extent to which planning objectives are being implemented, planning goals are being achieved, and the institutions mission is being realized.

Breakout Session Using your own Institutional Mission Statement, create at least five planning goals and associated measurable planning objectives that will help assess the extent to which that mission is being realized.

Linking Planning and Assessment
Two Case Studies at The Same Institution

The First Cut: Creating a Culture of Planning

University of Delaware Strategic Planning: 1990 - 2000

University of Delaware Mission Statement
“The University reaffirms its historic mission to provide the highest quality education for its undergraduate students, while maintaining excellence in selected graduate programs. The faculty are responsible for helping students learn to reason critically and independently, gain knowledge of the diverse culture and environment in which they live, and develop into well-informed citizens and leaders. To accomplish these goals, the University provides a learning setting enriched by undergraduate student research, experiential learning, and study-abroad programs. The University places high priority on the education of qualified Delaware residents and provides opportunity for a diverse group of citizens to participate in postsecondary education. Since the University is located in a state with a small population, providing programs of quality and diversity requires a community of student-scholars that reaches beyond the boundaries of the state, one that reflects the nation’s racial and cultural diversity. “The University of Delaware also aspires to excellence in graduate education, the heart of which is scholarship and research. The creation, application, and communication of knowledge is a primary goal of the institution and of every faculty member, providing the substance for creative, informed teaching. Research is typically based on cooperation between faculty and students, whereby faculty mentors teach students to conduct independent research and to master problem-solving techniques. Through involvement of undergraduates in faculty research, the University creates a special bond between its undergraduate and graduate programs. “The University is also committed to providing service to society, especially in Delaware and the neighboring region. Public service is a responsibility of every academic unit. In addition, each faculty member is responsible for service to the University community and to his or her profession. The University emphasizes practical research, provides extension services, and works to solve problems confronting the community. (University of Delaware, Middle States Self-Study Report, April 2001)

Case Study – University of Delaware
In 1987, the University embarked on a comprehensive, long-range planning process, termed Project Vision. Over a period of 18 months, the campus developed a planning document with a broad spectrum of planning goals and measurable objectives embracing all aspects of University operations. In Fall of 1988, the President who initiated Project Vision suddenly resigned. At the same time, the Delaware economy along with that of the entire mid-Atlantic region was plunging into deep recession

Rather than let 18 months of planning activity go for naught – even though resources would be scarce for the foreseeable future – a panel of distinguished senior faculty was assembled to review the Project Vision planning document and to cull out those goals and objectives that were clearly consistent with, and essential to furthering the University mission. The resulting document, Focused Vision, was economical when compared with its progenitor, both in terms of words and resource requirements. However, the economy would still clearly preclude anything even remotely approaching implementation. In 1990, the University hired its 25th President, David P. Roselle.

In order to maintain planning momentum, the President consulted with his senior staff, the faculty, and appropriate constituencies across campus to determine those areas that required immediate attention. From these consultations, the President articulated four strategic initiatives that would constitute the focus of decision-making and resource allocation activity in the immediate future. Those initiatives were competitive compensation for faculty and staff; enhanced access to the University for undergraduates through expanded availability of financial aid; a more student-centered campus environment; and renovation and rehabilitation of campus facilities.

These priorities were not a “wish list.” They grew out of a careful examination of empirical data provided by the University’s Office of Institutional Research and Planning and other data sources. Consider the following: When compared with the 24 Category Doctoral I universities in the states contiguous to Delaware, and the District of Columbia, in 1991 the average salary for all three major faculty ranks at the University of Delaware ranked near the bottom of the list. The Student College Selection Survey indicated that students were receiving offers of more aid from admissions competitors, and that the aid packages had more grants and fewer loans than University aid packages. Not surprising, the University was at a competitive disadvantage for academically talented students.

University scores on the ACT Student Opinion Survey suggested that the institution had considerable room for improvement with respect to student satisfaction with programs and services, and with a number of areas in student life. The University was looking at in excess of $200 million in deferred maintenance to its buildings and grounds.

A critical factor in moving forward with these initiatives was getting the campus to understand that the economy was in recession and that there would be no immediate or massive infusions of new resources. Colleges and universities have multiple revenue streams – tuition, state appropriation in the case of public institutions, contracts and grants, gifts, etc. While growing revenue streams is an important strategic initiative, so too is the commitment to not balance budgets on the backs of students through inordinately large tuition increases. Resource reallocation would be the primary source of funding the four strategic initiatives, and it was critical that the campus understand from where funds were reallocated, and why.

The University went on public record in 1991:
Average total compensation for faculty at each academic rank would be at or above the median within five years for the 24 Category I Doctoral Universities identified as salary peers. Total undergraduate financial aid from all sources would increase by 100 percent within five years. Student satisfaction with programs and services at the University, as measured through the ACT Student Opinion Survey would demonstrate significant gains within five years. The University would commit itself to a policy of annually setting aside at least 2 percent of the replacement value of the physical plant, to be used for facilities renovation and rehabilitation.

Results - Salaries

Results – Financial Aid

Results – Student Satisfaction

Results - Facilities By 2000, the University had renovated every classroom in its entire building inventory, retrofitting most with state-of-the-art teaching technology. An aggressive program of fundraising enabled not only the aforementioned renovation and rehabilitation, but also the construction of several new classroom and student services buildings. The University is now on a cycle of planned maintenance, as opposed to deferred maintenance.

Results – From an Accreditation Perspective
“ The University of Delaware has every reason to take enormous pride in what it has accomplished over the past 10 years. A decade ago, it was coming out of a period of considerable turmoil. Today, the University is seen as a national model for the integration of information technology in every aspect of university life: teaching and learning, research and service, academic support, and campus administration. It has created a physical plant that has few, if any, peers among public universities and would be the envy of most private colleges. These substantial achievements could not have happened without extraordinary leadership from the senior administration.” “ Better than almost any university we are familiar with, Delaware has a clear sense of what it wants to be, namely, a university that offers high quality undergraduate education with targeted areas of excellence in graduate education and research.” " The review team was enormously impressed by the high level of morale that pervades the faculty, staff, and students. Almost without exception, the people we spoke to take great pride in being part of the University.” Middle States Evaluation Team, 2001

The University submitted its Periodic Review Report to the Commission on Higher Education in May 2006, outlining its strategies for incorporating systematic assessment of student learning outcomes into all facets of campus planning. The Commission thanked the University for its Report, commended the University on its content, and reaffirmed accreditation.

What Was Not Apparent to MSCHE…
Despite an exemplary self study and PRR process, planning was not yet institutionalized into a formal, iterative process at the institution. The four strategic initiatives articulated in 1990 continued to be the benchmarks against which the University measured success well into the 2000 decade. The University needed to move forward with a new strategic planning effort that would more substantively incorporate planning into the organizational fiber of the institution.

The Second Cut: Institutionalizing Planning

University of Delaware Strategic Planning: 2007 - ???

The Path to Prominence On July 1, 2007, Patrick T. Harker succeeded David Roselle as the 26th President of the University. As former Dean of the Wharton School at the University of Pennsylvania, President Harker was intimately familiar with the concept of strategic planning. He immediately appointed a Strategic Planning Committee led by two distinguished senior faculty and comprised largely of individuals from the academic as opposed to administrative side of the institution.

The Path to Prominence During the academic year, the Strategic Planning Committee met with over 100 constituent groups from within, and external to the University. These information-gathering sessions were part of a comprehensive SWOT analysis. In April 2008, the Report of the Strategic Planning Committee was delivered to the President. In July, 2008, the President presented the Path to Prominence, the strategic planning document that would guide institutional activity for the next several years.

The Path to Prominence: The Strategic Initiatives
A Diverse and Stimulating Undergraduate Academic Environment A Premier Research and Graduate University Excellence in Professional Education The Initiative for the Planet The Global Initiative An Engaged University

Each of the six initiatives was operationalized with measurable goals and planning objectives. The Path to Prominence document and supporting materials were broadly circulated via the University website: for maximum transparency and accountability with respect to what was being accomplished. In Fall 2008, the U.S. and Delaware economies began to unravel, placing severe financial constraints on full implementation of the strategic initiatives.

While each of the six initiatives remain viable and will receive attention regardless of economic circumstances, hard decisions needed to be made with respect to where resources would be allocated. It was determined that three initiatives would be the primary (but not exclusive) focus of resource allocation: - A Diverse and Stimulating Undergraduate Academic Environment - A Premier Research and Graduate University - Excellence in Professional Education

The Path to Prominence: Moving Forward
The University decided to take advantage of the MSCHE decennial accreditation reaffirmation process, and utilize the “Special Topics Model” of self study as a means of documenting progress in implementing these three key strategic initiatives. The scheduled 2011 site visit by an MSCHE Evaluation Team affords the University not only the opportunity to demonstrate compliance with the 14 standards in Characteristics of Excellence; it provides us with a team of expert consultants in the topical areas on which we have elected to focus. Their feedback on the content of the Special Topics Self Study will prove invaluable in moving the institution forward.

The Path to Prominence: Tactical Issues
In any strategic planning process, there needs to be a clear linkage between human and fiscal resource allocation and the initiatives outlined in the strategic plan. The University of Delaware has adopted a responsibility centered budget management model that is designed to financially reward activity that supports full implementation of key strategic initiatives. The University has also designated an Associate Provost for Institutional Effectiveness, tasked with overseeing assessment activity that measures the effective implementation of The Path to Prominence.

The Path to Prominence: Tactical Issues
The Steering Committee for the 2011 MSCHE has tasked itself with responsibility for design of a formal, iterative institutional strategic planning process that will, on an ongoing basis, continuously evaluate the implementation of existing strategic initiatives, assess the continuing relevance of initiatives over time, and develop new initiatives, as appropriate, in the face of a changing external environment and/or advent of specific strategic opportunities. The intent is to institutionalize the strategic planning process so that it is truly an organizational function rather than the creation or vision of any particular individual.

Considerations When Creating a Strategic Planning Process

“Planning to Plan:” Organizing Your Institution for the Planning Process

Organizational Structure
The Planning Committee: Who Serves? How Representative? What is the Scope of Responsibilities: If the Planning Committee is to be effective, it must be “blessed” by the Chief Executive Officer and the Chief Academic Officer. The President and Provost need not be members of the Committee, but ideally will attend the initial meeting, and will formally receive the Committee Report.

Organizational Structure
Who staffs the Planning Committee? Institutional Research Budget Office Others Faculty Staff

Planning Tools Surveys: Students Surveys: Employees
Admitted Student Questionnaire Entering Student Needs Assessment Survey Student Satisfaction Survey Post-Graduation Placement Survey Alumni Survey Surveys: Employees Employee Satisfaction Survey Faculty Activity Analysis Campus Climate Survey

Planning Tools Institutional Studies: Admissions Monitoring Analyses
Faculty Workload Analyses Budget Support Data Persistence/Graduation Analyses Compensation Studies: Market/Equity Financial Ratio Analyses Benchmarking Studies Economic Impact Studies

Planning Tools Academic Program Review Programmatic Self-Examination
Internal and External Review Teams Focus on Several Levels of Quality Programs With National/International Distinction Programs of Regional Distinction (Meet Local Needs) Programs That, With an Infusion of Resources, Could Achieve Distinction Programs Targeted for Retrenchment or Elimination

Planning Tools Environmental Scan: Demographic issues b. Market issues
c. Political issues d. Economic issues e. Technological issues f. Natural disasters g. Other

Planning Tools – SWOT Analysis
Take an honest look at your institution within the context of a SWOT analysis (Strengths; Weaknesses; Opportunities; Threats) Leveraging Situation - Opportunity + Strength Constraining Situation - Opportunity + Weakness Vulnerable Situation – Threat + Strength Problematic Situation – Threat + Weakness

In the Final Analysis, Planning is Directed at Answering Four Basic Questions:
Who are the markets we are trying to serve? What services must be in place to fully serve those markets? What is the institutional “branding” that will enable our college or university to appeal to those markets? How will we know if we are successful in serving those markets?

Who are the markets we are trying to serve?
Largely Dictated by Mission Baccalaureate Colleges: Primarily undergraduate students (what kinds?) Master’s Institutions: Undergraduate students, selected graduate students, targeted state/community partnerships Doctoral Universities: Undergraduate students, graduate students, research/public service contractors/grantors, broad range of regional, national, international partnerships Community Colleges: Matriculated undergraduate students, occasional students with specific training needs, general interest students, local businesses, county government

What services must be in place to fully serve those markets
What services must be in place to fully serve those markets? Dictated by Markets Academic Support Services (Library, computing, advising, tutoring, etc.) Student Support Services (Residence life, counseling, health services, student center, recreation services, etc.) Institutional Support Services (General administrative support, sponsored research, extension office, alumni/governmental relations, etc.)

What is the institutional “branding” that will enable our college or university to appeal to those markets? Market Specific University of Delaware: “A Teaching University” Trenton State College : “The College of New Jersey” University of Phoenix: “The University for Working Adults”

How will we know if we are successful in serving those markets
How will we know if we are successful in serving those markets? Middle States Assessment Standards Standard 7: The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards. Standard14: Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals.

Breakout Session How would you respond to the four key planning questions within the context of your institution’s mission?

In the Final Analysis, Planning is Directed at Answering Four Basic Questions:
Who are the markets we are trying to serve? What services must be in place to fully serve those markets? What is the institutional “branding” that will enable our college or university to appeal to those markets? How will we know if we are successful in serving those markets?

What Should Be Assessed Under Standard 7?

You do not have to measure everything.
PRIORITIZE what is to be measured within the context of your institution’s culture and needs.

Students Admitted Entering Continuing Non-Returning Graduating Alumni

Environmental Issues Student and Faculty Engagement
Student and Staff Satisfaction Employee Productivity Compensation - Market - Equity Campus Climate Economic Impact

Some Basic Measures

A Typical Admissions Monitoring Report

Drilling Down Why do some students to whom we extend an offer of admission choose to attend our institution? Why do other students to whom we extend an offer of admission choose to attend a different school? How is our institution perceived by prospective students within the admissions marketplace? What sources of information do students draw upon in shaping those perceptions? What is the role of financial aid in shaping the college selection decision?

Continuing/Returning Students
Student Satisfaction Research - ACT Survey of Student Opinions - Noel-Levitz Student Satisfaction Inventory ACT Survey of Student Opinions Student use of, and satisfaction with 21 programs and services typically found at a college or university (e.g. academic advising, library, computing, residence life, food services, etc.) Student satisfaction with 43 dimensions of campus environment (e.g., out-of-classroom availability of faculty, availability of required courses, quality of advisement information, facilities, admissions and registration procedures, etc.) Self-estimated intellectual, personal, and social growth; Overall impressions of the college experience NOTE: Survey is available in four-year and two-year college versions.

What About Non-Returning Student Research?

What About Non-Returning Student Research? Drilling Deeper…..
Commercial instruments exist, but response rates tend to be low, and reported reasons for leaving politically correct – personal or financial reasons. For the last several years, we have administered the Survey of Student Opinions during the Spring term to a robust sample of students across freshman, sophomore, junior, and senior classes. The following Fall, the respondent pool is disaggregated into those who took the Survey and returned in the Fall, and those who took the Survey, did not return in the Fall, and did not graduate. Test for statistically significant differences in response patterns between the two groups.

Student Engagement College Student Expectations Questionnaire (CSXQ)
College Student Experiences Questionnaire (CSEQ) National Survey of Student Engagement (NSSE)

Graduating Students

Alumni Research Commercially prepared instruments exist. Decide if they meet your needs or if you have to develop your own. Decide early on why you are doing this research: Are you assessing the continuing relevance of the college experience? Are you cultivating prospects for the Development Office? Both? Be up front if fund raising is a component. Decide the which classes you need to survey; don’t go after every living alumnus unless you are a very young institution.

Non-Student Measures of Institutional Effectiveness: Teaching Productivity, Instructional Costs, and Externally Funded Scholarship

Transparency in Assessment

What About Administrative Productivity???

10 Key Strategic Indicators for Institutional Health
Revenue Structure Expenditure Structure Excess (Deficit) of Current Fund Revenues Over Current Fund Expenditures Percentage of Freshman Applicants Accepted and Percentage of Applicants Who Enroll Ratio of Full Time Students to Full Time Faculty

10 Key Strategic Indicators for Institutional Health
Institutional Scholarship and Fellowship Expenditures as a Percent of Total Tuition and Fees Income Tenure Status of Full Time Faculty Percent of Total Full Time Equivalent Employees Who Are Faculty Estimated Maintenance Backlog as a Percentage of the Replacement Value of Plant Percent of Living Alumni Who Have Given At Any Time During The Past Five Years

Think of the Budget as Buckets of Money Two Kinds of Buckets:
Revenues Expenditures

Major Revenue Streams Tuition and Fees Governmental Appropriations
Contracts and Grants Endowment Income Sales and Services of Educational Activities Auxiliary Operations Other

Major Expenditure Categories
Instruction Research Public service Academic Support Student Services Institutional Support Operation and Maintenance of Plant Student Aid Mandatory Transfers Non-Mandatory Transfers Auxiliary Enterprises Note: The term “Education and General Expenditures” refers to the core costs of doing business related to the institution’s mission of teaching, research, and service. It does not include non-mission related activities which typically fall under the umbrella of auxiliary enterprises.

Ratio Analysis: Revenue Contribution Ratios Expenditure Demand Ratios

Tuition Dependency Ratio
Net Tuition Education and General Expenditures

Other Important Revenue Contributors
State Appropriation Gifts Endowment Income

Expenditure Demand Ratios

Instructional Demand Ratio
Instructional Expenditures Total Operating Income

Institutional Support Demand Ratio
Institutional Support Expenditures Total Operating Income

IPEDS Peer Analysis System
Useful Resource IPEDS Peer Analysis System

Institutional “Dashboards” that report on key success indicators can be an succinct means of reporting on basic measures of institutional effectiveness.

Claims of institutional effectiveness are stronger when focal institution’s measures on important measures are compared with those of peer institutions.

Choosing Peer Groupings
Scientific – Cluster Analysis or Other Multivariate Tool Pragmatic - e.g. Compensation Peers “Whatever!” - e.g. Admissions Peers

We are extending the Dashboard concept to include key variables related to the University’s new Strategic Plan that enable us to compare our position vis-à-vis actual peers and aspirational peers.

Dashboard Variables Being Collected
STUDENT CHARACTERISTICS Percent of Accepted Applicants Matriculated Total Minority Percentage in Undergraduate Student Body Freshman to Sophomore Retention Rate Four Year Graduation Rate Six Year Graduation Rate RESEARCH ACTIVITY Total R&D Expenditures per Full Time Faculty Total Service Expenditures per Full Time Faculty FINANCE University Endowment as of June 30 Alumni Annual Giving Rate INSTRUCTION Full Time Students per Full Time Faculty Total Degrees Granted Total Doctoral Degrees Granted Percent of Faculty With Tenure Percent of Faculty That Are Full Time Percent of Women Among Full Time Faculty Percent of Minorities Among Full Time Faculty Percent of Minorities Among Full Time Staff Total Faculty Who Are National Academy Members

What Should Be Assessed Under Standard 14?

Assessing Student Learning Outcomes
I’ll provide only a brief overview, as there are others (Linda Suskie, Trudy Banta, Jeff Seybert) who are far better versed than I am. That said, understand that assessment of student learning is at the core of demonstrating overall institutional effectiveness. Assessment of student learning is a direct response to the inadequacy of student grades for describing general student learning outcomes.

According to Paul Dressel of Michigan State University (1983), Grades Are:
“ An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material. ”

There is no “one size fits all” approach to assessment of learning across the disciplines
None of these should be applied to evaluation of individual student performance for purposes of grading and completion/graduation status. 1. Standardized Tests General Education or Discipline Specific State, Regional, or National Licensure Exams 2. Locally Produced Tests/Items “Stand Alone” or Imbedded 3. Portfolios/Student Artifacts Collections of Students’ Work Can Be Time Consuming, Labor Intensive, and Expensive 4. Final Projects Demonstrate Mastery of Discipline and/or General Education 5. Capstone Experiences/Courses Entire Course, Portion of a Course, or a Related Experience (Internship, Work Placement, etc.)

Tying It All Together……

Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes
Start from institutional Mission: The Mission Statement should define the institution and its core values, communicate what business(es) it will engage in, and convey a sense of its unique character. Understand the Market and create a “Brand”: Market and branding decisions are important in today’s competitive environment, and should naturally flow from the mission. Avoid rushing into new markets without careful research/planning. Long Range and Strategic Planning: these types of plans should set the stage, articulating directions chosen to help the institution achieve its mission. Focus here is on what to do.

Tactical and operational Planning: These plans have a shorter time horizon and focus on how to achieve the goals and purposes that have been set. Integrate Academic, Resource, and Facilities Elements: Academic decisions should be the driving force, but good planning is holistic in approach. Set Priorities and Establish a Timetable: Decisions about priorities and timelines should recognize the “art of the possible” for a unique institutional environment.

Plans as Guidelines, not Monuments: The process is much more important than the plan itself. Change is constant. Identify and Implement Appropriate Linkages: Results from assessments should flow to budget decisions and into new rounds of planning. Department/division work should link to institutional work. Leadership is Imperative: none of this will happen without direct support of campus executives.

This is About Change Management: Respect the institution’s culture and history; be realistic in expectations; work toward the sustainable. Encourage Broad Participation and Wide Communication: These characteristics provide a solid foundation for a collegial process, and improve prospects for a successful implementation. Student Learning Outcomes Assessments and assessment of Institutional Effectiveness: Both types of assessment are important in determining if the institution is achieving its goals.

Develop a Capacity for Institutional Research: Internal research and external benchmarking is essential in assessment, but the function can be institutionally organized in many ways. Judge Progress and Re-Calibrate the Plan: Consider how the institution is doing in reaching its goals and purposes, and whether processes are all they should be. Do Not Re-Invent the Wheel: Use external resources such as the Society for College and University Planning and other topical higher education organizations and data bases.

Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and.

Similar presentations

Presentation on theme: "Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and.

Similar presentations

Presentation on theme: "Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and."— Presentation transcript:

Similar presentations

About project

Feedback