1. Institutional Effectiveness Presented By Claudette H. Williams

2. Institutional Effectiveness: A Definition
A systematic, explicit, and documented process of measuring performance against the mission in all aspects of an institution.
A commitment to continuous improvement is at the heart of an ongoing planning and evaluation process. It is a continuous, cyclical process that is participative, flexible, relevant, and responsive.
SACSCOC Resource Manual (2012), p. 13.
A commitment to continuous improvement is at the heart of an ongoing planning and evaluation process. It is a continuous, cyclical process that is participative, flexible, relevant, and responsive.
Continuous Quality Enhancement
“The concept of quality enhancement is at the heart of the Commission’s philosophy of accreditation. . .
Although evaluation of an institution’s educational quality and its effectiveness in achieving its mission is a difficult task. . . , an institution is expected to document quality and effectiveness in all its major aspects” (SACS, 2001/2004, p. 5)
Purpose
Establish an approach to ensure and facilitate continuous quality enhancement
Provide evidence of continuous quality enhancement

3. Why Engage in Institutional Effectiveness?
To respond to internal and external demands
To determine performance in terms of expectations and standards using objective measures
To use resulting information to document, explain, and improve performance

4. Institutional Effectiveness
Involves planning and evaluation (integrated)
Addresses institutional mission, goals and outcomes (comprehensive)
Applies to all programs, services and activities in the institution (institution-wide)
Is cyclical (ongoing)
Occurs within a clearly defined and structured system (research-based)

5. Institutional Effectiveness
Continuing improvement in institutional quality
Collection, analysis and use of data
academic, student, administrative and operational activities
Findings demonstrate achievement of mission
Dynamic process
information is used to make decisions about all aspects of the institution’s activities

6. Institutional Effectiveness: Core Requirement 2.5
The institution engages in ongoing, integrated, and institution-wide research- based planning and evaluation processes that
(1) incorporate a systematic review of institutional mission, goals, and outcomes; (2) result in continuing improvement in institutional quality; and
(3) demonstrate the institution is effectively accomplishing its mission. (Institutional Effectiveness)

7. Core Requirement 2.5: Issues and Documentation to Consider
(1) incorporate a systematic review of institutional mission, goals, and outcomes;
Describe and provide documentation of
the institution’s planning and evaluation process
systematic, ongoing, integrated, research-based (data-based) process
how planning and evaluation reviews are conducted
the history of sustained planning and evaluation cycles, including the use of results for improvement
how the process is related to accomplishment of the mission
how the process is related to the budget
engagement of internal and external stakeholders
Required documentation
Description of the institutional effectiveness process
Documentation that shows that the process includes a systematic review that results in continuing improvement and demonstrates the extent to which an institution accomplishes its goals
Other documentation
Evidence of linkage of institutional effectiveness to institutional mission
Institutional plans and budgets that demonstrate the linkage of assessment findings to planning at all levels
Strategic institution-wide plans that drive the mission
Minutes from appropriate units, committees, task forces charged with coordination of institutional effectiveness and evidence of braod-based involvement of faculty, staff, students and other stakeholders in the institutional effectiveness process
Documentation that relates to institutional effectiveness, such as budget preparation instructions, minutes of budget presentation meetings, annual reports, annual assessment updates, institutional effectiveness reports
Recent examples of how institution-wide planning/effectiveness has affected the institution
Sampling – must be
representative of the institution’s mission
a valid cross-section of programs from every school or division
A compelling case as to why the sampling and assessment findings are an appropriate representation of the institution’s educational programs
Sampling does not preclude the institution from having data and analysis available for all programs and units.

8. Core Requirement 2.5: Issues and Documentation to Consider, continued
(2) result in continuing improvement in institutional quality; and
Describe and provide documentation for
planning and evaluation processes resulting in continuing improvements in institutional quality
decision-making for improvement are informed by data from various sources about programs/units and services
extent to which institutional goals are being accomplished

9. Core Requirement 2.5: Issues and Documentation to Consider, continued
(3) demonstrate the institution is effectively accomplishing its mission.
Describe and provide documentation for
how systematic reviews of the institutional mission, goals, and outcomes are being incorporated in planning and evaluation processes
how planning and evaluation processes demonstrate that the institution is accomplishing its mission
appropriate support (budgetary and institutional research) for assessment programs institution-wide

10. Institutional Effectiveness: Comprehensive Standard 3.3.1
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:
educational programs, to include student learning outcomes
administrative support services
academic and student support services
research within its mission, if appropriate
community/public service within its mission, if appropriate

11. Comprehensive Standard 3. 3. 1. 1
Comprehensive Standard educational programs, to include student learning outcomes
For each educational program, describe and provide documentation for
effectiveness of the program and student learning
learning outcomes that are defined in measurable terms
assessment activities
assessment of outcomes are conducted periodically, results analyzed and reviewed, and used for improvements

12. Comprehensive Standard 3. 3. 1. 1
Comprehensive Standard educational programs, to include student learning outcomes, continued
For each educational program, describe
why sampling and findings are an appropriate representation of the institution’s programs, if appropriate
assessment instruments and why selected (multiple methods used?)
whether programs have been successful in achieving intended learning outcomes
whether program improvements made as a result of assessment findings
how the institution uses assessment results to improve educational programs
Required Documentation, if applicable
• Documentation of expected outcomes for educational programs and for student learning
outcomes
• Documentation of the evaluation of those outcomes
• Evidence that the student support services and programs effectively meet the needs of
students of all types
• Documentation of the use of the findings from assessment to improve the institution
• If sampling is used, (1) how the sampling is representative of the institution’s mission,
(2) documentation of a valid cross-section of programs, and a (3) case as to why
sampling and assessment findings are an appropriate representation of the institution’s
programs.
Reference to Commission Documents, if applicable
“Distance and Correspondence Education”
Cross References to other related Standards/Requirements, if applicable
Core Requirement 2.5
Comprehensive Standard 3.5.1
Federal Requirement 4.1

13. Comprehensive Standard 3.3.1.2 . . . administrative support services
For each administrative support service unit, describe and provide documentation for
Institution’s administrative support service units
Expected outcomes that are clearly defined in measurable terms
assessment of expected outcomes
periodic reviews used for improvements
use of assessment results to improve administrative support services
assessment instruments used and why selected
why sampling and findings were an appropriate representation of administrative units, if used

14. Comprehensive Standard 3.3.1.3 . . . academic support services
For each academic support service unit, describe and provide documentation for
expected outcomes that are clearly defined in measurable terms
assessment of expected outcomes
how periodic reviews are used for improvements
use of assessment results for improvement
assessment instruments used and why selected
why sampling and findings were an appropriate representation of the institution’s academic support service units

15. Comprehensive Standard 3. 3. 1. 4
Comprehensive Standard research within its mission, if appropriate
Describe and provide documentation for
the institution’s definition of research within its mission
institutional research outcomes in relation to its mission
expected outcomes that are clearly defined in measurable terms
assessment of research outcomes
periodic reviews used for improvement
how the institution uses assessment results to improve research
assessment instruments used and why selected
why sampling and findings were appropriate, if used
How sampling was representative of the institution’s research mission
faculty research and scholarship contributing to and benefiting the
institution’s research mission
how research contributes to the intellectual mission of the institution
How does the institution define research within its mission?
• Has the institution articulated its research outcomes in relation to its mission?
• How are expected outcomes clearly defined in measurable terms?
• What is the evidence of assessment activities for research?
• How are periodic reviews used for improvement of effectiveness?
• How does the institution’s use of assessment results improve research?
• What assessment instruments were used and why were they selected? Were
multiple assessment methods used? If so, describe.
• If the institution used sampling, why were the sampling and findings an appropriate
representation of the institution’s research mission?
• How does the faculty’s research and scholarship contribute to and benefit the
institution’s research mission?
• How does research contribute to the intellectual mission of the institution?
Documentation
Required Documentation, if applicable
• Definition of institution’s research mission
• Documentation of expected outcomes for its research mission
• Documentation of the evaluation of those outcomes
• Documentation of the use of the findings from assessment to improve the institution
• If sampling is used, (1) how the sampling is representative of the institution’s mission,
(2) documentation of a valid cross-section of units, and a (3) case as to why sampling
and assessment findings are an appropriate representation of the institution’s research
mission

16. Comprehensive Standard 3. 3. 1. 4
Comprehensive Standard research within its mission, if appropriate
Describe and provide documentation for
Institutional definition of community/public service
community/public service outcomes in relation to its mission
expected outcomes that are clearly defined in measurable terms
assessment of community/public service activities
use of assessment results to improve community/public
service
assessment instruments used and why selected?
why sampling and findings are an appropriate
representation of the institution’s community/public service mission
Relevant Questions for Consideration
• How does the institution define community/public service?
• Has the institution articulated its community/public service outcomes in relation to
its mission?
• How are expected outcomes clearly defined in measurable terms?
• What is the evidence of assessment activities for community/public service?
• How are periodic reviews used for improvements?
• How does the institution’s use of assessment results improve community/public
service?
• What assessment instruments were used and why were they selected? Were
multiple assessment methods used? If so, describe.
• If the institution used sampling, why were the sampling and findings an appropriate
representation of the institution’s community/public service mission?
Documentation
Required Documentation, if applicable
• Definition of institution’s community and public service mission
• Documentation of expected outcomes for its community and public service mission
• Documentation of the evaluation of those outcomes
• Documentation of the use of the findings from assessment to improve the institution
• If sampling is used, (1) how the sampling is representative of the institution’s mission,
(2) documentation of a valid cross-section of units, and a (3) case as to why sampling
and assessment findings are an appropriate representation of the institution’s
community and public service mission

17. Other Principles that Incorporate Institutional Effectiveness Activities
Comprehensive Standards
Federal Requirements

18. Questions Answered by Assessment
What are we trying to do?
Why are we trying to do this?
How well are we doing it?
How do we know how well we are doing?
How do we use the information?
Does using the information result in improvements?
Do the improvements we make work?

19. Goals of Assessment To improve performance To make informed decisions
Individually
As a department/division/unit
As an institution
To make informed decisions
To improve the quality of experiences for all
To promote continuous improvement

20. Levels of Assessment Institution Program The Classroom/Courses
Academic
Non-academic
The Classroom/Courses

21. Institutional Level Joint responsibility of several offices Assesses
General campus characteristics
Campus-wide issues
Institutional outcomes from Mission and Goals
Analyzes and disseminates findings to institution’s community and selected agencies
Monitors and encourages use of findings after dissemination

22. Program Level Responsibility of program staff
Assesses program goal attainment
Analyzes data and disseminates information
Staff
Offices of Institutional Effectiveness and Research
Other offices that may request or use findings
Implements recommendations from findings
Monitors improvement

23. Classroom/Course Level
General Education
Rationale
Course Description
Content
Standards alignment
Pedagogy – diversity
Outcomes – knowledge, attitudes, dispositions
Assignments – in/out of class
Assessment – methods, diversity
Analysis, findings and use of findings for improvement

24. Effective Assessment Meaningful Clear Manageable Focused Flexible
Specific
Truth-seeking
Appropriate
Objective
Reliable
Ethical
Valid
Systematic
Standards oriented

25. Successful Assessment
Involves everyone
Is goal/objective oriented
Addresses Student Learning Outcomes
Data collection is careful and deliberate
Analyzes and reflects on data
Disseminates results
Monitors use of results for improvement

26. Institutional engagement in institutional effectiveness
Necessary for
Institutional success
Accreditation
Continuous successful operation of the institution in its present structure

27. CORE REQUIREMENT 2.5
The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that
(1) incorporate a systematic review of institutional mission, goals, and outcomes;
(2) result in continuing improvement in institutional quality; and
(3) demonstrate the institution is effectively accomplishing its mission.

28. Types of Evidence to Consider
Outline, Chart, Calendar, or Description of Planning and Evaluation Processes
Evidence of integration of existing processes, e.g., budgeting, unit planning and assessment, performance evaluations, etc.
Strategic Plan including environmental scans, updates, metrics, annual reports, etc.
Assessments of the mission
Institutionally administered surveys and evidence of use
Minutes – e.g., Assessment Committee, Budget Committee, Executive Leadership, Board, etc.
Evidence of a culture of continuous improvement

29. Institutional Effectiveness (CS 3.3.1) and Outcomes
Two boys are walking down the street. The first boy says, “I’ve been really busy this summer. I’ve been teaching my dog to talk.”
His friend responds, “Wow! I can’t wait to have a conversation with your dog.”
The first boy shakes his head. “I said I’ve been teaching him. I didn’t say he learned anything.”
from Mary J. Allen (2004)
Assessing Academic Programs in Higher Education
Anker Publishing, Inc.: Bolton, MA.
At least two things to notice – being busy and the disconnect between teaching and learning
Being busy is not the same thing as being effective
Teaching is not the same thing as learning