1. Writing Measurable Student Learning Outcomes
Dr. Timothy S. Brophy
Director of Institutional Assessment
University of Florida

2. Today’s Goals
Describe and explain SACS accreditation expectations for academic program assessment
Identify and apply steps for developing measurable student learning outcomes
Develop and/or refine student learning outcomes for your degree program

3. What is SACS and Why is it Important to be Accredited?
SACS-COC = the Southern Association of Colleges and Schools’ Commission on Colleges
SACS is the Federally-approved accrediting body for southern region of the US
SACS develops policies and standards that operationalize Federal Regulations
Federal Student Aid is tied to our reaffirmation – without accreditation we lose this important funding source

4. Three Most Recently Available Years
Selected Student Financial Aid Data for University of Florida Undergraduate Students for the
Three Most Recently Available Years
Academic Year
All undergraduate students
Pell Grants
$ ,894,352
$ ,996,944
$ ,733,613
Federal Loans
$ ,969,318
$ ,795,534
$ ,470,923
Full-time, first-time degree/certificate-seeking undergraduate students
Federal Grants
$ ,512,300
$ ,400,395
$ ,826,734
$ ,471,084
$ ,097,580
$ ,014,885
Other Federal Grants
$ ,041,216
$ ,302,815
$ ,811,849
$ ,651,786
$ ,457,421
$ ,362,376
Source: IPEDS Student Financial Aid Component
$106,204,536

5. Historical Context THE TRANSITION: NOW: THEN: 2003 Self Study
Volume 1:
Six required sections
Volume 2:
International Focus report
THE TRANSITION:
2003-present
Evolving Standards and Expectations
Paradigm shift
NOW:
Data–driven Continuous Improvement
Compliance
Quality Enhancement
Student Learning
2003: basically a series of tables – standard in left column, brief narrative in center column, links to data in the right column
2013: Online, webbased report with narratives and links to extensive data to show long term improvement efforts
Introduction
Letter from President Charles Young
Preface
Table of Contents
Members of Self-Study Committees
Organizational Charts
Section 1: Principles and Philosophy of Accreditation
Institutional Commitment and Responsibilities in the Accreditation Process
Application of the Criteria
Separately Accredited Units
Conditions of Eligibility
Initial Membership
Representation of Status
Section 2: Institutional Purpose
Institutional Purpose
Section 3: Institutional Effectiveness
Planning and Evaluation: Educational Programs
Planning and Evaluation: Administrative and Educational Support Services
Institutional Research
Section 4: Educational Program
General Requirements of the Educational Program
Undergraduate Program
Graduate Program
Publications
Distance Learning Programs
Continuing Education, Outreach and Service Programs
Student Records
Faculty
Consortial Relationships and Contractual Relationships
Section 5: Educational Support Services
Library and Other Learning Resources/Purpose and Scope
Instructional Support
Information Technology Resources and Systems
Student Development Services
Intercollegiate Athletics
Section 6: Administrative Processes
Organization and Adminstration
Institutional Advancement
Financial Resources
Physical Resources
Externally Funded Grants and Contracts
Related Corporate Entities

6. Scope of the SACS Principles
12 Core Requirements
35 subcomponents
14 Comprehensive Standards
73 subcomponents
9 Federal Requirements

7. SACS Standard
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional effectiveness)
educational programs, to include student learning outcomes

8. What SACS Expects
Completion of at least one complete SLO assessment cycle
Documented evidence that the “improvement cycle” is complete
Evidence and documentation of compliance with all requirements and standards

9. The UF Assessment Process
Establish Goals and Outcomes
Assessment Planning
Implement the Plan and Gather Data
Interpret and Evaluate the Data
Modify and Improve
UF MISSION
Institutional Effectiveness
Assessment

10. Academic Assessment Plan
Mission Alignment
Student Learning Outcomes
Curriculum Maps (UG)/ Assessment Timelines (Grad/Prof)
Assessment Cycle
Methods and Procedures
Assessment Oversight
Hand out the AAP and SLO guides

11. Student Learning Outcomes at UF
March 29, 2007 – Board of Governors regulation 8.016, “Academic Learning Compacts”
“Research indicates that university students are served best when students and faculty fully engage in a teaching-learning partnership, and this partnership is all the more meaningful if it is made as clear as possible to students what it is they will learn and how program faculty will assess that learning. Therefore, the Board has determined that universities must develop “Academic Learning Compacts” and related assessment processes to define and demonstrate student achievement in baccalaureate degree programs in the State University System.”
BOG Regulation , revised , “Student Learning Outcomes Assessment”

12. Identifying SLOs and Program Goals

13. Definitions
Student Learning Outcomes (SLOs)describe student learning – what students will know and be able to do as a result of completing a UF program
Program Goals do not describe student learning – instead, they describe programmatic elements, such as admission criteria, acceptance and graduation rates, etc

14. Outputs or outcomes?

15. definitions
Outputs describe and count what we do and whom we reach, and represent products or services we produce. Processes deliver outputs; what is produced at the end of a process is an output.
An outcome is a level of performance or achievement. It may be associated with a process or its output. Outcomes imply measurement - quantification - of performance.

16. Outcomes and outputs: What is the difference?
This distinction is important, especially in the development and review of Student Learning Outcomes.
We seek to measure outcomes as well as their associated outputs; however, SLOs focus on outcomes.
For example, while we produce a number of new graduates (the output), it is critical that we have a measure of the quality of the graduates as defined by the college or discipline (the outcome).
Effective Student Learning Outcomes describe, in measurable terms, these quality characteristics by defining our expectations for knowledge, critical thinking, and communication for UF undergraduates, and knowledge, skills, and professional behaviors for graduate and professional students,.

17. characteristics of effective student learning outcomes
Focus on what students will know and be able to do.
All disciplines have a body of core knowledge that students must learn to be successful as well as a core set of applications of that knowledge in professional settings.
Describe observable and measureable actions or behaviors.
Effective SLOs present a core set of observable, measureable behaviors. Measurement tools vary from quizzes and tests to complex rubrics.
The key to measurability: an active verb that describes a observable behavior, process, or product
A framework for developing SLOs: Bloom’s Taxonomy (see Table 3 in your Student Learning Outcomes handout)

18. Verbs and phrases that complicate measurability
Understand
An internal process that is indicated by demonstrated behaviors – OK for ALCs but not recommended for program or course SLOs
Appreciate; value
Internal processes that are indicated by demonstrated behaviors closely tied to personal choice
Become familiar with
Focuses assessment on “becoming familiar,” not familiarity
Learn about, think about
Not observable; demonstrable through communication or other demonstration of learning
Become aware of, gain an awareness of
Focuses assessment on becoming and/or gaining – not actual awareness
Demonstrate the ability to
Focuses assessment on ability, not achievement or demonstration of a skill

19. Direct or Indirect assessment?

20. Definitions
Direct assessments of student learning are those that provide for direct examination or observation of student knowledge or skills against measurable performance indicators.
Indirect assessments are those that ascertain the opinion or self-report of the extent or value of learning experiences

21. Developing Measurable SLOs: A Three-level Model (carriveau, 2010)
This model allows you to develop assessments that measure the outcomes, and that then connect directly to the program learning goals
Course-level Student Learning Outcome
these are determined by the faculty and specify course-level, observable products or demonstrations
Program-level Student Learning Outcome
these describe what students will do to demonstrate they have met the learning goals
Program Learning Goal Level – programs establish learning goals for the degree
these are described in the Academic Learning Compact, Program mission, or Catalog entry

22. Level 1: Establishing Learning Goals for the Degree
Learning Goals – these are found in the Academic Learning Compact in the description of the major
Example: Materials Science and Engineering
The major enables you to develop an understanding of materials systems and their role in engineering. Emphasis is placed on the ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering; to design and conduct experiments, as well as to analyze and interpret data; and to design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability.
Source: UF Undergraduate Catalog,

23. Level 1: Learning Goals based on the MSE ALC
Students who complete the MSE degree will:
Understand materials systems and their role in engineering
Apply knowledge of mathematics, science and engineering principles to materials science and engineering to design and conduct experiments, as well as to analyze and interpret data
Design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability

24. Level 2 – Program Student Learning Outcomes for MSE
Content Knowledge
Apply knowledge of mathematics, science and engineering principles to materials science and engineering.
Design and conduct materials science and engineering experiments and analyze and interpret the data.
Critical Thinking
Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints.
Communication
Communicate technical data and design information effectively in speech and in writing to other materials engineers.

25. MSE: Connecting Goals to Outcomes
ALC Learning Goals:
Understand materials systems and their role in engineering
Design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability
Student Learning Outcomes:
Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints.
Communicate technical data and design information effectively in speech and in writing to other materials engineers
Goal
SLO

26. MSE: Connecting Goals to Outcomes
ALC Learning Goals:
Understand materials systems and their role in engineering
Apply knowledge of mathematics, science and engineering principles to materials science and engineering to design and conduct experiments, as well as to analyze and interpret data
Student Learning Outcomes:
Apply knowledge of mathematics, science and engineering principles to materials science and engineering.
Design and conduct materials science and engineering experiments and analyze and interpret the data
Level 1
Level 2

27. Connecting Program SLOs to Courses MSE Curriculum Map
Additional Assess-ments
Content Knowledge
EMA3050
EMA3066
EMA4714
EMA3080C
EMA3513C #1 I R A
Senior exit survey #2
Critical Thinking
EMA4223 #3
Communi-cation
EMA3013C #4
Assessments in the boxes marked A are conducted using specific homework, exam, or assignment questions aligned with that SLO.
Source: MSE Academic Assessment Plan

28. Level 3 – Course level SLOs
These are determined by the faculty to teach the course
However, these should directly relate to the program SLOs

29. Writing student learning outcomes (use with the UF Student Learning Outcome Guide)
Review the Academic Learning Compact. List the learning goals for the program that are in the ALC.
Review the current SLOs for your area with your program faculty for recency, relevance, and rigor.
Examine the SLOs for the Knowledge Type (see Table 1) and Cognitive Processes level (see Table 2) they engage. The majority of the SLOs should be in the upper three levels of the Cognitive Processes Dimension – Analyze, Evaluate, and Create. The Taxonomy template in Figure 1 may help with this process.
Cross-reference your SLOs with the list of verbs/actions associated with their corresponding cognitive dimension levels (see Table 3), and replace any “verbs and phrases to avoid” with appropriate verbs from Table 3.
Write the SLO concisely and clearly.

30. Additional Resources
Carriveau, R. (2010). Connecting the dots – Developing student learning outcomes and outcomes-based assessments. Denton, TX: Fancy Fox Publications
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.) San Francisco, CA: Jossey-Bass.
Walvoord, B. (2010). Assessment clear and simple: A practical guide for institutions, departments, and general education (2nd ed.). San Francisco, CA: Jossey-Bass.
You can always find our UF-specific Institutional Assessment resources at our website,