1. The assessment process
Steve Culver
Director, Assessment and Analytics
Summer 2017
College & Department Workshop

2. WHAT IS ASSESSMENT? WHAT IS ASSESSMENT?
A systematic processes for determining whether or not a program is meeting its outcomes and then utilizing this information to make program decisions.
Grading is not necessarily assessment.
Grades are related to achievement in a class;
assessment demonstrates achievement related to outcomes
of the program. Grades need to be broken down
into learning outcome components to be used
for assessment purposes.

3. WHY DO ASSESSMENT? WHY ASSESSMENT?
Identify strengths and areas for improvement
Provide student learning evidence for stakeholders, including discipline-specific and regional accreditors
Highlight program contributions to stakeholders
Encourage collaboration among program faculty
Create a program vision and ideal
Encourage curriculum review
BUT the PRIMARY REASON is to improve the teaching and learning process

4. STEPS IN THE PROCESS: WHAT IS ASSESSMENT?
“establishing clear, measurable expected outcomes of student learning;
ensuring that students have sufficient opportunities to achieve those outcomes;
systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations; and
using the resulting information to understand and improve student learning.”
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). San Francisco: Jossey-Bass

5. WHAT IS ASSESSMENT?

6. WHAT ARE THE SACSCOC REQUIREMENTS?
The Southern Association of Colleges and Schools Commission on Colleges (SACSCOC) Comprehensive Standard:
“3.3.1 The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:
educational programs, to include student learning outcomes” (p.27).
Educational programs implies any credentialed program including certificates
Southern Association of Colleges and Schools Commission on Colleges (2011, December, p. 27). The principles of accreditation: Foundations for quality enhancement. Retrieved from

7. WHAT DOES SACSCOC CONSIDER WHEN EXAMINING THIS STANDARD?
“How are expected outcomes clearly defined in measurable terms for each educational program?
What assessment instruments are used and why were they selected?
Have the programs assessed the extent to which they have been successful in achieving their learning outcomes?
If called for, have program improvements been made as a result of assessment findings?”
Southern Association of Colleges and Schools Commission on Colleges (Second Edition 2012, p.55). Resource manual for the principles of accreditation: Foundations for quality enhancement. Retrieved from

8. STARTS WITH THE MISSION STATEMENT
A concise statement about the values and purpose of the program
The foundation a program uses to help define its program goals and outcomes
Consistent with institutional and college mission
Structure of a program mission statement:
“The mission of (name of your program or unit) is to (your primary purpose) by providing (your primary functions or activities) to (your stakeholders).” Additional clarifying statements.

9. STUDENT LEARNING OUTCOMES
PROGRAM & STUDENT LEARNING OUTCOMES
PROGRAM OUTCOMES
Reflects the services the program provides or student achievement in program areas (e.g., retention, graduation rates, employment statistics)
Example: Full-time students who declare XYZ as a major will complete the program in 4 years.
STUDENT LEARNING OUTCOMES
Reflects the knowledge, skills, abilities that students are expected to acquire as a result of being in the program.
Example: Graduates should be able to explain how past events in the area of XYZ impact society now.

10. A GOOD PROGRAM OUTCOME:
PROGRAM & STUDENT LEARNING OUTCOMES
A GOOD PROGRAM OUTCOME:
Defines program services
Is meaningful to the program
Focuses on what your programs wants to accomplish or a service it provides
Specifies program student achievement areas
Can be reliably and validly measured
A specific action or achievement related to program mission that can be improved upon
Examples:
Within two years of graduating from the program, students will obtain licensure.
The department of XYZ will increase its graduation rates.
The MA program will develop new courses to meet discipline-specific accreditation requirements.

11. PROGRAM & STUDENT LEARNING OUTCOMES
At A&T, the University’s goal is that all degree programs will achieve excellence and recognition for high quality teaching and learning, including the achievement of a satisfactory external review and/or accreditation:
At least one program outcome should be in relation to this goal.
Strategic Plan
A&T Preeminence
2020
Embracing Our Past,
Creating Our Future

12. A GOOD STUDENT LEARNING OUTCOME:
PROGRAM & STUDENT LEARNING OUTCOMES
A GOOD STUDENT LEARNING OUTCOME:
A specific behavior or knowledge students are expected to be able to demonstrate as a result of the program
Focuses on what you want students to know or be able to do / narrowly focused, measurable
Completes the statement: at the end of the program, students will be able to …..
Narrowly focused, measurable, not a “bundled” statement
Appropriate cognitive level for the degree (Bloom’s taxonomy)
Examples:
Students will be able to compare and contrast the style and form used in Monet’s earlier and later works.
Students will be able to explain the logic of the field-field approach to American anthropology.

13. PROGRAM & STUDENT LEARNING OUTCOMES
At A&T, the University has adopted four goals, so programs should identify at least one learning outcome for each goal.
Students completing this degree program will exhibit effective communication skills (written, oral, interpersonal) appropriate for professionals in this field of study.
Students completing this degree program will effectively use quantitative and qualitative analytical problem-solving skills appropriate for professionals in this field of study.
Students completing this degree program will demonstrate a level of discipline-specific expertise (knowledge, skills, and professionalism) appropriate for professionals in this field of study.
Students completing this degree program will demonstrate ability to engage productively in the review and conduct of disciplinary research appropriate for professionals in this field of study.

14. Programs should have a mix of student learning and program outcomes:
OUTCOMES—HOW MANY?
Programs should have a mix of student learning and program outcomes:
4 to 7 student learning outcomes
2 to 3 program outcomes
Programs with discipline-specific accreditation requirements (e.g., ABET, AACSB, etc.) might have more outcomes
Each program should measure at least 2-3 student learning outcomes and 1-2 program outcomes every year.
All outcomes should be measured at least twice in five years.

15. CURRICULUM MAPPING CURRICULUM MAPPING
Allows programs to align learning outcomes with courses
Exposes gaps in the curriculum
Assists assessment planning by showing where and when assessment can be (may already be) implemented
Improves communication among faculty
Improves program coherence
Increases likelihood that students will achieve outcomes

16. An example of what a curriculum map might look like
CURRICULUM MAPPING
An example of what a curriculum map might look like
Student Learning Outcome (SLO)
Required Introductory Courses (100 / 200 level)
Required Upper Level Courses (300 /400 level)
Concentration Specific Requirements
Additional Required Academic Experience
BIOL 1006
BIOL 2010
BIOL 3025
BIOL 4030
BIOL 4064
BIOL 3018
Internship (required)
SLO#1: Explain the core biological concepts related to evolution and principles of genetics. I
R, A R
SLO #2: Critically analyze biological research and findings.
Continue with the rest of your CORE learning outcomes
SLO Concentration Example: Explain the use of cells and biological materials in biochemistry.
Complete for each concentration outcome
“I” indicates where the concept is first being introduced to the students, “R” represents where the concept is being reinforced, and “A” indicates the assessment point.

17. WHAT TO CONSIDER WHEN CHOOSING A MEASURE
MEASURES
WHAT TO CONSIDER WHEN CHOOSING A MEASURE
compatible with the outcomes
provide reasonably accurate, useful information
yield results specific enough to show where improvements could be made
match the cognitive level of the outcome
Find out what activities faculty are already doing in their courses
Find out what is being done institution-wide that might map to your outcomes
Utilize one activity for several outcomes when possible
Use capstone points in the curriculum

18. BE CAREFUL WHAT YOU ASK FOR…..
MEASURES
BE CAREFUL WHAT YOU ASK FOR…..

19. DIRECT MEASURES – tangible, visible, observable
Student learning outcomes:
Student artifacts (e.g. capstone projects, portfolios, presentations, case studies, etc.) examined by learning outcome and scored with a rubric
Externally reviewed exhibitions, performances, or projects scored by learning outcome
External evaluation of performance during internships based on outcomes.
Commercially-developed tests, locally-developed tests, national licensure exams or professional exams if examined by learning outcome
Program outcomes:
Number of students meeting certain criteria (e.g., % attending graduate school) or number of projects or initiatives implemented (e.g., courses developed, grants obtained, etc.)
Enrollment numbers / number of majors

20. INDIRECT MEASURES – infer student knowledge, not directly observable
Survey research: National or local instruments
Exit surveys / senior surveys, alumni or employer surveys
National Survey of Student Engagement (NSSE), The College Senior Survey (CSS), etc.
Faculty surveys
Course evaluations
Employer satisfaction studies and advisory boards
Exit interviews and student focus groups
Self-assessments
Peer ratings

21. METHODOLOGY MAPS CAN HELP WITH PLANNING:
MEASURES
METHODOLOGY MAPS CAN HELP WITH PLANNING:
An example of what an assessment methodology map might look like
Student Learning Outcome (SLO)
Dept Direct Assessment Methods
Dept Indirect Assessment Methods
Capstone course projects
Written course assignments
Standardized subject matter test
Graduating senior survey
Dept exit interviews
Alumni survey
SLO#1: Explain the core biological concepts related to evolution and principles of genetics.
BIOL 4030 – collected by professor every time course is taught
Conducted by the department every year
Conducted by OSPIE every year
Conducted by members of the curriculum committee every spring

22. (Decided to assess to determine if action plan is working)
MEASURES
TIMELINE MAPS CAN HELP WITH PLANNING:
An example of what an assessment timeline map might look like
Student Learning Outcome (SLO)
SLO#1: Explain the core biological concepts related to evolution and principles of genetics.
BIOL capstone project
Standardized test
Standardize d test
SLO#2: Critically analyze biological research and findings.
(outcome not met)
(Decided to assess to determine if action plan is working)

23. TARGETS
WHAT IS A PERFORMANCE TARGET?
Defines the program’s achievement expectations for each outcome-method pair
Is usually expressed as percentages or numbers expected
Is achievable but rigorous
Is realistic for the program level
Examples:
Student learning outcome: 80% of students will meet or exceed expectations
on the rubric items for this outcome.
Program outcome: 80% of students who declare XYZ as a major
will complete the program in 4 years.

24. ACTION PLANS
WHAT MAKES A GOOD ACTION PLAN?
Action plans:
Describe the changes a program intends to implement to address curriculum or program deficiencies identified as a result of the assessment
Describe specific programmatic changes with timelines if possible
Should be discussed among the faculty to involve them in the improvement or planning process
Changes to curriculum, assessment plan or methods, pedagogical practices, implementation of new technology or assignments

25. CLOSING THE LOOP
SEVERAL WAYS TO “CLOSE THE LOOP”
#1: Assessment results show that program has achieved its outcome.
Data were collected, results show outcome achieved given the specific target; program plans to continue to measure outcome in future assessment cycles.
#2: Assessment results show that program has not achieved its outcome.
Data were collected, results show outcome not achieved given the specific target; program developed/implemented action plan to improve performance; program examined impact of action plan by re-measuring and collecting data on outcome in a future cycle.
#3: Program has determined that the assessment plan is ineffective and needs to be revised.
Data were collected; program determined that plan was not accurately measuring the outcome or that outcome needed revision; program documented changes to be made in the plan and re-measured in future cycle.

26. REPORTING

27. Taskstream as the repository for our work
Reports due:
Assessment plans for the year - August 25th
Assessment results report - June 29th
Two major areas in Taskstream:
Standing requirements
includes mission, program/student learning outcomes
Annual Assessment Plan
includes assessment results report and assessment plans for the year

28. Summary
Summary and Review
Mission statement
Student learning and program outcomes
Curriculum mapping
Measures and Targets
Action plans
Documentation and Reporting