1. Program Assessment Processes for Developing and Strengthening
Jo Lynn Autry Digranes
Coordinator for Assessment

2. Presentation Outline Definition of Assessment
HLC Statement on Student Learning, Assessment, and Accreditation – Fundamental Questions
Student Learning Outcomes
Evidence of Learning
Analysis and Use
Shared Responsibility
Evaluation and Planning
Informing Stakeholders
Process for Developing an Assessment Plan for an Academic Program – Process model from: Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

3. Assessment of Student Learning
The systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning.
Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

4. HLC Statement on Student Learning, Assessment, and Accreditation
Fundamental Questions for Conversations on Student Learning
Six fundamental questions serve as prompts for conversations about student learning and the role of assessment in affirming and improving that learning:
How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?
What evidence do you have that students achieve your stated learning outcomes?
In what ways do you analyze and use evidence of student learning?
How do you ensure shared responsibility for student learning and for assessment of student learning?
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
In what ways do you inform the public and other stakeholders about what students are learning---and how well?
Higher Learning Commission. (2007) Statement on Student Learning, Assessment and Accreditation. HLC Website:

5. HLC Fundamental Questions
How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?
OCU Mission Emphasis
Scholarship
Service
Culturally rich community
Moral and spiritual development
Rigorous curriculum
Effective leaders

6. HLC Fundamental Questions
What evidence do you have that students achieve your stated learning outcomes?
Stated learning outcomes should be measureable.
Data can come from both direct and indirect measures, but always incorporate direct assessment.

7. HLC Fundamental Questions
In what ways do you analyze and use evidence of student learning?
Use multiple measures of direct and indirect assessment.
Do you utilize evidence for reflecting upon program outcomes?
Do you utilize evidence for indication of student learning?
Do you utilize evidence for planning and change?

8. HLC Fundamental Questions
How do you ensure shared responsibility for student learning and for assessment of student learning?
How many faculty members are involved?
How many courses are assessed?
How many students are assessed?
How often are learning outcomes assessed?
Are external stakeholders involved in assessment, such as in service learning or internships?

9. HLC Fundamental Questions
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
What is the plan for improvement, if needed?
How does the plan for improvement link to strategic planning or budget requests?
How do you know that last year’s plans worked?

10. HLC Fundamental Questions
In what ways do you inform the public and other stakeholders about what students are learning---and how well?
How are students informed of assessment results?
How are internal stakeholders informed of assessment results?
How are external stakeholders informed of assessment results?

11. The Process Analyze task, audience, and purposes.
Envision the departmental assessment report or plan.
Plan carefully for departmental collaboration in assessment.
Establish responsibility for assessment.
Articulate departmental learning goals.
Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

12. The Process Conduct an assessment audit.
Strengthen the department’s assessment processes.
Recommend actions to improve student learning.
Construct records and reports.
Establish ongoing oversight for assessment.
Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

13. Analyze Task, Audience, and Purposes
Reviewing the department’s current assessment practices and recommending changes
Reviewing the assessment data about student learning and recommending changes in curriculum, pedagogy, and other aspects to improve learning
Audience – Department, students, institution, accreditation agency
Purposes – Improvement, reporting, preparing for an accreditation visit

14. Envision the Departmental Assessment Report or Plan
Envision, early in the process, what the final plan and report will need.
Information such as:
Learning Objectives
Number of courses/students assessed
Types of assessment measures utilized
Results
Recommendations from analysis of results

15. Plan Carefully for Departmental Collaboration in Assessment
Assessment should assist the department to be more clear about aims and more effective and cost efficient in achieving them.
The first step may be to form a group for discussion and brainstorming. Focus questions may include:
What exactly is being asked in regard to assessment?
What can the department and the members gain from the experience?

16. Establish Responsibility for Assessment
The chair of the department may chair the effort, appointing an assistant chair, or appoint a chair and/or form a committee.
A committee must include full-time faculty, but can also include adjuncts, non-tenure faculty, and students.

17. Articulate Departmental Learning Goals
Identify if there are distinct student populations with somewhat different learning goals.
Identify existing goal statements.
Work from the mission and goal statements of the university and college/school. Indicate the explicit link.
Goals need to be specific to imply student performance and criteria for evaluation.

18. Examples of Goal Levels
Institutional: Students will communicate effectively orally and in writing.
General Education Curriculum: Students will write essays in which they select and defend a position on a debatable issue, analyze a text, propose research, or define a problem and suggest solutions.
Composition Course: Students will write a 5 to 7 page argumentative essay in which they select and defend a position on a debatable issue, support their position with evidence from their readings, and address counterarguments.
Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

19. Examples of Goal Levels
History Course: “Students will be able to write historical arguments in which they define a debatable issue in the field, take a position, defend the position with appropriate historical evidence, and address counterarguments.”
This type of statement implies a senior history course student could write a historical argument and faculty could utilize criteria implied in the statement to evaluate the performance.

20. Conduct an Assessment Audit
Identify where in classes the departmental goals are being taught and assessed.
A matrix could assist with this task, identifying goals, classes that teach the goals, types of assessment measures utilized, frequency of teaching and assessing the goals, results of the assessment, and changes that have been made due to assessment.

21. Conduct an Assessment Audit
Identify assessment beyond the individual classroom, both direct and indirect. These may include:
National or state exam
Review of senior projects by external evaluators
Retention and graduation statistics
Student evaluations
Alumni surveys
Placement
Program review

22. Conduct an Assessment Audit
Identify how data are used.
Is all available data used for analysis and recommendations?
Putting it all together
Assessment in individual classes
Other assessment measures
How data are used

23. Strengthen the Department’s Assessment Processes
Are learning goals well stated?
Are learning goals being taught in a sensible sequence? (Bloom’s Taxonomy)
What about goal disparity by course section?
Can you use student evaluations? (IDEA)
Can you build on the grading process?

24. Strengthen the Department’s Assessment Processes
Are you using institutional data effectively?
What are your structures and processes for feedback?
How to link data and action? (Link to strategic planning)
How do resources support change? (Link to budget)
How is the effectiveness of change assessed?

25. Direct and Indirect Assessment
Direct assessment involves an analysis of products or behaviors that demonstrate the extent of students’ mastery of learning outcomes.
Indirect assessment involves people’s opinions, and these opinions can richly supplement what is learned in direct assessment studies.
Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

26. Direct Assessment Examples
Standardized tests
Locally developed tests
Embedded assignments and activities
Portfolios
Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

27. More Direct Assessment Examples
Final Projects – such as senior thesis, undergraduate research project, senior art show or music recital
Capstone Experiences – such as student teaching, internship, cooperative educational experience
Middaugh, M.F. (2010). Planning and assessment in higher education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

28. Indirect Assessment Examples
Surveys
Interviews
Focus Groups
Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

29. Recommend Actions to Improve Student Learning
When providing recommendations, ensure that they will be “heard, owned, and acted upon.”
If there are many areas that could be recommended, just focus on one or two areas.

30. Construct Records and Reports
There is a current report form that has been slightly modified. There will be additional work on the form.
Identify how records are kept.
Ensure that departmental learning goals and assessment procedures are appropriately visible – such as on website or in brochures.

31. Establish Ongoing Oversight for Assessment
Identify who is responsible for ongoing oversight -- may keep the committee going.
Assessment of student learning should be embedded into the infrastructure of the department and the institution.
A “way of life” or “doing business”

32. Questions?
Thank You!