Program Assessment Processes for Developing and Strengthening Jo Lynn Autry Digranes Coordinator for Assessment

Presentation Outline Definition of Assessment HLC Statement on Student Learning, Assessment, and Accreditation – Fundamental Questions Student Learning Outcomes Evidence of Learning Analysis and Use Shared Responsibility Evaluation and Planning Informing Stakeholders Process for Developing an Assessment Plan for an Academic Program – Process model from: Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

Assessment of Student Learning The systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning. Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

HLC Statement on Student Learning, Assessment, and Accreditation Fundamental Questions for Conversations on Student Learning Six fundamental questions serve as prompts for conversations about student learning and the role of assessment in affirming and improving that learning: How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students? What evidence do you have that students achieve your stated learning outcomes? In what ways do you analyze and use evidence of student learning? How do you ensure shared responsibility for student learning and for assessment of student learning? How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning? In what ways do you inform the public and other stakeholders about what students are learning---and how well? Higher Learning Commission. (2007) Statement on Student Learning, Assessment and Accreditation. HLC Website: http://ncahlc.org/Information-for-Institutions/publications.html

HLC Fundamental Questions How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students? OCU Mission Emphasis Scholarship Service Culturally rich community Moral and spiritual development Rigorous curriculum Effective leaders

HLC Fundamental Questions What evidence do you have that students achieve your stated learning outcomes? Stated learning outcomes should be measureable. Data can come from both direct and indirect measures, but always incorporate direct assessment.

HLC Fundamental Questions In what ways do you analyze and use evidence of student learning? Use multiple measures of direct and indirect assessment. Do you utilize evidence for reflecting upon program outcomes? Do you utilize evidence for indication of student learning? Do you utilize evidence for planning and change?

HLC Fundamental Questions How do you ensure shared responsibility for student learning and for assessment of student learning? How many faculty members are involved? How many courses are assessed? How many students are assessed? How often are learning outcomes assessed? Are external stakeholders involved in assessment, such as in service learning or internships?

HLC Fundamental Questions How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning? What is the plan for improvement, if needed? How does the plan for improvement link to strategic planning or budget requests? How do you know that last year’s plans worked?

HLC Fundamental Questions In what ways do you inform the public and other stakeholders about what students are learning---and how well? How are students informed of assessment results? How are internal stakeholders informed of assessment results? How are external stakeholders informed of assessment results?

The Process Analyze task, audience, and purposes. Envision the departmental assessment report or plan. Plan carefully for departmental collaboration in assessment. Establish responsibility for assessment. Articulate departmental learning goals. Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

The Process Conduct an assessment audit. Strengthen the department’s assessment processes. Recommend actions to improve student learning. Construct records and reports. Establish ongoing oversight for assessment. Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

Analyze Task, Audience, and Purposes Reviewing the department’s current assessment practices and recommending changes Reviewing the assessment data about student learning and recommending changes in curriculum, pedagogy, and other aspects to improve learning Audience – Department, students, institution, accreditation agency Purposes – Improvement, reporting, preparing for an accreditation visit

Envision the Departmental Assessment Report or Plan Envision, early in the process, what the final plan and report will need. Information such as: Learning Objectives Number of courses/students assessed Types of assessment measures utilized Results Recommendations from analysis of results

Plan Carefully for Departmental Collaboration in Assessment Assessment should assist the department to be more clear about aims and more effective and cost efficient in achieving them. The first step may be to form a group for discussion and brainstorming. Focus questions may include: What exactly is being asked in regard to assessment? What can the department and the members gain from the experience?

Establish Responsibility for Assessment The chair of the department may chair the effort, appointing an assistant chair, or appoint a chair and/or form a committee. A committee must include full-time faculty, but can also include adjuncts, non-tenure faculty, and students.

Articulate Departmental Learning Goals Identify if there are distinct student populations with somewhat different learning goals. Identify existing goal statements. Work from the mission and goal statements of the university and college/school. Indicate the explicit link. Goals need to be specific to imply student performance and criteria for evaluation.

Examples of Goal Levels Institutional: Students will communicate effectively orally and in writing. General Education Curriculum: Students will write essays in which they select and defend a position on a debatable issue, analyze a text, propose research, or define a problem and suggest solutions. Composition Course: Students will write a 5 to 7 page argumentative essay in which they select and defend a position on a debatable issue, support their position with evidence from their readings, and address counterarguments. Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

Examples of Goal Levels History Course: “Students will be able to write historical arguments in which they define a debatable issue in the field, take a position, defend the position with appropriate historical evidence, and address counterarguments.” This type of statement implies a senior history course student could write a historical argument and faculty could utilize criteria implied in the statement to evaluate the performance.

Conduct an Assessment Audit Identify where in classes the departmental goals are being taught and assessed. A matrix could assist with this task, identifying goals, classes that teach the goals, types of assessment measures utilized, frequency of teaching and assessing the goals, results of the assessment, and changes that have been made due to assessment.

Conduct an Assessment Audit Identify assessment beyond the individual classroom, both direct and indirect. These may include: National or state exam Review of senior projects by external evaluators Retention and graduation statistics Student evaluations Alumni surveys Placement Program review

Conduct an Assessment Audit Identify how data are used. Is all available data used for analysis and recommendations? Putting it all together Assessment in individual classes Other assessment measures How data are used

Strengthen the Department’s Assessment Processes Are learning goals well stated? Are learning goals being taught in a sensible sequence? (Bloom’s Taxonomy) What about goal disparity by course section? Can you use student evaluations? (IDEA) Can you build on the grading process?

Strengthen the Department’s Assessment Processes Are you using institutional data effectively? http://starnet.okcu.edu/Pages/Default.aspx What are your structures and processes for feedback? How to link data and action? (Link to strategic planning) How do resources support change? (Link to budget) How is the effectiveness of change assessed?

Direct and Indirect Assessment Direct assessment involves an analysis of products or behaviors that demonstrate the extent of students’ mastery of learning outcomes. Indirect assessment involves people’s opinions, and these opinions can richly supplement what is learned in direct assessment studies. Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

Direct Assessment Examples Standardized tests Locally developed tests Embedded assignments and activities Portfolios Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

More Direct Assessment Examples Final Projects – such as senior thesis, undergraduate research project, senior art show or music recital Capstone Experiences – such as student teaching, internship, cooperative educational experience Middaugh, M.F. (2010). Planning and assessment in higher education. San Francisco, CA: Jossey-Bass A Wiley Imprint.

Indirect Assessment Examples Surveys Interviews Focus Groups Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.

Recommend Actions to Improve Student Learning When providing recommendations, ensure that they will be “heard, owned, and acted upon.” If there are many areas that could be recommended, just focus on one or two areas.

Construct Records and Reports There is a current report form that has been slightly modified. There will be additional work on the form. Identify how records are kept. Ensure that departmental learning goals and assessment procedures are appropriately visible – such as on website or in brochures.

Establish Ongoing Oversight for Assessment Identify who is responsible for ongoing oversight -- may keep the committee going. Assessment of student learning should be embedded into the infrastructure of the department and the institution. A “way of life” or “doing business”

Questions? Thank You!