1. Practicing Meaningful Learning Outcomes Assessment at UGA Department of Crop and Soil Sciences August 10, 2015 Dr. Leslie Gordon Associate Director for Assessment

2. FAQs about Assessment are programs are required to assess student learning? should all faculty be involved? Why.. is expected for assessment of graduate education? What.. do you focus your assessment efforts? Where..

3. Two Views of Assessment Ewell, P. (2009). Assessment, Accountability and Improvement: Revisiting the Tension. NILOA Occasional Paper #1 http://www.learningoutcomeassessment.org/documents/PeterEwell_005.pd

4. Why is LOA required of your program?

5. SACS-COC Core Standards

6. Why is LOA required of your program?

7. Undergraduate, graduate and professional degree and certificate programs

8. Why is LOA required of your program? UGA Academic Affairs Policy 4.12 http://provost.uga.edu/documents/Assessment_G uidelines_revised_20140624.pdf Faculty-driven Comprehensive (grad, undergrad, certificate) Faculty-reviewed  Evaluation as part of 7-yr Program Review

9. Why is LOA required of your program? UGA Academic Affairs Policy 4.12 http://provost.uga.edu/documents/Assessment_G uidelines_revised_20140624.pdf Faculty-driven Comprehensive (grad, undergrad, certificate) Faculty-reviewed  Evaluation as part of 7-yr Program Review

10. Why Is Faculty Involvement Important? Faculty are most qualified to influence the curriculum and its enactment make meaning of assessment evidence use evidence of student learning to demonstrate effective teaching ensure that assessment is integrated with, not a burden upon, faculty activity in teaching and scholarship

11. What is learning outcomes assessment? Assessment is the ongoing process of establishing clear, measurable expected outcomes of student learning ensuring that students have sufficient opportunities to achieve the outcomes systematically gathering, analyzing, and interpreting evidence using results to improve student learning (Suski, 2009, p. 4)

12. Define student learning outcomes Choose a measure and set criteria Do the assessment and analyze the evidence Enact improvements based on the evidence Report and plan for future assessment What is learning outcomes assessment? The Assessment Cycle

13. Best Practice Basics* Learning outcomes should be clear (action-focused), measurable, and student-focused Assessment measures should – clearly address the degree to which students attain the learning outcomes – include direct evidence – be independent from course grades and evaluations Faculty involvement should be clear and consistent *See June 11 th Presentation to Crop & Soil, which covered this in more detail

14. Best Practice for Graduate Assessment The same principles apply! Graduate learning outcomes focus on the knowledge, skill/ability, and perhaps disposition of students. The measures should reveal evidence of learning, not suggest that learning has taken place. Faculty involvement should be broad and consistent

15. What Measures to Use?

16. What is the Focus of Graduate Education Assessment? -Comprehensive knowledge -Research technique -Communication skill -Professional behaviors -Retention & graduation -Exam pass rates -Thesis / dissertation completion -Course / program satisfaction

17. What Measures to Use? When deciding what measures to use, first answer the question “Where will we see evidence that the student is demonstrating x,y,z learning outcomes?”

18. Where to Find Evidence of Learning? Some major approaches to measurement selection: Direct and Indirect Formative and Summative Embedded and Add-on

19. Where to Find Evidence of Learning? Direct and Indirect Measures Direct measures yield tangible, visible, self-explanatory evidence of learning. Indirect measures yield signs that students are probably learning.

20. Where to Find Evidence of Learning? Formative and Summative Measures Formative measures focus on “in process” learning that can be used to inform changes (to teaching, to student behaviors, etc.). Summative measures focus on the “end state” of learning, that students attain after the course or program.

21. Where to Find Evidence of Learning? Embedded and Add-on Measures Embedded assessments are those that are embedded into course work but that also provide information for program or institutional goals. Add-on assessments collect information about student learning independently from course/program assignments.

22. Your turn! Identify the approach(es) for each of the measures below. A. Direct B. Summative C. Embedded 1.Exam questions in research methods course 2.Discussion with advisor/major prof. about literature review 3.Dissertation defense 4.Alumni survey

23. Where to Find Evidence of Learning? Student Learning Assessment Engagement, or Service Scholarship Retention Job placement Grade distributions Teaching skills

24. Where to Find Evidence of Learning? Student Learning Assessment Engagement, or Service Scholarship Retention Job placement Grade distributions Teaching skills Indicators of success: Related, but not equal to student learning

25. Recommendations Regarding Scope and Manageability Focus on the essential learning outcomes of the program Assess what is directly under faculty control (monitor the rest) Adjust assessment as dictated by results or disciplinary demands

26. What is required for LOA reporting? DocumentsSubmitted by Assessment Plan All degree and certificate programs Brief Report on Learning Outcomes Assessment Undergraduate degree and certificate programs only Full report on Learning Outcomes Assessment Undergraduate and graduate degree and certificate programs See also “Program Review Outline for Evaluation of LOA”

27. What is required for LOA reporting? The Assessment Plan Each academic unit should establish and regularly maintain an assessment plan. The assessment plan documents a number of core elements that are defined by the faculty who teach in the program.

28. What is required for LOA reporting? The Assessment Plan Defines the learning outcomes for undergraduate students in your program (usually 4-6). Describes the methods the department will use to collect data that measure the attainment of the learning outcomes (usually 4-6). Describes the methods the department will use to analyze data that reflect the attainment of the learning outcomes. Describes how faculty will be involved in the various steps of the assessment process. Describes how the department will use information gathered from the analysis to improve its assessment practice and, more importantly, student learning.

29. What is required for LOA reporting? Examples of assessment plans: http://oap.uga.edu/assessment/plan/

30. What is required for LOA reporting? The Full Report includes information on ALL of the program's learning outcomes faculty analysis of the data improvements enacted or planned as a result of that analysis. The full report is submitted in the third and sixth years. – Graduate programs are asked to submit only full reports. – Undergraduate programs will also submit the full report (years 3 and 6) in addition to the brief reports (years 1,2,4,5).

31. What is required for LOA reporting? Examples of assessment reports: http://oap.uga.edu/assessment/plan/

32. What is required for LOA reporting? Upload plans and reports to the Academic Planning System (APS) https://aps.uga.edu/APS/ https://aps.uga.edu/APS/ This system houses Learning Outcomes Assessment plans and reports Program Review self studies Strategic plans for your unit/department, Annual Reporting documents for schools and colleges. Access to APS is granted by request. Contact Leslie Gordon, Associate Director for Assessment, at gordonls@uga.edu if you think you need access. gordonls@uga.edu

33. When do you report? October 1 st Annual reporting deadline for all programs, any type of report. Spring reminder issued to deans, academic associate deans, department heads, assessment coordinators

34. When do you report?

35. Assessment cycle Items dueSubmitted by Years 1-2 Brief Report on Learning Outcomes Assessment Undergraduate degree programs only Year 3 Full report on Learning Outcomes Assessment Undergraduate and graduate degree programs Years 4-6Repeat cycle described above Year 7 Academic Program Review Assessment reports are concurrently evaluated by a faculty subcommittee of the Program Review and Assessment Committee (PRAC) – evaluation and recommendations produced Undergraduate and graduate degree programs

36. Who reviews LOA practices? Subcommittee of Program Review and Assessment Committee (PRAC) 10-12 members Work is focused exclusively on LOA Follows evaluation criteria drawn from commonly accepted LOA best practices Issues report and recommendations See also “Sample LOA Evaluation Report”

40. Where can you find help? Maki, P. and Borkowski, N. (2006) Banta, T. Jones, E. and Black, K. (2009)

41. Where can you find help? Dr. Leslie Gordon Associate Director for Assessment Assessment (re)design consultation Informal evaluation of LOA Department/College presentations Interactive workshops Classes and Faculty Learning Communities And more! Begin your assessment redesign now!

42. Where can you go for help?

43. Where can you find help? OAP website > Student Learning Outcomes Assessment http://www.oap.uga.edu/assessment/sloa/

44. Questions?