1. 1 C-99 Assessing Student Learning in Graduate Degree Programs C-99 Assessing Student Learning in Graduate Degree Programs Bob Smallwood, University of Alabama Loraine Phillips, Texas A&M University SACS COC Annual Meeting Orlando, Florida December, 2011

2. 2 Session Outline C. Accrediting Agency Influences I. Graduate Program Assessment Requirements & Expectations D. Unique Graduate-level Assessment Challenges II. Typical Student Learning Outcomes at the Graduate Level A. at Texas A&M University III. Typical Graduate-Level Assessment Approaches IV. Assessment Resources for Graduate Program Assessment A. Books B. at University of Alabama B. Websites V. Examples of Graduate Level Assessments A. Program Outcomes and Measures B. Student Learning Outcomes and Measures

3. 3 I. Graduate Program Assessment Requirements & Expectations A. at Texas A&M University 1. External Review & Site Visit 2. Internal Review Academic Program Review (every 8 years)  Periodic, formal, evaluative, action-oriented  Focused on overall program “health” using internal and external data as evidence  Ongoing reporting requirements to ensure accountability  Outcomes-based assessment focused on learning  Continuous and ongoing process  Includes annual reporting

4. 4 I. Graduate Program Assessment Requirements & Expectations B. at University of Alabama 1. Requirements: a. At least Three Program Outcomes (common across all programs) 2. Expectations: 3. Reports: a. Annual Outcomes Assessment Plan (due Sep 10) b. Annual Outcomes Assessment Report ( due Jul 15) b. At least Two Student Learning Outcomes - the program will improve and sustain a high level of recognized quality - the program will build and sustain an optimal level of annual program enrollments and degree completions - the program will be highly valued by its program graduates and other key constituencies it serves - discipline-specific content knowledge - discipline-specific methodological skills a. At least Two Assessment Measures per Program Outcome b. At least Two Direct Assessment Measures per SLO c. Two year Assessment cycle for each SLO (Outcome-Measure- Result-Improvement)

5. 5 I. Graduate Program Assessment Requirements & Expectations C. Accrediting Agency Influences From 3.3.1.1  Develop graduate program outcomes Develop graduate student learning outcomes Assess achievement of outcomes Develop improvement initiatives Assess impact of improvements From 3.6.1  Masters and Doctoral student learning outcomes need to be progressively more advanced in academic content From 3.6.2  Student learning outcomes need to address discipline knowledge and research or professional practice skills From 3.4.12  Maybe student learning outcomes ought to include related technology skill development (technology literacy) From 4.2  Maybe student learning outcomes need to be informed by and aligned with professional organization standards, licensing bodies, or specialized accreditors

6. 6 I. Graduate Program Assessment Requirements & Expectations D. Unique Graduate-level Assessment Challenges 1. Small sample size compromises statistically sound inferences and conclusions 2. Curriculum often customized to meet unique needs and goals of the graduate students…..limited subject matter common core 3. Generalizations about the graduate student experience are limited due to the varied nature of the programs

7. 7 Session Outline C. Accrediting Agency Influences I. Graduate Program Assessment Requirements & Expectations 7 D. Unique Graduate-level Assessment Challenges II. Typical Student Learning Outcomes at the Graduate Level A. at Texas A&M University B. at University of Alabama

8. 8 II. Typical Student Learning Outcomes at the Graduate Level 1. Demonstrate knowledge in the field 2. Conduct research projects 3. Solve problems related to the field 4. Communicate effectively in written and oral forms 5. Use a variety of sources and evaluate multiple points of view to analyze and integrate information 6. Use appropriate technologies to communicate, collaborate, conduct research, and solve problems, and to conduct reasoned arguments

9. 9 Other Typical Graduate-level Student Learning Outcomes? 1.Content knowledge 2.Research skills 3.Problem-solving skills 4.Communication skills 5.Evaluate, analyze, integrate 6.Use technologies to 7.? 8.? 9.?

10. 10 Session Outline C. Accrediting Agency Influences I. Graduate Program Assessment Requirements & Expectations D. Unique Graduate-level Assessment Challenges II. Typical Student Learning Outcomes at the Graduate Level A. at Texas A&M University III. Typical Graduate-Level Assessment Approaches B. at University of Alabama

11. 11 III. Typical Graduate-Level Assessment Approaches 1. Comprehensive written exams 2. Dissertations, Theses, Scientific papers 3. Course-embedded Projects, Term Papers, or Case Studies 4. Course-embedded Exam Questions 5. ? 6. ?

12. 12 Other Typical Graduate-level Assessment Approaches? 1.Comprehensives 2.Dissertations, Thesis 3.Course Embedded Projects 4.Course Embedded Exams 5._ 6._ 7._ 8._ 9._

13. 13 Session Outline C. Accrediting Agency Influences I. Graduate Program Assessment Requirements & Expectations D. Unique Graduate-level Assessment Challenges II. Typical Student Learning Outcomes at the Graduate Level A. at Texas A&M University III. Typical Graduate-Level Assessment Approaches IV. Assessment Resources for Graduate Program Assessment A. Books B. at University of Alabama B. Websites

14. 14 IV. Assessment Resources for Graduate Program Assessment

15. 15 Books: a. Resources for Assessment in General b. Resources for Graduate Program Assessment IV. Assessment Resources for Graduate Program Assessment Websites: a. Department Assessment Manuals b. Internet Assessment Resources c. Selected Sites Related to Graduate Program Assessment

16. 16 Session Outline C. Accrediting Agency Influences I. Graduate Program Assessment Requirements & Expectations D. Unique Graduate-level Assessment Challenges II. Typical Student Learning Outcomes at the Graduate Level A. at Texas A&M University III. Typical Graduate-Level Assessment Approaches IV. Assessment Resources for Graduate Program Assessment A. Books B. at University of Alabama B. Websites V. Examples of Graduate Level Assessments A. Program Outcomes and Measures B. Student Learning Outcomes and Measures

17. 17 V. Examples of Graduate Level Assessments A. Program Outcomes and Measures 1. At A&M: Outcome: Students will progress through the program in a timely manner. Measures: The program will track enrollment, semester and graduation rate 2. At UA: Outcome: The Chemistry PhD program will build and sustain an optimal level of annual program enrollments and degree completions. Measures:  Number of students in the PhD program for the last three fall semesters;  Number of PhD degrees awarded last August + December + May commencements  Relation of number of PhD degrees awarded to ACHE viability standards

18. 18 V. Examples of Graduate Level Assessments B. Student Learning Outcomes and Measures 1. At A&M: Outcome: We expect graduate students to acquire the skills and intellectual habits of problem solvers. They should be able to identify a problem, select the appropriate problem-solving tools, assess the need for further information, gather information cost- and time-effectively, and frame solutions that meet professional and regulatory standards. Measures: The instructor for the FINC 672 or 673 course will include exam questions that assess them and use a rubric to score them

19. 19 V. Examples of Graduate Level Assessments B. Student Learning Outcomes and Measures 2. At UA: (PhD in Electrical Engineering) Outcome: Graduates will demonstrate expert skills and abilities in electrical and computer engineering including complex problem solving, analytical reasoning, advanced research and communication. Measures:  Results of direct, course embedded assessments (focused on advanced skills in a specialization area) will be an average of 3.0/4.0 or better for all (rubric) assessments performed in focus courses in embedded systems (ECE580, ECE 586), devices and materials (ECE 530, ECE 539) and electo-mechanical systems (ECE551, ECE579)  100% of graduates will have independently researched a topic and produced a dissertation that has been peer reviewed and accepted by a graduate advisory committee consisting of at least 5 graduate faculty members in the area of the research specialization

20. 20 Session Outline C. Accrediting Agency Influences I. Graduate Program Assessment Requirements & Expectations D. Unique Graduate-level Assessment Challenges II. Typical Student Learning Outcomes at the Graduate Level A. at Texas A&M University III. Typical Graduate-Level Assessment Approaches IV. Assessment Resources for Graduate Program Assessment A. Books B. at University of Alabama B. Websites V. Examples of Graduate Level Assessments A. Program Outcomes and Measures B. Student Learning Outcomes and Measures

21. 21 Questions or Comments

22. 22 Concurrent Session Evaluaton C-99 Assessing Student Learning in Graduate Degree Programs Roll TideGig ‘em Aggies Welcome to the SEC