Institutional Planning, Assessment & Research 2010 Institutional Planning, Assessment & Research Assessment Review Committee Report HCAS – Graduate Programs Heather Ries May 4, Institutional Planning, Assessment, and Research

Mentoring/Review Process Committee Selection & Composition –Chair –1 Assoc. Dean (Cindy Putnam Evans) –Faculty UAC’s representing each of the broad areas natural science & math social sciences, humanities (PhD program) –Susan Beck Frazier –Belinda Patterson Institutional Planning, Assessment & Research

Mentoring/Review Process Cont’d Institutional Planning, Assessment & Research Meetings - November 1 (Organizational meeting) - November 18 th (After 1 st review) - December 9 th (Last minute questions) Primary and Secondary Reviews - Faculty Members’ Suggestion - 21 programs and 13 certificates - Reviewers confer before entering final review

Component Data DevelopingAcceptableProficient Outcome Means of Assessment Criteria for Success Results Actions Taken Follow-Up to Actions Taken Institutional Planning, Assessment & Research

Data Visualization Institutional Planning, Assessment & Research

Best Practices – “Closing the Loop” Molecular Biology and BioTechnology Outcome: “Communication. Each student is expected to demonstrate effective oral and written communication.” Means of Assessment: “Evaluation of components of the student’s seminar on their thesis research, and the thesis itself according to a rubric. Components of the seminar evaluated are Speaking and Graphics; components of the thesis evaluated are Writing, Figures, and Tables.” Criteria for Success: “Each student’s average score for each component is at least 80% out of a possible 100.” Institutional Planning, Assessment & Research

Best Practices cont’d results: “Thesis research seminars of 9 students were evaluated. The cohort average for oral communication (Speaking; Seminar Rubric) was 90.7 out of 100. However, two individuals did not meet the minimum standard for oral communication. Scores of the remaining 7 ranged from 89.9 – 100. All students met the minimum for visual communication (Graphics). “Theses of 9 students were evaluated. One student did not meet the standard for writing quality. Scores of the remaining students ranged from 83.3 – The cohort average for writing quality was 88.8 out of 100. One student also did not meet the minimum for part of the visual communication (figures). All students met the minimum standard for communication via tables Institutional Planning, Assessment & Research

Best Practices Cont’d Actions Taken (based on analysis of results): “An acceptable performance of the great majority of students led the faculty to maintain the current pedagogy and curriculum, and the process of mentoring by the student’s faculty advisor and other members of the thesis committee, to provide constructive feedback on students’ learning and research accomplishments.” (YIKES – Not a best practice!!!) Institutional Planning, Assessment & Research

Best Practices Cont’d Initiative/Change/Strategy Implemented for Program/Unit improvement: –New actions taken for all outcomes, including the one above. –New actions taken for communication outcome: Past two years – 10-11% have substandard speaking skills with an additional 20% of the students slightly above the minimum standard. Early and more comprehensive assessment of speaking skills (during first semester as part of a required research course). Similar evaluation during proposal defense. Additional training in oral communication to those needing it. Developing course in communication and graphics. For writing, more rigorous review of proposals. Coursework assigned to address writing deficiencies as needed Institutional Planning, Assessment & Research

Substantive Changes Program/Unit: Geology MS Description of Changes: –Means of Assessment based on thesis –For used rubrics to evaluate thesis, thesis defense –In moved away from rubrics –faculty reluctance –Used more general means such as “student passed” thesis defense. –After review moving back to rubrics for Justification for Changes: –Realized that needed to evaluate for specific criteria so that student learning could be improved in specific areas Institutional Planning, Assessment & Research

Rubric and Review Process Feedback Feedback on the rubric and review process. –What worked for your ARC Multiple meetings Primary and secondary reviews Providing specific feedback (including suggested verbage) Faculty members increased “assessment awareness” – What difficulties were encountered? Rubric criteria open to interpretation Maintaining reviewer consistency Institutional Planning, Assessment & Research

Rubic and Review Process Feedback cont’d ARC’s process and progress towards addressing comments for “developing” and “acceptable” components received in the 2013 assessment review. –Follow up mostly from Associate Deans and not ARC –Bringing chairs back in Assessment orientation sessions for Chairs Chair review and sign-off –Follow up with recalcitrant departments and Phone –Meetings with individual UAC’s to address questions Institutional Planning, Assessment & Research

Preparations for Notice to all UAC’s –Importance of correct “actions taken” –Specific example Dean’s strategy meeting Call for ‘interim reports’ on Actions Taken - About 2/3 responded - Provided feedback via and meetings - Encouraged remaining 1/3 to attend workshops Series of workshops held throughout April –Handouts showing new format –Tracdat screenshots –Table with positive and negative examples Institutional Planning, Assessment & Research