1. Academic Assessment 2009-2010 Looking forward and backward

2. Big goals for this year 1. Adopt a statement of General Education goals (in process) 2 Develop guidelines and rubrics for objectives of Gen Ed

3. Example of Gen Ed assessment http://www.dsu.edu/documents/asse ssment/academic-assessment/sge- goal4-plan-2008.docxhttp://www.dsu.edu/documents/asse ssment/academic-assessment/sge- goal4-plan-2008.docx

4. Examples of GE areas that will need goals, objectives, and outcomes Ethics in Action Communication Multiculturalism and Gender Global Awareness (Culture) CreativeExpression Religious Studies Problem Solving and Critical Thinking

5. Examples of good practices Goal 1: Demonstrate factual and theoretical knowledge of Chemistry. Objective 1.1 : Describe the structure and composition of matter. Outcomes: 1.1.1 On course examinations that deal with the structure and composition of matter, students will correctly answer 75 percent of questions. *1.1.2 On the ETS exam, Chemistry majors will score within 10 points or higher than the national average for the year in which they take the exam. *1.1.3 On the comprehensive exam questions that deal with factual and theoretical knowledge, chemistry majors will demonstrate depth and breadth in knowledge across the fields of chemistry and score at least…75 out of 100 points….

6. Results: Goal I, Objective 1.1: On the ETS exam in chemistry, 3 majors scored an average of 138.6 which is within 10 points of the National Average for the period of August 2006-June 2008 thus satisfying outcome 1.1.2. On the comprehensive exam, the average score was 87.5 on questions related to structure and composition of matter. This satisfies outcome 1.1.3.

7. GOAL 1: Senior BA studio art and design majors will demonstrate technical competence, knowledge, and creativity in the realization of discipline-based studio arts. OBJECTIVE 1: Senior BA studio art and design majors will create artworks demonstrating facility and knowledge of a range of materials and techniques. OUTCOME 1: Artworks will provide evidence of technical competence and craft aspect of art works. This outcome is satisfied by a score of 3 or above according to discipline rubrics. OBJECTIVE 2.1: Senior BA studio art and design majors will effectively and practically resolve problems of composition, design, color, subject matter, draftsmanship, materials, and processes in order to create arts works in two or three dimensions. OUTCOME 2.1: Artworks will provide evidence of and an understanding of the formal elements related to the creation of art works in two or three dimensions. This outcome is satisfied by a score of 3 or better according to discipline rubrics.

8. RESULTS Goal 1 (Objective 1). Senior BA Studio Art and Design majors averaged a score of 3.9 on discipline rubrics, thereby satisfying the requirement of this outcome. Senior BFA Studio Art and Design majors averaged a score of 4.13 on discipline rubrics, thereby satisfying the requirement of this outcome. Goal 1, (Objective 2.1). Senior BA Studio Art and Design majors averaged a score of 3.6 on discipline rubrics, thereby satisfying the requirement of this outcome. Senior BFA Studio Art and Design majors averaged a score of 4.05 on discipline rubrics, thereby satisfying the requirements of this outcome.

9. Examples of flawed conclusions Example 1 “Furthermore, there appears to be a disconnect between the rubric and the student’s grades for the assignment. Given these results, it appears we need to better educate our instructors about the use of rubrics so that the rubric average will not be restricted to such a narrow range for such a wide range of results (in terms of the assignment grade).”

10. What’s wrong with that? Conclusions based on one assignment Too much emphasis on the grade and not on the skills expected Conclusion calls for changes in dept. procedure, not in student learning

11. EXAMPLE 2 GOAL 1: X majors will be trained to write effective papers that apply aspects of X Objective 1.1: students will be able to demonstrate proficiency in research and writing in the field of X.. Outcome 1.1: On papers, X majors will score at least a 3 (out of a possible 5) on a standardized grading rubric in each of the following subcategories: a) Writing quality including grammar, organization and style. b) Content quality including thesis development, supporting evidence, and assessment of counter-arguments. Outcome 1.2: On the required Senior Thesis, all X majors will earn a grade of B or better, signifying a score of 4 out of 5 on the paper grading rubric described above. Result: Outcome 1.2 : The X majors each produced a senior thesis. During the course of the semester their work was critiqued and reworked. Each paper got an A.

12. Issues: Outcome 1.2 is flawed: Rubrics should be based on a variety of “tests”, not one score Conclusions are based on a grade. Papers were not subjected to the criteria mentioned as evidence of success No evidence a rubric was used, despite what is said.

13. What do we learn from all this? Assess student learning, not departmental policies or procedures (that will be part of a dept. review process) Collect all the data you can every year, but do yearly reports on only part of the plan Avoid basing conclusions on little data, repeat samples are best. Use rubrics—adopt standardized ones if available. If not, create. If the plan doesn’t give you useable results, change it!