1. How to Develop an Assessment Plan for an Academic Program Ann Boudinot-Amin Director of Assessment Institutional Research October 23, 2011

2. What is Assessment? PLAN: Develop clear learning outcomes DO: Offer courses, programs, or learning experiences CHECK: Assess achievement ACT: Use results

3. Why Do We Do It? Outstanding communications and management tool Required by MSCHE Required by discipline-specific accrediting bodies like AACSB, ABET, ACEJMC, CSAB, and others Provides evidence that AUC is actively monitoring its progress towards its goals and achieving its mission.

4. What does AUC Require? Develop and implement effective and ongoing outcomes assessment.  At the program level  At the course level  Assessment plan  Report on results of assessment on an annual basis (Nov. 1).  Use results to inform planning, budgeting, etc.

5. Effective Programs: Specify:  Learning outcomes for the core courses  Learning outcomes for the sequence core  Learning outcomes for specialities within sequences Build curriculum on essential competencies (program outcomes), not other way round.  Course learning outcomes should be based on these.

6. Effective Assessment Should: Define learning outcomes Result in a written assessment plan using multiple direct and indirect measures Maintain contact with alumni to assess their experiences and to gain feedback Include members of professions in assessment processes Collect and report data from assessment and use the data to improve curriculum and instruction. Be feasible, implementable, simple, meaningful, and cost-effective.

7. How Do We Do It? Develop a program assessment plan:  Mission statement   Key learning outcomes  KEY knowledge, skills, values, and attitudes that students gain from a learning experience. Clear and measurable Focus on ends, not the means  “Upon graduation, students will (action verb) (knowledge, skills, values)  Tied to university outcomes

8. Learning Outcomes Lesson Course Program Institutional How We Develop Learning Outcomes How Students Achieve Learning Outcomes Adapted from Huba and Freed (2000), p. 108.

9. University Learning Outcomes Professional Skills  AUC graduates will synthesize discipline-based knowledge with a broad-based liberal arts education. They will be proficient in the tools of their discipline as well as the tools of research and learning; make decisions that reflect the highest standards of ethical conduct and professional behavior; and understand the importance of life- long learning. Advanced Communication Skills  AUC graduates will be fluent in English and will be able to write and speak effectively in a variety of settings. AUC graduates will be able to communicate in Arabic, establish rapport in groups, be adaptable to new circumstances, work both independently and in collaboration with others, and function effectively as leaders.

10. University Learning Outcomes Critical Thinking  AUC graduates will be independent learners, adept at using current technologies to access information and applying strong quantitative, analytical, and critical thinking skills to analyze and synthesize complex information to solve problems. Cultural Competence  AUC graduates will have an understanding and appreciation of Egyptian and Arab culture and heritage, as well as an understanding of international interdependence, cultural diversity, and consideration for values and traditions that may differ from their own. In addition, AUC graduates will have an aesthetic awareness of the various modes of human artistic expression and will be able to collaborate effectively in a multicultural context.

11. University Learning Outcomes Effective Citizenship  AUC graduates value service to their local community and to broader causes at the national and international level.

12. Relation of Program Outcomes to University Outcomes 12345 understand the principles and laws of freedom of speech and press, including the right to dissent, to monitor and criticize power, and to assemble and petition for redress of grievances; demonstrate an understanding of the history and role of professionals and institutions in shaping communications; demonstrate an understanding of the diversity of groups in a global society in relation to communications; understand concepts and apply theories in the use and presentation of images and information; demonstrate a knowledge of professional ethics and apply these principles in pursuit of truth, accuracy, fairness and diversity; think critically, creatively and independently; conduct research and evaluate information by methods appropriate to the communications professions in which they work; write correctly and clearly in forms and styles appropriate for the communications professions, audiences and purposes they serve; critically evaluate their own work and that of others for accuracy, fairness, clarity, appropriate style and grammatical correctness; apply basic numerical and statistical concepts; apply tools and technologies appropriate for the communications professions in which they work.

13. Developing an Assessment Plan Direct Measures  Pre- and post-testing  Projects in capstone courses evaluated using rubrics  Portfolios and e-portfolios Portfolios and e-portfolios  Projects, essays, videos or presentations evaluated using rubrics  Embedded exam questions related to outcomes

14. Developing an Assessment Plan Indirect Measures  Grade distribution stats  Student retention and graduation  Probation and dismissal  Internships and placement  Student performance in contests  Student surveys, exit interviews, alumni surveys  Advisory Board interviews  Employer evaluations

15. Example of Senior Survey

16. Developing an Assessment Plan For each learning outcome:  Learning Opportunities  Assessment Methods  Targets/Benchmarks  Timeline  How Will Results Be Used?

17. Assessment Plan Program:Date: Assessment Coordinator:Ext.Email: Program Mission Statement: Program Outcomes Is this a student learning outcome ? (2-8 max) Learning Opportunities: What courses or experiences provide students with the (primary) opportunity to achieve this outcome? Assessment Methods/Measures: List the direct and indirect methods you will use to measure how well students are achieving this outcome. Target Levels/ Benchmarks When Will Assessment Be Conducted and Reviewed? How Will Results Be Used and Communicated? MARCH 1

18. Appendix 7: Sample Completed Annual Assessment Report – Program Level Adapted from UVa Civil Engineering Program Assessment Report, Assessment Guide: Seven Steps to Developing and Implementing a Student Learning Outcomes Assessment Plan. University of Virginia, Office of Institutional Assessment and Studies. 29 April 2007. Program: B.A., Sociology Date: Assessment Coordinator: Susan SmithExtEmail: Program Mission Statement: The mission of the undergraduate program in Sociology is to….. Program OutcomesSLO ? Learning Opp. Assessment Methods/Measures Target Levels/ BenchmarksWhen Will Assessment Be Conducted and Reviewed? How Will Results Be Used and Communicated? Students who complete this program should have the ability to: Communicate ideas in a clear and logical manner in oral presentation Y SOC xxx Capstone Final presentation in the capstone course, graded using scoring guidelines. At least 80% of final presentations in the capstone course are rated as good or better Each semester All assessment results will be reviewed by faculty during a special faculty meeting at the end of each semester. The results will be used to recommend changes to the curriculum, development of rubrics and other assessment tools, changes to syllabi, etc. Alumni survey At least 80% of respondents indicate that they are confident of their ability to express ideas in a clear, coherent manner in oral presentation Every two years, beginning 3/07. Apply sociological concepts and principles to individual experiences and the social world. Y SOC xxx Capstone Panel of faculty members evaluate graduating seniors' student portfolios developed throughout the program using a rubric. At least 80% of students' portfolios are rated as good or better in achieving this outcome. Each semester Alumni survey At least 80% of respondents indicate that they are confident of their ability to apply sociological concepts. Every two years, beginning 3/07. Explain the major foundations of modern sociological theory and show how each of these foundations influences current sociological theories. Y SOC xxx Capstone Final exam questions in capstone course, graded using a rubric. At least 80% of senior theses are rated as good or better Each semester Graduating seniors exit interview At least 80% of respondents indicate that they believe this learning outcome was achieved. Each semester

19. Assessment Report -- Program Level Program:Date: Assessment Coordinator:Ext.Email: Program Mission Statement: Program Outcomes this a student learning outcome ? (2-8 Learning Opportunities: What courses or experiences provide students with the (primary) opportunity to achieve this outcome? Assessment Methods/Measures: List the direct and indirect methods you will use to measure how well students are achieving this outcome. Target Levels/ Benchmarks Results/ Findings: Use of Results: How were results used to improve student learning, planning, decision-making, and/or resource allocation? November 1

20. Appoint an assessment coordinator  Curriculum matrix  Conduct assessment inventory Agree on program assessment plan Discuss implementation Assess at the course level  Develop course learning outcomes and plan Getting Started

21. Assessment Timeline Year 0: Getting started, submit plan Year 1-5: Implementation and annual reports Year 6: Program review

22. Institutional Support Office of Institutional Research (OIR) Center for Learning and Teaching (CLT) Teaching Enhancement Grants

23. What is a Portfolio? A folder of student work Students participate in choosing  Selection criteria provided by faculty  Ex.: one assignment that shows research skills, one writing skills, one ability to use a concept, etc. Evaluated using criteria Illustrate growth Continually updated, can substitute Include written student reflections on significance, contribution  Develop ability to synthesize what they’ve learned. Within a course or across program Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Bolton, MA: Anker Publishing.

24. What is a Portfolio? To make it a program requirement:  Tell students as they enter the program.  Give them written guidelines developed by program faculty.  Provide interim reviews – review with faculty advisor once each semester/year  Make it part of the capstone  Recognize and honor students with outstanding portfolios  Allow students to take it with them  Collectively review using guidelines.  E-portfolios E-portfolios

25. Capstones 400-level course, 3 credits Should integrate all learning in the major, plus draw on core Should prepare students for transition from undergraduate program to whatever will follow Should include strong component of reading/writing at senior level Should include a strong component of oral presentation at a senior level. Assessed using guidelines/criteria To establish capstone status, project proposal and syllabus prepared and submitted to Chair, Core Curriculum committee.

26. Internships Academic credit: for 3 cr. hrs., generally 150-200 hours of work. Paid? Rarely How? Student submits proposal, views approved list, conducts successful interview. Must be supervised Supervisor ratings using guidelines/rubrics If capstone, must include substantial written project AND oral presentation. Student should reflect on experience, through evaluation, journal, portfolio of work, daily log, etc. Examples

27. Action Items Approve program plan Develop course learning outcomes, faculty agree Submit syllabi with outcomes, develop assessment plan for each course. Send approved plan to Dean, cc OIR Committee for internship design? Committee for portfolio design? Committee for capstone design? Advisory board?