1. The Missing Link: Development of Programmatic Outcomes Christopher Meseke,Ph.D. Park University

2. What is Assessment?  “As a whole, assessment is a framework for focusing faculty attention on student learning and for provoking meaningful discussions of program objectives, curricular organization, pedagogy, and student development” (Allen, 2004).  Assessment is the quality control of the educational process

3. Assessment is -- first and foremost -- about student learning.

4. Assessment Levels  Student  Classroom  Course  Program  College  Division  Institution

5. Common Reactions to Assessment Initiatives Ignoring it Bribing someone else to do it Complaining about it Losing sleep over it Sitting down and writing it

6. Levels of assessment: quality control and assurance  Institutional/Programmatic  National/State Licensure Exams  Certain academic programs (Nursing, Engineering, Physical Therapy, Social Work, etc)  Accreditation Council for Graduate Medical Education (ACGME)  Higher Learning Commission of the North Central Association of Colleges and Schools

7. Assessment is important to the accreditation process  The Higher Learning Commission of the North Central Association of Colleges and Schools (HLC-NCA) requires it.  Assessment of student learning provides evidence at multiple levels: course, program, and institutional.  Assessment of student learning includes multiple direct and indirect measures of student learning.  Assessment results inform improvements in curriculum, pedagogy, instructional resources, and student services.

8. Interpretation of the HLC-NCA Statement on Assessment on Student Learning, 2003.  It is the faculty members who must take ownership of the assessment process. Their buy-in, ownership, and implementation is directly related to both the mission of the institution and attitudes, knowledge, and skills required for the student to successfully complete the program requirements.

9. Goals of Program Assessment Planning  Measure student learning, not teaching, curricular content, processes or resources  Measure things that are important to us  Involve faculty in development of the assessment program, processes and instruments  Use multiple measures to produce valid and reliable data (triangulation)  Make it manageable and cost affordable

10. Goals of Program Assessment Planning  Map assessment data back to the curriculum for improvement  Produce annual reports of assessment outcomes that show the data, interpretation of data, and improvement plans where indicated  Same assessment procedures for all campuses and sister programs

11. Modified Hatfield Assessment Model Meta-Competencies/Institutional Goals objective Competency Learning Outcome objective Competency indicators Degree to which Competency is achieved Learning events

12. Programmatic Competencies  A measurable, complex behavioral statement may be written for each Key Component (meta-competency) and competency.  Statements reflect what we think we can actually measure in an educational environment considering time and resources.

13. Program Assessment Learning Competencies  Develop a set of Learning Goals (Meta- competencies) which represent the attributes of the graduate  General Education (Competencies)  Discipline (Competencies)  Course (Outcomes)

14. KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Identify Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower Order Bloom’s verbs

15. KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Identify Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Higher Order Bloom’s verbs/Upper division Course / Program outcomes

16. An Example of Departmental/Programmatic Competencies  Students will be able to:  Demonstrate biological knowledge appropriate for the course level.  Demonstrate a working knowledge of the scientific method.  Demonstrate the ability to communicate scientific concepts and findings, both in oral and written format.  Apply interdisciplinary knowledge to the biological sciences.  Demonstrate an awareness of ethical issues in the life sciences.

17. Mapping Competencies to Courses

18. Program Competencies Course 1 x x x Course 2 x x x x Course 3 x x x Course 4 x x x x x x x x Course 5 Mapping Outcomes to Courses

19. Program Competencies Course 1 x x x Course 2 x x x x Course 3 Course 4 x x x x x x x x Mapping Outcomes to Courses Course 5

20. Triangulation of Assessment Strategies: Direct Measures (Pick 2)  Licensure Scores  GRE/GMAT/MCAT/other  Departmental Exit Exams  Portfolio  Core Assessment  Capstone Experience  Others

21. Triangulation of Assessment Strategies: Indirect Measures (Pick 1)  Advisory Boards  Senior Survey  Employer/Professional School Survey  Focus Groups/Interviews  Other

22. Three-Year Assessment Plan  All departments/programs should be assessed on three-year cycles  Not all department/program competencies should/can be assessed  Focus on 2-3 competencies at most  Establish meaningful criteria  At end of three-year cycle examine data for meaningful trends  Only after the three-year cycle are changes made, only if needed

23. An example of a three-year assessment cycle Three-Year Schedule of Program Assessment Plans Submissions to the Assessment Committee 2013-20142014-20152015-2016 Program G/UG Accounting UG X Adult Education G X Athletic Training UG X Biology UG X Business Administration G X Business Core (CPC) UG X Chemistry UGX Communication Arts UG X Communication and Leadership G X Computer Information Systems UG X Criminal Justice UG X Economics UG X Early Childhood Education (Educational Studies) G/UG X Educational Leadership G X Elementary Education UG X Engineering UG X English UG X Finance UG X Fine Arts UG X Geography UG X Graphic Design UG X Healthcare Leadership GX Healthcare Management UG X History UG X Human Resource Management UGX Information and Computer Science UG X Interior Design UG X International Business UG X International Center for Music G/UG X Language and Literacy G X Legal Studies UG X Liberal Education UG X Liberal Studies UG X Logistics UGX Management UG X Management Information Systems UG X Marketing UG X Mathematics UG X Middle and Secondary Education UG X Modern Languages UG X Nursing UG X Political Science UG X Psychology UG X Public Administration (BPA) UG X Public Affairs (MPA) G X Social Psychology UGX Social Work UG X Sociology UG X Teacher Leadership G X

24. Summary  Choose an Assessment Model  Identify (develop) programmatic competencies  Write program level competencies based upon the Institutional meta-competencies.  Establish key performance indications (KPOs)based upon the competencies  Identify 3 measures for each competency (2 direct, 1 indirect) and set success criteria (may be different across various university units)  Develop data gathering and reporting mechanisms and templates  Develop close-the-loop mechanisms

25. Big Mistakes in Assessment Assuming that it will go away Trying to do too much, too soon Expecting to get it right the first time Not considering implementation issues when creating plans Borrowing plans and methods without acculturation

26. Big Mistakes in Assessment Demanding statistical research standards Doing it for accreditation instead of improvement Confusing assessment with student learning Making assessment the responsibility of one individual Assuming collecting data is doing assessment

27. Thank you  A special thanks to Dr. Susan Hatfield, for her gracious input.  Questions?