1. Presented by: Dr. Sue Courtney Janice Stoudemire, CPA, ATA, ABA Associate Degree Board of Commissioners Copyright Protected: Material can not be use or reproduced without permission from Janice Stoudemire Sunday 10:00 am-11:30 am Performance Results and Students Outcomes ACBSP

2. Session Goals 1. Performance results and student outcomes will be discussed. 2. Completing the assessment cycle. 3. Outcomes chart in the QA report. 4. Developing student outcomes.

3. Five Performance Areas for QA Report 1. Student learning 2. Student/stakeholder focused 3. Budgetary, finance, market performance 4. Faculty/staff focused 5. Organizational effectiveness

4. Six Items to Report on for Each Indicator in Each Area 1. Performance measure (define) 2. Description of measurement instrument 3. Areas of success 4. Analysis and action taken 5. Results of action taken (occurs in following year) 6. Graph of trends

5. Let’s start with Student Learning Results

6. Quick review from yesterday!

7. Two Types of Assessment Direct Assessment: Measures a specific competency attainment. Measures a specific competency attainment. Includes Student Learning Outcomes. Includes Student Learning Outcomes. Examples: Pre/post exams, case studies/projects, etc. Examples: Pre/post exams, case studies/projects, etc.

8. Two Types of Assessment Indirect Assessment: Defined as something that happens to a student as a result of one’s educational experience. Defined as something that happens to a student as a result of one’s educational experience. Includes Behavioral Outcomes. Includes Behavioral Outcomes. Measured by retention rates, surveys, etc. Measured by retention rates, surveys, etc.

9. An outcome is…. “what” you expect students to achieve. “what” you expect students to achieve. the knowledge and skills your students should master. the knowledge and skills your students should master. is the “end”, not the means! is the “end”, not the means!

10. A competency …. demonstrates "how" students can achieve an outcome. demonstrates "how" students can achieve an outcome. is the specific activity used to measure a student’s mastery of the outcome. is the specific activity used to measure a student’s mastery of the outcome. is a “means” to the end. is a “means” to the end.

11. Student Learning Results Measures a specific competency attainment. Use “Direct Assessment”

12. Student Learning Results Warning: Just “passing” a course is not a good indication of outcomes attainment (the course is NOT the competency!) Course-embedded assessment (tests, quizzes, specific activities). Course-embedded assessment (tests, quizzes, specific activities). Pre and post tests. Pre and post tests. Case studies or cumulative problem(s). Case studies or cumulative problem(s).

13. Student Learning Results Capstone performances*: (course, seminar, internship, project, or thesis). Capstone performances*: (course, seminar, internship, project, or thesis). Performance on MFT or licensure examinations. Performance on MFT or licensure examinations. Individual/group projects (may include simulations). Individual/group projects (may include simulations). Professional performance Professional performance

14. How do you report these? Student learning outcomes are integrated into the ACBSP standards. Therefore when you complete an initial accreditation or reaffirmation, you will be addressing these in various sections of the report.

15. How do you report these on QA? QA report contains a simple chart that captures the complete assessment process for each competency! Just fill in the six columns for each competency.

16. Analysis of Results Performance Measure (Competency) Description of Measurement Instrument Areas of Success Analysis and Action Taken Results of Action Taken (occurs in the following year) Attach Graph of Trends for 3-5 Years (Attach Trend Graph)

17. Performance Measure (Competency) Description of Measurement Instrument Measure overall attainment of course goals. What do you think of…….. X

18. Performance Measure (Competency) Description of Measurement Instrument Measure overall attainment of course goals. Pretest/post test What do you think of…….. X

19. Areas of Success Analysis and Action Taken Average improvement X

20. Areas of Success Analysis and Action Taken Average improvement Administered test; not pleased with results; will redo test X

21. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) 27% gain X

22. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) 27% gain No graph

23. Performance Measure (Competency) Description of Measurement Instrument Peachtree software proficiency What do you think of…….. X

24. Performance Measure (Competency) Description of Measurement Instrument Peachtree software proficiency Students will demonstrate proficiency in Peachtree applications What do you think of…….. X

25. Areas of Success Analysis and Action Taken Students have demonstrated proficiency X

26. Areas of Success Analysis and Action Taken Students have demonstrated proficiency Benchmark met X

27. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) Instructor will continue to emphasis Peachtree X

28. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) Instructor will continue to emphasis Peachtree ?

29. Performance Measure (Competency) Description of Measurement Instrument Accounting students will demonstrate the understanding of the components and inter- relationships of financial statements.

30. Performance Measure (Competency) Description of Measurement Instrument Accounting students will demonstrate the understanding of the components and inter- relationships of financial statements. Success is based on a grade of 70 or better on an accounting case completed by each student enrolled in ACC 201. The goal is 80% of the students will meet this standard.

31. Areas of Success Analysis and Action Taken For the past three semesters, the standard was met. Overall, 80% of the 61 enrolled students in ACC 201 courses met the standard.

32. Areas of Success Analysis and Action Taken For the past three semesters, the standard was met. Overall, 80% of the 61 enrolled students in ACC 201 courses met the standard. Although overall the standard was met, on a semester- by- semester basis, Fall 2005 resulted in only a 50% success rate. This was mainly due to the fact that several students did not fully complete the case and therefore skewed the results.

33. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) The department chair is investigating why the completion rate was so low in the fall. He will meet with the instructors to determine what can be done in the future to ensure students complete the assignment. The department will continue to collect the data and track the results.

34. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) The department chair is investigating why the completion rate was so low in the fall. He will meet with the instructors to determine what can be done in the future to ensure students complete the assignment. The department will continue to collect the data and track the results.

35. Performance Measure (Competency) Description of Measurement Instrument Accounting students will demonstrate knowledge of how management uses information from the accounting system to aid in operating the business enterprise.

36. Performance Measure (Competency) Description of Measurement Instrument Accounting students will demonstrate knowledge of how management uses information from the accounting system to aid in operating the business enterprise. Success is based on the successful completion of a comprehensive management accounting problem set by students enrolled in ACC 115. The goal is 80% of the students will successfully complete the problem set with a score of 70% or greater.

37. Areas of Success Analysis and Action Taken For the past two terms, 79% of the 42 students completing ACC 115 achieved a score of 70% or better.

38. Areas of Success Analysis and Action Taken For the past two terms, 79% of the 42 students completing ACC 115 achieved a score of 70% or better. Overall the benchmark was missed by 1%. Further analysis revealed that in the spring, the success rate was 66% and in the fall it was 90%. Changes were made in the fall to the Excel format of the problem set which may explain the increase in success rate. Note: Course was not offered in summer.

39. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) Will continue to monitor the success rates to determine if the changes made in the format will result in continued success.

40. Results of Action TakenAttach Graph of Trends for 3-5 Years (Attach Trend Graph) Will continue to monitor the success rates to determine if the changes made in the format will result in continued success.

41. Performance Measure (Competency) Description of Measurement Instrument Students will demonstrate overall knowledge of the learning objectives for introductory accounting, management, and marketing courses.

42. Performance Measure (Competency) Description of Measurement Instrument Students will demonstrate overall knowledge of the learning objectives for introductory accounting, management, and marketing courses. Success is based on the grades achieved on the standardized department final exams administered in all ACC 111, ACC 101, MGT 101 and MKT 101 classes each term. The goal is 75% of the students will pass the exam.

43. Areas of SuccessAnalysis and Action Taken In the areas of management and marketing, the standard was not only met but exceeded.

44. Areas of SuccessAnalysis and Action Taken In the areas of management and marketing, the standard was not only met but exceeded. In accounting neither ACC 111 nor ACC 101 results met or exceeded the standard. At the end of each year, the exams were reviewed to determine which areas the students were having difficultly. A faculty member has been charged with the task of compiling the data results and identity which learning objectives need additional attention.

45. Results of Action TakenAttach Graph of Trends for 3-5 Years (Attach Trend Graph) Once the areas have been identified, teaching methodology will be reviewed. In addition, trends for each faculty member will be reviewed and discussed with the faculty member. Will continue to monitor the success rates to determine if the changes made in the format will result in continued success.

46. Results of Action TakenAttach Graph of Trends for 3-5 Years (Attach Trend Graph) Once the areas have been identified, teaching methodology will be reviewed. In addition, trends for each faculty member will be reviewed and discussed with the faculty member. Will continue to monitor the success rates to determine if the changes made in the format will result in continued success.

47. Performance Measure (Competency) Description of Measurement Instrument Management and marketing students will be able to communicate effectively through active listening and in written and/or oral communications.

48. Performance Measure (Competency) Description of Measurement Instrument Management and marketing students will be able to communicate effectively through active listening and in written and/or oral communications. Success is based on management and marketing students, who are enrolled in BUS 130, successfully completing a business résumé, portfolio and presentation. The goal is 80% of the students will achieve a score of 80% or higher on each element.

49. Areas of SuccessAnalysis and Action Taken The standard was partially met. In the area of the of a business résumé, the students consistently exceeded the 80% benchmark.

50. Areas of SuccessAnalysis and Action Taken The standard was partially met. In the area of the of a business résumé, the students consistently exceeded the 80% benchmark. In the other two areas, writing portfolio and presentation, the students did not meet the standard. Between the 2003-2004 academic year, the department changed the prerequisite for BUS 130 from ENG 100 to ENG 101. Instructor teaching Bus 130 will meet to determine what can be done to improve the scores on the presentation component.

51. Results of Action Taken Attach Graph of Trends for 3-5 Years (Attach Trend Graph) There was marked improvement in the writing portfolios as a result of this change. However, it is still below the 80% goal. Instructors teaching BUS 130 will meet to determine what additional learning resources can be offered to the students.

52. Results of Action TakenAttach Graph of Trends for 3-5 Years (Attach Trend Graph) There was marked improvement in the writing portfolios as a result of this change. However, it is still below the 80% goal. Instructors teaching BUS 130 will meet to determine what additional learning resources can be offered to the students.

53. Performance Measure (Competency) Description of Measurement Instrument Marketing students will demonstrate knowledge and application of various marketing principles and strategies as applied to the physical market place.

54. Performance Measure (Competency) Description of Measurement Instrument Marketing students will demonstrate knowledge and application of various marketing principles and strategies as applied to the physical market place. Success is determined by students enrolled in MKT 270 successfully completing an on-line market research project. The goal is 80% of the students completing the plan will achieve a score of 80% or higher on the project.

55. Areas of SuccessAnalysis and Action Taken Data collection was started in 2005. The results and analysis will be included on the next report. N/A

56. Let’s play “create competencies and measurement instruments”!

57. Guidelines In groups of three or four, create 2 different competencies for any business course. Be sure you identify the course! In groups of three or four, create 2 different competencies for any business course. Be sure you identify the course! Remember, competency demonstrates "how" students can achieve an outcome. It is the specific activity used to measure a student’s mastery of the outcome. Remember, competency demonstrates "how" students can achieve an outcome. It is the specific activity used to measure a student’s mastery of the outcome. For each competency developed 2 different measurement instruments with success criteria. For each competency developed 2 different measurement instruments with success criteria. You have 15 minutes! You have 15 minutes!

58. Additional Resources Baldrige National Quality Program at http://www.quality.nist.gov Continuous Quality Improvement Network (CQIN) at http://www.cqin.org Academic Quality Improvement Project (AQIP) at http://www.aqip.org Council for Higher Education Accreditation (CHEA) at http://www.chea.org Middle States Commission on Higher Education at http://www.msache.org/mainstudents.html

59. Happy Assessing! Any Questions?