1. Design and Implementation of a Course Review Process
Adam M. Persky, Kimberly H. Deloatch, Wendy C. Cox, Mary T. Roth McClurg, Pamela U. Joyner
UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC
INTRODUCTION
RESULTS
Quality assurance is the systematic monitoring and evaluation required to ensure standards of quality are being met.
One of the principal mechanisms for assuring quality of learning and teaching is peer review of teaching and evaluation of the curriculum (learning, teaching and assessment methods) (Dill et al. 1996; Massy et al. 1994).
Horsburgh, (1998) investigated factors that impact student learning through a quality assurance process and found that the most important factors were the curriculum, the instructors, how the teachers taught and facilitated learning, and the assessment practices they adopted.
The course review at UNC focused on:
learning objectives,
course content,
degree of student-centered activities,
assessments methods consistent with learning objectives and course goals,
consistency in coordination, and;
course policies.
FIGURE 1
TABLE 1
Number and types of recommendations made to course directors
Schematic of the course review process.
RECOMMENDATION
TOTAL RECOMMENDATIONS (N= 93)
(% of all recommendations)
NUMBER OF COURSES
with Recommendation
Learning Objectives (update, clarify, make explicit)
25 (23.4) 19
Active Learning (increase the amount)
20 (19.6) 16
Assessment Methods (modify, improve, provide more feedback)
16 (18.3) 17
Content (change, update, remove, re-sequence, reduce)
11 (11.5) 11
Course Policies (add, modify, clarify)
6 (6.7) 6
Course Direction (ensure consistency between instructors/integrate or align content with other courses)
2 (1.9) 2
Other (readings, Blackboard issues, assignments)
13 (17.3) 13
Office of Professional Education and Office of Assessment
Assessment & Curriculum Committees
Assessment Committee
Course Directors
Full review of course
Joint meeting to discuss reviews
Formal letter to course director / division chair
Review of comments; written reply
Review of comments/plan of course director
Recommendations implemented
Post semester follow-up
Annual assessment
6 weeks
4 weeks
2 weeks
Semester
2 weeks
METHODS
6 weeks
2 weeks
Description of Process
TABLE 3
TABLE 2
Survey results from the COURSE DIRECTORS. Data presented as % response (n=13, 50%)
Survey results from the COURSE REVIEWERS.
Data presented as % response (n=22, 60%).
Each required course (n=30) was reviewed by a team of two faculty members
Most of faculty were on the Curriculum or Assessment Committee.
Each team contained at least one clinical faculty member
The review and reports were based on a standardized rubric developed by the Committees which addressed:
course layout and integration,
learning outcomes,
assessment of learning,
resources and materials, and;
learner interaction
Data used in the review process included a self-reflection and teaching goals inventory, course content on Blackboard, curricular mapping information, student course evaluations, and a student completed version of the rubric.
Agree
Disagree
Neutral
The review process was objective.
95.0
5.0
0.0
The rubric was beneficial during the course review process.
85.0
10.0
By reviewing other courses, I learned how to improve my own course or teaching.
90.0
The review process helped me become more familiar with the School's curriculum.
The time commitment to review a course was burdensome.
35.0
45.0
20.0
The one day format to discuss all the review reports was productive.
55.0
40.0
I prefer the discussion of review reports to be over two or three days not the one-day current format.
15.0
The course review process is an important part of the quality assurance process for the professional program)
Overall I was satisfied with the course review process from the REVIEW TEAM perspective.
Agree
Disagree
Neutral
The feedback I received was objective.
84.6
0.0
15.4
The course review provided constructive feedback.
69.2
30.8
The recommendations were consistent with areas already recognized by myself as course director.
61.5
23.1
The time required to prepare the self-reflection documents was burdensome.
46.2
7.7
I would support a course review process every 3-4 years as part of the curricular quality assurance.
66.7
25.0
8.3
For the next offering of the course, the academic year, I plan to implement the majority o...XXXX
The course review process is an important part of the quality assurance process for the professional...XXXXX
Overall I was satisfied with the course review process from a COURSE DIRECTOR standpoint.
53.8
38.5
CONCLUSION
Assessment of Process
The overall process was assessed by two methods:
the ability the process to identify areas for improvement and
an attitudinal survey completed by the course reviewers and the course directors
There was some feeling among course directors the review was burdensome on their time (Table 3). However the median time was 2 hours (range <1 hour to 17 hours).
The course review process was able to identify a variety of areas for course improvement (Table 1).
Faculty are supportive of the course review process (Table 2 & 3).
Considerations for future iterations of the course review may include more incorporation of student feedback
Review team members felt the review process helped them become familiar with the curriculum and how to improve their own courses (Table 2).
Next course review will focus on elective courses and pharmacy practice experiences.
Course directors felt the feedback was objective and constructive (Table 3).
Assessment Committee working to identify methods for continuous course review as opposed to a cyclical process every few years.
REFERENCES
Dill DD. Is There An Academic Audit in Your Future? Reforming Quality Assurance in U.S. Higher Education. Change. 2000;32(4):34-41.
Massy WF, National Center for Postsecondary Improvement SCA. Energizing Quality Work: Higher Education Quality Evaluation in Sweden and Denmark. Project 6, Quality and Productivity in Higher Education 1999.
Horsburgh M. Course approval process. Quality Assurance in Higher Education. 2000;8(2):96-9.