1. Mingming.Shao@dsu.edu Haomin.Wang@dsu.edu Dakota State University

2. Quality Assurance  Standards to guide instructors in designing and developing online courses. Standards

3. Quality Assurance  Standards to guide peer reviewers to evaluate online courses and suggest improvement. Standards

4.  Course navigation.  Class communication.  Specification of learning objectives.  Alignment between learning objectives, learning activities, and assessment measures.  Appropriate use of media, resources and technology tools.

5. Review Background (1)  Since 2006, South Dakota EUC (Electronic University Consortium) adopted Quality Matters standards. Quality Matters  In the system level, 3% of the online courses are randomly selected each semester for quality assurance reviews.

6. Review Background (2)  At the university level, all newly developed or redesigned courses are reviewed after they are taught for one semester.  All online courses are assigned a future three-year review date based on the date of initial or last review.

7. Review Roles instructor Peer Reviewers Instructional Designer Coordinator

8. Review Roles  Each online course is reviewed by two faculty members, in communication with the course instructor.  An instructor may need to serve as a reviewer of courses taught by other instructors.  Instructional designer coordinates the process and writes a review report.

9. Review Procedures  Select courses.  Self review.  Alert the coordinator and reviewers when self review is complete.  Peer review.  Reviewer feedback to the coordinator and instructor.  Course instructor responds and makes improvement.

10. Review Process

11. Issues  Rubric cumbersome to use.  Delay in sending feedback.  Difficulty for coordination, data analysis, and reporting.  Time-consuming and inefficient. Time-consuming

12. Solution: A Web Portal coordinator instructorsreviewers A web portal

13. Our Solution  Database support User login information and access logs. Review standards and related annotations and examples. Review input, both self review and peer review. Review status, in progress or completed.

14. Our Solution (continued)  Consistency: Same rubric for self review, peer review, and coordinator reference.  Immediacy: Update instantly reflected for all users.  Convenience: Annotations and examples as links for each standard.  Coordinator can monitor the reviewer process in real time.

15. Instructor’s Interface  Sees a list of courses to be self reviewed.  Can complete a self review in multiple sessions.  View links of Start, Edit and View.  View links of reviewers’ feedback.

16. Instructor’s Review Process  Select Yes or No.  View the annotations and examples.  Review points can be automatically calculated.  Submits a self review, sending an e-mail notification to coordinator and the assigned reviewers.

17. Reviewers’ Interface  A list of courses to be reviewed.  Links to self reviews available.  Can complete a review in multiple sessions.  View links of Start, Edit and View.

18. Reviewers’ Review Process  Select Yes or No.  View the annotations and examples.  Review points can be automatically calculated.  Upon review submission, an e-mail alert sent to both coordinator and instructor.

21. Coordinator’s Interface  View a list of all courses, instructors, reviewers, system reviewed courses with progress marks indicating.  Can check self reviews and review progress or completed.  can re-open a course to allow re- submitting of a self review or review.

23. Benefits (1)  Cleaner and scalable interface for the rubric. Annotations and examples hidden by default and available at a click.  Instructors and reviewers can save a partially completed self review or review and return later to continue.  Self review makes the instructor more self conscious of course design and quality assurance.

24. Benefits (2)  When a self review or review is completed, relevant persons will be automatically notified by e-mail.  Much easier for the coordinator to monitor the review progress, and facilitate communication between instructor and peer reviewers.  Communication between instructor and peer reviewers is beneficial to both sides in their own teaching.

25. Benefits (3)  Much easier for the coordinator to collect review data and get the review findings.  The findings help us how to define our trainings in course design and improvement.

26. Lesson Learned  Instructional designers offer help to instructors before, during and after self review.  Self review should start early.  Instructors provide notes in self reviews to guide reviewers.  Just-in-time improvement encouraged.  Share best practices.

27. Upgrade Plan  Integration with additional instructional design and faculty support resources.  Integration with distance student services and online tutoring.