Quality Matters : Inter-Institutional Quality Assurance in Online Learning Teaching with New Technologies May 2007

Session Goals Provide an overview of the critical materials, processes, and administrative elements of the Quality Matters online course quality assurance program. Describe the Peer Course Review process. Explore lessons learned in creating a quality assurance program and applications of the QM process.

About Quality Matters

Peer Course Review Feedback Course Course Meets Quality Expectations Course Revision Instructional Designers Institutions Faculty Course Developers National Standards & Research Literature Rubric Faculty Reviewers Training Quality Matters: Peer Course Review Process

Factors that Affect Course Quality

Factors Affecting Course Quality Course design QM REVIEWS THIS Course delivery (i.e. teaching, faculty performance) Course content Course management system Institutional infrastructure Faculty training and readiness Student engagement and readiness

Underlying Principles of QM The QM toolset and process are: –based in national standards of best practice, the research literature and instructional design principles –designed to promote student learning –part of a faculty-driven, peer review process Process designed to ensure all reviewed courses will eventually meet expectations Collegial review process, not an evaluation process

For Our Purposes, Quality Is… More than average; more than “good enough” An attempt to capture what’s expected in an effective online course at about an 85% level Based on research and widely accepted standards 85 %

QM Process Provides Institutional toolset and process to meet quality expectations: –Online course design –Student learning –Improved instruction –Assessment and feedback loops –Professional development

What’s In It For Institutions … External validation Strengthen accreditation package Raise QA as a priority activity Access to a sustainable, replicable, scalable QA process Inform online course training & practices Provide professional development activities

What’s In It For Faculty … Improve online courses External quality assurance Review other courses and gain new ideas for own course; expand professional community Participation useful for professional development plan and portfolio

What this process is NOT Not about an individual instructor (it’s about the course) Not about faculty evaluation (it’s about course quality) Not about “win/lose” or “pass/fail” (it’s about continuous improvement in a supportive environment)

Design vs. Delivery The faculty member is integral to both design and delivery. Course Design … is the forethought and planning that a faculty member puts into the course. Course Delivery … is the actual teaching of the course, the implementation of the design. QM is about DESIGN - not delivery or faculty performance

Quality Matters Toolset

The Instructor Worksheet Worksheet

Instructor Worksheet Includes info such as: –Institutionally mandated objectives, materials, practices, policies –Materials outside course pages –Types of interaction used & instructor’s statement on the appropriateness of interaction in the course –Additional items that may require review

The Rubric

Eight standards: –Course Overview and Introduction –Learning Objectives –Assessment and Measurement –Resources and Materials –Learner Interaction –Course Technology –Learner Support –Accessibility Key components must align.

What is Alignment? Critical course elements work together to ensure that students achieve the desired learning outcomes.

The Rubric Essential Standards (14) Essential Standards (14)

General Standard I: Course Overview and Introduction 1.1: Navigational instructions make the organization of the course easy to understand. 1.2:A statement introduces the student to the course and to the structure of the student learning, and, in the case of a hybrid course, clarifies the relationship between the face-to-face and online components.

General Standard II: Learning Objectives (Competencies) II.I: The course learning objectives describe outcomes that are measurable. II.2:The module/unit learning objectives describe outcomes that are measurable and consistent with the course-level objectives.

General Standard III: Assessment and Measurement III.1:The types of assessments selected measure the stated learning objectives and are consistent with course activities and resources. III.2:The course grading policy is stated clearly. III.3:Specific and descriptive criteria are provided for the evaluation of students’ work and participation.

General Standard IV: Resources and Materials IV.I:The instructional materials support the stated learning objectives. IV.2:The instructional materials have sufficient breadth, depth, and currency for the student to learn the subject.

General Standard V: Learner Interaction V.1:The learning activities promote the achievement of stated learning objectives. V.2:Learning activities foster instructor- student, content-student, and if appropriate to this course, student-student interaction. V.3:Clear standards are set for instructor response and availability (turn-around time for , grade posting, etc.)

General Standard VI: Course Technology VI.I:The tools and media support the learning objectives and are appropriately chosen to deliver the content of the course.

General Standard VII: Learner Support No 3-point elements This standard has no “essential” 3-point elements because it’s primarily concerned with academic support services, student support services and technical support services….usually thought to be the primary responsibility of the institution and not the individual instructor.

General Standard VIII: Accessibility VIII.1:The course acknowledges the importance of ADA requirements. To meet this standard, the course must include BOTH of these elements: The course must be offered using software that is accepted as “ADA compliant.”. AND The course should include a brief statement that clearly tells students how to access ADA services at the institution.

The Rubric Scoring Scoring

Rubric Scoring StandardsPointsRelative Value 143Essential 122Very Important 141Important TOTALS 4080 Team of three (3) reviewers One score per standard based on team majority Assigned point value; not sliding scale

To Meet Expectations… A course must achieve: “Yes” on all 14 of the 3-point “essential” standards. A minimum of 68 out of 80 points 68/80 = 85%

Online & Hybrid Courses Rubric designed for application to fully online and hybrid/blended courses Same set of standards apply to both How we achieve the standards may differ For hybrids, focus on pedagogical integration of online and F2F components

Recommendations

Peer Course Review Process

The Peer Review Team 3 faculty peer reviewers: –must be experienced online instructors –must attend QM training –one MUST be external to the course’s originating institution –there must be a subject matter expert (SME) on the team. NOTE: The SME could also be the external reviewer. AND Faculty course developer: –access to rubric prior to review –involved in pre-review discussions –consulted during review

About the Course QM is designed to review “mature” courses (taught at least two semesters) QM logo indicates year course met expectations Triggers for subsequent reviews: Faculty request More than 3 years since original review New textbook or instructor Professional or accreditation review pending

About the Review On average, a course review takes 7-10 hours; active review period approximately 3 weeks Factors affecting review time include Reviewer familiarity with the discipline Reviewer familiarity with the CMS Reviewer familiarity with the QM review process Organization of the course

Review Outcome If meets expectations: –Recognized by Quality Matters –Notifications distributed –ID support provided if requested If does not yet meet expectations: –ID support provided if requested –Team Chair and ID approve revisions –Course meets expectations

Roles and Responsibilities Faculty Developer Peer Reviewers Team Chair

QM To Date

QM in Transition 2003 – August 2006 –QM project funded by FIPSE grant money –materials and some services freely available August 2006 and beyond –QM project funded by MarylandOnline –Some limited materials will be freely available –Other materials available to individuals and institutions at nominal fees –Institutional membership affords full access to materials and services

QM to Date Overall Participation –Individuals & programs from 150 institutions across 30 states Course Reviews Peer Reviewer Rubric Training –900+ trained

Quality Matters Quality MattersApplications

Multiple Uses of QM Reported Uses of QM System: Guidelines for initial online course development Quality assurance of existing courses Ongoing faculty professional development Institutional reaccredidation packages Formation of distance learning policies & steering committees

Thanks to YOU… Thanks to YOU… Quality Matters!