1. Defining and Implementing Quality Assurance Standards for Online Courses Lawrence C. Ragan, Director, Instructional Design/Development, The Pennsylvania State University Christina M. Sax, Professor & Assistant Dean, Social, Behavioral, Natural, & Mathematical Sciences, University of Maryland University College

2. Presentation Agenda Session Objectives Participant Activity QM Overview: Circle and Rubric Penn State Approach/Status Story of Convergence QM Attractive Features PSU Adoption/Adaptation QM Opportunities a. Models b. Institution/Institute

3. Session Objective Provide an overview of the development, implementation and administration of an inter- institutional quality assurance system and the application within a single institution.

4. Participant Activity 2-minute drill –Turn to person next to you and discuss: –What characteristics would you use to assess the quality of the vehicle you may be purchasing? Feedback from Teams Issues of Definition--single vs group

5. How do we … identify & recognize it? motivate & instill it? assess & measure it? insure it? assure it? Quality Matters Quality does matter to … students faculty administrators institutions consortia accrediting agencies legislators tax-payers

6. Quality Matters: Inter-Institutional Quality Assurance in Online Learning Grantor: FIPSE –Grant period: 9/03 – 8/06 –Award: $509,177 Grantee: MarylandOnline –Statewide consortium: 14 community colleges, 5 senior institutions http://www.QualityMatters.org

7. FIPSE Interested Because … Quality assurance of online courses is important Voluntary inter-institutional assurance has never been done before This can serve as a national model Quality Matters!

8. Peer Course Review Feedback Course Course Meets Quality Expectations Course Revision Instructional Designers Institutions CAO’s AR’s Faculty Course Developers National Standards & Research Literature Rubric Faculty Reviewers Training Incoming process

9. For Our Purposes, Quality Is… More than average; more than “good enough” An attempt to capture what’s expected in an effective online course at about an 85% level Based on research and widely accepted standards 85 %

10. Major Themes develop inter-institutional consensus - criteria & process online course QA assure & improve course quality positively impact student learning faculty-centered activities promote voluntary participation and adoption ensure institutional autonomy replicable, reliable, and scalable processes foster sharing of materials and expertise create opportunities for training and professional development

11. What Quality Matters is NOT Not about an individual instructor (it’s about the course design) Not about faculty evaluation (it’s about course quality) Not a win/lose, pass/fail test (it’s about a continuous improvement process in a supportive environment)

12. Quality Matters Rubric Based on –research literature –nationally recognized standards of best practice –instructional design principles Used by review teams to: –assess course quality in 8 key areas (40 review elements) –provide feedback to faculty course developer –provide guidance to instructional design support team

13. Rubric Scoring StandardsPointsRelative Value 143Essential 122Very Important 141Important TOTALS 4080 Team of three reviewers One score per standard based on majority Two criteria to meet quality expectations: “Yes” to all 14 Essential Standards Receive at least a total of 68 points

14. Review Teams Teams composed of 3 reviewers: –1 from home institution, 2 from others –1 from same discipline, 2 from others –May be either faculty and or ID/IT (practitioners) –mix of CC & 4 yr schools –mix of large & small schools –mix of public & private schools Course author--resource for review team

15. Reviewer Rubric Training Focus on: –Application of rubric to course review –Interpretation of review elements –Providing constructive feedback –Competency-based

16. Course Reviews To date, 50% meet expectations on initial review –Instructional design support provided Identified 11 common areas for improvement –Target for course development/revision, faculty training instructor's self-introduction netiquette expectations learning objectives stated at the module/unit level self-check/practice activities with feedback Interaction links to school's academic support ADA issues

17. What’s In It For Institutions … External validation process Strengthens institution’s accreditation package Raise QA as a priority activity Gain access to a sustainable, replicable, scalable QA process Inform online course training & practices Provide professional development activities

18. What’s In It For Participants … improve your online course instructional design support external quality assurance expand professional community chance to review other courses gain new ideas for your own course participation useful for annual evaluations, promotion applications, professional development plan/requirements

19. National Participation Scope: –individuals from 70 different institutions (including the 19 MOL schools) in 14 different states –Over 250 faculty trained to review online courses using the rubric –National external partners & advisory board Use of QM System: –online course development, review, and revision, faculty training, formation of distance learning policies & steering committees, institutional reaccredidation packages

20. Awards - 2005 WCET Outstanding Work (WOW) Award USDLA 21st Century Best Practice Award Maryland Distance Learning Association (MDLA) Best Program Award

21. PSU Approach to Quality Standards History: WC started in 1998 with 4 courses grew to 150 by 2002 Course design and development initiatives from units external to WC “Some” variability of design and development practices Compromise of “PSU Quality” If delivered via WC then held responsible for ensuring quality

22. Definition of “Quality” The user (learner) of a system (educational) has a reasonable opportunity for success.

23. PSU Online Quality Standards Charge by Provost to eLearning Council to identify standards of quality for online learning Built upon technical standards and pedagogical guidelines developed by WC Forwarded to Provost and accepted as University policy by the University Web Task Force

24. Implementation Distributed to Deans as policy (probably stopped there) Do exist as record somewhere and accessible if you can find them No enforcement “teeth”-- no implementation system established No mechanism for reviewing and updating Some may claim “academic freedom” and standards need not apply

25. PSU Interest in QM Standards PSU eLearning Council examining the potential of QM model at PSU Subcommittee organized to look at various adaptation options Primary points of interest: –Externally validated –Well defined rubric –Established “system” –Research-based criteria

26. Potential Options 1.PSU can fully accept and adopt the QM system for quality assurance. Courses would be submitted to QM for evaluation and ultimately the QM seal of approval. 2.PSU can adopt the QM system to the PSU environment including approval of reviewers and QM process. PSU would negotiate with the QM board on the adaptation process in order to maintain the QM seal of approval. 3.PSU can adopt the QM system to the PSU environment including approval of reviewers and QM process. PSU would not seek the QM seal of approval. 4.Penn State would maintain the current model of Technical Standards and Pedagogical Guidelines.

27. QM: Looking Ahead Adapt rubric & process for other formats –Hybrid/blended, face-to-face, continuing education, commercial, professional training Adapt rubric & process for specific institutional needs Explore the “QM Institution” concept Assess the impact on student learning through research projects

28. QM: Looking Ahead Diversify Training Program –Additional review training & train-the-trainer Annual rubric update cycle Sustainability plan Develop partnerships & business opportunities

29. www.qualitymatters.org For more information: Kay Kane Kkane@pgcc.edu