2. The 3 rd Annual Quality Matters Conference Presentation Nov. 8, 2011 4:45PM Presenter: Dr. Marcia L Ashbaugh

4. Why Leadership? Whos Following? What does Leadership have to do with Quality? The STUDY

5. …and join the audience in a quick poll PLEASE TURN ON CELL PHONES Would you recommend an online course to a student who is struggling in a traditional classroom? Text VOTE and the number of your response to: 52627. Type VOTE, space, (YES = 6321, NO = 6322) OR, go to: http://m.smspoll.net now!http://m.smspoll.net

6. POLL results

7. Faculty vote no No significant difference (flawed research) Online course quality questioned (Naidu) Are designers competent? leaders? (Beaudoin, Sims & Koszalka) New mode of learning (Beaudoin, Dede, IPad) New designs, new situations: designers need re-tooled Context a schedule, brief outline of the course content, PowerPoint slides of the lecturers notes, and sometimes, sample examination papers (Naidu, 2003, p. 349) and not much more!

8. Significance Leadership: Futures mindset, informed choices, purposeful change Expanded notions of leadership Elevated quality of academic programs Change in designer status Evolution in fields duty to education

9. PURPOSE Query designers in practice for quality predictors

10. Research QUESTIONS What instructional design leadership competencies are identified as critical to creating quality online learning designs? What are the characteristics of courses created by participants who have identified critical leadership competencies? How do students evaluate the quality of courses by those using the identified critical leadership competencies? 1 2 3

11. In-depth interviews Instructional design experts (6) 610 AECT members Semi-structured, open-ended questions Triangulated with personal documents (online designs) Assured anonymity Study METHODOLOGY

12. Leadership Competencies Modified RATER Quality Online Courses Strategy Vision Productivity Personality/ Communication

13. Major RESULTS Competencies for Overall Design Leadership Strategy55% Vision29% Others16% RQ2 Analysis What are the characteristics of courses created by participants who have identified critical leadership competencies? RQ2 Analysis What are the characteristics of courses created by participants who have identified critical leadership competencies?

14. Leadership competencies for the design structure Instructional strategies by category Leadership COMPETENCIES in use

15. QM Rubric Triangulates DATA QM Category Points possible Points achieved % of standards met QM minimum quality percentage % +/- of minimum Learning objectives 9.0 8.2 91.1% Assessments and measurements 13.0 11.4 87.7% Learning engagement 10.0 8.6 86.0% Total 32.0 28.2 88.1% 82.9% + 5.2%

16. Max Score Actual Score Course Topic: Students as prosumers of online learning, designers, researchers 2.1-Course learning objectives describe outcomes that are measurable 32Objectives were clear, organized, specific, most were measurable but not all, and section was not labeled – reduced score by -1 ObjectiveAuthentic tasks 2.5-The learning objectives address content mastery, critical thinking and core learning skills 22Objectives included descriptors: explore, critically examine, create, participate, write, experience (not measurable though goal was value- based learning) ObjectiveInteraction 3.1-Assessments measure the stated learning objectives, are consistent with activities 33Participation: 30% Design project: 35% Scholarly Paper: 35% AssessmentLearner control tasks 3.3-Specific, descriptive criteria provided for evaluation of student work 32Course evaluation rubric with graphic example Detailed grading chart AssessmentProblem-solving 5.4-Requirements for student interaction are clear 22Course participation section included percent of grade, due dates, specific instructions and expectations, expected outcomes StrategyTheory-based instruction 5.1-Activities promote achievement of stated learning objectives 33Activity: Design learning experience in technology- enhanced environment ActivityValues-based instruction Applied RUBRIC Stated Strategies: QM Standard Course Component OASA

17. Process Organize Score Tally Benefits Standardizes Fair Reliable (Trustworthy) Limitations Interpretations QM definitions Lack of design standards

18. Audience FEEDBACK Does pedagogical quality need improvement? How will the use of the QM rubric in research help?

19. Future USES 1. More research Aman (2009) Slovick (2011) 2.? 3.?

20. SHOUTOUTS