1. 7th Annual QM National Conference
The Noel Levitz Priority Survey for Online Learners at NMSU: A Baseline for Future Research
7th Annual QM National Conference
November 1-4, 2015
Sharon Lalla, Ed.D. & Susie Bussmann, Ph.D. Instructional Innovation & Quality (IIQ), New Mexico State University

2. Start a research project to get a baseline about online course quality and services

3. Research Questions
What are student perceptions about quality in the online courses that have been influenced by QM versus NonQM?
How satisfied are our online learners with issues that are really important to them at NMSU in the areas of institutional perception, academic services, student services, instructional services, enrollment services, and course quality?

4. Our Alternative Hypothesis…
Courses influenced by Quality Matters will have a positive effect on quality at NMSU

5. Nov. 2014 Nov. 2014 How did we start? Get Buy-in from Colleges (Pilot)
Add 4 questions
Nov.
2014

6. How did we start? Jan. 2015 Feb.- May Submit IRB Application
Collaborate with Noel Levitz; Distribute Survey
Feb.- May
Instutional Analysis IA department, integrity and anonymity

7. The Instrument: Noel Levitz PSOL
26 NL Questions
6 QM and 4 NMSU Questions
Importance to me
1 - not important at all 2 - not very important 3 - somewhat unimportant 4 - neutral
5 - somewhat important 6 - important 7 - very important N/A - does not apply
My level of Satisfaction
1 - not satisfied at all 2 - not very satisfied 3 - somewhat dissatisfied 4 - neutral
5 -somewhat satisfied 6 - satisfied 7 - very satisfied N/A - not available/not used
The Noel Levitz Priority Survey for Online Learners at NMSU

8. Questions about Quality
QM Questions
This institution’s online courses are of high quality.
Courses have clear instruction about getting started and where to begin.
Technologies required in my courses are readily available, provided or easily downloaded.
Navigation in the online courses is logical, consistent, and efficient.
Course learning objectives, and instructions on how to meet them, are made clear.
Courses are well-designed, readable and not distracting.

9. Questions about Quality
Our Questions:
The purchase of course materials is a worthwhile investment.
The learning experience is engaging and supportive.
Courses use a variety of instructional materials such as audio, video, transcripts, etc.
Course assignments are meaningful and relevant.

10. Target Population
3,270 online learners in Business, Health & Social Services, and Education Colleges, Spring 2015
Susie 1-10: Last Slide for Susie
Two surveys – can we see noticable Impact on qm and non qm
Survey distributed to three colleges in Spring out of 3,270 students responded to both surveys
NonQM: 424 out of 2750 (15%) response
QM: 109 out of 520 (21%) responses

11. Data Sample N=533 (16%) out of 3,270 Responses 109/520 (21%)
Two surveys – can we see noticable Impact on qm and non qm
Survey distributed to three colleges in Spring out of 3,270 students responded to both surveys
NonQM: 424 out of 2750 (15%) response
QM: 109 out of 520 (21%) responses

12. Demographics Female 73% 19-34 Age 63% Masters 34% Rent or Own 79%
Dona Ana County 63%
Online
54%
Single 62%
4=Year
43%
Full-time 48%
19-34 Age 63%

13. Satisfaction with Online Services
The smaller the gap, the greater the satisfaction

14. Online Services Strengths
Registration for online courses is convenient.
Billing and payment procedures are convenient for me.
The technologies required in my courses are readily available, provided or easily downloaded.
Instructional materials are appropriate for program content.
Challenges
The institution's online courses are of high quality.
Faculty provide timely feedback about student progress.
There are sufficient offerings within my program of study.
The quality of online instruction is excellent.

15. Qualitative Responses
Please enter any comments you would like to share with this institution.
154 (29%) responses

16. Findings: About Quality
History of Online Courses at NMSU
Conflicted Responses
Excellent, Fair, Poor
Across colleges
“I do not think that this survey is set up very well to obtain accurate results. Half my classes have great instructors, who respond timely and have very nicely set up online classes. Other instructors are terrible and do not respond and often times the online course conflicts with the syllabus. As a result, it is difficult to answer about all classes with just one question. I either give a higher rating than some online courses deserve or a lower rating than others deserve. “

17. Learner Voice on “Quality”
Course Organization
Instructor Feedback
Instructor Interaction
Instructor Response
Instructor (Social) Presence
Instructor Engagement
Work Overload
Canvas Training
Online Teacher Training (Design/Delivery)
Consistency
Time put in to class
Validates importance of QM and some not qm

18. Learner Voice on “Quality” (cont’d.)
Course Organization
Some of the courses have been well organized and others have not been and have been difficult to navigate.
Instructor Feedback
On a positive note though, my instructor has been very timely with the grading of assignments and always provides practical and meaningful feedback.
Instructor Interaction
I have VERY good and VERY bad experiences in online classes. Some classes are perfectly fine, some are exceptionally great, and a couple have been wastes of time; incorporating out of date textbooks, lack of professor-student interaction/feedback.
Time put in to class

19. Learner Voice on “Quality” (cont’d.)
Instructor Response
For example, the instructors do not reply to s in a timely manner or at all (I have yet to receive replies from my current professors on s I sent several weeks ago…
Instructor (Social) Presence
I have been very pleased with the program thus far. The teachers and staff are excellent and I loved coming down to see the University even though I am taking my online classes in Canada. I feel connected to the University and am so pleased I choice NMSU.
Instructor Engagement
Many faculty are not very engaged in student learning.
Time put in to class

20. Learner Voice on “Quality” (cont’d.)
Work Overload
The on-line classes seem to require so very much work that it is really hard to take a full-time schedule compared to taking in-class classes on campus since I have done both.
Canvas Training
My only other pet peeve, which should not be misconstrued to be less irritating than the previous, is PROFESSORS WHO DON'T KNOW HOW TO USE CANVAS OR EVEN BE AN ONLINE INSTRUCTOR!!!
Online Teacher Training (Design/Delivery)
A few instructors might benefit from training on how to facilitate online courses and the importance of timely feedback.
Time put in to class
Gap Analysis
Strengths in online services
Challenges in online services
Mixed qualitative responses regarding quality of courses

21. Results of Goal/Hypothesis…
Goal: Start a research project to get a baseline about online course quality and services
✓ About online course services
✓ Student perceptions of quality
Alternative Hypothesis: Courses influenced by Quality Matters will have a positive effect on quality
Unproven

22. Closing the Loop
Disseminate the results to administration, faculty, students.
Disseminate the results to academic support, student support, instructional, and enrollments services for service improvement.
Add online teaching to our professional dev.
Revisit lens for further research.

23. What’s Next? Redesign: Pilot study in Business college
Research trends in retention, student satisfaction, and grade improvement in QM recognized courses
Work with NMSU System-Wide QM Coordinators to evaluate COP
Possibly conduct Noel Levitz PSOL for entire university

24. Thank you for attending.
Questions?

25. New Mexico State University
Contact Information
Sharon Lalla, Co-Principal Investigator
Susie Bussmann, Co-Principal Investigator
New Mexico State University