1. Graduate Student Satisfaction with an Online Discrete Mathematics Course Amber Settle, CTI, DePaul University joint work with Chad Settle, University of Tulsa CCSC:MW September 24, 2005

2. Satisfaction with distance learning Distance learning is popular with CTI students Distance learning is popular with CTI students There are 8 M.S. and 1 M.A. degree online (from 10 M.S., 1 M.A., and multiple joint degrees) There are 8 M.S. and 1 M.A. degree online (from 10 M.S., 1 M.A., and multiple joint degrees) Distance learning students are 21% of the student population Distance learning students are 21% of the student population It has been asserted that while outcomes are similar in DL and traditional classes, DL classes are less satisfying to students (Carr 2000) It has been asserted that while outcomes are similar in DL and traditional classes, DL classes are less satisfying to students (Carr 2000) Is DL less satisfying for CTI students? If so, how? Is DL less satisfying for CTI students? If so, how?

3. The test case The course The course CSC 415: Foundations of Computer Science CSC 415: Foundations of Computer Science Discrete mathematics including propositional and predicate logic, proofs by induction, basic algorithms, asymptotic analysis, recurrence relations, graph theory Discrete mathematics including propositional and predicate logic, proofs by induction, basic algorithms, asymptotic analysis, recurrence relations, graph theory 9 sections between Fall 2001 and Fall 2003 9 sections between Fall 2001 and Fall 2003 The format The format Traditional (3 sections) Traditional (3 sections) Sibling DL: Runs parallel to a traditional class; entire classroom interaction is recorded automatically (2 sections) Sibling DL: Runs parallel to a traditional class; entire classroom interaction is recorded automatically (2 sections) Sibling DL Sibling DL Pre-recorded DL: High production quality independent of any live class; broken into five modules (4 sections) Pre-recorded DL: High production quality independent of any live class; broken into five modules (4 sections) Pre-recorded DL Pre-recorded DL

4. Course evaluations Conducted every quarter for every CTI course Conducted every quarter for every CTI course Mandatory for all students Mandatory for all students Online using secure login; anonymous Online using secure login; anonymous Completed during the 8 th and 9 th week of 10 week quarter Completed during the 8 th and 9 th week of 10 week quarter Results are withheld from instructor until grades are submitted; results are then published on the CTI web site Results are withheld from instructor until grades are submitted; results are then published on the CTI web site Consists of 22 multiple choice questions Consists of 22 multiple choice questions 10 questions about course-related factors; 12 questions about instructor-related factors 10 questions about course-related factors; 12 questions about instructor-related factors Ratings on a scale from 0 to 10; a higher number indicates greater satisfaction; 0 indicates the question is not applicable Ratings on a scale from 0 to 10; a higher number indicates greater satisfaction; 0 indicates the question is not applicable

5. Statistical analysis Ordinary least squares regression: Ordinary least squares regression: X 2i = 0 for traditional, X 2i = 1 for DL X 2i = 0 for traditional, X 2i = 1 for DL If  2 is statistically different from 0, it indicates a difference in how DL students view the course vs traditional students If  2 is statistically different from 0, it indicates a difference in how DL students view the course vs traditional students Total of 100 data points Total of 100 data points 80 traditional students 80 traditional students 20 pre-recorded DL students 20 pre-recorded DL students Scores of 0 (not applicable) were dropped from each question Scores of 0 (not applicable) were dropped from each question Q i =  0 +  1 X 1i +  2 X 2i + u i Question i Constant TimeDL Error

6. Course-related questions 1. Was this course well organized? 2. Do you feel the course objectives were accomplished? 3. The amount of work you performed outside of this course was: 4. How difficult was this course material? 5. The textbook for this course was: 6. Supplementary reading for this course was: 7. The assignments for this course were: 8. What is your overall estimate of this course? 9. How valuable was this course in terms in your technical development? 10. Would you recommend this course to another student?

7. Course-related results QuestionTimeDLQuestionTimeDL Q-CR1-0.164** (0.075) 0.200 (0.393) Q-CR60.046 (0.164) 0.302 (0.910) Q-CR2-0.120 (0.079) -0.023 (0.416) Q-CR7-0.071 (0.101) 0.237 (0.533) Q-CR30.064 (0.086) 0.221 (0.460) Q-CR8-0.179** (0.089) 0.594 (0.467) Q-CR40.305*** (0.102) -0.549 (0.530) Q-CR9-0.199* (0.110) 0.620 (0.573) Q-CR50.185 (0.144) -0.261 (0.758) Q-CR10-0.202** (0.092) 0.067 (0.486) Coefficient estimates are presented with standard errors in parentheses. *Statistically significant at the 10% level of a two-tailed test. **Statistically significant at the 5% level on a two-tailed test. ***Statistically significant at the 1% level of a two-tailed test.

8. Instructor-related questions 1. How would you characterize the instructor’s knowledge of this subject? 2. How would you characterize the instructor’s ability to present and explain the material? 3. Does the instructor motivate student interest in the subject? 4. How well does the instructor relate the course material to other fields? 5. Did the instructor encourage participation from the students? 6. Was the instructor accessible outside of class?

9. Instructor-related questions continued 7. What was the instructor’s attitude? How did he/she deal with you? 8. How well did the instructor conduct, plan, and organize classes? 9. Were the instructor’s teaching methods effective? 10. How fair was the grading of the homework and exams of this course? 11. Would you take this instructor for another course? 12. Rate the teaching effectiveness of this instructor as compared to other faculty in the department.

10. Instructor-related results QuestionTimeDLQuestionTimeDL Q-IR1-0.010 (0.072) 0.065 (0.379) Q-IR7-0.115 (0.084) 0.013 (0.437) Q-IR2-0.069 (0.069) -0.029 (0.360) Q-IR8-0.110 (0.084) 0.042 (0.442) Q-IR3-0.194** (0.083) -0.016 (0.437) Q-IR9-0.181** (0.088) 0.570 (0.454) Q-IR4-0.189* (0.098) 0.043 (0.515) Q-IR100.010 (0.087) 0.304 (0.451) Q-IR5-0.267** (0.102) 0.481 (0.654) Q-IR11-0.096 (0.089) 0.411 (0.461) Q-IR6-0.053 (0.090) -0.183 (0.468) Q-IR12-0.114 (0.082) 0.268 (0.443) Coefficient estimates are presented with standard errors in parentheses. *Statistically significant at the 10% level on a two-tailed test. **Statistically significant at the 5% level on a two-tailed test.

11. Summary of results Similarities in evaluations: Overall scores Similarities in evaluations: Overall scores None of the coefficients for instructor-related or course- related questions were significantly different from 0 for DL sections None of the coefficients for instructor-related or course- related questions were significantly different from 0 for DL sections Differences in evaluations: Not applicable response rate Differences in evaluations: Not applicable response rate Q-IR5 (Encourage participation) Q-IR5 (Encourage participation) DL: 60% DL: 60% Traditional: 6% Traditional: 6% Q-IR12 (Teaching effectiveness) Q-IR12 (Teaching effectiveness) DL: 15% DL: 15% Traditional: 2.5% Traditional: 2.5%

12. Conclusions and future work Potential explanations for results Potential explanations for results Pre-recorded DL is better organized which compensates for the lack of interaction (Swan 2001) Pre-recorded DL is better organized which compensates for the lack of interaction (Swan 2001) DL students are not watching the recordings DL students are not watching the recordings Small DL sample size Small DL sample size Second study with Java I and II courses Second study with Java I and II courses Larger data set Larger data set Course-related differences: DL students feel the class is less organized and that course objectives were not accomplished as well Course-related differences: DL students feel the class is less organized and that course objectives were not accomplished as well Instructor-related differences: Nine out of twelve questions were statistically different from zero for DL students Instructor-related differences: Nine out of twelve questions were statistically different from zero for DL students

16. Back