1. Online vs Face to Face: Is There a Significant Difference in Learning Outcomes?
Elaine Gerber & AJ Kelton,
Montclair State University
& Tracy Chu,
Brooklyn College, CUNY

2. Abstract
Our aim is to better understand whether and how instructional modality (online vs face-to-face) impacts student learning. More specifically, this presentation compares survey results and final grades from an in-person face-to-face class with those from an online class, where the professor, course, and semester are the same. While much has been written about online learning, it is rare for the same course to be taught by the same professor in the same semester at the same institution. Thus, these circumstances provided a unique opportunity for comparison. Consistent with extant research, our preliminary findings suggest that there is no significant difference in learning outcomes, according to modality. What did appear significant, however, is that the online students worked more hours and experienced more life obstacles. Despite working more hours and having more obstacles, the online students performed as well in the course overall, as students who didn’t experience as many challenges or spend as many hours in paid employment. This suggests that certain attributes of online learners, such as being highly organized, a “self-starter,” or previously performing well in school, may be better predictors of success than the modality used to deliver course content. This research is ongoing in order to gather a larger sample with greater predictive power. Our presentation at the RAUL Showcase 2014 will highlight findings from the first round of data collection (Spring 2013).

3. A “natural experiment”
Research Design
A “natural experiment”
Same course
Same professor
Same school
Same semester
RQ: Is there a significant difference in learning outcomes by modality?
Learning outcome = final grade for course
COURSE: Anthropology of Multicultural America (ANTH 110) was taught face-to-face and online, during the Spring 2013 semester at Montclair State University (MSU) in New Jersey. The course satisfies a GEN ED requirement on the campus of a large 4-year, mainly undergraduate public institution of higher education, and therefore, attracts students from across the campus.  
The course description, objectives, reading list, and full syllabus can be provided……

4. Same Course?! Similar / shared the same: course objectives, readings
enrollment size (35 students each)
Instructor, semester, university
Differences:
“lecture” content
nature, amount, and quality of peer-based discussion
presence of the instructor
amount of writing involved
weights attributed to various assignments
type of student taking each modality (below)
Selection Bias re enrollments / type of students: the online version of the course only became available shortly before the semester was about to begin, and therefore, the students enrolling in it were those, who for one reason or another, enrolled “late” and may have needed a last-minute course to complete their schedule.  

5. Other limitations? How similar is the course really? (see above)
Planning / Intentional course structure for research design vs retro-fitting to work with existing course offerings
Intellectual property issues
Self-reports, and the problems that go with it
Small sample size
Student characteristics / attributes

6. Sample Bias?
WE THOUGHT THAT STUDENTS WHO WERE INTERESTED IN THE TOPIC MIGHT HAVE MORE INTERNAL MOTIVATION TO LEARN….

7. Data Collection
Subjects were recruited from the full class roster of all students who were enrolled in ANTH (face-to-face) and ANTH (online) at Montclair State University, Spring 2013
Students were given 2 points on their final exam for participating; they also had the option to complete a short assignment in lieu of participating, in order to receive the same extra credit.  
The professor did not know which students had completed the survey and which had done the optional assignment, only which students were to receive the extra two points on their final exam, until after grades were submitted to the Registrar.  

8. Data Analysis
Data were then exported from the Limesurvey database into SPSS for analysis.  Most questions were close-ended and could therefore be analyzed using this program to measure frequencies and other descriptive statistics.  We used T-tests for correlations and a multiple regression analysis, which I’ll talk more about below.
The few open-ended questions were analyzed “by hand,” by reading and re-reading the data in either MS Word or Excel.  There was not sufficient open-ended content to warrant using a qualitative software analysis program, such as The Ethnograph.

9. Preliminary Findings Still preliminary, ongoing
Collecting “round 2” data Spring 2014
Nonetheless……..
What did we find?

10. Statistical Findings:
Only significant difference between the online and F2F samples is that: online students worked more hours, and experienced more obstacles! But, they did just as well in the course overall…

11. Grades by modality Final Exam Course Grade F2F 85.23 84.5 Online 84.13
85.5
Note:  grades were slightly lower for students who did not participate in the survey.
We also see that the greatest number of lowest scores in the course overall, came from the F2F class.   Twelve of the lowest 14 scores (86%) were all from the F2F section.

12. T-test (variables by modality)
Age -- ns But .075, so almost sig
Academic Credits - Current Semester   ns But .055, so almost sig

13. T-test (variables by modality)

14. Obstacles
Are there any obstacles to learning that you faced this semester? (check all that apply):
Working too many hours
Commute to class and/or parking often caused me to miss class
I had family emergencies
I had personal issues
I had housing problems
I was too busy
Access to computers & Internet were not reliable
Course content too intellectually difficult or presented too poorly to learn
Other (please explain):  ______________________________

15. Predictors of Learning Outcome
When looking at the entire sample overall (n=41) , the only variables that correlate with FINAL GRADE are RACE and GPA. When looking at each modality separately, in the online class (n=17), RACE was correlated to FINAL GRADE, but GPA was not. However in the F2F class (n=24), GPA was correlated to Final Grade, and RACE was not.

17. When GPA is controlled for, race is no longer a significant predictor of course grade (Model 1 & Model 2). Moreover, modality does NOT predict course grade, even when controlling for race and GPA (Model 3). Controlling for modality and race, only significant predictor of course grade is previous GPA (Model 3).

19. Discussion
What is going on here?

20. Other Predictors of Outcome?
Do you consider yourself… (select all that apply):
very organized
highly motivated to learn
an independent worker
able to multi-task
good with time management
focused & goal-oriented
comfortable with technology
other: _________________

21. Implications …And, thanks!
Not all students are the same, no universal student
Implications for academic advising:
Retention
Time to graduation
Questions?
Suggestions?
…And, thanks!