1. An Empirical Study of In-Class Labs on Student Learning of Linear Data Structures Sarah Heckman Teaching Associate Professor Department of Computer Science North Carolina State University ICER 2015

2. Problem ICER 2015 CSC116CSC216CSC316 7-8 sections 33 students 1 instructor 2 TAs Lecture/Lab 1-2 sections 70-90 students 1 instructor 2-3 TAs Lecture 1-2 sections 70-90 students 1 instructor 2-3 TAs Lecture Transition! Retention! Lab? In-Class Labs? Do Nothing?

3. Research Goal To increase student learning and engagement through in-class laboratories on linear data structures Hypothesis: active learning practices that involve larger problems would increase student learning and engagement ICER 2015 In-class Labs > Pair & Share

4. Research Questions Do in-class laboratories on linear data structures increase student learning on linear data structures exam questions when compared to active-learning lectures? Do in-class laboratories on linear data structures increase student engagement on linear data structures exam questions when compared to active-learning lectures? ICER 2015

5. Active Learning in CSC216 “engaging the students in the process of learning through activities and/or discussion in class, as opposed to passively listening to an expert” [Freeman, et al. 2014] Control: Active Learning Lectures –2-5 pair & share exercises per class –Submitted through Google forms Treatment: In-class Labs –Lab activity for the entire lecture period –Pre-class videos introduced topic ICER 2015

6. Study Participants MetricSection 001Section 002 # Enrolled85102 Participants (completed course) 4960 Dropped/Withdrawn (consenting only) 34 Women910 Meeting TimeTH 2:20-3:35pMW 2:20-3:35p ICER 2015 Self-selected into section during standard registration period Populations were similar as measured by a survey on experience with tooling and self-efficacy.

7. Methods Quasi-Experimental –Counter-balanced design –Learning measured through exams –Engagement measured through observations of class meetings ICER 2015 Lists Array Linked Iterators Exam 1 001 002 Replication Materials: http://people.engr.ncsu.edu/sesmith5/ 216-labs/csc216_labs.html Replication Materials: http://people.engr.ncsu.edu/sesmith5/ 216-labs/csc216_labs.html Observed Class Meetings

8. Student Learning – Exam 1 Part 4: Method Tracing with ArrayLists Part 5: Writing an ArrayList method ICER 2015 ItemPointsS001 Mean S001 SD S002 Mean S002 SD p-value E1 P4#853.631.564.351.45< 0.010 E1 P4#954.181.074.571.090.016 E1 P4#1052.632.403.452.180.149 E1 P41510.453.9712.373.74< 0.010 E1 P52017.764.018.254.090.233

9. Student Learning – Exam 2 Part 3 – Linked Node Transformation Part 5 – Writing a LinkedList Method ICER 2015 ItemPointsS001 Mean S001 SD S002 Mean S002 SD p-value E2 P3169.435.8511.806.41< 0.010 E2 P52011.804.1412.584.210.412

10. Student Learning – Exam 3 Comprehensive 3 hour final exam Stack Using an ArrayList Queue Using a LinkedList ICER 2015 ItemPointsS001 Mean S001 SD S002 Mean S002 SD p-value E3 Array108.312.458.462.490.313 E3 Linked108.362.538812.450.221 E3 Score10585.0229.1787.2328.920.372

11. Student Engagement Observations for ArrayList and LinkedList class meetings Observers were graduate students and a colleague participating in a Teaching and Learning seminar Counts of students off topic during lecture and exercise portions of the class Some inconsistent use of the observation protocol ICER 2015

12. Student Engagement ICER 2015 ObservationClass Type # Off Topic – Lecture # Off Topic – Exercise Questions of Teaching Staff 1Lab5732 2Lecture624912 3Lab104350 4Lecture4616--- 5Lecture--- 6Lab51033 7Lecture52542 8Lab165--- Lab Average916.338.3 Lecture Average53.339.77 Lecture / Lab5.92.40.2

13. Threats to Validity External Validity –Two sections of the same course, taught by the same instructor, in the same semester, and same time of day –Replication needed in other contexts to generalize further –Could provide additional data points in future meta- analyses ICER 2015

14. Threats to Validity Internal Validity –Selection bias: students selected their own sections Initial surveys shows groups were similar –Confounding factors Materials shared between groups Effect size – only 6 in-class labs –Differential Attrition Bias Considered “soft-drops” in the study –Experimenter Bias Participants were not revealed until after the semester was over ICER 2015

15. Threats to Validity Construct Validity –Exams as Measures of Learning Exam 1 and Exam 2 were similar, but not the same, between sections Exam 3 was common Does exam really measure student learning? –Survey Wording may be confusing for prior tool experience Efficacy questions not a validated instrument –Observation Protocol as Measure of Engagement Inconsistent use by observers ICER 2015

16. Discussion Did in-class labs increase student learning? –No, at least not as measured by exam questions –Both control and intervention were active learning Maybe a simple active learning intervention is enough –Comparisons with earlier semesters may show more Did in-class labs increase student engagement? –Yes and No –The atmosphere in the classroom was fantastic –But many questions were technology and not concept Completion – 72% of students earned a C or higher –Not reaching the higher levels of completion we expect from active learning literature ICER 2015

17. Future Work Additional Work on Fall 2014 Data –Compare results on final exam with previous courses –Incorporate analysis of other measures of learning – projects, exercises, etc. Starting in Fall 2015 –Additional in-class labs → Lab-based course –Measure types of questions asked during in-class labs –Use labs as a way to encourage best practices (frequent commits to version control, TDD) ICER 2015 Replication Materials: http://people.engr.ncsu.edu/sesmith5/ 216-labs/csc216_labs.html Replication Materials: http://people.engr.ncsu.edu/sesmith5/ 216-labs/csc216_labs.html

18. Thank You! Questions? Comments? Concerns? Suggestions? ICER 2015