1. The Effects of Engaged Learning on Student Academic, Personal, & Civic Development Dana Natale Assistant Director, Center for Career Development and Community-Based Learning Courtney Hopkins Graduate Student, Department of Psychology Valerie Sessa Assistant Professor, Department of Psychology Montclair State University Bringing Theory to Practice 4th Annual National Working Conference January 18-19, 2007 New Orleans, Louisiana

2. Research Questions: Does engaged learning, in the form of service- learning, impact the academic and civic development, and well- being of students? Does engaged learning, in the form of service- learning, impact the academic and civic development, and well- being of students? What is it about engaged learning that impacts the academic and civic development, and well-being of students? What is it about engaged learning that impacts the academic and civic development, and well-being of students?

3. Research Design/Measures Quantitative Data Quasi-experimental, non-equivalent control group design Quasi-experimental, non-equivalent control group design Freshman students Freshman students Pre-Test & Post-Test Measures consisting of standardized questions about participants well-being, as well as their academic and civic development Pre-Test & Post-Test Measures consisting of standardized questions about participants well-being, as well as their academic and civic development –CES-D: Center for Epidemiological Studies-Depression –Satisfaction with Life –AUDIT: Alcohol Use Disorders Identification Test Qualitative Data Will use critical reflections-experimental group only Will use critical reflections-experimental group only Will offer insight into any changes we see in the experimental group as a result of the service-learning Will offer insight into any changes we see in the experimental group as a result of the service-learning

4. Sample Experimental Experimental –Total student=80/176 –Consented=68/153 –Completed=63/132 (lost 5 btw pre & post) – Final sample=63 Control Control –Total students=96/176 –Consented=85/153 –Completed=69/132 (lost 16 btw pre & post) – Final Sample=69

5. Sample Demographics Experimental Group Experimental Group –Race W=30 W=30 NW=33 NW=33 –Sex* M=17 M=17 F=46 F=46 –Average Age=18.5 –Average H.S. GPA=3.12 –Average SAT=1408 –Living Arrangements* Campus=41 Campus=41 Off-Campus=21 Off-Campus=21 Missing=1 Missing=1 –Note 2/3 of this group did not meet MSU admission standards. Control Group Control Group –Race W=42 NW=27 –Sex* M=27 F=42 –Average Age=18.5 –Average H.S. GPA=3.04 –Average SAT=1482 –Living Arrangements* Campus=26 Off-Campus=42 Missing=1

6. Caveats Regarding Analyses Received post-test data right before winter break Received post-test data right before winter break –These are preliminary analyses. We plan to add more data and proceed much more slowly before drawing any firm conclusions. –Concentrated on Well-being and Civic Development measures (not Academic development).

7. Student Well-being CES-D CES-D Life Satisfaction Survey Life Satisfaction Survey AUDIT AUDIT

8. Does service-learning impact the level of depression of students? No, it does not appear to No, it does not appear to CES-D CES-D –Step-wise regression equation Step 1: Regressed CES-D post-test on control variables (Sex, living arrangements, CES-D pre-test): R 2 =.40, p=.000; only CES-D pretest was significant (ß=.62, p=.000). Step 1: Regressed CES-D post-test on control variables (Sex, living arrangements, CES-D pre-test): R 2 =.40, p=.000; only CES-D pretest was significant (ß=.62, p=.000). Step 2: Entered group type (experimental vs control): ΔR 2 =.01, n.s. Step 2: Entered group type (experimental vs control): ΔR 2 =.01, n.s. Service-learning during the semester did not appear to make a difference in the level of depression of the students surveyed. Service-learning during the semester did not appear to make a difference in the level of depression of the students surveyed.

9. Does service-learning impact the life satisfaction of students? No, it did not appear to No, it did not appear to Life Satisfaction Life Satisfaction –Step-wise regression equation Step 1: Regressed Life Satisfaction post-test on control variables (sex, living arrangements, Life Satisfaction pre-test): R 2 =.41, p=.000; only Life Satisfaction pretest was significant (ß=.64, p=.000). Step 1: Regressed Life Satisfaction post-test on control variables (sex, living arrangements, Life Satisfaction pre-test): R 2 =.41, p=.000; only Life Satisfaction pretest was significant (ß=.64, p=.000). Step 2: Entered group type (experimental vs control): ΔR 2 =.00, n.s. Step 2: Entered group type (experimental vs control): ΔR 2 =.00, n.s. Service-learning during the semester did not appear to make a difference in the life satisfaction of the students surveyed. Service-learning during the semester did not appear to make a difference in the life satisfaction of the students surveyed.

10. Does service-learning impact the drinking habits of students? Yes, to a small extent Yes, to a small extent AUDIT AUDIT –Step-wise regression equation Step 1: Regressed AUDIT post-test on control variables (Sex, ethnicity, living arrangements, AUDIT pre-test): R 2 =.75, p=.000; only AUDIT pretest was significant (ß=.86, p=.000). Step 1: Regressed AUDIT post-test on control variables (Sex, ethnicity, living arrangements, AUDIT pre-test): R 2 =.75, p=.000; only AUDIT pretest was significant (ß=.86, p=.000). Step 2: Entered group type (experimental vs control): ΔR 2 =.01, p=.01, ß=.111, p=.01). Step 2: Entered group type (experimental vs control): ΔR 2 =.01, p=.01, ß=.111, p=.01). –Service learning during the semester did appear to make a difference in the drinking habits of the students surveyed. Drinking among students engaged in service learning slightly increased over the course of the semester.

11. Student Civic Development Created 3 sets of scales (from survey, using principle components analysis) Created 3 sets of scales (from survey, using principle components analysis) Behavior: Behavior: –General volunteering –Child-centered volunteering –Health Education volunteering –Engaged in discussion –Religious-based volunteering –Vulnerable population volunteering Attitudes: Attitudes: –General positive/negative –Blame the Victim Perceived Importance: Perceived Importance: –Social justice –Socio-economic –Education

12. Does service-learning impact student civic behavior? Yes. Yes. 6 behavior variables: General, Child-centered, Health Ed., Discussion, Religious-based, Vulnerable populations. 6 behavior variables: General, Child-centered, Health Ed., Discussion, Religious-based, Vulnerable populations. 6 separate stepwise regression equations 6 separate stepwise regression equations –Step 1: Regressed post-test behavior variables on control variables (Sex, living arrangements, and pre- test scores). –Step 2: Entered group type (experimental vs control)

13. Does service-learning impact student civic behavior? (continued) General volunteering* General volunteering* – Step 1: R2=.62, p=.000; only pretest was significant, ß=.76, p=.000 – Step 2: ΔR2=.17, p=.000; ß=.44, p=.000 Child-centered volunteering* Child-centered volunteering* – Step 1: R2=.21, p=.000; only pretest was significant, ß=.41, p=.000 – Step 2: ΔR2=.25, p=.000; ß=.51, p=.000 Health Education volunteering* Health Education volunteering* – Step 1: R2=.15, p=.000; living arrangement and pretest were significant, ßs=.18, p=.03 and.37, p=.000 – Step 2: ΔR2=.12, p=.000; ß=.35, p=.000 Engaged in discussion Engaged in discussion – Step 1: R2=.44, p=.000; only pretest was significant, ß=.65, p=.000 – Step 2: ΔR2=n.s. Religious-based volunteering Religious-based volunteering – Step 1: R2=.44, p=.000; only pretest was significant, ß=.67, p=.000 – Step 2: ΔR2=n.s. Vulnerable population volunteering Vulnerable population volunteering – Step 1: R2=.32, p=.000; only pretest was significant, ß=.57, p=.000 – Step 2: ΔR2=n.s.

14. Does service-learning impact the student attitudes toward civic issues? No, it does not appear to No, it does not appear to 2 attitude variables: General positive/negative and Blame the Victim. 2 attitude variables: General positive/negative and Blame the Victim. 2 separate stepwise regression equations 2 separate stepwise regression equations –Step 1: Regressed post-test attitude variables on control variables (Sex, Ethnicity, living arrangements, and pre-test scores). –Step 2: Entered group type (experimental vs. control) Neither of the 2 regressions were significant. Neither of the 2 regressions were significant.

15. Does service-learning impact the perceived importance of civic issues? No, it does not appear to No, it does not appear to 3 importance variables: Social justice, Education, and Socio-Economic. 3 importance variables: Social justice, Education, and Socio-Economic. 3 separate stepwise regression equations 3 separate stepwise regression equations –Step 1: Regressed post-test importance variables on control variables (Sex, Ethnicity, living arrangements, and pre-test scores). –Step 2: Entered group type (experimental vs control) None of the 3 regressions were significant. None of the 3 regressions were significant.

16. Discussion In our students, we saw clear changes in behaviors: In our students, we saw clear changes in behaviors: –Well-being: Slight increase in drinking (AUDIT). –Civic Development: Increase in volunteering behaviors including general, child-centered, and health education. In our students, we did not see changes in psychic measures: In our students, we did not see changes in psychic measures: –Well-being: No changes in Depression (CES-D) or Life Satisfaction. –Civic Development: No changes in attitudes or perceived importance.

17. Discussion (continued) Data makes sense. Data makes sense. Leads us to additional questions: Leads us to additional questions: –Will the behavior changes continue/evolve/progress/increase over time? –Will behavior changes lead to attitudinal changes?

18. Learnings Inclusion of the IRB in all phases of project development greatly improved research design and facilitated IRB approval. Inclusion of the IRB in all phases of project development greatly improved research design and facilitated IRB approval. Providing invited faculty members with sufficient information regarding the study and how it would benefit both the institution and the students facilitated recruitment of students. Providing invited faculty members with sufficient information regarding the study and how it would benefit both the institution and the students facilitated recruitment of students. Data was collected during class time, contributing to our high response rate. We had the biggest loss of students (6 students refused) in the one class where we collected data after class time. Data was collected during class time, contributing to our high response rate. We had the biggest loss of students (6 students refused) in the one class where we collected data after class time. A total incentivization of $20 ($10 for pre & $10 for post) greatly increased participation. A total incentivization of $20 ($10 for pre & $10 for post) greatly increased participation. Multiple reminders of the date on which the post-test would be distributed (flyers, class visits, emails) decreased attrition. Multiple reminders of the date on which the post-test would be distributed (flyers, class visits, emails) decreased attrition. We were surprised, and a bit disappointed, to learn students were not as interested as we expected in self-scoring the AUDIT, CES-D, and Satisfaction with Life measures. We were surprised, and a bit disappointed, to learn students were not as interested as we expected in self-scoring the AUDIT, CES-D, and Satisfaction with Life measures.

19. For More Information Contact: Contact: –Dana Natale nataled@mail.montclair.edu nataled@mail.montclair.edu nataled@mail.montclair.edu –Courtney Hopkins hopkinsc1@mail.montclair.edu hopkinsc1@mail.montclair.edu hopkinsc1@mail.montclair.edu –Valerie Sessa sessav@mail.montclair.edu sessav@mail.montclair.edu sessav@mail.montclair.edu