1. In search of engagement: Describing the drivers of secondary student science engagement in the United States and Finland Justin Bruner Michigan State University

2. Outline  Background and Research Questions  Defining Engagement Using Flow Theory  Methods  Experience Sampling Methods  Data and analysis  Between Country Differences  Correlations  T-Test of between country difference  Conclusion

3. Background  Much of the international comparative policy discussions have focused on two main themes:  Access MDG and EFA CCT programs But what happens when kids get to school?  Achievement PISA, PIRLS, TIMSS But there is more to learning than just a test score…  This work will focus on engagement with an emphasis on science classes

4. Research Questions  What is driving student engagement in United States and Finnish classrooms?  What are the differences in engagement between United States and Finnish students?  Out of School?  In school?  In Science?

5. Defining Engagement Using Flow Theory  Flow Theory (Csikszentmihalyi, 1990)  Applies to many domains of life  “Time flies by”  Try to capture the elements that lead to a flow state or the “optimal experience”  Increased feelings of challenge and skills in the task  “Challenge of activity”  “Your skills in the activity” Low to High (4 point) Source: Csikszentmihalyi, 1990

6. Defining Engagement  Also add Interest  Adapt Flow for school settings (Shernoff et al., 2003) Increased feelings of interest in the task –Concentration and Affect also a part of their model »Did not ask these “Is this activity interesting?” –Not at all to very much (4 point)  Therefore a student is considered engaged when they show increased levels of challenge, skill, and interest  Lab example of our student  How this scales up from student to teacher/classroom to school to country

7. Experience Sampling Method (ESM)  Capture daily routines and changes on an individual level  Random signaling to avoid “stacking the deck”  Scheduled science response  Make sure at least one science observation per day  Creates a nested data structure of responses within students  Need more power for classrooms, schools, subjects

8. Table 1 - Descriptive Statistics United States Within country percentFinland Within Country PercentTotal Number Students147112259 Number Beeps495434438397 Out School Beeps178636%195957%3745 In School Beeps316864%148443%4652 Science Beeps56511%57617%1141 Total Engaged Beep66515%51015%1175 Total Engage Out-Sch Beep22513%23612%461 Total Engage In-Sch Beep44014%27318%713 Total Engage Sci Beep6211%12422%186

9. Table 2 – Correlations with Engagement PartialSemipartPartial2Semipart2SigN US All Challenge0.4710.4330.2220.1880***4422 Skill0.3150.2690.0990.0720***4422 Interest0.2770.2330.0770.0550***4422 Finland All Challenge0.3950.3550.1560.1260***3325 Skill0.2640.2260.0700.0510***3325 Interest0.3120.2710.0970.0730***3325 US Out School Challenge0.5410.5060.2930.2560***1588 Skill0.2750.2250.0760.0500***1588 Interest0.2320.1880.0540.0350***1588 Finland Out School Challenge0.4580.4210.2100.1770***1882 Skill0.1980.1650.0390.0270***1882 Interest0.2780.2370.0780.0560***1882 US In School Challenge0.4350.3950.1890.1560***2834 Skill0.3340.2890.1110.0840***2834 Interest0.3010.2570.0900.0660***2834 Finland In School Challenge0.3240.2790.1050.0780***1437 Skill0.3270.2810.1070.0790***1437 Interest0.3720.3270.1390.1070***1437 US Science Challenge0.4600.4270.2120.1820***498 Skill0.3030.2620.0920.0690***498 Interest0.2790.2390.0780.0570***498 Finland Science Challenge0.3590.3120.1290.0970***558 Skill0.3280.2820.1070.0790***558 Interest0.3490.3020.1220.0910***558

10. Table 3 – t-test of between country differences US MeanFinland MeanUS SEFinland SET-valueP-value All Engagement0.1500.1530.0050.006-0.360.72 Challenge000.0150.01701 Skill000.0150.01701 Interest000.0150.01701 Out School Engagement0.1420.1250.0090.0081.410.159 Challenge-0.141-0.2330.0240.0212.840.005*** Skill0.036-0.0230.0250.0231.750.081* Interest0.0740.0170.0260.0241.620.104 In School Engagement0.1550.1900.0070.010-2.880.004*** Challenge0.0780.018 0.025-7.20<0.001*** Skill-0.0200.0280.0180.025-1.530.125 Interest-0.041-0.0220.0180.023-0.660.512 In Science Engagement0.1240.2220.0150.018-4.19<0.001*** Challenge0.1030.3140.0430.040-3.60<0.001*** Skill-0.0940.0150.0460.039-1.820.069* Interest-0.1330.0910.0430.035-4.06<0.001*** Notes: Means and standard errors are based on standardized person means. *=.10 significance, **=.05 significance, and ***=.01 significance.

11. Findings of analysis  Overall and out of school challenge is the biggest driver of engagement in both countries  In the United States it is same for in school and in science  In Finland it is interest in school and equal in science  In both countries students feel lower levels of challenge and skill out of school and lower levels of interest in school  US students show very low interest in science  Finnish students show higher levels of engagement in school, in science, and for each emotion in science  US students show higher levels outside of school but the difference is only significant for the emotions, not engagement

12. Future Steps for Analysis  More in depth exploration of other predictors of engagement  Explore and test other possible definitions  More exploration of other correlates of engagement  See what are the other related emotions when students are engaged (or not)  Look at student background and teacher characteristics  Gender, SES, teacher experience  Need to explore the nested structure of the dataset  Requires methodology that is not well known

13. Future Steps for Project  Collecting more detailed science teacher lesson data  Link it directly with the ESM data  Better understand what is happening in science classroom when students are engaged or not  Create a more explicit partnership with teachers  Show them their students data  Create some PD workshops utilizing that information

14. Thank you!  I appreciate your time and interest  Please feel free to contact me Justin Bruner Michigan State University brunerju@msu.edu

15. Literature Review  Why science?  Declining number of students are studying science beyond requirements globally (Osborne et al., 2003).  United States fills about 90% of H-1B Visas in STEM professions (Rothwell and Ruiz, 2013)  STEM jobs offer many employment premiums over other professions (Langdon et al., 2011)  What is happening?  Gender differences  Teacher differences  Quality of teaching  Curriculum  Cultural Diffences

16. Literature Review  Students see science as too difficult and have difficulty seeing it outside the classroom (Lavonen et al., 2008, Lavonen and Laaksonen, 2009, Osborne et al., 2003)  Girls generally experience science in more negative way than boys (Griffith, 2010, Osborne et al., 2003, Uitto et al., 2006, Reigel-Crumb and Moore, 2013)  Science teachers vary in quality  Connected instruction, academic rigor, lively teaching (Cooper 2013)  Cultural Differences (Tsai and Yang, 2011)  Asian students show higher performance but low interest in science. Cultural mediating factors.