1. Modelling Mathematics Achievement of Ontario’s Francophone Students TIMSS-R 1999 Researchers: Marielle Simon Renée Forgette-Giroux Assistants: Nathalie Loye, Sarah Plouffe, Robin Tierney, Danielle Higgins 2005 CESC – SSHRC Symposium

2. The Issue Francophone students in minority language settings in Canada, particularly in Ontario, tend to achieve results significantly below those of anglophone students on national and international assessments. Francophone students in minority language settings in Canada, particularly in Ontario, tend to achieve results significantly below those of anglophone students on national and international assessments.

3. Research Questions 1.Are the following factors associated with the achievement of francophone students in a minority language setting on the TIMSS-R 1999? a) Student characteristics b) Teaching practices c) Assessment practices d) Use of information and communications technologies (ICT) 2.What are the interactions among these factors and the students’ achievement in mathematics on TIMSS-R 1999?

4. Study Framework Student Characteristics Information and Communications Technology Teaching and Assessment Practices Achievement in Mathematics

5. Study Variables Student mathematics achievement on the TIMSS-R 1999 (1) * Student mathematics achievement on the TIMSS-R 1999 (1) * Variables taken from the background questionnaires of teachers and students Variables taken from the background questionnaires of teachers and students  Student characteristics (38)  Teaching practices (51)  Assessment practices (19)  Access to, and use of, information and communications technologies (13) * Number of variables

6. Study Sample Initial SampleFinal Sample Grade 7 or 8 students in Ontario’s French- language schools 1186683 Mathematics teachers associated with the students sampled 7536

7. Data Analysis Techniques Used I. Logical analysis of the questionnaires II. Descriptive analysis III. Exploratory factor analysis IV. Confirmatory factor analysis V. Structural equation modelling VI. Regression analysis

8. Results Confirmatory Factor Analyses Factor 1 Expectations (mother, peers) 6 variables Factor 2 Educational level of parents 2 variables Factor 3 Success attributed to beliefs (ability, luck) 6 variables Factor 4 Access to a computer at home 3 variables Factor 5 Attitude toward mathematics 2 variables Student Characteristics Excellent fit…! Fit indices: CFI: 0.965 * TLI: 0.975 RMSEA: 0.070 ** * CFI > 0.95 ** RMSEA < 0.08

9. Factor 1 Access to ICT (at school) 2 variables Factor 2 Access to ICT (in classroom) 2 variables Information and Communications Technologies Reasonable fit …! Fit indices: CFI: 0.998 TLI: 0.996 RMSEA: 0.191 Results Confirmatory Factor Analyses

10. Factor 1Expectations (mother, peers) 7 variables Factor 2Educational level of parents 2 variables Factor 3Success attributed to beliefs (ability, luck) 2 variables Factor 4Success attributed to hard work 2 variables Factor 5Access to a computer at home 3 variables Factor 6Attitude toward mathematics 6 variables Factor 7Access to ICTs (school)2 variables Factor 8Access to ICTs (home)2 variables Student Characteristics and Use of ICTs Excellent fit…! Fit indices: CFI : 0,981 TLI : 0,975 RMSEA : 0,062

11. Results Structural Equation Modelling CFI: 0.979 TLI: 0.985 RMSEA: 0.064 Expectations of mother Expectations of peers Expectations of student Employment expectations University expectations Math for pleasure Importance in life Mother’s education level Father’s education level Educational level of parents Being gifted Having good luck Success attributed to beliefs Much work Memorization Success attributed to hard work Computer at home Modem at home Internet at home Computer technology at home Likes mathematics Math is easy Good at math Math is hard for me No talent for math Not my best subject Attitude toward mathematics Computer (classroom) Software (classroom) Computer (school) Software (school) ICT at school R 2 : 0.515 Achievement in Mathematics 0.522 -2.841 46.126 -4.389 14.020 22.644 10.297 6.333 Student Characteristics and Use of ICTs Expectations and values ICT in the classroom

12. Results Confirmatory Factor Analyses Teaching Practices Poor fit… Fit indices: CFI: 0.812 TLI: 0.840 RMSEA: 0.127 Factor 1 Traditional types of tasks (homework) 4 variables Factor 2 Alternative types of tasks (homework) 6 variables Factor 3 Working alone or with teacher 3 variables Factor 4 Alternative teaching methods 4 variables Factor 5 Traditional teaching methods 3 variables Factor 6 Working in a large group 2 variables

13. Results Confirmatory Factor Analyses Results Confirmatory Factor Analyses Assessment Practices Poor fit… Fit indices: CFI: 0.812 TLI: 0.840 RMSEA: 0.205 Factor 1 Traditional assessment methods 3 variables Factor 2 Alternative assessment methods 4 variables Factor 3 Assessment for learning 2 variables Factor 4 Assessment of learning 3 variables

14. Frequency of homework Exercises Problems from the textbook Reading the textbook Traditional types of homework Project in groups Real applications Alternative types of homework Small research project Working alone/with teacher Computer Working alone or with the teacher Work in large group Large group interactions Working all together in a group Computer used by teacher Work in small groups Alternative teaching methods* Teacher explains rules Teacher gives examples Achievement in Mathematics 25.464 -46.204 6.8 -9.237 -13.678 10.27 Results Structural Equation Modelling Teaching Practices Length of homework Individual project Oral reports Keeping a diary Projects in small groups Overhead projector - student Teacher explains problem Traditional teaching methods* * As perceived by the student R 2 : 0.294 CFI: 0.805 TLI: 0.826 RMSEA: 0.128

15. Results Regression Analyses All the regression analyses done on the teaching practices variables confirm the results of the structural equation modelling analyses. When compared, the following factors show contrary effects: Traditional type of homework (Textbook problems, reading the textbook, exercises, etc.) Alternative type of homework (Projects, real applications, oral reports, keeping a diary, etc.) Traditional teaching methods (Lectures by the teacher, explanations by the teacher, etc.) Alternative teaching methods (Working on projects, group work, etc.)

16. Conclusions The CFA and SEM models combining student characteristics and ICT use result in excellent fit indices. These models would explain 52% of Ontario’s French- language minority students’ achievement in mathematics on TIMSS-R 1999. The factor that has the most influence on student achievement is students’ attitudes toward math followed by the attribution of success to beliefs (ability, luck). Traditional and alternative instructional practices seem to have an opposite impact on student achievement in math (to be continued…).

17. Study Limitations Sample size Background questionnaires Problems associated with the analysis of categorical variables Distribution of variables (asymmetry and kurtosis) Little variance in the achievement variable for some classes due to their small size Nested effect of variables not taken into account with these types of analyses

18. On a more positive note… Useful and significant results Presentations of results at various conferences Contribution to the pool of knowledge on large- scale assessments Establishment of a learning assessment research unit

19. Dissemination Higgins, D., Plouffe, S., Simon, M., Forgette-Giroux, R. (mai, 2004). Analyse logique des évaluations à grande échelle en mathématiques en contexte francophone minoritaire au Canada. Communication présentée au congrès de l’ACFAS,Montréal. Plouffe, S., Tierney, R.D., & Turcotte, C. (April, 2005). Impact of classroom teaching and assessment practices and the use of technology on the achievement of French-language minority students in Ontario on the 1999 TIMSS- R. Poster presentation at the National Council of Measurement in Education (NCME) Graduate Student Poster Session in Montreal. Simon, M. & Plouffe, S. (avril, 2005). Réflexion sur le processus d’analyse des données d’évaluation à grande échelle. Conférence au Colloque International sur la Recherche en Éducation Minoritaire (CIREM), Ottawa.

20. Dissemination (cont’d) Plouffe, S., Simon, M., & Loye, N. (mai, 2005). Facteurs déterminants sur le rendement en mathématiques des élèves francophones minoritaires de l’Ontario aux études à grande échelle – Premiers résultats. Conférence à la Société Canadienne pour l’Étude de l’Éducation (SCÉÉ), London, Ontario. Simon, M, & Forgette-Giroux, R. (July, 2005). Modeling the effects of teaching practices on French-language minority student achievement on large-scale assessment programs: Findings and Issues. Paper submitted for presentation at the Education Assessment Association of the Americas (EAAA), Brasilia. Plouffe, S., Turcotte, C., Simon, M, & Forgette-Giroux, R. (Novembre, 2005). Limites des questionnaires contextuels lors de l’analyse de données des programmes nationaux et internationaux d’évaluation du rendement. Communication prévue à la rencontre annuelle de l’Association pour le Développement de la Mesure et de l’Évaluation en Éducation (ADMÉÉ), Québec.