1. A New Look at the Evaluation of Sociological Theories in International Large Scale Educational Assessments Daniel Caro and Andrés Sandoval-Hernandez CIES 2012 Annual Conference April 25, San Juan, Puerto Rico

2. Rationale Two important criticisms of LSA studies are Studies lack a solid theoretical basis This affects the selection, operationalization and explanation of the relations between variables e.g., extended use of SES indicators Techniques for developing and testing theories (EFA and CFA) have several limitations, for example: EFA lacks model fit indices CFA is too restrictive This study addresses these criticisms

3. Purpose Evaluate sociological theories of economic, cultural, and social capital postulated by Bourdieu, Bernstein, and Coleman Investigate if theoretical models are reflected by the data of international student assessments Demonstrate advantages of ESEM Explore factor structure Evaluate model fit Test MI across countries

4. Theoretical Framework Economic capital (Bourdieu, 1983) Economic capital is defined as the command individuals have over economic resources The concept is commonly understood as exchange values, like income and assets that can be easily transformed into cash Cultural capital (Bernstein, 1971; Bourdieu, 1983) Cultural capital is accounted by the cultural long-lasting dispositions embedded in the human mind and body, as well as in cultural goods and educational credentials Cultural capital can appear in three states: objectified, institutionalized, and embodied

5. Theoretical Framework Social capital (Coleman, 1988) Social capital are all those aspects of the social structure that can be used by actors as resources to achieve their interests Family social capital is represented by family relations that enhance the transmission of other structural resources (e.g. dispositions towards culture)

6. EFA and CFA In general, EFA for generating theory CFA for testing theory CFA limitations (Marsh, 2009, 2010) : The zero cross-loading restriction Statistically, leads to inflated interfactor correlations Also theoretically problematic, e.g. number of books reflects both cultural capital and economic capital Often well-defined EFA structures are not supported by CFA In practice researchers perform exploratory CFA

7. Exploratory Structural Equation Modeling (ESEM) Marsh et al. (2009, 2010) introduced ESEM, an integration of EFA and CFA Essentially, ESEM is EFA with Standard errors Model fit indices MI tests ESEM is appropriate if model fit is better than in CFA To our knowledge, ESEM has not been yet applied to sociological theories

8. Data PIRLS 2006 42 educational systems 21,000 students (500*42) 36 items from parent questionnaire PISA 2009 14 countries 7,000 students (500*14) 66 items from student and (optional) parent questionnaire

9. Analytical Strategy EFA with all items Selected items (indexes) for 3 factors 14 in PIRLS 2006 12 in PISA 2009 CFA and ESEM with selected items Model fit Interfactor correlations Cross-loadings Measurement invariance (MI) tests by country’s HDI

10. Children's book Parents' reading for enjoyment Parents' reading for enjoyment Parental attitudes to reading Parental attitudes to reading Parents' reading at home Books at home Visits to library / booksstore Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Childrens’ own room Financial status Parental education Cultural Capital Social Capital Economic Capital PIRLS 2006: CFA results 0.73 0.19 0.30 CFI= 0.77 TLI=0.71 RMSEA=0.07

11. Children's book Parents' reading for enjoyment Parents' reading for enjoyment Parental attitudes to reading Parental attitudes to reading Parents' reading at home Books at home Visits to library / bookshop Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Childrens’ own room Financial status Parental education Cultural Capital Social Capital Economic Capital PIRLS 2006: ESEM results 0.73 0.41 0.19 0.08 0.30 0.24 CFI= 0.77 0.96 TLI=0.71 0.93 RMSEA=0.07 0.03

12. PIRLS 2006: CFA and ESEM factor loadings

13. Home educational resources Cultural possessions Parents ' reading motivation Books at home Visits to library / bookstore Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Household overcrowding Parental education Cultural Capital Social Capital Economic Capital PISA 2009: CFA results 0.680.28 0.36 CFI= 0.87 TLI=0.83 RMSEA=0.07

14. Home educational resources Cultural possessions Parents' reading motivation Books at home Visits to library / bookstore Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Household overcrowding Parental education Cultural Capital Social Capital Economic Capital PISA 2009: ESEM results 0.68 0.38 0.28 0.17 0.36 0.15 CFI= 0.87 0.96 TLI=0.83 0.93 RMSEA=0.07 0.05

15. PISA 2009: CFA and ESEM factor loadings

16. Measurement Invariance Tests Two groups of countries PIRLS 2006: low/medium HDI versus high HDI PISA 2009: high HDI versus very high HDI

17. Discussion Evaluated theories of economic, cultural, and social capital are reasonably reflected by the data ESEM performs better than CFA to capture these theories Better model fit Greater discriminant validity (lower interfactor correlations) Cross-loadings are statistically and theoretically significant Educational/cultural possessions and parental education for cultural and economic capital Parent-child cultural interactions for social and cultural capital

18. Discussion But theoretical models seem not to hold between countries SES measures (economic capital) are not comparable This raises theoretical and methodological questions Theoretical Are other theories more appropriate? Global or contextualized theories? Methodological What should be the groups of comparison? Can ESEM manage comparisons among a large number of groups/countries?

19. Future Steps Theoretical Consider competing theories, e..g., rational action theory Reproduction theories for developing countries and rational action theory for developed countries? Methodological MI Identify problematic variables for MI Countrywise comparison matrix Evaluate MI within regions What to do if no MI?

20. Questions? Thank you for your attention! :)