A New Look at the Evaluation of Sociological Theories in International Large Scale Educational Assessments Daniel Caro and Andrés Sandoval-Hernandez CIES 2012 Annual Conference April 25, San Juan, Puerto Rico

Rationale Two important criticisms of LSA studies are Studies lack a solid theoretical basis This affects the selection, operationalization and explanation of the relations between variables e.g., extended use of SES indicators Techniques for developing and testing theories (EFA and CFA) have several limitations, for example: EFA lacks model fit indices CFA is too restrictive This study addresses these criticisms

Purpose Evaluate sociological theories of economic, cultural, and social capital postulated by Bourdieu, Bernstein, and Coleman Investigate if theoretical models are reflected by the data of international student assessments Demonstrate advantages of ESEM Explore factor structure Evaluate model fit Test MI across countries

Theoretical Framework Economic capital (Bourdieu, 1983) Economic capital is defined as the command individuals have over economic resources The concept is commonly understood as exchange values, like income and assets that can be easily transformed into cash Cultural capital (Bernstein, 1971; Bourdieu, 1983) Cultural capital is accounted by the cultural long-lasting dispositions embedded in the human mind and body, as well as in cultural goods and educational credentials Cultural capital can appear in three states: objectified, institutionalized, and embodied

Theoretical Framework Social capital (Coleman, 1988) Social capital are all those aspects of the social structure that can be used by actors as resources to achieve their interests Family social capital is represented by family relations that enhance the transmission of other structural resources (e.g. dispositions towards culture)

EFA and CFA In general, EFA for generating theory CFA for testing theory CFA limitations (Marsh, 2009, 2010) : The zero cross-loading restriction Statistically, leads to inflated interfactor correlations Also theoretically problematic, e.g. number of books reflects both cultural capital and economic capital Often well-defined EFA structures are not supported by CFA In practice researchers perform exploratory CFA

Exploratory Structural Equation Modeling (ESEM) Marsh et al. (2009, 2010) introduced ESEM, an integration of EFA and CFA Essentially, ESEM is EFA with Standard errors Model fit indices MI tests ESEM is appropriate if model fit is better than in CFA To our knowledge, ESEM has not been yet applied to sociological theories

Data PIRLS educational systems 21,000 students (500*42) 36 items from parent questionnaire PISA countries 7,000 students (500*14) 66 items from student and (optional) parent questionnaire

Analytical Strategy EFA with all items Selected items (indexes) for 3 factors 14 in PIRLS in PISA 2009 CFA and ESEM with selected items Model fit Interfactor correlations Cross-loadings Measurement invariance (MI) tests by country’s HDI

Children's book Parents' reading for enjoyment Parents' reading for enjoyment Parental attitudes to reading Parental attitudes to reading Parents' reading at home Books at home Visits to library / booksstore Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Childrens’ own room Financial status Parental education Cultural Capital Social Capital Economic Capital PIRLS 2006: CFA results CFI= 0.77 TLI=0.71 RMSEA=0.07

Children's book Parents' reading for enjoyment Parents' reading for enjoyment Parental attitudes to reading Parental attitudes to reading Parents' reading at home Books at home Visits to library / bookshop Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Childrens’ own room Financial status Parental education Cultural Capital Social Capital Economic Capital PIRLS 2006: ESEM results CFI= TLI= RMSEA=

PIRLS 2006: CFA and ESEM factor loadings

Home educational resources Cultural possessions Parents ' reading motivation Books at home Visits to library / bookstore Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Household overcrowding Parental education Cultural Capital Social Capital Economic Capital PISA 2009: CFA results CFI= 0.87 TLI=0.83 RMSEA=0.07

Home educational resources Cultural possessions Parents' reading motivation Books at home Visits to library / bookstore Academic communication Communication & Interaction Cultural communication Literacy support activities Parental ocupational status Household overcrowding Parental education Cultural Capital Social Capital Economic Capital PISA 2009: ESEM results CFI= TLI= RMSEA=

PISA 2009: CFA and ESEM factor loadings

Measurement Invariance Tests Two groups of countries PIRLS 2006: low/medium HDI versus high HDI PISA 2009: high HDI versus very high HDI

Discussion Evaluated theories of economic, cultural, and social capital are reasonably reflected by the data ESEM performs better than CFA to capture these theories Better model fit Greater discriminant validity (lower interfactor correlations) Cross-loadings are statistically and theoretically significant Educational/cultural possessions and parental education for cultural and economic capital Parent-child cultural interactions for social and cultural capital

Discussion But theoretical models seem not to hold between countries SES measures (economic capital) are not comparable This raises theoretical and methodological questions Theoretical Are other theories more appropriate? Global or contextualized theories? Methodological What should be the groups of comparison? Can ESEM manage comparisons among a large number of groups/countries?

Future Steps Theoretical Consider competing theories, e..g., rational action theory Reproduction theories for developing countries and rational action theory for developed countries? Methodological MI Identify problematic variables for MI Countrywise comparison matrix Evaluate MI within regions What to do if no MI?

Questions? Thank you for your attention! :)