Shudong Wang NWEA Liru Zhang Delaware Department of Education Exploring Impact of Student Perceptions of Family, School, and Individual Factors on Student Achievement Gap Shudong Wang NWEA Liru Zhang Delaware Department of Education Paper presented at the NCSA National Conference on Student Assessment June 20-22, 2016, Philadelphia, Pennsylvania.

I. Introduction Achievement Gap (AG) * Indicates statistically significant difference Male Female Two-Sample t statistics* Content N Mean Std Dev Read 3934 53.1 11.8 4076 56.4 10.9 -13.03 Math 41.7 15.9 40.6 3.04 Science 36.5 12.4 0.19 Social S 27.3 11.5 -0.07 African American White Two-Sample t statistics* Content N Mean Std Dev Read 2599 50.0 11.5 4619 57.6 10.4 -27.59 Math 33.1 13.8 45.7 15.0 -36.34 Science 30.0 10.7 40.3 11.1 -38.53 Social S 22.0 9.9 30.3 -32.67

Major factors that affect AG In general, AG is a historical, economical, and sociopolitical phenomenon More specifically, there are family, school, and individual student factors

Some of Variables Classified as School, Family, and Student Factors School Factors: School funding, school safety and disciplinary, high mobility, student/teacher ratio, percent students receiving free/reduced lunch, curriculum content and instructional materials, classroom activities, teacher’s perception (expectation, experience, and behavior), teaching methods and instruction, peer pressure, etc. Family Factors: Family background (parental schooling, income, single-parent families), parenting strategies and participation, parent encouragement and support, poverty, substandard housing, poor nutrition, hunger, and low birth weight, high violence and substance abuse, etc. Student Factors: Peer culture, social network, work ethic, motivation, effort, belief , gender, television watching, the “Summer Effect”, etc.

Effects Causes Student perceptions of family factor, Rest of Factors Effects Causes High mobility Parental income School funding Hunger Teaching methods and instruction Achievement Parenting strategies, participation Student/teacher ratio Motivation Single-parent families Parent encouragement and support Social network Curriculum content and instructional materials Teacher’s perception Work ethic Classroom activities Parental schooling Peer pressure Belief Substandard housing High violence and substance abuse School safety and disciplinary Poor nutrition Student perceptions of family factor, school factor, and student factor Percent students receiving free/reduced lunch Peer culture Summer Effect Low birth weight Effort Poverty Television watching

Purposes of This Study To investigate the effect of student perceptions of family, school, and student factors on a statewide achievement test by gender and race.

Research Questions: Do student perceptions of family, school, and student factors affect their overall achievement measured by Reading, Math, Science, and Social Science? Is the effect the same for Caucasian and African American students? Is the effect the same for Male and Female students?

II. Method 1. Data source and instrument A sample of 7980 grade 8 students scores on reading, math, science, and social science were collected from a statewide summative assessment that was given in conjunction with the student survey questions. There are 22 questions in the survey; 21 questions used either a 3-point or 4-point Likert scale to indicate frequency and degree of agreement with each survey statement. Survey questions were classified into three overlapping categories: Family Factor (e.g., encouragement, support), Student Factor (motivation, effort), and School Factor (e.g., classroom activities, instructional materials, curriculum content).

2. Data analysis The theoretical model tested in this study specified causal relationships between achievement and several latent variables, such as family, school, and student (motivation and effort) factors by using Structural Equation Modeling (SEM). Both Confirmatory Factor Analysis (CFA) and Structural Equation Modeling (SEM) methods was used in this study to analyze relationships among several latent constructs across gender and race. CFA: A Measurement Model specifies relationships between the latent constructs and their indicator variables and for both overall achievement results (4 content areas) and survey (21 questions). SEM: A Structural Model (Causal model) specifies causal relationships between the latent constructs themselves. SEM can simultaneously test and determine whether Measurement Model + Structural Model (combined model) as a whole, provides an acceptable fit to data. All data were analyzed using Proc Calis (SAS Institute, Cary, NC).

Figure 1. Measurement Model for Achievement eR1 Reading R1 eR1 R1 eR1 M1 eM1 M1 eM1 Math M1 eM1 M1 eM1 Achievement S1 eS1 S1 eS1 Science S1 eS1 S1 eS1 Sub-Test Scores O1 eO1 O1 eO1 SOS O1 eO1 O1 eS1

Figure 2. Measurement Model for Student Factor eMo6 Mo10 eMo10 Motivation Mo16 eMo16 eM22 Mo22 Student Factor Survey Question Ef9 eEf9 Ef15 eEf15 Effort Ef20 eEf20 eEf21 Ef21

Figure 3. Measurement Model for Family and School Factors F1_r1 EF1 F1 EF1 F2_r2 EF2 F2 EF2 F3 EF3 F3 EF3 Family School F4 EF4 F4 EF4 F5 EF5 F5 EF5 F17 EF17 F17 EF17 Survey Question

Figure 4. Structural Model Regression weight or Standardized regression coefficient Family Motivation Achievement Student Effort School

Figure 5. SEM model (Combined Model)

III. Results 1. CFA results for survey Table 1 presents the rotated factor pattern and loadings of each survey questions on a given factor.

Table 1. Varimax - Rotated Factor Loading (Pattern)* Question number Factor1 Factor2 Factor3 Factor4 Student Motivation 10 83 * 4 2 3 Student Motivation 16 81 * 8 2 4 Student Motivation 22 77 * 7 4 5 Student Motivation 6 41 * 20 36 * 5 Student Effort 9 5 79 * 8 7 Student Effort 15 13 72 * 7 11 Student Effort 21 2 69 * 2 7 Student Effort 20 10 33 * 3 27 * Family 7 -2 4 66 * -2 Family 1 16 11 61 * 6 Family 3 19 13 53 * 1 Family 13 2 -15 45 * 27 * Family 5 4 -7 45 * 13 Family 17 2 11 30 * 10 Family 2 20 23 26 * -5 Family 4 9 -3 29 * 7 School 12 4 2 5 77 * School 8 1 -3 9 75 * School 11 11 34 * 8 50 * School 18 -1 11 9 30 * School 19 2 3 -10 11 *: Values are multiplied by 100 and rounded to the nearest integer. Values greater than 0.25 are flagged.

2. Structural Equation Modeling 2.1 First baseline model without concern on across group was tested 2.2 Then baseline model were tested across gender (Male vs. Female) and ethnicity (White vs. Africa American). Table 2 shows the fit results between data and model is acceptable. Table 2. Goodness-of-Fit of Model* Model df 2 GFI NFI SRMR RMSEA Overall 517 8637.1 .94 .93 .07 .04 Male 4283.5 Female 4557.8 White 5278.5 African American 2680.4 .06 *: All 2 statistics are significant at α = 0.01.

Table 3. Standardized Effects (Absolute Value) on Overall Achievement Group Path Estimate Standard Error t Value Pr>|t| R-Squared Overall Student -0.04 0.05 -0.92 0.36 0.10 Family 0.30 0.04 6.76 <.00 School 0.13 0.02 8.09 Male -0.06 0.06 -0.97 0.33 0.08 0.26 4.35 0.15 6.57 Female 0.34 0.73 0.14 0.31 5.33 0.12 5.27 White -0.08 -1.07 0.28 0.07 4.69 0.03 1.16 0.25 African American 0.42 0.67 2.21 0.09 3.50 Among 3 factors (Family, School, Student), only two factors (Family, School) have significant direct effects on achievement. For example, for 1-point increase on Family standard deviation, one can expect, on average, an increase of 0.30 on Achievement standard deviation for overall group. 2. Family factor has the most impact on Achievement. 3. All three factors explain about 10.3% of the variability on student’s Achievement.

IV. Conclusions 1. The results of this study suggest that Family factor and School factor defined by this study have statistically significant impact on students overall Achievement. 2. All three factors explain about 10.3% of the variability of student’s achievement for whole group. 3. Among three factors, the Family factor has the largest effect on overall Achievement. 4. These results may vary across different groups.

Thank you ! For any question: Shudong.wang@NWEA.org