Adolescent Literacy and Academic Behavior Self-Efficacy Survey (ALAB) Sharon deFur, Virginia SPDG Evaluation March 2013

Virginia’s SPDG School wide initiative focused on improving adolescent literacy based on the KU-CRL Content Literacy Continuum Began with 4 schools: 2 middle and 2 high schools School-based literacy teams Intensive all Faculty and Administrator Professional Development on intervention(s) including follow-up, coaching, implementation monitoring by administrators and coaches, data collections School-wide student assessment to identify students in need of interventions beyond classroom routines Teacher evaluation expectations for evidence of use of interventions Opportunities for career advancement with additional professional development for teachers School wide initiative focused on improving adolescent literacy based on the KU-CRL Content Literacy Continuum Began with 4 schools: 2 middle and 2 high schools School-based literacy teams Intensive all Faculty and Administrator Professional Development on intervention(s) including follow-up, coaching, implementation monitoring by administrators and coaches, data collections School-wide student assessment to identify students in need of interventions beyond classroom routines Teacher evaluation expectations for evidence of use of interventions Opportunities for career advancement with additional professional development for teachers March 2013

Adolescent Literacy & Academic Behavior Self- Efficacy Survey (ALAB) deFur & Runnells, 2011, Runnells & College of William and Mary, 2012 Goals – To develop a validated student self-efficacy measure linked to reading and writing literacy achievement enabling academic behaviors Intent – To provide SPDG schools and teachers with a literacy self-efficacy measure that could be used to identify change in student literacy confidences and motivation Result – 28 item survey for use Self-report from 0 – 9 reflecting self confidence Goals – To develop a validated student self-efficacy measure linked to reading and writing literacy achievement enabling academic behaviors Intent – To provide SPDG schools and teachers with a literacy self-efficacy measure that could be used to identify change in student literacy confidences and motivation Result – 28 item survey for use Self-report from 0 – 9 reflecting self confidence March 2013

Why Self-Efficacy Focus? Positive self-efficacy These students work harder, persist longer, persevere in the face of adversity, have greater optimism and lower anxiety, and achieve more. The degree of self-efficacy explains more than 25% of the variance in the prediction of academic performance (Pajares, 2006). March 2013

Pajares (2006) defined self-efficacy as the belief and confidence students hold about their ability to succeed at a given task. Important to this construct is that self-efficacy is context specific. Self-Efficacy refers to confidence in, or beliefs about, one’s ability to perform a skill and can result in a self-fulfilling prophecy (Hunter & Csikszentmihalyi, 2003; Pajares, 2006; Schunk & Meece, 2006 ). Self-efficacy strengthens as a function of the continuum of skill development from basic to mastery (Lodewyk & Winne, 2005 ). Self-efficacy is widely used to predict and explain student achievement (Feldman, Kim, & Elliott, 2011; Mucherah & Yoder, 2008; Pajares, Johnson, & Usher, 2007; Phan, 2011) March 2013

Study 1 - Validation of the Adolescent Literacy and Academic Behavior Self-Efficacy Survey (ALAB) Development and Site Selection /Analyses Processes DEVELOPMENT – Developed based on self- efficacy assessment process from the work of Pajares & Urdan (2006) and Bandura (2006) as well as the work on adolescent literacy of Scammacca et al. (2007), Torgesen et al. (2007), Deshler & Hock (2007), KU-CRL CLC. – Field tested with 11 youth – Expert review of survey – 28 item survey DATA COLLECTION – Surveys sent to schools with a request to collect information from a representative sample – Data entered into SPSS – Data analyses included descriptive statistics, principal component factor analyses, and reliability analyses

Response Scale Scale from 0 to 9 Thinking about school-related tasks in any classroom, how confident are you that you can… March 2013 Not sure I can do this Maybe I can do this Pretty sure I can do this Real sure I can do this

Comparison of validation studies Study 1 (deFur & Runnells, 2011) 28 item survey with 0 – 9 scale, no items dropped out N = 271, grades 6 – 11, eight schools, all participating in the SPDG literacy intervention at varying levels of implementation stages Near equal distribution of males and females Non-random selection & Non- controlled participation >8% self-reported having IEPs, no data on race/ethnicity Reliability.96 for Total Self- Efficacy Factor analysis yielded four correlated factors 28 item survey with 0 – 9 scale, no items dropped out N = 271, grades 6 – 11, eight schools, all participating in the SPDG literacy intervention at varying levels of implementation stages Near equal distribution of males and females Non-random selection & Non- controlled participation >8% self-reported having IEPs, no data on race/ethnicity Reliability.96 for Total Self- Efficacy Factor analysis yielded four correlated factors Study 2 (Runnells, 2012) 28 item survey with 0 – 9 scale N = 1110, grades 7-9, one SPDG participating school division that did not participate in Study 1; in initial implementation stage of intervention – 17% ELL – ~13% SWD ~ 20% ELL Near equal distribution of males and females 39% Hispanic; 32% White; 15% Black; 5% Asian; 5% Other Reliability.98 for Total Self-Efficacy Factor analysis yielded five correlated factors (original four plus one new factor) Correlation with literacy measure 28 item survey with 0 – 9 scale N = 1110, grades 7-9, one SPDG participating school division that did not participate in Study 1; in initial implementation stage of intervention – 17% ELL – ~13% SWD ~ 20% ELL Near equal distribution of males and females 39% Hispanic; 32% White; 15% Black; 5% Asian; 5% Other Reliability.98 for Total Self-Efficacy Factor analysis yielded five correlated factors (original four plus one new factor) Correlation with literacy measure March 2013

Factor Analysis – Study 2; N = 1,110 (Runnells & College of WM, 2012) Total Efficacy M=6.3 Reading M=6.3 Writing M=6.4 Application M=6.0 Self- Regulation M=6.7 Strategic Learning M=5.9 March 2013

Factor Survey Item Examples 1.READING – Read my textbooks 2.WRITING – Write good sentences 3.APPLICATION – Use diagrams or pictures to remember what I am learning 4.SELF-REGULATION – Complete my homework on time 5.STRATEGIC LEARNING – Ask questions in class March 2013

Student Literacy Perceptions by Program Classification (Runnells & WM, 2012) March 2013

ALAB Precautions & Considerations Significant correlation (p<.05) with the Measure of Academic Progress (MAP) and Reading and Writing Self-Efficacy, but low effect sizes Determined internal validity and reliability, but not test re-test reliability Have not yet used longitudinally to assess utility in intervention evaluation Maturation and time in school impacts self-efficacy – overall, 9 th grade students rated themselves as more confident than 8 th, 8 th grade students were more confident than 7 th grade students Students with disabilities (SWD) and Students who are English Language Learners (ELL) expressed lower self-efficacy for some factors (total self-efficacy, writing self-efficacy, & reading self- efficacy) Students who were ELL, but not disabled, had self-efficacy scores comparable to SWD March 2013

Disclaimer Funding for this project was partially provided by a grant from the Virginia Department of Education and the USDOE Virginia‘s State Personnel Development Grant # H This project was found to comply with appropriate ethical standards and was exempted from the need for formal review by the College of William and Mary. The opinions expressed herein do not reflect those of the Virginia Department of Education or the USDOE. March 2013