1. Examining Teacher Quality and Teacher Preparation for English Language Learners: the Use of Multiple Data Sources Maria Coady, Alice Jo, Ramandeep Brar & Yang Qi Florida Association of Teacher Educators October 8, 2011

2. Overview of ELL Preparation Nationally, most teachers are inadequately prepared to teach ELLs (Gándara et al., 2005; Walker et al., 2004) –Karabenick & Noda (2004) report that teachers lack basic foundational knowledge about ELL issues, despite the fact that 88% teach ELLs – –Colon-Muniz, Brady and SooHoo (2010) reported that teachers who had been prepared to teach ELLs at universities performed better in their practice in teaching ELL students. Florida’s requirements to prepare mainstream teachers of ELLs (through in-service since 1990, and pre-service since 2001) are unique.

3. ELLs in Florida Context FDOE reported 231,801 ELLs in 2009-2010 (8.8% of total enrollment) Additionally, FDOE reported 204,287 former ELLs in 2009-2010 (7.7% of total enrollment) Most ELLs (66%) are enrolled in the elementary grades, including 40% enrolled in grades K-2 (2009-2010) Source: http://www.fldoe.org/aala/omsstat.asp http://www.fldoe.org/aala/omsstat.asp

4. ELL Achievement in Florida

5. The Florida Context: Preservice Teacher Preparation Requirements Teacher education programs have prepared candidates through an “infused” ESOL endorsement program since 2001 – –Minimum of 2 ESOL stand-alone courses taught by ESOL faculty – –ESOL Performance Standards are addressed and assessed in the program – –ESOL content is “infused” in coursework throughout the program – –Field experience requirement (varies by course)

6. Project DELTA (Developing English Language and Literacy Through Teacher Achievement) Federally funded, post-training assessment grant to examine the impact of an ESOL-infused elementary education program on teacher practices and ELL achievement through –Quantitative data from Education Data Warehouse (EDW) linking teacher and student files –Survey of UF graduates’ sense of preparedness and efficacy in teaching ELLs –Interviews with UF teacher graduates –Case study observations in elementary, mainstream math and reading classes and follow up interviews with those teachers

7. DELTA Research Questions How (and how well) do teacher graduates from a (ESOL infused) teacher preparation program (ProTeach) work with ELLs? A. What practices do teacher graduates use to facilitate learning (language, literacy & content) for ELLs in their classrooms? (Case Studies) B. What do teacher graduates believe they have learned in their teacher preparation program? (Survey, Interviews) C. How well do English learners in the classrooms of UF’s teacher graduates (across various preparation pathways) perform on 3 rd grade state standardized tests? (EDW Metadataset)

8. Theoretical Framework: Quality Teachers of ELLs

9. Teacher Quality and Teacher Effectiveness Terms frequently used interchangeably Current climate to associate teachers with student performance as one method of teacher evaluation (quantitative measures) Annual Evaluation 100%=50% student growth + 50% in- person evaluation 50% instructional evaluation = 20% teacher growth + 30% teacher status score

10. Our Focus Today... Present findings from a large-scale, mixed methods study using data from Florida’s Educational Data Warehouse (EDW), case study teachers of ELLs, program graduates (survey and interviews); Discuss the use of mixed methods to explore quality teachers of ELLs; Demonstrate how various methods illuminate specific characteristics of quality teachers of ELLs.

11. Mixed Methods Both quantitative (EDW, survey) and qualitative (observations, interviews, document analysis, survey) methods are used in data collection and analysis. –EDW –Survey –Observations –Interviews –Documents

12. The Education Data Warehouse Only a handful of states have statewide datasets matching students to teachers for any length of time (NC, WA, NY) Florida Department of Education has developed the EDW as a large dataset related to schools in Florida http://edwapp.doe.state.fl.us/home.aspx http://edwapp.doe.state.fl.us/home.aspx

13. EDW Metadataset www.edwapp.doe.state.fl.us/home.aspx www.edwapp.doe.state.fl.us/home.aspx

14. Final Dataset for Analysis -Non-ESE, non-retained ELLs, not enrolled in ESOL course, in grades 3, 4, and 5 with complete FCAT and enrollment data matched to –Individual non-UF teachers (n= 71,194 students; 23,985 teachers) –Individual UF teachers (n=1,100 students; 358 teachers)

15. DELTA – EDW Findings Do different UF teacher preparation paths make a significant difference in ELL student achievement? UF teacher preparation pathway codes (n=12) were aggregated into four groups: –A [5 Year ESOL Endorsement] –B [5 Year ESE + ESOL Endorsement] –C [5 Year No ESOL Endorsement] –D [4 year Bachelor’s degree]

16. DELTA -- EDW Findings Student Mean FCAT Score UF Teacher Preparation Paths MathReading A: 5 Year ESOL306.42 * 282.64 * B: 5 Year ESOL + ESE310.42294.52 C: 5 Year No ESOL291.53 * 270.64 * D: 4 Year Bachelor’s307.26278.82

17. Survey Methods and Design Five sections of the survey with 10-12 items in each (total 49 statements)  Social and cultural dimensions of teaching for ESOL students  Content area instruction  Language and literacy development  Curriculum and classroom organization  Assessment issues in teaching ESOL students

18. Survey

19. Survey Questions 1. In what instructional areas related to ELLs do teacher education program graduates feel most prepared? Least prepared? 2. In what instructional areas related to ELLs do teacher education program graduates feel most effective? Least effective?

20. Survey Findings Teachers feel most prepared and effective: Teachers feel most prepared and effective: a. creating a welcoming and affirming (valued) atmosphere in the classroom b. using graphic organizers Teachers feel least prepared and effective using students’ first language as a resource for learning (bilingual materials, home resource) Teachers feel least prepared and effective using students’ first language as a resource for learning (bilingual materials, home resource) Implications for the development of bilingualism and learning theory (connecting background to new learning) Implications for the development of bilingualism and learning theory (connecting background to new learning)

21. Interviews 19 teacher-graduates, conducted by telephone, explored teachers’ beliefs –Interview data (n=19) Criteria for participation: –Graduate of UF elementary education program with infused ESOL endorsement (two stand-alone ESOL courses completed) –Two or more years of teaching experience – Experience teaching elementary ELLs Interview Questions: - ESOL Preparation - ESOL Preparation - Languages Other Than English (LOTE) - Languages Other Than English (LOTE) - Program Recommendations - Program Recommendations

22. Interview Findings Overall positive evaluation of the program (all but 1 graduate felt positive about their preparation, especially compared with some of their colleagues) Emphasis on central role of field experiences in building confidence and developing competence in teaching ELLs Recommendations to expand field experiences, ensure elementary field placements, connect theory and practice, and provide access to teaching resources Importance of LOTE proficiency and/or experiences to connect with ELLs and make classrooms more comfortable and accessible

23. Case Studies Teacher Cases(n=6) teacher graduates in 5 school districts in north/central Florida Graduated from an ESOL-infused pre-service teacher preparation program Classroom observation (math & reading over 18 month period) Classroom observation (math & reading over 18 month period) –Audio, video, observation protocol, material artifacts, photos

24. Case Studies Findings

25. Case study teachers – observed no interactions in which teachers planned for ELLs and made specific modifications based on language proficiency level of the ELL; Teachers’ generally addressed language demands of classroom/text/activities in a spontaneous manner: –Feedback, Questioning, Instruction, Management Initiation, Response, Evaluation (IRE) pattern of language use Preliminary findings of two case study teachers

26. Case Study Teacher

27. Use of Mixed Methods in Determining Teacher Quality Different research methodologies are complementary, such that they paint a more holistic picture of teacher graduates’ work, beliefs, and effectiveness with ELs. The methodologies used highlight distinct findings to answer the research questions The methodologies may not provide enough answers (“why” or “how”); other methods may need to be employed

28. Conclusion Multiple methods paint a picture that addresses questions of quality teachers of ELLs that may not be evident by simply using quantitative (assessment) data; Multiple methods seek to address not only relationships between variables (teachers and student) but illuminate how and why; Multiple methods allow for improvement in teacher education and programmatic feedback.

29. Questions? And Contacts Maria Coady, Ph.D. mcoady@coe.ufl.edu Project DELTA website http://education.ufl.edu/project-delta/