1. Improving gender, racial, and social equity in elementary science instruction and student achievement: The impact of a professional development program RHETA E. LANEHART KATHRYN M. BORMAN THEODORE BOYDSTON BRIDGET A. COTNER REGINALD S. LEE ALLIANCE FOR APPLIED RESEARCH IN EDUCATION AND ANTHROPOLOGY UNIVERSITY OF SOUTH FLORIDA

2. Overview of the Study Replication and Outcomes of the Teaching SMART Program in Elementary Science Classrooms (US DOE IES)  Design: Randomized Control Trial  Sample:  10 treatment; 10 control schools  Approximately 300 teachers and 9000 students  Mixed method Purpose of the study  To document the efficacy of a professional development program

3. Overview of the Teaching SMART Program RESOURC ES TS project staff 3 yr research- based PD model 100+ lesson plans aligned curriculum and based on standards science supplies and equipment ACTIVITI ES Train the trainer model In-class visits Mirror coaching Ongoing technical assistance Formal evaluation OUTPU TS Beginning, Intermedia te, and Advanced levels 90 hours of PD for Site Specialists 60 hours of PD for teachers SHORT- TERM OUTCOMES ↑ student- centered learning ↓ teacher- centered learning Equity-based teaching strategies IMPACT More students, females and minorities, pursuing STEM studies and STEM careers

4. Focus of this Analysis Evaluation  Equity  Gender, racial, social  Student attitudes toward science  Science achievement data Definition of equity  all students, regardless of age, sex, cultural or ethnic background, disabilities, aspirations, or interest and motivation in science, should have the opportunity to attain high levels of scientific literacy (NSES, 1996) Teaching SMART Strategies to Promote Equity  Cooperative learning  Job roles  Student exploration  Open inquiry

5. Research Question To what extent does teacher participation in the Teaching SMART professional development program improve students’ outcome in science?  Does teacher participation in Teaching SMART improve student science outcomes?  Does teacher participation in Teaching SMART increase students’ interest in science?  Does teacher participation in Teaching SMART improve achievement outcomes for female, low-income, or minority students?

6. STEM Pathways: Science, Technology, Engineering & Mathematics Status of historically underrepresented groups in the STEM pathway  Passed over or opted out of the STEM pathway Status of females, minorities, and low income groups pursuing STEM degrees  Not the same rate as men, White and Asian students, and students with higher SES (Tyson, 2007).

7. Science Achievement Males outperform females at grade proficiency levels for 4 th, 8 th, & 12 th grade state science assessments. Black, Hispanic, and American Indian students in 4 th, 8 th, & 12 th grades are 4-5 times more likely to have lower science scores on average as well as a smaller percentage of students reaching grade proficiency levels Students on free/reduced lunches are 3 times more likely to experience lower science scores SEI, 2006

8. Science Achievement NSF, 1999

9. Science Achievement NSF, 1999

10. Attitude toward science and science achievement Social Capital (Bourdieu, 1977)  Important relationships forged during students’ academic career Agency (Foucault, 1980;Gramsci, 1971)  Student perceptions of their own power in educational choices.

11. Measurement of Attitude and Achievement Performance gap and negative attitude toward science  Study of kindergarten children found that boys and girls differed in their “motivational-related beliefs” about science (Patrick, et al. 2009).  A large racial/ethnic gap in science knowledge occurs during the first 2 years of school among black children, with scores 1 SD below that of white children (Chapin, 2006).  Low SES students entering kindergarten have cognitive scores 60% lower than high SES students(Lee & Burkham, 2002).

12. Statement of Purpose Evaluation  Student attitudes as indicated by student survey responses  Student achievement as indicated by student responses on the PASS Multiple Choice assessment

13. Data Collection Fall, 2005 (baseline), Spring, 2006, Spring, 2007, & Spring, 2008 Measure of Outcome  Multiple choice assessment  Partnership for the Assessment of Standards- based Science (PASS)  Items for the multiple choice  n=28 for 3 rd grade  n=29 for 4 th & 5 th

14. Sample 3 rd grade cohort  1037 students: 598 treatment & 439 control Teachers  249 teachers: 135 treatment & 114 control Schools  19 schools: 9 treatment & 10 control

15. School Demographics at Baseline

16. Attrition Overall attrition = 27.9% Differential attrition = 3.9% Adapted from WWC, 2008

17. Item Response Theory (IRT)  IRT models for binary data  Item responses  1 = correct  0= incorrect  Two parameter logistic (2PL) model  Measures latent traits: item difficulty & item discrimination  BILOG_MG3 software  Multiple choice scores scaled from -3 to +3

18. Hierarchical Linear Modeling (HLM) Multi-level model used to analyze nested data  Three-level model with students nested within classrooms (teachers) nested within schools. Mixed Model  Y ij ( Scaled Multiple Choice Score) = γ 000 + γ 001 * FCAT + γ 010* Gifted + γ 020* Reading Baseline + γ030* Math Baseline + γ100* Time + γ 101* Group*Time + γ 102 * Migrant * Time + γ 110* Teacher Science Emphasis * Time + γ 120 * Non-White*Time + γ 200 * Self-efficacy + r 0 + r 1 * Time + r 2 * Self-efficacy + u 00 + u 10 * Time + u 20 * Self-efficacy

19. HLM Variables Level-1: Student  Time: Time-point of data collection: 0,1,2,or 3  Self-Efficacy: Composite variable of a factor loading derived from Likert scale type student survey responses that conceptualized self-efficacy Level-2: Teacher  Gifted: Average percentage of gifted students in a teacher’s classroom  Baseline Reading Score: Average value of students’ baseline reading score  Baseline Mathematics Score: Average value of students’ baseline mathematics score in a teacher’s classroom  Teacher Emphasis on Science: Score on teacher survey based on responses that conceptualized teacher emphasis on science in the classroom  Non-White: Average percentage of non-white students in a teacher’s classroom

20. HLM Variables Level 3  FCAT Score  Average school score on the Florida Comprehensive Assessment Test for 5 th grade science  Group  Treatment or control school  Migrant Status  Average percentage of migrant students at a school  Female  Average percentage of females at a school

21. Results HLM  Baseline  Average initial scaled MC score for the control schools was -0.05(p < 0.06)  Scaled MC scores of the treatment schools were 0.03 units higher (3%) than the control group (p = 0.50)  Classrooms with a higher percentage of gifted students (0.60, p<0.001), high baseline reading (0.0004, p<01), baseline math scores (0.002, p<0.001)had higher scaled MC scores.  Schools with higher student self-efficacy (0.22, p<0.001) and FCAT (0.01, p<0.01)scores had higher scaled MC scores.  Schools with a higher percentage of female students (-0.14,p=0.50) had lower scaled MC scores.

22. Results Linear Growth  Science achievement outcomes  Average rate of change for control schools was -0.07 (p < 0.001)  Treatment schools’ average rate of change was (0.09, p < 0.008) significantly higher than control schools per unit change

23. Results A positive rate of change for treatment schools and a negative rate of change for control schools.  0.02 Slope tx - (-0.07) Slope ctrl = 0.09 (treatment effect)

24. Results Linear Growth  Students’ attitude toward science  Change in student self-efficacy was higher in treatment schools than control schools.  Schools with low student self-efficacy scores at baseline had lower scaled MC scores overtime.

25. Results Linear Growth o Does teacher participation in Teaching SMART improve achievement outcome for female, low- income, or minority students? o Classrooms in schools with a high percentage of non-white students (-0.18, 0<0.01) had a decrease in scaled MC scores overtime. o However, classrooms in treatment schools with a high percentage of non-white students had a rate of change in scaled MC scores equivalent to control classrooms with a low percentage of non- white students.

26. Results Schools with a high percentage of migrant students (-0.72, p<01) had a decrease in scaled MC scores overtime o However, treatment schools with a high % of migrant students had a rate of change in scaled MC scores greater than control schools with a high % of migrant students.

27. Results Schools with a high percentage of females had an increase in scaled multiple choice scores (0.22, p=0.25). o Treatment schools with a high percentage of females had the greatest rate of change in scaled MC scores overtime.

28. Conclusions Teacher participation in the Teaching SMART professional development program  Improves science achievement outcomes  Improves student attitude towards science  Improves science achievement outcomes among female, minorities, and low income groups.

29. Discussion The influence of teachers is a crucial element in bridging the gap between students’ achievement and initiating educational change (van Driel, et al. 2001). Schools must be a catalyst for providing an even playing ground for inequities in social capital. Teaching SMART  Sustained duration  Active learning opportunities  Collaboration  Standards based curriculum

30. Questions? Kathryn Borman, borman@cas.usf.eduborman@cas.usf.edu Maressa Dixon, mdixon83@gmail.commdixon83@gmail.com Bridget Cotner, bcotner@cas.usf.edubcotner@cas.usf.edu Ted Boydston, boydston@cas.usf.eduboydston@cas.usf.edu Vanessa Hein, vhein@mail.usf.eduvhein@mail.usf.edu Rheta Lanehart, rlanehar@cas.usf.edurlanehar@cas.usf.edu Reginald Lee, rlee@cas.usf.edurlee@cas.usf.edu Thank you!