1. The Center for Educational Effectiveness, Inc. 1 Staff Perceptions of School Conditions What Are They? Do They Reflect Reality? WERA March 2008 Pete Bylsma bylsmapj@comcast.net Greg Lobdell greg@effectiveness.org

2. The Center for Educational Effectiveness, Inc. 2 Field-based research, service, and data-centric tools to support School & District Improvement Partnerships with 500 Schools in 100 districts in WA Assist all schools & districts in OSPI School & District Improvement programs –What we do & how we do it varies based on serving districts from 80 students K-12, to districts over 30,000 K-12. WASL Analysis and “Educational Growth” repository –WASL growth data (student cohorts) for schools serving 380,000 students The largest database of school effectiveness information in the state of Washington (Nine Characteristics of High Performing Schools) –40,000 Staff responses –110,000 Students (30% from homes where English is not primary language) –38,000 Parents (30% from homes where English is not primary language) Schools of Distinction: Creation and Design for OSPI and ongoing research into what’s happening in these buildings Data and the Center for Educational Effectiveness

3. The Center for Educational Effectiveness, Inc. 3

4. 4 Partnership Expand Capacity Provide Value-add services Expertise in core areas Partnership is critical CEE Services OSPI ESDs Districts Schools & Classrooms Demographics & Community Characteristics Perceptual Academic Achievement Contextual-Program and Process

5. The Center for Educational Effectiveness, Inc. 5 Who We Serve

6. The Center for Educational Effectiveness, Inc. 6 Who We Serve

7. The Center for Educational Effectiveness, Inc. 7 Opening the CEE Repositories for Research Activities What? –Educational Effectiveness Survey TM : Staff, Student, and Parent editions Why? –Expands CEE capacity to get research back into the field –Brings organizational effectiveness data from 189,000 educational stakeholders in WA into critical conversations at all levels –Improves our collective practice – impacts future research and development activities

8. The Center for Educational Effectiveness, Inc. 8 The EES Staff Repository Demographics and descriptive variables –Respondents: Position, School Level, Department, Length of service (both “this school” and “in education”), and ethnicity (listed as “optional”) –Extensive research variables are attached at CEE as the data enters the repository –N= 40,236 staff respondents as of March 1, 2008

9. The Center for Educational Effectiveness, Inc. 9 Staff Descriptors

10. The Center for Educational Effectiveness, Inc. 10 Staff Years of Service

11. The Center for Educational Effectiveness, Inc. 11 Staff Ethnicity & Gender School Makeup- Students

12. The Center for Educational Effectiveness, Inc. 12

13. The Center for Educational Effectiveness, Inc. 13

14. The Challenge Time is the educational leaders best friend and worst enemy Dissemination and use of broadly applicable information and research takes far too long and often doesn’t reach the people who need it most Let’s not forget: –Ocam’s Razor: “one should not increase, beyond what is necessary, the number of entities required to explain anything” –Often shortened as: “all else being equal, the simpler solution is preferable” (The “K.I.S.” principle)

15. Staff Perceptions of School Conditions What Are They? Do They Reflect Reality? WERA March 2008 Greg Lobdell Pete Bylsma

16. Study Rationale Self assessments are one element of the school improvement process. State Board of Education requires survey data from staff, students, and family members to inform their annual school improvement plans. Staff typically examine their schools on the dimensions of effective schools and reflect on the results to address perceived shortcomings. No attempt to examine survey results across multiple schools, no norms when reviewing results

17. Concerns About Survey Results Self perceptions reflect self awareness Self awareness may not reflect reality Psychologists have documented our tendency to inflate ourselves and be blind to our shortcomings or suffer from “groupthink” If educators don’t have accurate perceptions of their condition, they won’t to identify their problems, which will lead to efforts not focused on the right solutions Needed changes won’t occur, outcomes may not improve. Lead to discouragement, less effort to improve in the future, and a belief that external conditions are to blame for the problems

18. Phases of Learning Unconscious Competence Conscious Incompetence Unconscious Incompetence Conscious Competence Conscious of Unconscious Competence

19. Matrix of Perceptions of School Quality Unrealistic Unaware of limitations Realistic Aware of success Realistic Aware of limits/success Realistic Aware of limitations Unrealistic Unaware of success Staff Perceptions of School Quality (consciousness) Student Outcomes adjusted for student characteristics (competence) HIGH MEDIUM LOW MEDIUM HIGH

20. Objectives and Scope Determine staff perceptions of school characteristics based on a school’s –Grade level (elementary, middle, high, multiple grades, alternative) –Size (number of staff) –Socioeconomic status (% eligible for free/reduced lunch) –Student performance (avg. % met on grade 4/7/10 reading & math WASL) Results from 430 public schools in CEE database (only known data set across schools in multiple districts) –Not a representative sample: more likely to be poor, low performing –Schools enrolled about 25% of all students in the state –Surveys reflect the views of more than 19,000 staff –Only included results from first survey administration School-level analysis only

21. Survey Contents/Structure 86 statements covering 9 characteristics of high-performing schools (plus 14 other statements) Range of 5-16 statements per characteristic 5-point Likert scale 1.Almost always true 2.Often true 3.Sometimes true 4.Seldom true 5.Almost never true

22. Survey Results: All Schools Scale: 1 Almost never true 2 Seldom true 3 Sometimes true 4 Often true 5 Almost always true (All results are preliminary)

23. Survey Results By Grade Level On every characteristic, elementary had highest scores, HS the lowest

24. Survey Results By Size Larger schools had lower perceptions, but only because HS were larger and had lower scores No variation by size at the grade level

25. Survey Results By SES Weak negative relationship at elementary and middle school levels (perceptions decline as poverty increases), no relationship for professional development Only one characteristic significant at HS level (PD, +) SES had little effect in multiple regression Strongest Relationship Middle School Results for “High Standards and Expectations” (R-square =.177) Most lines were very flat

26. Survey Results By WASL Positive relationship at elementary and middle school levels (perceptions increase as WASL increases), but no relationship for professional development or for any HS results Results at elementary level

27. Survey Results By WASL Strongest relationship related to “Family and Community Involvement” at elementary and middle school levels

28. Perceptions and WASL Crosstabs Schools grouped into three groups based on achievement controlling for SES Schools grouped into three groups based on overall perception score for all characteristics Student Outcomes (competence) Staff Perceptions of School Quality (consciousness) (N=379)

29. Summary and Implications On average, school staff rate themselves fairly high Most results seem to reflect actual conditions and are consistent with research on effective schools –Elementary schools score the highest, high schools the lowest –Perceptions increase as performance increases Perceptions do not increase as school size decreases, not affected much by SES High schools and PD remain enigmas Some schools appear to have inaccurate perceptions Need other assessments/surveys to gain less subjective data Schools can use these trends to compare themselves to others We need ways to compare similar schools to each other

30. Further Study Needed We still need to understand … Perceptions of alternative schools and those serving multiple grades What is happening with professional development activities Why similar schools generate such a wide range of perceptions How perceptions in the same school change over time How perceptions at the staff level (e.g., new vs. experienced teachers) What is happening in schools with unrealistically high & low ratings How staff perceptions compare to student and family views How the results are being used at the school and district levels