Presentation on theme: "SECOND REPORT ON THE ACPP: TEACHER PERSPECTIVES & IMPLEMENTATION LESSONS PRESENTED AT THE STATE OF EDUCATION IN ARKANSAS 2008 Marc J. Holley, MEd Doctoral."— Presentation transcript:
SECOND REPORT ON THE ACPP: TEACHER PERSPECTIVES & IMPLEMENTATION LESSONS PRESENTED AT THE STATE OF EDUCATION IN ARKANSAS 2008 Marc J. Holley, MEd Doctoral Fellow Dept. of Education Reform University of Arkansas April 22, 2008 1
Why try merit pay? Of all the inputs we can control, teacher quality matters most for student success. “…a good teacher will get a gain of 1.5 grade level equivalents while a bad teacher will get.5 year for a single academic year.” “…having five years of good teachers in a row (1.0 standard deviation above average, or at the 85 th percentile) could overcome the average seventh-grade mathematics achievement gap between lower-income kids (those on the free or reduced-price lunch program) and those from higher-income families.” Hanushek, E.A. & Rivkin, S.G. (2004). Similar findings by Sanders, W.L. & Rivers, J.C. (1996)
Compensation Reform as Policy Tool The Single Salary Schedule which rewards teachers for credential and experience does not align pay with the outcome we care about most – Student Achievement. Teacher compensation reform focused on pay-for- performance can have motivational (short term) and compositional effects (long range)
Achievement Challenge Pilot Project Description (ACPP) ACPP - Little Rock, Arkansas Merit pay program for all staff members in a school Reward personnel solely for increases in student achievement 3-Year Program Created in 2004-05 at Meadowcliff Elementary Expanded to include Wakefield Elementary, 2005-06 Expanded to include 3 more schools (Romine, Mabelvale, & Geyer Springs), 2006-07
ACPP – Well Designed Plan Straightforward – Based on growth, not complicated statistics Non-Competitive for Teachers – Everyone could earn Significant Awards - up to $10,000 for teachers Focus on Growth of Students – address student placement Everyone Participated – from secretaries to principals
Achievement GAINS! …BUT … An Unqualified Success??? Evaluation Focus Two years of modest positive student gains (Winters et al., 2008); (Barnett et al., 2007) Should we think about other costs and benefits when evaluating a program? What can we learn from implementation?
The Hypotheses We Tested: Proposed Benefits Possible advantages of merit pay: 1) Leads to greater innovation in teaching 2) Leads to teachers working harder 3) Leads to greater salary satisfaction
The Hypotheses We Tested: Proposed Drawbacks Possible disadvantages of merit pay: 1) Leads to a degraded school climate 2) Leads to an increase in counterproductive competition 3) Leads to the neglect of low- performing students
Teacher Views: Research Questions Teacher Surveys (Compelling Control Groups) 1) Do teachers in schools with a long-term merit pay (IA) program have differing attitudes and perceptions than teachers in new merit pay schools (LA)? 2) Do the attitudes and perceptions of teachers in schools with a long-term merit pay program (IA) change over time? Teacher Interview 3) Were teachers supportive of merit pay and the ACPP, and did their attitudes and behaviors change as a result of the merit pay program? 4) Did teachers feel the goals of the ACPP were fair and attainable, and how would they improve the program?
Teacher Surveys: Methods Surveys were administered to teachers in Fall ’06 & Spring ’07 at all five participating schools Survey participation was voluntary Teachers responses were measured on a scale of 1 – 4, with a higher score representing stronger levels of agreement with that question Questions were grouped into constructs that examined attitudes about oft-cited positives and negative associated with merit pay
Teacher Surveys Comparison of IA vs. LA – Fall 2006 ConstructIA Mean Score LA Mean Score DifferenceItI-valuep 1) I am innovative.2.96 0.000.030.97 2) I work hard.3.063.09-0.030.270.79 3) I am satisfied with my salary.2.341.880.463.470.00 4) My school does not suffer from negative competition. 3.322.214.171.1240.08 5) My school has a positive work environment. 3.042.570.474.390.00 6) I view low-performing students as a positive challenge. 3.012.560.454.060.00 7) I am an effective teacher.3.252.840.413.700.00 RQ1: Do teachers in schools with a long-term merit pay (IA) program have differing attitudes and perceptions than teachers in new merit pay schools (LA)? * * * *
Teacher Surveys: Comparison of IA vs. IA Fall 2006 to Spring 2007 ConstructIA Fall Mean Score IA Spring Mean Score DifferenceItI-valuep 1) I am innovative.2.962.910.050.57 2) I work hard.3.063.15-0.090.610.54 3) I am satisfied with my salary.2.342.44-0.100.630.53 4) My school does not suffer from negative competition. 3.333.35-0.020.160.87 5) My school has a positive work environment. 3.042.960.080.650.52 6) I view low-performing students as a challenge. 3.012.8126.96.36.199 7) I am an effective teacher.3.253.210.040.320.75 RQ2: Do the attitudes and perceptions of teachers in schools with a long-term merit pay program (IA) change over time? …ATTITUDES PERSISTED AFTER FIRST YEAR
Synthesis of Survey Findings Predictions Concerning Merit PayDoes Prediction Hold? IA vs. LA Fall 2006 Does Perception Continue? IA vs. IA Spring 2007 Conclusion Concerning the ACPP’s Effect on Teacher Attitudes 1) Teachers will report more innovation. NoYesNot associated 2) Teachers will report working harder. NoYesNot associated 3) Teachers will report more salary satisfaction. Yes Possible association 4) Teachers will report increased counterproductive competition. NoYesNot associated 5)Teachers will report a more negative work environment. NoYesNot associated 6) Teachers will report an increased view of low-performing students as burdensome. NoYesNot associated 7) Teachers will report being more effective. Yes Possible association
Teacher Interviews: Methods Conducted in Fall 2007 after bonuses were disbursed Questions explored the same areas as the surveys : Teacher salary satisfaction was not covered in the interviews 43 interviews were conducted (40 teachers, 3 principals) at the 5 ACPP schools Responses were coded as positive, negative, or neutral A positive response would advocate for performance pay Ex. Do teachers work harder at your school as a result of merit pay? Yes = positive response No = negative response No answer = neutral response 21 coded as positive, 16 were negative, & 6 were neutral
Teacher Interviews: Results QuestionPositive Responses Negative Responses Overall Attitude Confirms Survey Results Did teachers initially support the ACPP? 178PositiveNA* Did teachers support merit pay as a reform? 1511PositiveNA* Did the ACPP encourage teachers to become more innovative in the classroom? 510NegativeYes Did the ACPP encourage teachers to work harder? 1310PositiveNo Did the ACPP lead to counterproductive competition amongst teachers? 101PositiveYes Did the ACPP have a negative effect on the climate of the school? 713NegativeNo Did the ACPP encourage teachers to work with lower- performing students? 62PositiveYes Did teachers feel that the ACPP was fair during the 2006-07 school year? 413NegativeNA* RQ3: Were teachers supportive of merit pay and the ACPP, and did their attitudes and behaviors change as a result of the merit pay program? RQ4: Did teachers feel the goals of the ACPP were fair and attainable, and how would they improve the program? Highlighted are different findings. *Not evaluated on the survey
Teacher Bonus Amounts: 2004-2007 SchoolYearTotal Bonus $Highest Teacher Bonus Lowest Teacher Bonus* Average Teacher Bonus % of Teachers Receiving Bonuses Meadowcliff2004-05$134,800$8,600$1,800$5,417100% Meadowcliff2005-06$200,926$7,300$3,700$5,92893% Wakefield$228,300$9,200$4,000$6,709100% Meadowcliff2006-07$101,535$5,100$1,100$3,153100% Wakefield$51,716$7,600$300$2,33548% Mabelvale$39,550$6,400$450$2,11156% Geyer Springs$64,530$7,600$350$3,10592% Romine$12,450$5,200$450$2,71327% * Other than $0
Ways to Improve the Program: Teacher Responses “I think it should have been consistent across the board. The rules changed.” “I don’t know if it was a lack of understanding, miscommunication, or it was literally changed after we were told that it would be this way, and then it wasn’t done that way. It was done a different way. That caused a lot of conflict.” “I like the pretesting in the Fall when they [the students] got here. I like the post-testing at the end of the year because that told me how good of a teacher…or what I needed to work on.” “I believe that no teacher should have gone without a check.” “They [teachers] would have to be in on the planning/making. And you would have the exact same discussion with them…what do you think? What do we want? And they would have to have so much ownership of it.” “Have one set of rules for the entire district. Everything has to be transparent. Our situation wasn’t transparent.”
Conclusions: Teacher Views of Merit Pay 1) Teachers initially supported the ACPP and were somewhat supportive of merit pay as a compensation reform over time. 2) Teachers did not associate merit pay with being more innovative, and responses varied in regards to working harder. 3) Teachers did not associate merit pay with counterproductive competition or viewing low-performing students as a burden, but did feel that the school environment became more negative after bonuses were disbursed. 4) Teachers reported higher salary satisfaction and felt more effective. 5) Teacher expectations play a significant role in the “success” of a merit pay program.
Take away message Merit pay that ties bonuses directly to student achievement gains can lead to student improvement. Merit pay does not necessarily lead to divisive competition and nor undermine collaboration. Participant expectations are very important, so implementers must communicate clearly and openly to set and maintain expectations for participants.
Contact Information: Marc Holley Office for Education Policy University of Arkansas http://www.uark.edu/ua/oep Email: firstname.lastname@example.org Phone: (479) email@example.com