1. Applying the Performance Equation to Evaluation Tom McKlin, Ph.D tom.mcklin@gatech.edu

2. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement Broad-strokes logic model behind many MSP programs

3. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement Broad-strokes logic model behind many MSP programs 1. How do we know when to do this?

4. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement Broad-strokes logic model behind many MSP programs 2. How do we know what gets in the way of this?

5. Performance Equation Performance = Internal + External + Resources Internal: skills/knowledge + inherent ability + motivation External: organizational system + incentives (ext. motivation) Resources: Cognitive supports + tools + physical environment P= I({s,k} + ia + m) + E(os + i) + R(cs + t + pe) Foshay, R. (2006). A technology for improving performance in schools. Retrieved January 15, 21007, from http://www.foshay.org/PDFs/TechnologyImprovingPerformance.pdf http://www.foshay.org/PDFs/TechnologyImprovingPerformance.pdf Wile, D. (1996, February) Why doers do. Performance and Instruction, 35(2), 30-35.

6. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement We can apply the performance equation anywhere P= I({s,k} + ia + m) + E(os + i) + R(cs + t + pe) Today, we’re applying it here:

7. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement Professional learning is just one predictor of good instruction Inherent ability Internal motivation Organization (School or District) Incentives Cognitive Supports Tools Environment

8. How do we know that teachers need a training solution? We rule out other overwhelming variables Teachers express a need to know more Many do not reach state-wide “highly qualified” standards Administrators observe a deficit We see lower-than-desired LMT or MOSART pretest scores: –One school district we worked with used a combination of MOSART, LMT, DTAMS, and Ad Hoc instruments

9. Example: Applying a professional learning solution Content Knowledge 6 th Math  7 th Math  8 th Math  11 th Math  6 th Science - 7 th Science - 8 th Science - 11 th Science - Based on those assessments, we may say that middle and high school math teachers experienced an increase in knowledge/skills. Increase = average teacher scores saw gains on 67% or more assessments and sub- assessments

10. Example: Applying a professional learning solution Content Knowledge New Standards (GPS) 6 th Math  7 th Math  8 th Math  11 th Math  6 th Science -  7 th Science -  8 th Science -  11 th Science -  The implementation of the Georgia Performance Standards washes out any gains we might otherwise see.

11. Example: Applying a professional learning solution Content Knowledge New Standards (GPS) Expectations 6 th Math  - 7 th Math  - 8 th Math  11 th Math  6 th Science -  7 th Science -  8 th Science -  - 11 th Science - 

12. School 1 School 2 School 3 School 4 School 5 School 6 School 7 School 8 School 9 School 10 School 11 School 12 School 13 We expected to see gains in 8 th grade math performance…do we? What is the effect of GPS implementation in 2006 in 6 th & 7 th grade math and science?

13. Example: Applying a professional learning solution P= I({s,k} + ia + m) + E(os + i) + R(cs + t + pe) = Professional learning + GPS Student Achievement

14. Example: Applying a professional learning solution P= I({s,k} + ia + m) + E(os + i) + R(cs + t + pe) = Professional learning + GPS Student Achievement What about the other variables?

15. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement Broad-strokes logic model behind many MSP programs LMT MOSART DTAMS Other ad Hoc Instruments Tells us whether content and pedagogical knowledge has actually improved

16. Professional learning in Math/Science pedagogy and content Increased teacher content and pedagogical knowledge Instructional improvement Improved student achievement Broad-strokes logic model behind many MSP programs Observations Focus Groups Interviews Details barriers that prevent teachers from changing their instruction

17. This teacher has received professional development on how to help students with disabilities succeed; however, she says it’s not enough. Proximity, removed from the testing setting, notes, no scantrons, things that are helpful sort of, but it doesn’t get to the heart of it. It doesn’t…. I just find those very minor. Extended time, well, every kid in my class gets extended time if they need it. They’re very cursory, I guess. Does it really help me teach them better? I don’t think so. Does it help them learn better? Probably not, maybe on the assessment - the separate testing, the scantrons - but it doesn’t really get to the heart of it. Not in science. Not for me at least.

18. When asked “why,” she and other teachers said things like: The classrooms are crowded. We no longer have a science lab in our school. The support teachers don’t know the science/math content. We don’t want to use hands-on learning because it will be too much of a distraction. There aren’t enough support teachers. The special needs teachers are good, but they also have to support English, Math, History, and other courses as well as science. Many students can’t understand the vocabulary in science. Our school is so large that kids have trouble getting to class on time. The cabinets are not accessible to students in wheelchairs. The ADD students distract the other students.

19. When asked “why,” she and other teachers said things like: The classrooms are crowded. (pe) We no longer have a science lab in our school. (pe) The support teachers don’t know the science/math content. (s/k) We don’t want to use hands-on learning because it will be too much of a distraction.(m) There aren’t enough support teachers.(os) The special needs teachers are good, but they also have to support English, Math, History, and other courses as well as science. (os) Many students can’t understand the vocabulary in science. (os) Our school is so large that kids have trouble getting to class on time. (os, pe) The cabinets are not accessible to students in wheelchairs. (pe) The ADD students distract the other students.(pe)

20. Disclaimer The instructional practices and assessments discussed or shown in these presentations are not intended as an endorsement by the U.S. Department of Education.

21. Applying the Performance Equation to Evaluation Tom McKlin, Ph.D tom.mcklin@gatech.edu