2. The context
Having pre-read pages of the Sutton Trust Report consider the questions on the next slide.

3. Consider the following:
What are the pros of gathering pupils’ opinions to make judgements on teacher effectiveness?
What are the cons of gathering pupils’ opinions to make judgements on teacher effectiveness?
What evidence is there from research about the use of using pupils’ opinions in making judgements on teacher effectiveness?

4. A summary
The two main methods of pupil surveys are questionnaires and focus groups.
These are based on the opinions that pupils build up over the year.
The best questions focus on the activities of the teacher rather than the pupils’ feelings towards their teacher.

5. Research findings 1
Pupil voice outcomes have been shown to have a greater correlation with future pupil outcomes than classroom observations.
A major concern has been that pupils may not distinguish between how much they like a teacher and how good they think a teacher is.
A second concern has been that pupil evaluations merely reflect their grades rather than their actual learning.
It has been found that pupils often prefer teachers who teach to the test.

6. Research findings 2
Questions related to classroom atmosphere are likely to be more indicative of a teacher’s teaching ability than a pupil’s like or dislike for a teacher. 
Surveys can provide useful feedback for teachers as well as evaluating performance.
Surveys should be seen as an additional piece of information for the evaluating of teacher performance.

7. Agreeing survey criteria
What aspects of teaching should we question the pupils about?

8. Example of pupil responses

9. Agreeing survey criteria
What type of teacher activity should we question the pupils about?
How do these areas link with our criteria for teaching and learning?

10. Focus groups/questionnaires
The two main methods of surveying pupils are focus groups and questionnaires.
What are the advantages and disadvantages of using focus groups and questionnaires for gaining evidence on teacher effectiveness?

11. Advantages and disadvantages
Focus groups
Questionnaires
Advantages
Probes pupils’ responses.
Mirrors Ofsted process.
Cheap to run.
Commercially available products supply moderated judgements.
Good supply of data.
Large evidence base.
Disadvantages
Small evidence base.
Time consuming.
Can be seen as controversial by teachers.
Can be expensive.
Time consuming to plan and analyse.

12. Focus group – effectiveness of teaching
Which lessons do you feel you are making the most progress in? How do you know this?
Which lessons do you feel you are making least progress in? How do you know this?
In which lessons is your work regularly assessed? Does the feedback help you progress?
In which lessons is your work not regularly assessed?
In which lessons do you feel you learn most effectively?
Which lessons are the most interesting?
In which lessons do you receive homework according to the timetable? Does the homework help you progress?

13. Focus group – effectiveness of teaching (particular subject)
Do you feel you are making progress in XXXX? How do you know this?
Is your work regularly assessed? Does the feedback help you progress?
Is there a climate where you feel you can learn effectively? Why?
Are the lessons interesting?
Do you receive homework according to the timetable? Does the homework help you progress?

14. Commercially available questionnaires
Which feedback relates to certain curriculum/subject areas?
What evidence does this give us in evaluating the quality of the different elements of teaching shown in Handout 4.3?

15. Criteria for pupil surveys
Highly effective
Effective
Developing
Less effective
Comments on progress
All pupils make progress
Most pupils make progress
Some pupils make progress
Few pupils make progress
Comments on assessments
Assessment is always regular and valuable
Most assessment is regular and valuable
Some assessment is regular and valuable
Assessment is rarely regular and valuable
Comments on climate
Classroom climate is always supportive of learning
Classroom climate is mostly supportive of learning
Classroom climate is sometimes supportive of learning
Classroom climate is often not supportive of learning
Comments on interest
Lessons are always interesting
Lessons are mostly interesting
Lessons are sometimes interesting
Lessons are rarely interesting
Comments on homework
Homework is always set and useful
Homework is usually set and useful
Homework is sometimes set and useful
Homework is rarely set or useful

16. An effective teacher Highly effective Effective Developing
Less effective
Comments on progress
All pupils make progress
Most pupils make progress
Some pupils make progress
Few pupils make progress
Comments on assessments
Assessment is always regular and valuable
Most assessment is regular and valuable
Some assessment is regular and valuable
Assessment is rarely regular and valuable
Comments on climate
Classroom climate is always supportive of learning
Classroom climate is mostly supportive of learning
Classroom climate is sometimes supportive of learning
Classroom climate is often not supportive of learning
Comments on interest
Lessons are always interesting
Lessons are mostly interesting
Lessons are sometimes interesting
Lessons are rarely interesting
Comments on homework
Homework is always set and useful
Homework is usually set and useful
Homework is sometimes set and useful
Homework is rarely set or useful

17. Plenary: consider your next steps
Do you wish to use focus groups or questionnaires to investigate teacher effectiveness?
Have you designed a focus group which you can trial with a certain group of pupils to look at a certain subject area?
Do you currently use a commercial questionnaire?
Have you linked this to your criteria of individual teacher effectiveness?
Would you like to design your own set of questionnaires to probe individual teacher effectiveness?