2. #TrainWithEase

3. School Performance and Accountability 16th November 2016
Our Speakers
Jamie Pembroke – School Data Expert
Paul Charman -FFT
James Burch – Angel Solutions
Sean Harford - Ofsted
Nick Capstick – White Horse Federation

4. Key Performance Measures: an in-depth guide W: E:

5. data

6. 1: Floor standards

7. Primary floor measures
65% or more meet expected standard in reading, writing and maths combined, or
Pupils make ‘sufficient’ progress in reading (score >= -5), and
Pupils make ‘sufficient’ progress in writing (score >= -7), and
Pupils make ‘sufficient’ progress in maths (score >= -5)

8. Secondary floor measure (Progress 8)
-0.5

9. This is what should happen
Ditch the attainment component
Remove writing TA from progress measures
Have single combined progress measure for Reading and Maths
This is what the floor standard should be based on

10. alternative floor standard (combined CVA measure)-4

11. 2: The coasting measure

12. (below 85% & any progress threshold)
Coasting
2014
(below all)
2015
2016
(below 85% & any progress threshold)
Attainment
L4+ RWM
85%
EXS RWM
Progress
2LP R
94%
VA R
-2.5
2LP W
96%
97%
VA W
-3.5
2LP M
93%
VA M
* Coasting if below all 4 measures in 2014 and 2015, and below 85% EXS and any one progress threshold in If below just one threshold in 2016, that threshold must be sig- to be defined as coasting.

13. Alternatively, remember these?
If school plots ‘significantly’ in this zone 3 years running then it’s coasting

14. 3: The use of teacher assessment in high stakes accountability measures

15. Reading test vs writing TA at KS2 2016

16. Use of writing results in floor standards
In 2016 only, if a school's performance at KS2 has dropped below the floor standard based on performance in writing alone, and in the absence of any other factors, the local authority or RSC should not issue a warning notice, except where the extent of the change in performance cannot be explained by the impact of the changes to primary assessment arrangements in this transitional year. (Schools causing concern guidance)
As this is the first year of schools working with the new interim assessment frameworks, the Minister for Schools has written to the Chief Inspector asking him to ensure that Ofsted inspectors take into account national performance and contextual factors when considering a school's performance in writing at KS2, which is used as part of the floor standard (Writing clarification document)

17. Writing TA is not used as part of the progress 8 baseline for KS4; only reading and maths tests scores are.

18. 4: The writing progress measure

19. 4.5: Setting targets based on these
Prior Attainment Group (PAG)
KS1 average points score
Average KS2 Reading Score for PAG
Average KS2 Writing Score for PAG
Average KS2 Maths Score for PAG 1
>0 to <2.5
77.48
77.39
78.23 2
>=2.5 to <2.75
80.79
80.11
81.53 3
>=2.75 to <3
82.92
82.05
84.14 4
>=3 to <6
85.58
84.01
87.50 5
>=6 to <9
87.87
85.94
90.05 6
>=9 to <10
90.76
89.14
92.11 7
>=10 to <12
93.96
92.60
95.55 8
>=12 to <13
95.77
94.21
97.69 9
>=13 to <14
97.26
96.69
98.33 10
>=14 to <14.5
98.38
98.22
99.77 11
>=14.5 to <15
99.23
100.66 12
>=15 to <15.5
100.62
100.75
101.50 13
>=15.5 to <16
102.46
101.67
102.14 14
>=16 to <16.5
102.60
102.06
103.58 15
>=16.5 to <17
104.13
102.78
104.50 16
>=17 to <18
105.56
104.10
104.97 17
>=18 to <19
106.81
104.74
106.33 18
>=19 to <20
107.96
105.77
107.54 19
>=20 to <21
109.04
106.11
109.41 20
>=21 to <21.5
111.58
108.68
110.57 21
>= 21.5
115.70
110.40
114.51
4.5: Setting targets based on these

20. Nominal Scores for teacher assessments
Not applied to HNM and EXS in reading and maths unless special consideration granted (80 points)
PKS codes:
BLW: 70
PKF: 73
PKE: 76
PKG: 79
Writing codes:
WTS: 91
EXS: 103
GDS: 113
NB: Only applied for purposes of progress measures

21. VA for all possible KS2 Writing outcomes
Prior Attainment at KS1
KS2 Estimate
VA for all possible KS2 Writing outcomes
Prior Attainment Group (PAG)
KS1 average points score
Average KS2 Writing Score for PAG
BLW
PKF
PKE
PKG
WTS
EXS
GDS 70 73 76 79 91
103
113 1
>0 to <2.5
77.39
-7.39
-4.39
-1.39
1.61
13.61
25.61
35.61 2
>=2.5 to <2.75
80.11
-10.11
-7.11
-4.11
-1.11
10.89
22.89
32.89 3
>=2.75 to <3
82.05
-12.05
-9.05
-6.05
-3.05
8.95
20.95
30.95 4
>=3 to <6
84.01
-14.01
-11.01
-8.01
-5.01
6.99
18.99
28.99 5
>=6 to <9
85.94
-15.94
-12.94
-9.94
-6.94
5.06
17.06
27.06 6
>=9 to <10
89.14
-19.14
-16.14
-13.14
-10.14
1.86
13.86
23.86 7
>=10 to <12
92.60
-22.60
-19.60
-16.60
-13.60
-1.60
10.40
20.40 8
>=12 to <13
94.21
-24.21
-21.21
-18.21
-15.21
-3.21
8.79
18.79 9
>=13 to <14
96.69
-26.69
-23.69
-20.69
-17.69
-5.69
6.31
16.31 10
>=14 to <14.5
98.22
-28.22
-25.22
-22.22
-19.22
-7.22
4.78
14.78 11
>=14.5 to <15
99.23
-29.23
-26.23
-23.23
-20.23
-8.23
3.77
13.77 12
>=15 to <15.5
100.75
-30.75
-27.75
-24.75
-21.75
-9.75
2.25
12.25 13
>=15.5 to <16
101.67
-31.67
-28.67
-25.67
-22.67
-10.67
1.33
11.33 14
>=16 to <16.5
102.06
-32.06
-29.06
-26.06
-23.06
-11.06
0.94
10.94 15
>=16.5 to <17
102.78
-32.78
-29.78
-26.78
-23.78
-11.78
0.22
10.22 16
>=17 to <18
104.10
-34.10
-31.10
-28.10
-25.10
-13.10
-1.10
8.90 17
>=18 to <19
104.74
-34.74
-31.74
-28.74
-25.74
-13.74
-1.74
8.26 18
>=19 to <20
105.77
-35.77
-32.77
-29.77
-26.77
-14.77
-2.77
7.23 19
>=20 to <21
106.11
-36.11
-33.11
-30.11
-27.11
-15.11
-3.11
6.89 20
>=21 to <21.5
108.68
-38.68
-35.68
-32.68
-29.68
-17.68
-5.68
4.32 21
>= 21.5
110.40
-40.40
-37.40
-34.40
-31.40
-19.40
-7.40
2.60

22. 5: the ostracism of CVA

23. DfE view of CVA in 2011
Contextual Value Added (CVA) goes further than simply measuring progress based on prior attainment by making adjustments to account for the impact of other factors outside of the school’s control which are known to have had an impact on the progress of individual pupils e.g. levels of deprivation. This means that CVA gives a much fairer statistical measure of the effectiveness of a school and provides a solid basis for comparisons.

24. Why we need CVA Fair, like for like comparisons
Compare achievement of your pupils with that of similar pupils in similar schools
Pupils with low start point: the SEN vs EAL issue
FFT use 13 different factors in their CVA model
By comparing like-for-like, CVA gets closer to the school effect on pupil progress.

25. 6: the progress loophole of despair

26. Included in Average score? Included in progress measure
Teacher Assessment
Test Status
Score
Included in Average score?
Included in progress measure
PKS
Did not take test
70-79 No
Yes
Did take test but not enough marks to achieve scaled score
Took test and enough marks to achieve scaled score
80+
HNM
Took test but not enough marks to achieve scale score
No score

27. 7: the one about getting rid of expected progress measures

28. the ‘[expected progress] measure has been replaced by a value-added measure. There is no ‘target’ for the amount of progress an individual pupil is expected to make.’ Primary Accountability Technical Guide, October 2016

29. Hmmmm

30. 8: Progress measures in 2020

31. Because…… The DfE didn’t collect scaled scores
Fewer outcomes than under levels
Therefore fewer prior attainment groups (possibly 13?) containing more pupils
All pupils in specific group have same ‘expectations’
Schools should consider alternative, more robust progress measures to counter negative messages from future data.

32. 9: colour codes in RAISEonline

33. This is statistically significant (sort of)
These are statistically significant based on 95% confidence interval
These are statistically significant based on 95% confidence interval + an arbitrary threshold

34. And this isn’t! Diff (no. of pupils)
Presents the gap from national average as a number of pupils.
E.g. If you have 20 pupils, each pupil is worth 5%. Therefore a gap of 15% equates to 3 pupils.
Gap 18% = 3 pupils (1 pupil = 5%): dark green
Gap 4% = 1 pupil (1 pupil 3%): light green
Gap -16% = 1 pupil (1 pupil 14%): light red
Gap -9% = 3 pupils (1 pupil = 3%): dark red

35. On the subject of statistical significance
Simply identifies a deviation from the mean that probably didn’t happen by chance
You cannot infer cause i.e. sig+/- is not necessarily due to school effect (CVA is a better indicator)
Red does not necessarily mean the school has done something wrong
Green does not necessarily mean the school has done something right
It’s just a threshold - a 0.01 point shift is all it takes for data to become ‘significant’ or not.

36. At least it’s the last year of RAISE

37. 10: recreating levels

38. Levels were removed because……
They were best-fit so pupils could have serious gaps but still be placed within a level
They led to a focus on getting pupils across the next threshold
Progress became synonymous with moving on to the next level, rather than developing deeper or wider understanding
They told us nothing about what a pupil could and couldn’t do
Pupils at opposite sides of level boundary could have more in common that pupils within a level.

39. These are levels! Band % Objectives Step/Point Y1 Secure 67-100% 3
Y2 Emerging
0-33% 4
Y2 Developing
34-66% 5
Y2 Secure 6
Y3 Emerging 7
Y3 Developing 8
Y3 Secure 9
Y4 Emerging 10

40. and this is just weird Band % Objectives Step/Point Y2 Emerging 0-25%4
Y2 Developing
26-50% 5
Y2 Secure
51-90% 6
Y2 Mastery
90-110% 7
Y3 Emerging
Please note: mastery is not a thing that applies to the ‘most able’ that happens at the end of the year.

41. 11: this!

42. ‘KS2 progress was not significantly below average
‘KS2 progress was not significantly below average* overall or for any prior attainment group in any subject. *and not below -3’

43. and relax

44. data