1. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Overview  Contextual VA  What is it?  What factors are used?  How is it calculated?  Self-Evaluation Analyses  Overview  Examples (NB most examples shown are KS2, but same principles apply at KS3 and 4)  Slide 21 example applies to secondary schools only  Interpretation  Limitations  Extensions

2. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Contextual Value-Added  Contextual Value-Added  compares  the progress made by each pupil  with  the average progress made by ‘similar pupils’ in ‘similar schools’  calculates  the average progress made by all pupils in the school  converts  the school’s value-added score into a PERCENTILE RANK

3. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged.  Factors used in new Contextual VA models:  Pupil Factors  Overall attainment in previous Key Stage  Variation between subjects at previous Key Stage  Variation between Test and Teacher Assessment  Gender  Month of Birth  School Factors  Demographic Data (linked to pupil PostCode)  FSM Entitlement  Average prior attainment for all pupils  Distribution (spread) of pupils’ prior attainment  Factors used in the ‘SX’ model include:  Mobility  Ethnicity  SEN Stage  EAL

4. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Value Added Models - Overview  Model PA (Prior Attainment)  Model SE (Socio Economic) Prior Attainment Gender Month of Birth Gender School Context Prior Attainment  3 VA models are used in Self-Evaluation Reports:  PA and SX models are used for value-added sections  PA and SE models are used for estimates  Model SX (School EXtended) Month of Birth Gender Pupil Context Prior Attainment Month of Birth School Context (E)

5. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. VA Models - Indicators Pupil PASESX School PASESX Mean Test Lev (Fine Grade) (  ) Mean Intake Test Lev Mean TA Level SD of Intake Test Lev Individual Subject Levels (FG) Gender % Girls Month of Birth EAL % EAL FSM % FSM (Percentile Rank) SEN Stage, Statemented % SEN, % Statemented Ethnicity Time in School Joined ‘late’ % Joined ‘late’ Geodemographic Data (Perc. Rank) Mainstream or Special / Unit  Nonlinear modelling and some multilevel modelling used to improve accuracy across whole attainment range

6. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Analyses – Outline

7. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Contextual VA (KS12)  Includes:  Value-added analysis which takes into account:  Pupils’ prior-attainment  Pupil and School Context factors.  Summary of 3 indicators for each Subject:  e.g. (KS2) %Level 4+, %Level 5+, Overall Level  Percentile rank  Indicators for pupil groups  Girls, Boys  Lower, Middle and Upper Prior Attainment  Highlights  Where VA is significant  Where floor target not met (e.g. KS2, L4+ below 65%)  Where match to prior attainment less than 85%  Enables  Initial overview of trends for 3 years  Evaluation of variation within and between core subjects

8. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Contextual VA (KS2 Core) Number of pupils in each of the 6 groups: Boys and Girls split by prior attainment into Lower, Middle and Upper (highlighting showing significant difference between actual performance and expected) Percentile rank of the difference between actual result and estimate, compared with all schools nationally. 1 = value-added is in the top 1% of schools, 100 = VA in the bottom 1% of schools. Middle 50% of schools are in the range 26 to 75.

9. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Value Added (3 Year Summary)  Includes:  Data for 3 years overall and each individual year  School Context measures  Unadjusted (Raw) results  Value Added (using prior-attainment only)  Value-Added (using prior-attainment and contextual factors)  Enables:  Indication of impact of school context, by comparing VA with and without contextual factors  Assessment of overall trends for broad indicators covering the Key Stage as a whole

10. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Overall Summary (Part 1) These percentile ranks show school’s raw results compared with national. No highlighting used here.

11. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Overall Summary (Part 2) These percentile ranks are based upon actual results compared to what would have been achieved if: a)pupils had made similar progress to similar pupils nationally b)pupils had made similar progress to similar pupils in similar schools nationally.

12. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Some combinations of arrows and boxes Consistently significantly above Significantly above, but declining Significantly above, but declining rapidly In line, but declining In line and no trend Consistently significantly below Significantly below, and declining Significantly below, and declining rapidly In line, and no trend due to variation (down then up) Significantly above and no trend In line, and improving rapidly Yr1Yr2TrendYr3

13. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Value Added (Significant Areas - Grid)  Includes:  Only areas where there is a significant difference  Where overall VA across 3 years is significantly high or low  Where the trend over the 3 individual years is significant  Where there are 10 or more pupils in the group  A range of groupings including by gender, prior- attainment, ethnicity, SEN stage and FSM entitlement.  Enables:  A more detailed evaluation of significant strengths and weaknesses and of 3 year trends

14. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Significant Areas (Grid) These boxes refer to the 3- year trend. See slide 12 for more information.

15. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Estimates Summary  Includes:  Estimates for attainment in future years  Based upon 3 VA models:  Prior Attainment (PA)  Prior Attainment + School Context (SE)  Top Quartile (of schools in similar contexts)  Also includes estimates which takes into account the school’s previous VA performance (averaged over 3 years)  Enables:  A discussion about estimates which:  is based upon consideration of a range  Indicates the likely outcomes if previous school performance is maintained

16. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Estimates (L4+) 3 types of estimate (PA, SE and TQ) shown in ascending order. Estimate in box based on your school’s historical performance.

17. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Estimates (L5+) 3 types of estimate (PA, SE and TQ) shown in ascending order. Estimate in box based on your school’s historical performance.

18. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Value Added (Significant Areas - Text)  Includes:  Only areas where there is a significant difference  Where overall VA across 3 years is significantly high or low  Where the trend over the 3 individual years is significant  Where there are 10 or more pupils in a group  A range of groupings including by gender, prior- attainment, ethnicity, SEN stage and FSM entitlement.  Looks at a limited subset of indicators (when compared to the ‘grid’ approach)  Enables:  A more detailed evaluation of significant strengths and weaknesses and of 3 year trends

19. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Significant Areas (Text) This is the same information as shown in slide 14, but in text version.

20. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Secondary Schools Only  The following report (KS234 CVA) shows value added from KS2-3, KS3-4 and KS2-4.  Each row relates to one particular cohort.

21. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. KS234 CVA In this example, performance at KS3 is significantly lower than could be expected. Performance from KS3 – KS4 is significantly better than expected. Overall, from KS2 – KS4, performance is significantly better than expected.

22. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Limitations of the Analyses  VA based only upon pupils matched  So need to be aware of this in schools with significant proportion of refugees or pupils for whom previous data is not available  Significant differences  Not shown where number of pupils falls below 10

23. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Extensions  The school may have access to more detailed reports, for example:  Analyses provided by FFT to LEAs which look at VA and estimates in greater detail, including pupil-level analyses.  Outcomes of analysis using PAT (Pupil Achievement Tracker) to examine performance of particular groups of pupils or specific aspects of the curriculum (using question-level analysis).  Other Value-Added Analysis (e.g. using MidYis, Yellis, CATS) – in such cases, a ‘triangulation’ approach might be helpful.

24. FFT Data Analysis Project – Supporting Self Evaluation  Fischer Family Trust / Fischer Education Project 2005. Extracts may be reproduced for non commercial education or training purposes on the condition that the source is acknowledged. Triangulation of Data Analysis A e.g. FFT Analysis B e.g. PAT Professional Judgement Basis for actionInvestigate Further Check Accuracy Challenge Assumptions