1. Analyse School Performance (ASP) Question Level Analysis (QLA) Inspection Data Summary Report (IDSR) Training for school leaders and governors
To login to wifi in this room:
Username: 2427-SICT-GUEST
Password: riverside2427

2. Agenda: A brief overview of evaluating pupil and school performance
A brief narrative on governor role in data analysis
An introduction to and use of Analysing School Performance (ASP) including question level analysis
An overview of the RAISE replacement – the Inspection Data Summary Report (IDSR)
Follow up work for governors, senior and middle leaders
Scope baseline from those in room – who are we? What levels of experience raise, new asp, idsr…
Governors – linked or ‘just’ general
Try to finish by 9pm
Outcomes – understand what asp has on it and its capabilities and limitations; start the journey towards understanding IDSR and therefore possible inspection trends; start the journey to ensuring all are clear that data is just a SIGNpost (next slide)
STOP ME if I use jargon etc
Questioning through pls

3. “Data is a signpost not a destination.”
Give our example – data variances – tell the story of the school, the impact you are having.
Equally however, good data alone is not a protection or a rationale to say good/outstanding!
“Data is a signpost not a destination.”
Amanda Spielman,
Chief Inspector of Schools

4. The Governance Handbook says the governing board must have access to "objective, high quality and timely data" in order to hold the school leadership to account. What does this mean in practical terms….. ….for sharing data – raw and processed ….for access to data streams ….for sharing processed/evaluative narrative ….for in-year tracking and data sharing

5. Questions to ask:
How does our attainment and progress compare to the national averages?
Did we meet the floor standard? What about the coasting standard? What is our 3 year coasting position?
Are there any gaps in specific groups of pupils? (For example, those with English as an additional language or special educational needs)
Are our results what you were expecting? If not, why not? How are you addressing this?
Are pupils making at least the progress expected year-on-year as they move through the school? If not, is there a link with the quality of teaching in different subjects and/or year groups?

6. Interrogating data:
How much detail should Governors expect in the answers from the headteacher and SLT when the governing board questions them about the school's data.
Governors should expect to hear:
Reasons for any disparity in attainment between groups of pupils
Details of what the school is doing to address any issues
IMPACT of those measures including on in-year/in-school data
The answers should be detailed and specific. For example, if governors ask why attainment in mathematics has dropped, they should not be satisfied with simply being told that it was just one pupil who brought the average down. They should ask for details of what happened with this particular pupil, and why he/she did not achieve as expected.
Governors should not be afraid of asking the headteacher for more details, or for clarification on anything confusing. It is the responsibility of the governing board to hold the headteacher to account, and good headteachers will welcome the challenge as long as it is constructive.
Understanding data will enable us to ensure the correct threads are challenged and written into improvement plans
ALSO to ensure that where data points down an avenue, we can use books, internal data etc to tell the whole story
Challenge what numbers might say on historic performance – do this yourself, but also be prepared ofsted will do this!
An understanding of what data is available and how it can be used will aid all school leaders and governors to have an informed an appropriately challenging discussion.

7. A few don’ts:
Governors should remain strategic – even if they know who is involved they should not discuss named individuals
Individual pupils should not be named in reports
Governors should not be able to access ASP to pupil level or take part in named pupil level question analysis

8. Sources of data:
Schools own tracking information across all year groups
preferably at least termly
includes relevant groups of pupils
includes attendance and exclusions
Includes evaluative commentary
External
ASP
LA record of visit table; similar from MAT/TSA teams
LA School Profile
Interactive reports on Perspective Lite
DfE 8 accredited suppliers eg Fischer Family Trust
DfE School Performance Tables
Link in to next slide and the added value from FFT

9. Comparing RAISE v ASP v IDSR v Provider:
FFT
Single year: analyse results & progress 
3 year trend: analyse results/progress from 2014 to 2017 X
Pupil group filtering for all reports/analyses
Pupil progress: value-added (VA) using DFE (where available)
Pupil progress: contextual value-added (CVA)
Statistical significance for all performance indicators
Percentile rank for all performance indicators
Disadvantaged pupils report
Pupil scatterplots
Pupil-level data: results and VA progress
Edit pupils: remove pupils and create custom groups
School context: current pupils (school-level)
School context: current pupils (pupil-level)
Create groups of schools to collaborate and share data
Limitations apart from FFT – this session is not on FFT, but as an ofsted accredited provider – they HAVE to respect information against FFT, targets set etc
Okay to move on to ASP itself?

10. Analyse School Performance
ASP
Analyse School Performance
This is a new tool
It replaces RAISE online
It provides a single year of data i.e outcomes
There are various levels of access which can be set up, but it is for SCHOOLS to set this up, not the LA or individual member of staff/governor:
Governor log-in for static tables
Anonymous log-in allows filter so they can’t see named pupils
Named log-in for staff
Governors may prefer to ask the school for summary report and for screen prints of filtered data
Thorough knowledge and use of ASP in effect will negate the IDSR – certainly it allows for a lot more useful probing etc

11. ASP Logging in to ASP - 1 Click on Login
Web site address is:
Click on Login

12. ASP Logging in to ASP - 2 Make sure you tick the box
After the first log in you will have to log in a second time!
Click and then repeat log in

13. Opening Screen and side tabs:
ASP
Opening Screen and side tabs:
This is the main landing page – shouts in your face and looks nice, nasty or neutral
Explain side tabs briefly
Do pay attention to the statistical significance scaling which links in to the confidence interval – if the confidence interval spans 0 then in effect average if entirely positive will be statistically above average etc – see next slide

14. ASP
Progress starting points indicates Average (Starting point is KS1 Prior Attainment)
Clearly reading is much higher. Why is this?
Progress calculations – everyone happy with how they are worked out?? – quickly explain prior attainment group (PAG) which gives typical scaled score and then +1 for the difference; net impact of adding all of these up and then dividing gives average progress
What this means is that on average pupils in the above school scored 4.67 scaled points more than pupils nationally with similar starting points
DRAFT ONLY

15. ASP
ASP Reports
Bottom links allow access to reports as listed on next page…

16. ASP
Provide candidates with lists

17. ASP
Tables
Remember at all times – this only points a sign post it does not tell the whole story
It must be balanced by in-school tracking over other year groups – for example the above school could be really strong in all aspects through KS2 since tracking and books etc in years 3-5 support the above or just possibly this could be out of school catch up, interventions, year 6 test teaching etc!!

18. ASP

19. ASP Filters Outliers can be filtered here to some extent
Also allows for comparatives:
disadvantaged to non-disad
send to all

20. ASP
Filters contd:
This is a really useful part of the interactive tool – OUTLIERS – ofsted now need to take into account small groups AND outliers

21. ASP

22. Pupil Breakdown – Progress Scores
ASP
Pupil Breakdown – Progress Scores
The ability to do this enables you to see outliers (and disqualify as OFSTED now will)
Also to see where any particular positive or negative scores may be

23. ASP Boy/Girl Again outliers
These are interactive if not on a governor login
You can hover and identify pupils etc

24. Pupil Breakdown – attainment scores
ASP
Pupil Breakdown – attainment scores

25. ASP KS2
ASP

26. ASP
ASP KS2

27. ASP KS2
ASP

28. ASP
ASP KS2 disadvantaged

29. ASP
ASP KS1/phonics

30. ASP KS1/phonics
ASP

31. ASP
Phonics
Each graph can tell a story – eg here – pass mark is 32, so….. 5 didn’t pass etc etc

32. ASP
ASP EYFSP

33. ASP
ASP EYFSP
Not in for 16/17 EYFS yet – beware!

34. Question Level Analysis
QLA
ASP QLA
Question Level Analysis
Part of ASP also allows for question level analysis
This is accessed via the front page – not on governor logins
It does allow for band and strand analysis eg on the above you can see that the school performed less favourably in fractions

35. QLA
You can drill into questions – look at them by type, view % attempting etc
THIS TOOL is really important….What should leaders and governors do – as a minimum from last year; what about tracking and predicting forwards?
Flag MLT training to follow
Queries, thoughts etc before we move on to IDSR?
Bit of time to play and read etc etc
COFFEE BREAK

36. IDSR
The new INSPECTION DATA SUMMARY REPORT (read alongside the IDSR general guidance)
This is THE core pre-inspection document – how will an inspector use this information – do all those who might be involved understand what it is suggesting as these may flag possible inspection threads?
What about in school data and data over time etc

37. IDSR
What is in the IDSR?
The IDSR contains data for context, trends over time, progress and attainment. It presents progress data from the main starting points. The disadvantaged group is also shown.
Inspectors should not report separately on small numbers (typically fewer than five) where individual pupils could be identified. Progress and attainment figures based on 10 or fewer pupils are displayed in grey throughout the report.
Averages and percentages for small cohorts are more affected by the performance of individual pupils. In order to inform areas to investigate on inspection, inspectors should look at:
 How cohorts have performed across time to see if there has been a consistent trend.
 The distribution of scores within a cohort, using the scatterplots in the IDSR. This includes identifying whether all pupils in the cohort were above or below similar pupils nationally.
No one data measure should lead to an overall judgement; these will inform areas to investigate on inspection. Therefore single measures, particularly about small cohorts, should only be commented on in the context of a range of measures and information.

38. IDSR Front Page - Overview
This is the front page – but we will explore its constituent parts…..

39. IDSR Areas To Investigate Not in order
Will include statements about outliers
NOT highlighted on actual
The areas to investigate may lead an inspection but you should also pre-think what else ; they should also be your starting points to ensure the back story is in SEF and they are build into SIP
Also be aware there is no trend or conversion information for ks1 and you should look to provide this as it is not available in ASP either

40. Areas to investigate contd:
The areas to investigate are shown on the front page of the IDSR. These replace the previous strengths and weaknesses. They are calculated based on the latest three years of data. Most of the areas to investigate are calculated based on only the latest year of data shown in the IDSR. Some are calculated using the latest three years to show any trends. The sentences are shown on a single page at the beginning of the IDSR.
The areas to investigate may indicate what inspectors may need to study on inspection. They can show: when schools are performing better or worse, consistencies across subjects, and how particular groups or subjects in a school are performing.
The areas to investigate are based on statements related to historic data in the grade descriptors and handbook. It is possible that there will be no areas to investigate flagged for a school.
Some sentences are customised to specify only the subjects and groups that match the school’s data.

41. Summary KS 2 Performance
IDSR
Summary KS 2 Performance
School information }
2017 Accountability Standards }
Although it was below the Coasting measures in it is not ‘Coasting’ because it has not been below for three consecutive years.

42. School Context - Overview
IDSR
School Context - Overview
Quickly go through
Your context can also help tell story of barriers and enablers

43. Year Group and Prior Attainment Overview
IDSR
Year Group and Prior Attainment Overview
Explain esp prior attainment as these are really significant bits of data eg on this data last years y6 were -1.2 below national prior for rdg etc

44. Key Stage 2 Absence and PA
IDSR
Key Stage 2 Absence and PA
This school you can see has improved attendance from 2015 and is now broadly around national, but PA rate is below national

45. IDSR
Trends over time
Dotted line denotes a change in methodology or calculations and therefore can not compare one year with another directly. But, can consider trend to national.
The number is the percentile ranking
Red border indicates significantly below the national value.
Cohort number shown in brackets.
Explain this page
Of course this page is a bit false as cohort is stated as 30 the whole way down – in yours, look at cohort number as a red box may be around a significant below which is actually an outlier
This data shows a school in reading with a trend of being around average and below overall but with pupils with low prior performing less well at ks2
2015 quintiles and percentile are based on value added measures.
2016 and 2017 quintiles and percentiles based on progress measures.

46. Overview - KS2 Reading Progress
IDSR
Overview - KS2 Reading Progress
Explain the chart on left and where bullet is and whiskers overlapping 0; sig below and sig above
Also that PAG is PAG group
The charts don’t show anything new – but on this vs ASP they try and do SOOO much on one page!
Scatterplot is complex – eg on right is that boys did less well generally….

47. IDSR Reading Attainment
Again explain bars for exs and for gds and where national lines fit
Explain again comparators disadvantaged with other; SEND with all
Explain the diff no is pupils to make the difference eg in the abnove 4 pupils down on rdg gds

48. English grammar, punctuation and spelling (EGPS)
IDSR
English grammar, punctuation and spelling (EGPS)
Still PAG group – note spelling separate on here so can be a very direct line of enquiry

49. IDSR
Science
Also note science – so again can be a direct line of enquiry AND ensure there is match to work, planning and evidence – good and bad – does the work in school reflect the outcomes this implies

50. IDSR Key stage 1: Reading
Remember – no trend info for key stage 1 – so if you have a ‘peculiar’ year – good or bad, you need to tell that story and ensure you clearly convey the ‘norm’
Difference number again
For EYFS the PAG is subject specific

51. IDSR
KS1 - Phonics

52. IDSR
EYFS Profile
There is trend information for EYFS

53. Time to look through, discuss, questions

54. Summary:
ASP
QLA – further training for middle leaders will follow within STAR
IDSR – personal analysis and ensure build into SEF/SIP
As governors – use of data strategically to check on and challenge performance and to track whether improvement plans are having impact
As leaders – to probe into detail to fine tune and examine in order to be specific around what needs to improve and build into planning, BUT, to always check that plans impact on pupils in school and that the current and relevant issues are in plan
Use of tools such as ‘ready reckoner’ and FFT to ensure predicted outcomes for current year build and improve
Target setting – asp ready reckoner, check against FFT, etc