1. Value Added and Distance Travelled and the Learner Achievement Tracker
Yvonne Fullwood
New Measures of Success
LSC, National Office

2. Context of the New Measures
Fair, valid and ‘fit for purpose’ measures
Four partners: LSC, DfES, ALI, OFSTED
‘Basket’ of measures to contextualise performance
Application across the sector
No additional bureaucratic burden

3. New Measures of Success
Qualification success rate (QSR)
Value Added and Distance Travelled measures (VA/DT)
Recognising and recording progress and achievement (RARPA)
4. Learner satisfaction
Learner destinations
Provider responsiveness to employers
Value for money (VFM)
Staff measures

4. Purpose of VA and DT for 16-19 learners
Improvement and inspection:
LSC: Value Added and Distance Travelled data for learners
Learner Achievement Tracker (LAT)
Provider Gateway
DfES accountability:
VA for 16 – 19 Learners included in the School and College Achievement Tables pilot

5. Overview (1)
Measures the progression in qualifications of learners based on their prior attainment
ie, how well do learners do post-16 based on their achievement at Key Stage 4?
Cover the majority of larger, approved qualifications taken by learners, including vocational and ungraded qualifications

6. Overview (2) VA for 16-19 learners will measure achievement in Level 3
graded qualifications, for example:
GCE A and AS level
AVCE
BTEC National Diploma
DT for learners will include other approved
Qualifications at levels 1,2 and 3 for example:
NVQ
Foundation and Intermediate GNVQ
GCSE
BTEC First

7. Learner Achievement Tracker (LAT)
Produces reports for use by teachers, managers, inspectors and LSC
Accessible via the Provider Gateway
Uses most robust statistical methods available and includes full national sample
Builds on proprietary systems currently available
Software and data free of charge to all LSC-funded providers and LAs
The LSC produced a prototype version of the Learner Achievement Tracker (LAT). The LAT is able to produce reports at different levels of aggregation and can be used by providers, inspectors and the LSC
The LAT is available on the Provider Quality Gateway.
The measures are calculated using a multi level modelling technique. More information on the statistical methodology will be available on the LSC website for the pilot year.
The LAT provides the facility of reporting VA and DT data, but whilst built on similar functionality of proprietary systems, the LAT does not replicate all the functionality or range of services as these systems maybe able to offer. Further development work will continue during the pilot year and following the launch to increase the usefulness of the system and the information it presents.
The software will be free of charge and will also include the entire national data set and provider data.

8. VA and DT data
Data used to calculate the VA and DT for 16 – 19 learners is gathered directly from the the awarding bodies
Additional data from the ILR is required to calculate DT for 16 – 19 learners
Providers will not be involved in collecting data for the measure
Providers will be involved in checking the accuracy of the VA data
In accordance with the commitment to reduce bureaucracy, the data used to calculate the VA and DT measures will be gained directly from the awarding bodies. This information is sent annually to the University of Bath who match the achievement data for qualifications achieved by learners post-16 to the learners prior attainment at KS4 (or equivalent) which they have collected in the previous year(s). The dataset is credibility checked by a further contractor and the prior attainment for each learner is calculated. The achievement data provided to the University of Bath is in grade format. Each qualification (and grade within a qualification) has been assigned a number of QCA points. The grades for each learners achievement are converted into QCA points, added up and then divided by the number of qualifications achieved to give an average prior attainment in QCA points.
The national performance is calculated using MLM and this information as well as the provider level data is fed into the LAT so the provider level performance can be calculated.
The process for producing DT information is the same as for VA apart from an additional data source that is needed, given the nature of the qualifications included in the DT measure. The additional data source required is the ILR which is needed to identify those learners who complete a qualification but that do not achieve.

9. Issues for further exploration
Learners with missing or no prior attainment
Context variables
Customising the LAT
Treatment of AS levels in the VA measure
Treatment of fails in the VA measure
Use of success rate for DT
A number of issues have been identified in discussions with providers, partners and stakeholders which are to be investigated further during the pilot year. These include:
FAILS – these are not included in the VA measure at present due to an issue with the QCA points scoring system. AT present, the distance between grades is not consistent between pass and fail. In the case of an A level, the distance in QCA points between grades is 30 points (A = 270; B = 240 etc.). However the difference between an E grade (140) and a fail (0) is 140 QCA points. As a result of this, fails can skew the VA results.
USE OF SUCCESS RATES FOR DISTANCE TRAVELLED – there are a number different platforms from which the Distance Travelled measure for WBL can be built, depending on whether achievement rate or success rate is used. As part of the work with the Adult Learning Inspectorate and the team leading on the Qualification Success Rate measure, analysis will be undertaken to identify which is the most suitable measure to use.
CONTEXT VARIABLES / SOFT INDICATORS – at present, the VA and DT measures do not contain context variables of soft indicators. Feedback from providers particularly, has indicated that this is something they wish to be included in the measure. Further modelling work will be undertaken to ascertain how these can be built into the measures and how the LAT can accommodate them in the calculation.

10. LAT piloting arrangements
January 2006
Updated Quality Improvement Packs available
Pilot LAT will be available via the Provider Gateway. This will contain Work-Based Learning Distance Travelled (DT) data and un-amended Value Added (VA) data for schools and colleges
March 2006
DT data for colleges and amended VA data for schools and colleges.
to April 2006
Short term evaluation.
November 2006
The LAT software released for reflection and quality improvement purposes. Ofsted continues to pilot the LAT.

11. More information http://www.lsc.gov.uk/nms
Measuring Success Newsletter
The Quality Improvement Pack
Implementing New Measures of Success
Local LSC

12. The Learner Achievement Tracker (LAT) Reports

13. LAT Output reports Type of LAT report What the report shows
Summary Charts
Overview of performance across different
qualifications and across different subjects
within a given qualification type compared to
the national average
National comparison
graph
Performance in a given qualification/subject
compared to the national average across
the full range of prior attainment
National chances charts
Chances of a learner with a given prior
attainment achieving a particular outcome

14. Summary Chart for all Qualifications

15. A Level Summary Chart

16. National Comparison Graph: A Level D&T Product Design

17. National Comparison Graph: A Level Maths
A Level Mathematics

18. National Comparison Graph: A Level Psychology

19. A Level D&T Product Design Chances Chart (40-46)
34%

20. A Level Mathematics Chances Chart (40-46)

21. A Level Psychology Chances Chart (40- 46)

22. Example of a Distance Travelled National Comparison Graph

23. Further Information
The Quality Improvement Pack contains detailed information about the interpretation and application of each type of LAT reports