1. DfE Data Exchange Project
The Analytical Review
The remit and my strategic recommendations for future data needs
Roger Plant, So Direct Ltd.
Co –Author, The DfE Analytical Review

2. The Analytical Review Terms of Reference Key factors
Conclusions and recommendations
This review asked that we find a system where only data essential to the Department and to a devolved self-improving system is collected. Such a system needs to be efficient, trusted and must maintain appropriate standards of quality.

3. Analytical Review: Key Factors to address
The number of different systems used to collect and deliver the data with each needing specific new programming, documentation and user training whenever data is changed.
The same data is being changed and checked in parallel by different people at different processing stages without modifications feeding back to the source (the delivery organisation which generated the data).
That the Department is dealing with ever increasing numbers of delivery organisations during this checking process. As more schools convert to Academies the Department must check data with thousands of schools individually rather than through 150 local authorities.
The fact that data held by schools and other frontline providers on their systems is not the same as that held by local authorities or the Department. This means that data has to be reconciled every time there is a data collection.

4. Analytical Review: High Level Vision
We need to move to model where:
Data can be automatically moved from one organisation to another with no manual intervention
Data is updated and shared on a real-time basis as part of day-to-day business processes
Data is available to all of the people who need it for decision making when they need it.

5. Analytical Review…more specifically…
To achieve this vision the Department should put in place the standards and technology that will enable the automatic flow of data. This can be done using off-the shelf technology that is widely used commercially, that will:
Collect only the data that is needed at a given moment rather than larger set-piece collections. This will support the gathering of timely and less burdensome data exchanges.
Automate data checking and validation, with responsibility for accuracy pushed back to source.
Provide real-time feedback to delivery organisations and third parties in a format that can be used for comparative analysis. The more data is used and is useful, the greater the incentive to increase accuracy and so the system becomes self-cleaning.
Enhance the efficiency and effectiveness of the data warehouse and the School Performance Data Programme as data will be more readily available and up-to-date.
Draw on existing industry-wide standards so that data sets can be efficiently matched together, whilst also allowing new categories of data to be added for use in schools, authorities and the Department at minimum cost.
Automate data collection wherever possible for all education and children’s services. For example, through agreement with schools, attendance data or free school meals data could be regularly accessed.
Support teaching and learning directly. The system will be able to cater for broadening data demands particularly in relation to performance and pedagogical data held in systems such as learning platforms.
Reduce the costs related to the management and use of data both within the Department and throughout the education and children’s services community.

6. Analytical Review: Further recommendations
The Department should source an interoperability system for the exchange, use and maintenance of all data.
An appropriate data model should be developed through collaborative design between users and system suppliers coordinated by the Information Standards Board.
The Star Chamber should be used to ratify the inclusion of data items in the data model. The remit and membership of Star Chamber should be reviewed as part of the changes to the system
Data should be gathered and used in real-time, as part of the day-to-day business processes. Some formal collections may still be timetabled (for example core pupil data - once a year) whilst others will be running regularly (for example attendance data gathering - weekly).
The data exchange process should include a data validation system designed to ensure that data is always validated at source.
The data validation service should be able to be used independently of data exchanges to help ensure the accuracy of source data.
The Department’s data warehouse should become the one-stop-shop for all education and children’s services data in England.
The system should provide analytical tools within a Departmental web portal offering controlled access to reports and data extractions for the community.

7. Learning from International Experience: With thanks to NSIP - Australia
Extracts have been taken from
NSIP PROGRAM REVIEW
A REVIEW FOR THE NATIONAL SCHOOLS
INTEROPERABILITY PROGRAM (NSIP)
STEERING GROUP 12 FEBRUARY 2013

8. AUSTRALIA - NATIONAL SCHOOLS INTEROPERABILITY PROGRAM (NSIP)
Achieving interoperability between education systems is increasingly crucial
The ability for education information and communication technology (ICT) systems to effectively interoperate is becoming essential to deliver national and multi-jurisdiction education policies and projects.
The evolving policy context is driving demand for further work on systems interoperability
There is agreement that demand for interoperability support is growing and is principally driven by four primary factors: a growing trend towards national education policy and projects; constrained public finances; technological advances creating new opportunities for education innovation; and desire for student-centred programs.

9. AUSTRALIA - NATIONAL SCHOOLS INTEROPERABILITY PROGRAM (NSIP)
Interoperability refers to the capacity for information systems to effectively exchange data using agreed standards and architectures. The concept of interoperability is not new, but there is a growing view that systems must be able to share information and resources to support common education objectives. Achieving interoperability requires a coordinated focus on technical standards, the design of systems that capture, store and transfer data, as well as the strategic decision making and business processes of system owners. Interoperability can be described through its practical applications, including:
Interoperability of systems that provide access to learning resources and teaching tools
Interoperability of systems that enable reliable and secure transfer of student data
Interoperability of systems that support online assessment and performance monitoring, including the linking of datasets required for reporting purposes
Interoperability of systems that will support online learning using products and services from multiple providers on an increasingly diverse range of devices.

10. NSIP Conclusions
1: Interoperability will remain a priority for national education projects, and will be difficult to achieve without a national forum bringing together the views of the full range of education stakeholders.
2: There is a strong case for a nationally representative interoperability entity to be retained, performing many or all of the functions currently undertaken by NSIP.
3: Interoperability support should apply to all national or multi-jurisdictional education projects involving technology components.
4: Understanding of the value of interoperability and underpinning issues that need to be solved should be broadened beyond those with specific ICT knowledge.

11. Analytical Review – going forward…
Data standards
We must go forward beyond Common Basic Data Set
Governance
We will establish protocols to cover data exchange so ensuring “Managed Interoperability”
Usage
We will ensure that data exchange has the potential to support all aspects of education management
In conclusion, we should work for a clearer articulation of the role and benefits of interoperability. This will increase understanding of its potential value among stakeholders and in turn will help to broaden support for systems interoperability in general, and potentially assist in clearing roadblocks in the delivery of relevant projects.

12. Data Exchange Phase One: Defining the solution requirements with confidence
Eight Core Activities…
1. Understanding Interoperability
& Learning from others
5. Legislation
2. Stakeholder Engagement
6. Interdependency 1: Definitions,
Data Models & Standards
3. Scoping of Data Exchange
7. Technical Architecture:
Documenting requirements
and appraising high level solutions
4. Demonstrations of Technologies &
input from implementers
8. Establishing the project
and governance

13. Data Exchange Project : What a data exchange means to us
Principles to note:
School / Provider MIS market &
relationship not affected
All MIS are still free to be
structured as they please
(yellow, green, brown etc)
All MIS need to be able to
talk the same language at some
point, hence the ‘common
language translator’ prior to
the exchange all being red. A
common language is needed
for all exchange approaches
Movement of data can be
out of, and into, a school / LA
/ DfE
The central exchange
technologies can be set up in
infinite different ways…figuring
out the best way is where we
need most help!
Data exchange

14. Data Exchange: Benefits in an afternoon…
Children
Quicker Identification of those going off role and not reappearing – supporting troubled families
Interaction with school nurses would reduce ‘falling between the gap’ when moving schools
Data held on the child automatically accessible should the child cross LA boundaries
e.g.: statement information immediately available
Quicker processes for decision making e.g. FSM
DfE
More timely evidence base to support decisions
Less cost associated with obtaining data
More frequent information
Ability to develop new collections more quickly
Future proof system easily adapted to handle
academisation programme and future initiatives
Together with SPDP, support greater use of data
in schools and research community
Increase internal monitoring.
Enable schools and LAs to drive benchmarking
and analytical agenda. Putting evidence in the
hands of local decision makers
Schools / Providers
Reduced collection burden from DfE
Greater recognition / consistency with
DfE / ‘published’ data
Greater access to comparative data
Common Transfer File always available,
not dependent on departing school sending
Allow existing system compatibility
Input once use many times. No need to
send several consumers the same data (eg LA, DfE
Ofsted)
Potential to speed up processes such as
HR data when teachers move to a new school
Fast and
simple data
movement
LAs
Reduced administrative costs for things like FSM entitlement
Less time chasing schools for data
More frequent attendance data = better understanding of impact of interventions
Better information to support capacity knowledge and admissions process
Allow existing system compatibility
Cross boundary information immediately available

15. Vision – a reminder
The vision for the ‘To Be’ state Data Exchange capability is that data is:
Moved from one organisation to another with minimal manual intervention and interoperable data exchange
Uploaded to an operational data store and shared on a near real-time basis as part of day-to-day business processes that require it
Available to all of the people who need it for decision making when they need it 15

16. High Level Functional Requirements
Code
Functional Requirements
FR1
The Data Exchange Capability (DEC) shall be capable of providing dataflow services in either direction between any two end points (including statutory data exchanges and sending notifications to interested parties)
An end point can be a school or a local authority or the Department.
We want to be able to exchange information flexibly between any two or group of end points to future proof the solution
FR2
The DEC shall be able to transfer any dataflow sets within maximum volume limits
We want a solution that is as flexible as possible to avoid cost and delay in setting up new data collections / exchanges 16

17. High Level Functional Requirements
Code
Functional Requirements
FR3
The DEC shall provide facilities for authorised users to define dataflow services in terms of:
Data flow contents
Schedule for collection
Performance targets
Source and destination end points (individual or multiple)
We need to be able to set up the dataflows, including what they contain, when they need to be sent, how quickly and who to. We don’t want to have to ask the solution provider to do this for us, because that will increase cost and introduce delay
FR4
The DEC shall require minimal manual intervention once a dataflow service has been defined and configured
We want to make the transfer of information as easy as possible. Ideally a school will only need to set up a dataflow once and it will happen automatically afterwards. 17

18. High Level Functional Requirements
Code
Functional Requirements
FR5
The DEC shall provide facilities for authorised users to add, maintain or remove end points
We need to be able to add in new schools or agencies that education sector users want to exchange information with quickly and easily and without having to pay the supplier to do this.
FR6
The DEC shall provide facilities for authorised users to define validation rules, including:
What the validation rule is
At what points it should be applied
Where validation errors should be reported to
Validation is important.
Ideally the school management information systems will validate information as it is entered, but if any get past this stage, we want to be able to catch them as soon as possible.
Over time new rules might be identified, so we want to be able to add these in as we go along, rather than needing to pay the supplier to make changes.
If an error is detected, then we need to make sure that the right contact is informed, so that the data can be corrected at source, improving data quality for all improved 18

19. High Level Functional Requirements
Code
Functional Requirements
FR7
The DEC shall provide capabilities for authorised users to:
Undertake ad-hoc queries of any data that they are authorised to access
Create, maintain, schedule, use and delete standard reports
Prepare ad-hoc reports
Once data has been collected then it needs to be possible to report on it.
Schools need to be able to view their own data and to see how they compare with (anonymised) others.
Local Authorities need to view information about schools under their control
The DfE need to undertake statistical and other analysis of the data.
In all cases we do not want to pay the supplier to develop reports.
FR8
The DEC shall provide capabilities for authorised users to write to end point MIS and other systems
This is required, for example, to enable local authorities to provide information on free school meals to schools. 19

20. High Level Non Functional Requirements
Code
Non Functional Requirements
NFR1
The DEC shall achieve a data transfer time of:
30 minutes or less (TBC) for < 10% of the defined dataflow services
24 hours or less (TBC) for 45% of the defined dataflow services
7 days or less (TBC) for 45% of the defined dataflow services
We have been told that there is a requirement for data to be transferred in ‘near real time’ but it isn’t clear what this means.
We think that 30 minutes is quick enough and that this would only be required for a small percentage of dataflows. Is this right? Are the other timescales reasonable?
NFR2
The DEC shall be scalable:
To manage peaks and troughs (in volume, capacity and pupil cohort sizes)
Such that it can, if required, take on other Department data flow sets
We want the DEC to be able to cope with higher levels of data exchange in the future if required, to avoid the need to start again. 20

21. High Level Non Functional Requirements
Code
Non Functional Requirements
NFR3
The DEC shall be able to support
40,000 end points (TBD)
5 million messages daily (TBD)
The DEC needs to be able to support at least 25,000 schools and a range of other organisations. It may be that some schools have multiple end points, so this number needs confirming.
As to the number of messages, it will depend on the way of working. If messages are sent after every change, then the number of messages may be higher but the messages will be smaller. At the other extreme, if schools sent a message with all changes once a week, then this would be closer to 5,000 /day.
NFR4
The DEC shall be:
Available 24 hours per day, 7 days per week (not including maintenance periods) with a 99% availability
Resilient to ensure no more than 30 minutes interruption of service for Data Exchange services
The DEC will be a core infrastructure element within the education sector and should have a correspondingly high availability 21

22. High Level Non Functional Requirements
Code
Non Functional Requirements
NFR5
The DEC shall only allow users to access data for which they have a business need and legal right to do so
We need to ensure that access to data is controlled, so that data providers can be reassured that the data that they provide is only accessed by those authorised to do so. The DEC will need to be designed and built with security in mind.
NFR6
The DEC must comply with the HMG IA requirements for information up to:
IL3 for individual messages
IL4 in aggregate
The requirements for security are published in the Security Policy Framework and supporting documents. The Business Impact Levels (IL) are a measure of the impact of the inappropriate release of information. This requirement defines the impact level of the data and this will affect the strength of the security controls that need to be put in place. 22

23. High Level Non Functional Requirements
Code
Non Functional Requirements
NFR7
The DEC shall record when data was gathered, from where and who entered it (TBD)
We need this audit trail to meet security requirements and for data quality management.
NFR8
The DEC shall provide an intuitive user interface that satisfies the requirements of the DDA and related accessibility standards (eg W3C)
We need the DEC to be easily usable by anyone that needs to use it as part of their role
NFR9
The DEC shall make use of open standard technologies
This is a requirement based on HMG ICT strategy. It means that it should be easier to maintain or upgrade the system in the future and avoid lock in to any particular supplier. 23