1. Rules Based Editing Maintaining quality in the drive for efficiency in the ONS
Kevin Moore
Head of Platforms Development and Support Branch
Business Data Division
The Office for National Statistics, UK

2. Outline Overview of ONS transformation RBE and vision
RBE Discovery details
RBE model, rules and their verification
Where next?

3. ONS transformation Shift from traditional paper-based surveys
Service Orientated Architecture
Digital by default
New data sources
Rationalisation and harmonisation of surveys/questions based on output need

4. Target ONS Enterprise Architecture

5. RBE vision
“To identify a standardised set of configurable and reusable rules based editing products and services that provide a consistency in the approach taken to data cleaning that fulfils the needs of data and output users.  In general, the rules should be applied as early in the process as possible, be efficient and automated by default
The use and result of using these products and services must be auditable to provide a transparent record that supports the data journey through the statistical production process”

6. Discovery details Six weeks Core team from across DC and MD Scope
All data sources
Includes comparison across sources
Micro and macro level
Links with other EA services

7. Out of scope Coding/Classification Orchestration of rules
Identification of new rules/methods
Storing/managing datasets

8. Discovery content Conceptual RBE model (how)
Identification of rules (what)
Verification
ONS
International
Classification of rules
Links with other services

9. Conceptual model Extended CORA model Built from ground up
Based on using “data sets”
2-stages: identification and action
Provides framework for implementation
Data management assumptions

10. RBE model NOTE: Lots of set up and administration required:
- Defining rule metadata
- Creating the Ruleset, Ruleset parameters
- Linking Rulesets with the Dataset (and its definition)

11. Identification of rules
CORA base, extended for social surveys
Stakeholder involvement
Definitions aligned to European work
Provides flexibility required

12. Model and rule verification 1/2
Users
Rule types confirmed with FSS and admin data
Opinion survey rules mapped to rule types
Census test questionnaire examined
Strategic activities
RBE implementation compared to current data collection (take-on) processes (CORA, CS, OR)
Consultation with eQ, SPP, Stat Methods, SBR, Metadata
Discussion with ONS Social Survey Division

13. Model and rule verification 2/2
International
Principles and rule classification aligned to ESSNet Validat Foundation methodology handbook
4 levels of validation rules
0: file structure/batch type
1: single variable constraint; is x > 0
2: comparison between instances of same output; period on period
3: comparison with different output/source; Survey T/O vs VAT T/O

14. Next steps Discovery work welcomed by DA (Feb)
Vehicle required to progress:
Census test 2019
Non-survey data collection platform
Metadata platform
Statistical methods library