1. Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis

2. Overview Introduction Different strategies for updating the business register Simulation method for testing updating strategies Simulation results Conclusions and comments

3. Introduction Inter-Departmental Business Register (IDBR) used by most ONS business surveys Key variables – employment, turnover, industry (SIC) Updated from different survey and admin sources Quality of updates affects survey estimates Desire to produce priority rules for updating Accuracy of updating strategies tested by simulation

4. Sources for updating the IDBR Employment: –Pay As You Earn tax data (PAYE) –Business Register Survey (BRS) –Imputed from turnover data (Annual) Turnover: –Value Added Tax data (VAT) –Imputed from employment data Standard Industrial Classification (SIC): –VAT –PAYE –BRS

5. Updating scenarios tested Employment: –Always favour PAYE –Always favour BRS –Favour BRS if less than 3 / 2 / 1 years old SIC: –Always favour PAYE –Always favour VAT –Always favour BRS –Favour BRS if less than 3 / 2 / 1 years old –Range of options for second priority if first unavailable Frequency of updates – monthly, quarterly, annually

6. Method for testing updating scenarios Assess effect on monthly turnover survey estimates Simulation method using four steps: 1.Simulate 12 months real world data 2.Create business register 3.Select and survey samples from register 4.Calculate estimates and compare to true values

7. 1. Simulate real world data (i) Use January IDBR data as starting point Analyse 12 months of IDBR data to estimate: –Probability of a business dying –Probability of changing SIC –Probability of a change in turnover –Probability of a change in employment –Probability of a new business being born Probabilities used to randomly assign characteristics to businesses each month

8. 1. Simulate real world data (ii) Changes in employment and turnover modelled based on observed means and standard deviations within strata Change in SIC randomly assigned based on probabilities of each type of SIC change Births and deaths also randomly introduced based on probabilities Monthly turnover data created by comparison with weighted monthly survey data from the same year

9. 2. Create business register Create register variables from real world for each updating source with matching (realistic) quality deficiencies Quality parameters for each source derived by comparing average changes in employment and SIC before and after main register update Variables for each updating scenario derived by adding random variation to real world value:

10. 3. Select and survey samples Samples drawn many times from the simulated business registers using typical business survey sample design Stratified by industry and employment Neyman allocation using annual turnover data Samples selected using Permanent Random Number sampling with different random start for each iteration and 15 month rotation period Turnover value collected for each sampled business

11. 4. Calculate and compare estimates Estimates of total turnover calculated for each sample using separate ratio estimator with employment as auxiliary MSEs calculated for each updating scenario by comparison with true turnover in the real world:

12. Simulation results Simulation very time consuming –Only possible to draw 516 samples –Just sufficient for convergence Best updating strategy: –Update IDBR monthly –Give preference to PAYE for employment –Use BRS for SIC if less than 3 years old, otherwise VAT A few other options were not significantly worse

13. Conclusions and comments Method very computer intensive, but gave useful results Project time limited, so constrained to using simple methods Potential to extend the model to better reflect business survey populations and updating processes Then possible to test a wide range of business survey methods: –Sample designs, rotation rates, estimation, variance estimation, outlier treatment, …