1. Business data linking recent UK experience

2. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection supervised by a Survey Control Unit –concerns over burden on business –exemptions from repeat surveys for smallest firms devolved political and statistical framework –government departments separate bodies –data sharing has purposes and limitations specified

3. the Business Data Linking project (BDL) begun in the late 1990s –core dataset: Annual Respondents Database –other datasets: R&D, skills, Community Innovation Surveys, e-commerce, New Earnings Survey… joint venture between ONS, OGDs*, academics academics on secondment work in a “safe setting” no access outside ONS outputs checked manually for disclosure checking *OGD: other government department

4. sample outputs solving the productivity problem? –UK multinationals as productive as foreign-owned firms –domestically-oriented firms even more unproductive? ecommerce lowers prices! –...perhaps... –actually seems to emphasise existing market conditions –competition increases - but monopolies get stronger too on-the-job versus general skills –linking skills and schooling data to firm data indicates a genuine productivity gain from general human capital

5. problems (1): “the ministry for adding things up” microdata quality suffers –statistical editing and block adjustment redefinition and interpretation of data or metadata –more problematic for micro users –eg SIC80-SIC92 longitudinal integrity –crucial to micro analysis, irrelevant to macro numbers –not designed into repeat surveys documentation –different focus

6. problems (2): sampling frames small firms –low probability of reselection –smallest excluded by design changes in census band voluntary surveys non-IDBR sample selection

7. problems (3): inconsistencies inconsistent across time –eg ICT and innovation surveys inconsistent across surveys –eg foreign ownership

8. problems (4): confidentiality linking complicates disclosure control –increases number of quality assurers linking across small samples –reduces frequencies –increases likelihood of disclosiveness no general government right to share data –explicit agreement needed to share data across OGDs

9. new developments timely electronic documentation automatic matching feedback into survey design integrated data and metadata system increasing awareness of benefits of microdata –increases value of data –lowers business burden –answers new questions –improves knoweldge of datasets

10. what have we learnt? enthusiastic data providers are the key –plan early for disclosure checking too –feed back check data version –may not be a ‘definitive’ file –and even ‘clean’ datasets need preparation time check micro validity - macro validity isn’t enough –duplicates and bad values –inconsistencies within and across datasets and time “useless” data can be useful when linked

11. finally... be prepared to take the lead don’t get stressed –recognise the data wasn’t collected for this purpose –enjoy the fact that is available talk about it

12. contact Felix Ritchie Business Data Linking Office for National Statistics 1 Drummond Gate London SW1V 1QQ felix.ritchie@ons.gov.uk bdl@ons.gov.uk