business data linking recent uk experience. business data in the uk common register (idbr) since...
DESCRIPTION
the Business Data Linking project (BDL) begun in the late 1990s –core dataset: Annual Respondents Database –other datasets: R&D, skills, Community Innovation Surveys, e-commerce, New Earnings Survey… joint venture between ONS, OGDs*, academics academics on secondment work in a “safe setting” no access outside ONS outputs checked manually for disclosure checking *OGD: other government departmentTRANSCRIPT
![Page 1: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/1.jpg)
Business data linkingrecent UK experience
![Page 2: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/2.jpg)
business data in the UK• common register (IDBR) since 1994• key law: Statistics of Trade Act 1947• data collection supervised by a Survey Control Unit
– concerns over burden on business– exemptions from repeat surveys for smallest firms
• devolved political and statistical framework– government departments separate bodies– data sharing has purposes and limitations specified
![Page 3: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/3.jpg)
the Business Data Linking project (BDL)• begun in the late 1990s
– core dataset: Annual Respondents Database– other datasets: R&D, skills, Community Innovation
Surveys, e-commerce, New Earnings Survey…• joint venture between ONS, OGDs*, academics• academics on secondment work in a “safe setting”• no access outside ONS• outputs checked manually for disclosure checking
*OGD: other government department
![Page 4: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/4.jpg)
sample outputs• solving the productivity problem?
– UK multinationals as productive as foreign-owned firms– domestically-oriented firms even more unproductive?
• ecommerce lowers prices!– ...perhaps...– actually seems to emphasise existing market conditions– competition increases - but monopolies get stronger too
• on-the-job versus general skills– linking skills and schooling data to firm data indicates a
genuine productivity gain from general human capital
![Page 5: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/5.jpg)
problems (1): “the ministry for adding things up”• microdata quality suffers
– statistical editing and block adjustment• redefinition and interpretation of data or metadata
– more problematic for micro users– eg SIC80-SIC92
• longitudinal integrity– crucial to micro analysis, irrelevant to macro numbers– not designed into repeat surveys
• documentation– different focus
![Page 6: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/6.jpg)
problems (2): sampling frames• small firms
– low probability of reselection– smallest excluded by design
• changes in census band• voluntary surveys• non-IDBR sample selection
![Page 7: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/7.jpg)
problems (3): inconsistencies • inconsistent across time
– eg ICT and innovation surveys• inconsistent across surveys
– eg foreign ownership
![Page 8: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/8.jpg)
problems (4): confidentiality• linking complicates disclosure control
– increases number of quality assurers• linking across small samples
– reduces frequencies– increases likelihood of disclosiveness
• no general government right to share data– explicit agreement needed to share data across OGDs
![Page 9: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/9.jpg)
new developments• timely electronic documentation• automatic matching• feedback into survey design• integrated data and metadata system
• increasing awareness of benefits of microdata– increases value of data– lowers business burden– answers new questions– improves knoweldge of datasets
![Page 10: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/10.jpg)
what have we learnt?• enthusiastic data providers are the key
– plan early for disclosure checking too– feed back
• check data version– may not be a ‘definitive’ file– and even ‘clean’ datasets need preparation time
• check micro validity - macro validity isn’t enough– duplicates and bad values– inconsistencies within and across datasets and time
• “useless” data can be useful when linked
![Page 11: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/11.jpg)
finally...• be prepared to take the lead• don’t get stressed
– recognise the data wasn’t collected for this purpose– enjoy the fact that is available
• talk about it
![Page 12: Business data linking recent UK experience. business data in the UK common register (IDBR) since 1994 key law: Statistics of Trade Act 1947 data collection](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b5d7f8b9ab0599abb7e/html5/thumbnails/12.jpg)
contact
Felix Ritchie
Business Data Linking Office for National Statistics 1 Drummond Gate London SW1V 1QQ