Evaluating the Effects of Business Register Updates on Monthly

Survey Estimates

Daniel Lewis

Overview

• Introduction

• Different strategies for updating the business register

• Simulation method for testing updating strategies

• Simulation results

• Conclusions and comments

Introduction

• Inter-Departmental Business Register (IDBR) used by most

ONS business surveys

• Key variables – employment, turnover, industry (SIC)

• Updated from different survey and admin sources

• Quality of updates affects survey estimates

• Desire to produce priority rules for updating

• Accuracy of updating strategies tested by simulation

Sources for updating the IDBR

• Employment:– Pay As You Earn tax data (PAYE)– Business Register Survey (BRS)– Imputed from turnover data

• (Annual) Turnover:– Value Added Tax data (VAT)– Imputed from employment data

• Standard Industrial Classification (SIC):– VAT– PAYE– BRS

Updating scenarios tested

• Employment:– Always favour PAYE– Always favour BRS– Favour BRS if less than 3 / 2 / 1 years old

• SIC:– Always favour PAYE– Always favour VAT– Always favour BRS– Favour BRS if less than 3 / 2 / 1 years old– Range of options for second priority if first unavailable

• Frequency of updates – monthly, quarterly, annually

Method for testing updating scenarios

• Assess effect on monthly turnover survey estimates

• Simulation method using four steps:

1. Simulate 12 months ‘real world’ data

2. Create business register

3. Select and survey samples from register

4. Calculate estimates and compare to ‘true’ values

1. Simulate ‘real world’ data (i)

• Use January IDBR data as starting point

• Analyse 12 months of IDBR data to estimate:– Probability of a business dying– Probability of changing SIC– Probability of a change in turnover– Probability of a change in employment– Probability of a new business being born

• Probabilities used to randomly assign characteristics to businesses each month

1. Simulate ‘real world’ data (ii)

• Changes in employment and turnover modelled based on observed means and standard deviations within strata

• Change in SIC randomly assigned based on probabilities of each type of SIC change

• Births and deaths also randomly introduced based on probabilities

• Monthly turnover data created by comparison with weighted monthly survey data from the same year

monthly survey turnover estimatesimulated annual turnover

actual IDBR annual turnover

2. Create business register

• Create register variables from ‘real world’ for each updating source with matching (realistic) quality deficiencies

• Quality parameters for each source derived by comparing average changes in employment and SIC before and after main register update

• Variables for each updating scenario derived by adding random variation to ‘real world’ value:

,

,

, ,

where unit value for a given source and age

real world value for unit

quality parameter for the source and age

N(0,1)

i sa

i real

sa

i

i sa sa ii real

Y i

Y i

Q

Z

Y Y Q Z

3. Select and survey samples

• Samples drawn many times from the simulated business registers using typical business survey sample design

• Stratified by industry and employment

• Neyman allocation using annual turnover data

• Samples selected using Permanent Random Number sampling with different random start for each iteration and 15 month rotation period

• Turnover value ‘collected’ for each sampled business

4. Calculate and compare estimates

• Estimates of total turnover calculated for each sample using separate ratio estimator with employment as auxiliary

• MSEs calculated for each updating scenario by comparison with true turnover in the ‘real world’:

2

1

ˆ( )N

ii

Y YMSE

N

Simulation results

• Simulation very time consuming

– Only possible to draw 516 samples

– Just sufficient for convergence

• Best updating strategy:

– Update IDBR monthly

– Give preference to PAYE for employment

– Use BRS for SIC if less than 3 years old, otherwise VAT

• A few other options were not significantly worse

Conclusions and comments

• Method very computer intensive, but gave useful results

• Project time limited, so constrained to using simple methods

• Potential to extend the model to better reflect business survey populations and updating processes

• Then possible to test a wide range of business survey methods:

– Sample designs, rotation rates, estimation, variance estimation, outlier treatment, …