Evaluating generalised calibration / Fay-Herriot model in CAPEX

Tracy Jones, Angharad Walters, Ria Sanderson and Salah Merad (Office for National Statistics)

Overview

• Introduction

• Generalised calibration estimation

• Fay-Herriot model

• Conclusions and further work

Introduction

• Quarterly survey of capital expenditure (Capex)– Sample size – 28,000– Stratified by industry and size– Main user is National Accounts– Many zeros and some very large values

• Aim to reduce costs and respondent burden – Reduce the sample size whilst maintaining quality

• Investigated two strategies– Calibration estimation in cut-off sampling– Fay-Herriot model

Current cut-off sampling

• Not sample businesses with < 20 employees

• G-weights adjusted to account for this

Sampled (20-299)

Fully enumerated (300+)

Not sampled (<20)

Extension of cut-off sampling

• Extend to a cut-off of < 50 employees

• Sample size reduced by about 9,000• Reduce bias introduced through cut-off sampling

Sampled (50-299)

Fully enumerated (300+)

Not sampled (<50)

Relationship between acquisitions and employment

Direct calibration

• Find set of weights wi such that:

distance (d,w) is minimised while

• Solution

Xx si

iiw

i

iT

ii cFdw

x

Generalised calibration (Deville 2002)

Xx

si

ii

iT

i c

xFd

Xxz si

iiT

iFd

The set of calibration equations

Can be generalised to yield the set of equations

Generalised calibration

• In context of cut-off sampling, Haziza et al

(2010) assumed a linear function F of the form

• And obtained weights

Esi

iii

iii d

-dwTE xz

zXX ˆ1~

iT

iTF z1z

Applying generalised calibration

• Cut-off set deterministically based on employment

• Consider two auxiliary variables:–x well correlated with variable of interest

• employment from the business register

–z well correlated with probability of being above the cut-off• turnover from the business register

Generalised calibration estimation

• 2008 sample data – Bands 2 to 4

• 3 Estimates – Ratio estimate using full sample data – Ratio estimate with extended g-weight adjustment– Generalised calibration estimate

• Relative difference compared to ratio estimate using full sample data

Results

Relative difference (in %) compared to ratio estimate using full sample data

Period Ratio estimate with g-weight adjustment for

band 2

Generalised calibration estimate

Q1 2008 3.3% 32.1%

Q2 2008 5.9% 41.6%

Q3 2008 7.8% 32.8%

Q4 2008 8.1% 37.0%

Industries with largest contribution to total acquisitions in size-bands 2-4

Summary – Extension of cut-off sampling

• Adjusted g-weights method performs better overall

• Generalised calibration estimation does not consistently improve on simple method in any industry

• Residual relationship between x and p

Fay-Herriot model

• Combine direct estimate with synthetic estimate

• Fay-Herriot aggregate level model fitted to obtain synthetic estimator

,direct i i i ix u e

i=1, 2, …,m

Fay-Herriot model - BLUP

)ˆ( ,2

2

idirectu

ui

V

iiidirectiicomb x ˆ1ˆˆ,,

Fay-Herriot model

• 2008 sample data

• Two variables - total acquisitions and total disposals

• Auxiliary variables for Fay-Herriot model – VAT turnover and expenditure

• Scaled estimates and auxiliary variables using the total number of employees

• Fitted mixed model

Plot of Residuals against Predicted (mixed model with no transformation)

Transformation

• Transformation needed

• Implementation of BLUP becomes complicated– noted by Chandra and Chambers, 2006

,log logdirect i i i ix u e

Plot of Residuals against Predicted(mixed model)

Plot of Residuals against Predicted(linear model without random effects)

Back transformation

• Used back transformation to obtain synthetic estimate (Chambers and Dorfman, 2003)

• Calculation of gamma - variance of random effects required back transformation

ie xisyn ee ˆ2/ˆ

,

2ˆ

Evaluating use of Fay-Herriot model

• Gamma very high – Gamma using back transformation may not be

suitable

• Investigated combined estimate using a fixed value for gamma

• Evaluation is via re-sampling – Reduced the sample size by 25% (about 6,000 units)– Repeated sub-sampling– Set gamma to 0.7 – Calculated a combined estimate

Evaluating use of Fay-Herriot model

• Estimated Bias and MSE of combined estimate

fullidirect

K

krunicomb

icomb KBias

k

,,1

,,

,

^ˆ

ˆˆ

Results

• Average of the direct estimates very similar to the direct estimate from the full sample

• Variance of the synthetic estimate is small

• Variance of combined estimate lower than variance of direct estimate from full sample

• Bias is high in most industries– Relative bias also large

• High bias ratio resulted in higher Mean Square Error in most divisions

Results – Acquisitions Q2 2008

Division

Percentage bias

(bands 2 to 4)

Percentage bias

(bands 2 to 5)

Bias ratio

Percentage

difference in MSE

Overall 21.0 5.9 7.1

20 3.5 1.9 0.3 -25

40 18.3 0.4 1.2 139

52 29.8 2.9 1.8 275

25 11.7 7.4 1.7 266

55 12.5 3.0 2.3 509

32 62.2 20.8 5.8 3371

15 28.7 5.5 5.3 2837

Conclusions and further work

• Cut-off sampling with g-weight adjustment performed best– Know this has bias

• More work to be done – Impact on growth– Modelling at unit level– Additional covariates– Alternative estimation methods

• Model-based direct approach (Chandra and Chambers, 2006)

Questions