handbook on precision requirements and variance estimation for ess household surveys denisa...

Handbook on Precision Requirements and Variance Estimation for ESS Household Surveys

Denisa Florescu, Eurostat

European Conference on Quality in Official StatisticsVienna, 3-5 June 2014

2

Contents

1. Standard formulation of precision requirements

2. Variance estimation methods and tools Good and bad practices

3. Approaches to compute standard errors for national and EU statistics

4. Guidance to assess the compliance to the requirements

3

Standard formulation of precision requirements

Two strategies

Precision thresholds to be met by a few main target national indicators

Minimum effective sample sizes to be ensured by National Statistical Institutes

• Quality of the output

• Recommended for regulations

• They ensure satisfactory precision for a few indicators, too

• Design requirements, not quality of the output

4

Standard formulation of precision requirements

Precision measures geared to the type of statistics

Relative precision measures Absolute precision measures

recommended for:

• Totals and means of continuous variables

• Proportions

• Ratios and changes close to 0

5

Impact of proportion on the minimum sample size needed to achieve a coefficient of variation of 5 %,

under simple random sampling

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

estimated proportion

Sample size

6

Standard formulation of precision requirements

For EU regulations, for:

proportions

overall national estimates and estimates of national breakdowns

estimates of level and net changes of estimates of level

Some versions e.g. precision expressed as model function of estimated proportions

7

Evaluation of methods and recommendations using various criteria e.g.:

Applicability to sampling designs and types of statistics: choice guided by a

developed matrix:

Types of statistics

Sampling designs

Linear Ratios Non-linear, smooth

Non-smooth

… … … … …

… … … … …

• Suitable methods

• Unsuitable methods

• References

Variance estimation methods

8

Type of data Approach Methods

Aggregated Decentralised in NSIs Various

Integratedburden shared by NSIs and Eurostat

Generalised variance functionsparameters provided by NSIs

Approaches to compute standard errors for national and European statistics

9

Type of data Approach Methods

Microdata Integratedburden shared by NSIs and Eurostat

Replication methods

Fully centralized

burden in Eurostat

Replication methods

Approaches to compute standard errors for national and European statistics

10

Guidance to assess compliance to requirements

Principles of transparency and tolerance

3 strategies:

Use of integrated or fully centralised approach in Eurostat

Trace systematic deviations on the basis of quality reports (metadata template proposed)

Fixed normative rules agreed in advance between NSIs and Eurostat

11

Thank you for your attention

Contact: ESTAT-Methodology@ec.europa.eu