MIP: ANALYSIS OF METADATA AND DATA REVISIONS

Item 11.5DMES 12-13 December 2013

Presented by Dario Buono (Eurostat, MIP team)

Topics

Safebook for MIP metadata

Analysis of revisions of MIP scoreboards

2

What for?

• The big question:• What about the quality of the MIP indicators?

• The small question:• How safe are the MIP indicators?

Our Response

3

Some insights:

Expert Opinion poll is not a survey: • No sampling• Assumption that the respondents are the real experts• Likelihood and impact• 166 answers in 2 weeksResults:• By country (about 2/3 of the 28 Member States)• By indicator (11 headlines)

4

Likelihood Analysis

1. Impact factor weighting2. Computing the total score3. Normalization (64<=x<=256)4. Computing the uncertainty (percentage)5. Computing the safe probability (equivalent to 1

minus the uncertainty)

5

Results

• Interesting relevant findings for: • Deflated House Price • Financial Accounts• Balance of Payment • Unit Labour Cost • Government Debt Croatia

6

Analysis of revision of MIP scoreboards

• Small revisions, likely to bring in noise rather than news

• Large revisions, likely to bring in news rather than noise

• Policy: Never revise data. In this case you have neither news nor noise

7

Vintages

• 2 vintages, 1 set of revisions• MIP headline data• Last 5 years (2007-2011)

8

Radar of revisions

9

Background references

• ESS guidelines on revisionshttp://

epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-RA-13-016/EN/KS-RA-13-016-EN.PDF

• OECD/Eurostat Task Force on “Performing Revisions Analysis for Sub-Annual Economic Statistics”

• http://www.oecd.org/std/40315546.pdf

1. Basic/ core measures: targeting users that require quick, easy to understand information

2. Additional/ advanced measures: targeting users that require more in-depth analysis

3. Sophisticated / special user measures: information for detailed research purposes.

10

Summary of statistics to be used for analysis of revisions (1)

1. Mean absolute revision2. Median absolute revision3. Arithmetic average or mean revision4. Statistical significance of the mean revision5. Median revision:6. % of positive revisions:7. % of negative revisions:8. % of zero revisions:9. Adjusted t-statistic for significance of mean revision10.Critical values of t statistic for significance of mean revision11.Standard deviation of revision12.Root mean square revision13.Quartile deviation

11

Summary of statistics to be used for analysis of revisions (2)14.Minimum revision15.Maximum revision16.Range of revision17.Skewness18.% sign (later) = sign (earlier):19.Acceleration / deceleration20.Relative mean absolute revision:21.Average absolute value of first published estimate22.Correlation between revision and earlier estimate (test if revisions are

„noise‟)23.Correlation between revision and later estimate (test if revisions are

„news‟)24.Serial correlation of revisions:25.Decomposition of the Mean Squared Revision

12

Dynamics of Standard deviation within MS of 11 MIP headlines

13

Dynamics of Standard deviation within MS of 8 MIP headlines

14

MAX and MIN revisions of 2011 data by individual Member State

Revisions of 2011 data upward downward

Current account (as %of GDP) SK CY

Net international investment positions (as %of GDP) LU FI

Share of world exports CZ DE

Real effective exchange rates (REER)  NA  NA

Unit labour cost RO LU

Private debt (as %of GDP) UK LU

Private credit flow (as %of GDP) IE EE

Deflated house price index EL RO

Government sector debt (as %of GDP) IE RO

Financial sector liabilities  NL  LU

Unemployment  NA IE

15

Open discussion: the way forward

• Safebook: • consistent results with robust methodology (fine

tuning of questions)• Further involvement of ESTAT domain managers

and ESS experts • Revision analysis done using:

• different stats • more vintages • higher frequencies

16

Time for questions

17