oecd short-term economic statistics working partyjune 26-28 2006 analysis of revisions for...
TRANSCRIPT
![Page 1: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/1.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Analysis of revisions for short-term economic statistics
Richard McKenzie OECD
OECD Short Term Economic Statistics Working Party
![Page 2: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/2.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Analysis of revisions for short-term economic statistics
• Main results from OECD revisions analysis study for the index of industrial production
– Country comments and reactions
– Preliminary results from a similar OECD study on Retail Trade Volume
• OECD Main Economic Indicators Original Release Data and Revisions Database
– Providing the tools for countries to do their own analysis
![Page 3: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/3.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Revisions analysis for IIP
• An attempt to analyse 3 dimensions of statistical quality and make comparisons across countries
– Timeliness
– Accuracy (assessed through revisions)
– Coherency (between IIP and GDP in industry)
• Also able to study the interactions between these quality measures
• Monthly snapshots of data published in MEI CD-Rom from February 1999 – March 2006
![Page 4: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/4.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Main reasons for revisions in IIP
• Late data or revised data from respondents; later correction of errors or imputations
• Different estimation methodologies for early estimates and lower sample sizes
• Updating of seasonal factors
• Benchmarking to other sources (e.g. National Accounts or annual structural surveys)
• Changes to base period and changes in statistical methodology, classifications etc.
![Page 5: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/5.jpg)
0
0.5
1
1.5
2
2.5
3
3.5
Mea
n Ab
solu
te R
evis
ion(
%)
3 Months Later 1 Year Later 2 Years Later
Mean absolute revision to first estimates of year-on-year growth rates
![Page 6: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/6.jpg)
Mean absolute revision to first estimates of year-on-year growth rates
0
0.5
1
1.5
2
2.5
3
3.5
Mea
n Ab
solu
te R
evis
ion(
%)
3 Months Later 1 Year Later 2 Years Later
![Page 7: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/7.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Are revisions statistically significant?
• Mean absolute size of revisions is important
– Revisions analysis quantifies this and provides a basis to review our compilation methods
• Ideally revisions should centre around zero over time (i.e. equally likely to be + or - )
– If this is not true then reasons causing this tendency in the compilation process must be found
![Page 8: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/8.jpg)
-0.8-0.6-0.4-0.2
00.20.40.60.8
11.21.41.61.8
Mea
n Re
visi
on(%
)
Month on Month Year on Year
Mean revision between first estimates and published one year later
![Page 9: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/9.jpg)
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
Mea
n R
evis
ion(
%)
Month on Month Year on Year
Mean revision between first estimates and published one year later
![Page 10: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/10.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Which short term measures should we advise users to focus on?
• Are first estimates of month-on-previous-month growth rates from the IIP reliable enough to enable informed decision making?
– Not in most countries! First estimates of MoM growth rates revised by 2/3 initial value on average after one year (95% countries revised by more than 2/5)
– Shows the value that revisions analysis can provide to users
• How to extract the latest signals in estimates without being misleading?
![Page 11: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/11.jpg)
40
50
60
70
80
90
100
Perc
enta
ge
% Sign(Later) = Sign(Earlier)
Reliability of MoM growth rates to signal expansion or contraction(sign of first estimate compared to that one year later)
![Page 12: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/12.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Do revisions affect identification of turning points?
• Very difficult to study
– As future turning points interact with past revisions
• OECD Composite Leading Indicators dates turning points in IIP using the Bry-Boschan routine
– Compared series of first estimates of YoY growth rates with those published two years later, at the BB turning point spots
– Where turning points were evident in the YoY growth rates series, preliminary estimates and those 2 years later usually coincided or were very close
![Page 13: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/13.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Timeliness and accuracy
• Is there a trade-off between timeliness and size of revisions for the index of industrial production?
• Might expect those countries which publish earlier to have higher revisions – at least in the following two months
– E.g. due to estimation based on lower response rates, less editing of errors, flash estimation techniques
![Page 14: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/14.jpg)
IIP mean absolute revision to first estimate of monthly growth rate after two months
SMFFF
SF
S
S
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Perc
ent
![Page 15: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/15.jpg)
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
2 months 1 year 2 months 1 year 2 months 1 year 2 months 1 year
Finland Mexico Portugal Spain
Mea
n Ab
solu
te R
evis
ion(
%)
Before
After
Mean absolute revision to year-on-year growth rates before and after an improvement in timeliness.
![Page 16: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/16.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Coherence analysis
• In concept the Index of industrial production should have a strong relationship to Value added in industry from the National Accounts
• Coherency test: compare annual growth rates of IIP (most recent series) to annual growth rates in value added in industry from the national accounts
– Maybe the countries with high revisions are more coherent (e.g. due to revisions caused by benchmarking?)
![Page 17: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/17.jpg)
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Correlation between annual growth rates for the IIP and GDP in industry from the annual national accounts
![Page 18: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/18.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Review of country comments to IIP revisions analysis study
• Comparisons with similar studies done by the national institute (or comments on results / methods used)
– Generally similar results for countries that had done their own studies (some exceptions)
– Need to look at other time windows and remove outliers
– Sign change analyse of MoM revisions
– Impact of seasonal effects should be separated
– Few comments on the validity of the timeliness analysis
![Page 19: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/19.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Review of country comments to IIP revisions analysis study
• Comments on reasons for revisions and national revisions analysis policies
– About 1/3 countries reported aspects of their revisions analysis policy (some with recent changes and efforts to be clear to users in this area)
– Need to be wary of revisions analysis as definitive measure of quality (one user emphasised the preference for transparency on statistical methods)
– Reasons for revisions covered earlier, all in line with those presented in the paper
![Page 20: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/20.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Review of country comments to IIP revisions analysis study
• Comments on the importance of performing revisions analysis for Short term statistics
– 2/3 countries stressed the importance of performing revisions analysis
– Many countries welcomed the provision of tools and a standardised approach which can save their resources
![Page 21: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/21.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Preliminary results from Retail Trade Volume revisions analysis
• Very similar results to analysis of IIP
– Mean absolute revisions increase the longer the revision interval
– Revision size similar across countries – with some countries noticeably higher and with mean revision statistically significant
– First estimate of MoM growth rates subject to large revision within one year
![Page 22: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/22.jpg)
Mean absolute revision to first estimates of year-on-year growth rates
0
0.5
1
1.5
2
2.5
AUS AUT BEL BRA CAN CHN CZE DEU DNK EMU ESP FIN GBR HUN
Mea
n A
bsol
ute
Rev
isio
n(%
)
3 Months Later 1 Year Later 2 Years Later
![Page 23: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/23.jpg)
Mean absolute revision to first estimates of year-on-year growth rates
0
0.5
1
1.5
2
2.5
IRL ITA J PN KOR MEX NLD NOR NZL POL PRT RUS SWE USA ZAF
Mea
n Ab
solu
te R
evis
ion(
%)
3 Months Later 1 Year Later 2 Years Later
![Page 24: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/24.jpg)
-0.8-0.6-0.4-0.2
00.20.40.60.8
1
AUS AUT BEL BRA CAN CHN CZE DEU DNK EMU ESP FIN GBR GRC HUN
Mea
n Re
visi
on(%
)
Month on Month Year on Year
Mean revision between first estimates and published one year later
![Page 25: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/25.jpg)
Mean revision between first estimates and published one year later
-0.4-0.3-0.2-0.1
00.10.20.30.40.50.6
IRL ITA JPN KOR MEX NLD NOR NZL POL PRT RUS SWE USA ZAF
Mea
n Re
visi
on(%
)
Month on Month Year on Year
![Page 26: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/26.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
OECD Main Economic Indicators Original Release Data and Revisions Database
• Full time series of data published every month starting from the February 1999 edition of the Main Economic Indicators for 21 key economic variables
• Access OECD revisions analysis studies for GDP, Index of Industrial Production and Retail Trade Volume
• Automated programs and detailed user guide allowing users to perform there own revisions analysis for any country / variable combination available in the database
• Information on reasons for revisions and references to international recommendations for establishing revisions analysis policies (IMF SDDS and OECD)
![Page 27: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/27.jpg)
![Page 28: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/28.jpg)
Live version
• http://stats.oecd.org/mei/default.asp?rev=1
![Page 29: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/29.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
Conclusions / Future work
• Engaging users on results of revisions analysis studies
– Especially the expected reliability of different measures from first estimates – should we do more work to promote more appropriate measures?
• How to promote the facilities developed by the OECD
– Large saving of resources for NSIs to perform revisions analysis, with OECD or their own data
• How to use the results from revisions analysis to improve compilation methods?
![Page 30: OECD Short-Term Economic Statistics Working PartyJune 26-28 2006 Analysis of revisions for short-term economic statistics Richard McKenzie OECD OECD Short](https://reader036.vdocuments.us/reader036/viewer/2022070323/56649d995503460f94a838e7/html5/thumbnails/30.jpg)
OECD Short-Term Economic Statistics Working PartyJune 26-28 2006
THE END