![Page 1: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/1.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Impact and timing of revisions for seasonally adjusted series relative to
those for the corresponding raw series
Richard McKenzie, Zakia Adam & Michela Gamba
OECD
OECD / Eurostat taskforce on performing revisions analysis for sub-annual economic
statistics
![Page 2: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/2.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Item VI of task-force Terms of Reference
“….. be able to break down the results of a revisions analysis study to quantify the impact of different (homogenous groups of) sources of revision. As a minimum the guidelines must describe how the effects of seasonal adjustment on revisions can be separated and quantified in relation to other reasons.
![Page 3: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/3.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Description of OECD study
• Compare revisions analysis for raw and SA series for selected countries, IIP & Retail trade
– Size of raw and SA revisions at different intervals
– Occurrence of revisions to raw and SA series
– Bias in revisions to raw series relative to SA series
– Revisions of raw and SA series for year-on-year growth rates
– Provide a tool which would allow countries to perform similar studies with their data
![Page 4: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/4.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Countries chosen
• Balance of EU and non-EU countries
• Include some countries where seasonal adjustment is performed by OECD (forward factor)
• Most importantly, need vintage series for all monthly snapshots with few missing updates
– Last criteria presented the biggest problem due to lack of availability of vintages of raw series
![Page 5: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/5.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Index of industrial production
• United States, Japan, Korea, Belgium, Czech Republic, Finland, France, Netherlands, Poland
• Finland, Belgium, Netherlands – raw data is in fact working day adjusted
• Belgium, Czech Republic, Poland – SA done by OECD (forward factor method)
![Page 6: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/6.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Retail Trade Volume
• Canada, Japan, Korea, Mexico, Norway, Sweden
• SA done by OECD for Mexico
![Page 7: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/7.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Size of raw vs SA revisions for month-on-previous-month growth rates
• Consider relative absolute mean revision (RMAR)
• Review for different revision intervals (is the relationship stable across time?)
![Page 8: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/8.jpg)
Relative mean absolute revision (RMAR) of IIP for USA
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
M2_P M3_P M3_M2 Y1_P Y2_P L_P Y2_Y1 Y1_M2
raw sa
![Page 9: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/9.jpg)
Relative mean absolute revision (RMAR) of IIP for Finland
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
M2_P M3_P M3_M2 Y1_P Y2_P L_P Y2_Y1 Y1_M2
raw sa
![Page 10: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/10.jpg)
Relative mean absolute revision (RMAR) of IIP for Korea
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
M2_P M3_P M3_M2 Y1_P Y2_P L_P Y2_Y1 Y1_M2
raw sa
![Page 11: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/11.jpg)
RMAR ratio between raw and SA for Y1_P
0.00
0.10
0.20
0.30
0.40
0.50
0.60
BEL
CZE
FIN
FRA
JPN
KOR
NLD
POL
USA
CAN
JPN
KOR
MEX
NOR
SWE
Index of Industrial Production Retail trade
![Page 12: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/12.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Size of raw vs SA revisions for month-on-previous-month growth rates
• Revisions of raw consistently smaller than SA by similar magnitude for different revisions intervals
– Ratio differs across countries, WDA is evident (larger revisions relative to true raw series)
• Similar picture for Retail Trade, but revisions to raw series even smaller in proportion of those to SA
• Overwhelming evidence that majority of revisions to SA series is from process of seasonal adjustment
• ….. But month on month growth rates of raw series are of little use to users / analysts …..
![Page 13: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/13.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Occurrence of revisions for raw and SA series
• How many months after first published values do we expect data to be revised?
– Raw series should only be affected by late data, fixing errors etc. (especially if direct from one survey with no models etc.)
– Revisions to SA series will depend on revisions to raw series and process of seasonal adjustment
• Revisions in longer term may also affect raw series (e.g. rebases, methodological changes, benchmarking etc.)
![Page 14: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/14.jpg)
Poland total non zero revisions for IIP
0
10
20
30
40
50
60
70
80
M1_P M2_M1 M3_M2 M4_M3 M5_M4 M6_M5 M9_M6 Y1_M9 Y2_Y1 Y3_Y2
occurrence_rawoccurrence_sa
![Page 15: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/15.jpg)
Japan total non zero revision for IIP
0
10
20
30
40
50
60
70
M1_P M2_M1 M3_M2 M4_M3 M5_M4 M6_M5 M9_M6 Y1_M9 Y2_Y1 Y3_Y2
occurrence_rawoccurrence_sa
![Page 16: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/16.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Occurrence of revisions for raw and SA series
• IIP
– occurrence of revisions similar for raw and SA for USA, Netherlands, Korea, Japan
– But raw less often revised than SA for Belgium, Czech Republic, Finland, France and Poland
• Retail trade
– Occurrence of revisions similar for Korea, raw a bit less for Japan and raw much less for Canada, Mexico, Norway & Sweden
![Page 17: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/17.jpg)
Total non zero revision for IIP for NON WDA countries (RAW)
0102030405060708090
100
M1_P M2_M1 M3_M2 M4_M3 M5_M4 M6_M5
CZE JPN KOR POL USA
Total non zero revision for IIP for WDA countries (RAW)
0102030405060708090
100
M1_P M2_M1 M3_M2 M4_M3 M5_M4 M6_M5
BEL FIN FRA NLD
![Page 18: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/18.jpg)
Total non zero revision for IIP for NON WDA countries (RAW)
0102030405060708090
100
M1_P M2_M1 M3_M2 M4_M3 M5_M4 M6_M5
CZE JPN KOR POL USA
Total non zero revision for Retail trade for all countries (RAW)
0102030405060708090
100
M1_P M2_M1 M3_M2 M4_M3 M5_M4 M6_M5
CAN JPN KOR MEX NOR SWE
![Page 19: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/19.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Occurrence of revisions: raw vs SA • Some evidence that revisions to raw series less
likely the further from first published data than for SA
• More evident for Retail than IIP and likelihood of revisions to raw for Retail trade is lower
– Reflecting again that Retail trade data more likely to be compiled from single survey in most countries
• Further work needed to understand the impact SA method has on occurrence and size of revisions
• OECD tool now developed to break down revisions on a month by month basis
![Page 20: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/20.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Are revisions statistically significant?
• Ideally revisions should centre around zero over time (i.e. equally likely to be + or - with mean revision close to 0)
– If this is not true then reasons causing this tendency in the compilation process should be found
• Should not expect a bias to exist in the raw series but not in the SA and vice versa
![Page 21: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/21.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Mean revision statistically significant for raw but not SA (& vice versa)
• IIP
– Some evidence found for Belgium and Korea but very much on the margin
• Retail trade
– Some evidence for Canada but mean revision still very small
– For Korea, raw series mean revision significant (at 10% and 5% level) for all revisions intervals but not for SA
> Mean revisions smaller but growing with length of interval (SA is in same direction and close to SIG)
![Page 22: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/22.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Which short term measures should we advise users to focus on?
• Are first estimates of month-on-previous-month growth rates of SA series for IIP and Retail Trade reliable enough to enable informed decision making?
– Not in most countries! First estimates of MoM growth rates revised by 2/3 initial value on average after one year (95% countries revised by more than 2/5)
– Shows the value that revisions analysis can provide to users
• Previous study suggested that users should focus on year-on-year growth rates for short-term analysis as these are more robust with respect to revisions
![Page 23: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/23.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Revisions to year-on-year growth rates
• Which series should year-on-year growth rates be calculated on?
– OECD Data presentation handbook recommends YoY growth rates to be calculated from raw or WDA series if available
• What about robustness of raw / WDA and SA YoY growth rates with respect to revisions?
![Page 24: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/24.jpg)
MAR of M3_P for IIP for WDA countries - YoY
0.00
0.50
1.00
1.50
2.00
2.50
BEL FIN FRA NLD
raw SA
![Page 25: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/25.jpg)
MAR of M3_P for IIP for Non WDA countries
0.00
0.10
0.20
0.30
0.40
0.50
0.60
CZE JPN KOR POL USA
raw SA
![Page 26: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/26.jpg)
MAR of M3_P for Retail trade
0.00
0.10
0.20
0.30
0.40
0.50
CAN JPN KOR MEX NOR SWE
raw SA
![Page 27: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/27.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Revisions to year-on-year growth rates: Conclusion / Future work
• Some weak evidence to suggest that YoY are more robust with respect to revisions for raw series compared to SA
– Thus supports recommendations in the handbook
– More empirical research would be interesting for users, particularly for WDA series
![Page 28: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/28.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Conclusions / Future work
• Relative mean absolute revisions (RMAR) for month-on-month growth rates are much lower for raw relative to SA series
– Process of seasonal adjustment accounts for more than half of RMAR for the SA series, sometimes up to 90% and indeed still more than 50% in comparison to WDA series
– …… but mean average revision is still generally higher for RAW series and month-on-month growth rates for raw series are not much use for short-term analysis
![Page 29: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/29.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Conclusions / Future work
• Likelihood of revisions to raw series (MoM growth rates) drops considerably after the first month but tends to stabilise at lower level in following months
• Likelihood of short-term revisions to SA series should depend on method used but more work needed to assess the relative impacts
• New OECD tool allows analysis of revisions to raw series on a month-on-month basis up to 6 months from first published value
![Page 30: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/30.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
Future work
• Disseminate results on OECD website, prepare paper and discuss implications for taskforce recommendations
• Use detailed revisions triangles to try and identify and quantify other reasons for revisions?
• OECD also conducting a study to look at the impact of improvements in timeliness on revisions
![Page 31: OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the](https://reader035.vdocuments.us/reader035/viewer/2022062715/56649d975503460f94a819a7/html5/thumbnails/31.jpg)
OECD Short-Term Economic Statistics Working PartyJune 25-27 2007
THE END