improvements to economic survey methodologies to reduce revisions in published estimates
DESCRIPTION
Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates. François Brisebois, Statistics Canada International Total Survey Error Workshop June 15, 2010. Outline. Revisions Context Revisions vs. Total Survey Error My stories about revisions - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/1.jpg)
François Brisebois, Statistics CanadaInternational Total Survey Error Workshop
June 15, 2010
Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates
![Page 2: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/2.jpg)
Outline Revisions Context Revisions vs. Total Survey Error My stories about revisions
1. Revisions in tax data2. Study about the sources of revisions3. Quality indicator incorporating revisions
Points for discussion
![Page 3: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/3.jpg)
Revisions Release of preliminary, revised and final figures
• Timeliness-accuracy trade-off • Users want both
Tracking of the size and direction of revisions helps assessing the trade-off• Coherence of signals from one vintage to another
![Page 4: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/4.jpg)
Context Sub-annual business surveys
* Monthly Survey of Manufactures• Monthly Food Services Survey* Monthly Wholesale and Retail Trade Survey• Quarterly Industry Revenue Indices (Services)
Used by the System of National Accounts
![Page 5: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/5.jpg)
Context Publication about 50 days after the reference
period for monthly surveys, about 90 days for the quarterly survey
Typical revision scheme:• Preliminary, revised 1, revised 2, annual revision
Quality indicators• Sampling variability• Nonresponse treatment
![Page 6: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/6.jpg)
Context All use two sources of data
1. Surveyed portion (census or survey)
2. Administrative portion Goods and Services Tax (GST) database
Includes sales of businesses Annually, quarterly or monthly
Calendarized to monthly data Data for a given reference month reprocessed every
month
![Page 7: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/7.jpg)
Revisions vs. Total Survey Error Sources of TSE
• Frame• Sampling• Measurement• Nonresponse
Focus on the last two• Measurement: Reported value revised• Nonresponse: Late reporting
“Longitudinal” dimension of total survey error
![Page 8: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/8.jpg)
My stories about revisions1. Revisions in tax data
• GST; same reference month reprocessed every month
• Is the quality of imputed data improving through processing (until we actually receive tax data)?
June YY July YY Aug YY Sept YY Oct YY ...Reference Month - Tax data received
- 1st processing -2nd processing - 3rd processing etc.
Example for a monthly remitter
![Page 9: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/9.jpg)
My stories about revisions2. Study about the sources of revisions
• Monthly Survey of Manufactures• Systematic downward revisions? • Examined main contributors to revisions
Survey/Admin * Reported/Imputed Main findings:
No significant trend in revisions « Survey - Reported » category showed highly unexpected
revisions Small downward trend in administrative data. Why?
Look for improvements (operational, methodological)
![Page 10: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/10.jpg)
My stories about revisions3. Quality indicator incorporating revisions
• Quarterly Industry Revenue Indices• Design = Census
Complex units = Collected data Simple units = Administrative data
• Quality indicator? No sampling error Nonresponse dealt with imputation Non-negligible revision rates for some industries
![Page 11: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/11.jpg)
My stories about revisions3. Quality indicator incorporating revisions
• Quality indicator combining three criteriaa) Combined reported rate of the survey and
administrative data portionsb) Variance due to imputationc) Revision rate
• Approach to be examined to expend to other designs
![Page 12: Improvements to Economic Survey Methodologies to Reduce Revisions in Published Estimates](https://reader035.vdocuments.us/reader035/viewer/2022081604/56813061550346895d962f5b/html5/thumbnails/12.jpg)
Points for discussion Magnitude of revisions:
• Should revisions be monitored more closely?Users’ #1 tool to evaluate/challenge quality
• How are revisions dealt with in your organisation? Overall indicator of quality
• How can we incorporate revisions into our measures of quality?Typically sampling error and nonresponse/imputation rates