ec.europa.eu · web viewthe benchmarking procedures improve the quality of the resulting hf data by...

ESS guidelines on temporal disaggregation,

benchmarking and reconciliationFrom annual to quarterly to monthly data

Version 27, October 2017

1ESS guidelines on temporal disaggregation, benchmarking and reconciliation

ForewordTemporal disaggregation and related methods are used to disaggregate low frequency time series to high frequency time series. Temporal disaggregation can be performed with or without one or more high frequency indicator series and can involve certain temporal constraints. Where temporal disaggregation is applied to more than one time series it is possible to also include constraints for contemporaneous consistency.The establishment of common guidelines for temporal disaggregation within the European Statistical System (ESS) is an essential step towards a better harmonisation and comparability of official statistics, especially the Principal European Economic Indicators (PEEIs). The ESS Guidelines on Temporal Disaggregation address the need for harmonisation expressed by many users such as European Commission services.The ESS Guidelines on Temporal Disaggregation present both theoretical aspects and practical implementation issues in a friendly and easy to read framework. They meet the requirement of principle 7 (Sound Methodology) of the European Statistics Code of Practice (CoP), and their implementation will also be in line with principles 14 (Coherence and Comparability) and 15 (Accessibility and Clarity). The guidelines also foster the transparency of temporal disaggregation practices by encouraging documentation and dissemination of practices. The guidelines include sections with a policy for seasonal adjustment, quarterly national accounts, labour market, and revisions consistent with the other guidelines on these topics. The specification of alternatives takes into account the operational issues.


AcknowledgmentsThe European Commission expresses its gratitude and appreciation for the work carried out by the members of the Task Force Temporal Disaggregation for authoring these Guidelines: XXXXXXX

The European Commission would like to thank the chairs of the DIME/ITDG Director Group, the ESS Working Group Methodology and of the Working Group National Accounts, XXXX, together with all members for their useful comments and support.

The European Commission would also like to thank all the members of the Task Force on Temporal Disaggregation, from Eurostat, from National Statistical Institutes, from the ECB and from National Central Banks who have contributed with their comments to improve the Guidelines.


Table of contentsFOREWORD........................................................................................................................................ 2

ACKNOWLEDGMENTS......................................................................................................................... 3

1. INTRODUCTION............................................................................................................................... 5

1.1 MOTIVATION FOR GUIDELINES..................................................................................................................51.2 SCOPE OF GUIDELINES............................................................................................................................51.3 COSTS AND RISKS...................................................................................................................................51.4 TERMINOLOGY.......................................................................................................................................61.5 METHODS............................................................................................................................................81.6 PRINCIPLES FOR TEMPORAL DISAGGREGATION..............................................................................................9

2. PRELIMINARY ANALYSIS................................................................................................................ 10

2.1: OBJECTIVE AND DESIGN OF DISAGGREGATION...........................................................................................102.2: ANALYSIS OF INDICATORS.....................................................................................................................112.3: DETAIL OF COMPILATION......................................................................................................................13

3. METHODOLOGICAL ASPECTS......................................................................................................... 14

3.1: PREFACE...........................................................................................................................................143.2: AVAILABLE METHODS...........................................................................................................................143.3: RELATED PROBLEMS............................................................................................................................173.4: MODEL SELECTION AND CHOICE OF INDICATORS........................................................................................20

4. BENCHMARKING........................................................................................................................... 21

4.1 CHOICE OF BENCHMARKING METHOD.......................................................................................................21

5. RECONCILIATION AND MULTIVARIATE BENCHMARKING................................................................24

5.1 CHOICE OF RECONCILIATION METHOD......................................................................................................24

6. SPECIFIC ISSUES............................................................................................................................. 26

6.1 CHOICE OF THE SOFTWARE.....................................................................................................................266.2 END OF SERIES....................................................................................................................................276.3 TEMPORAL DISAGGREGATION IN FLASH ESTIMATES.....................................................................................286.4 OUTLIERS IDENTIFICATION AND TREATMENT..............................................................................................296.5 TREATMENT OF SHORT SERIES.................................................................................................................306.6 BENCHMARKING AND RECONCILIATION IN CHAIN LINKED SERIES.....................................................................316.7 BENCHMARKING AND RECONCILIATION IN NATIONAL ACCOUNTS...................................................................336.8 BENCHMARKING AND RECONCILIATION IN LABOUR MARKET.........................................................................346.9 ADVANCED AND RECENT METHOD...........................................................................................................35

7. PRESENTATION.............................................................................................................................. 36

7.1 GENERAL REVISION POLICY AND RELEASE CALENDAR....................................................................................367.2 ACCURACY OF BENCHMARKING AND RECONCILIATION.................................................................................377.3 METADATA (?)....................................................................................................................................38

REFERENCES...................................................................................................................................... 39

ESS guidelines on temporal disaggregation, benchmarking and reconciliation

1. Introduction1.1 Motivation for GuidelinesThe European Statistical System (ESS) developed these guidelines to help data producers with deriving high frequency data (e.g. quarterly or monthly) from low frequency data (e.g. annual) while respecting the related temporal and accounting constraints. Typical applications are also known as benchmarking or reconciliation. With this aim, the guidelines identify the best practice among temporal disaggregation methods in order to:

achieve harmonisation across national processes; enhance comparability between results; increase the robustness of European aggregates.

The guidelines are aimed at European statistics (compiled by Eurostat) and country specific official statistics compiled by national statistical institutes (NSIs). The guidelines provide a consistent framework for temporal disaggregation, benchmarking and reconciliation, taking advantage of similarities in the process to define a common vocabulary to facilitate communication and comparison between practitioners.

1.2 Scope of Guidelines The guidelines attempt to cover important issues related to temporal disaggregation, benchmarking, and reconciliation from annual to quarterly to monthly data, from the choice of the methods, to revisions and documentation. Whether experts or non-experts, the framework remains the same, only the level of detail in the analysis varies. Each stage of the process is explained and options are described. Three alternative courses of action are highlighted: (A) Best alternative, (B) Acceptable alternative, and (C) Alternative to be avoided. These alternatives are characterised as follows:

(A) The best alternative should always be the target for producers. It can always be achieved with enough effort.

(B) The acceptable alternative should only be tolerated if time or resource issues prevent alternative (A).

(C) The alternative to be avoided should never be accepted, extenuating circumstances are not a valid excuse.

The objective of the guidelines is help producers move to alternative (A).

1.3 Costs and risksThe costs of applying alternative (A) may be significant as temporal disaggregation, benchmarking and reconciliation is time consuming in terms of human resources and requires a common and well defined IT structure. The risks of not applying alternative (A) are that inappropriate or low-quality temporal disaggregation, benchmarking and reconciliation can generate misleading results, e.g. over-smoothing which increases the probability of false signals leading to

1Introduction


misinterpretation of the dynamics of the data. This will negatively affect credibility and hence ultimately lead to reduced trust in official statistics.

1Introduction


1.4 TerminologyThe definitions in this section are applied throughout the guidelines. Some basic definitions of the types of data used in temporal disaggregation methods are provided followed by definitions of key methods related to temporal disaggregation.

Flow Definition: a flow is a measure of an (economic) phenomenon per time period.Examples: monthly turnover measured as the sum of daily turnover within each month, gross fixed capital formation, depreciation of capital.

Stock Definition: a stock is a measure of an (economic) phenomenon at a specific point of time. A stock is the result of cumulated flows.Examples: annual employment measured as the number people employed in an organization on the last day of each calendar year, capital stock at the end of a year.

Time seriesDefinition: a time series is a set of regular time-ordered observations of a quantitative characteristic of an individual or collective phenomenon taken at successive, in most cases equidistant, periods or points of time. Time series in official statistics are often either flow series or stock series.Examples: monthly turnover, quarterly unemployment, annual Gross Domestic Product.

Index seriesDefinition: an index series measures the relative size of a variable in a time period relative to a base period. A reference period is usually set equal to 100 and may or may not be the same as the base period.Examples: quarterly index of production measured as the ratio of quantity produced in each quarter relative to a base period (often a year) holding prices constant, chain-indices of GDP.

Averaged stock seriesDefinition: time series of averages relative to a quantity measured at a higher interval of time.Examples: annual employment measured as average of monthly data of employed persons in each calendar year. Quarterly average of monthly consumer prices.

ConstraintsDefinition: constraints limit to data provided in form of function (TO BE BETTER DESCRIBED). The function is usually of linear aggregation (simple or weighted sum, average, ...), but it can become nonlinear after nonlinear transformation of original data. Constraints are either binding or nonbinding respectively, when they are affected or not by errors.

1Introduction


Examples: Common binding linear aggregation constraint is that annual sum of quarterly estimates is equal to given annual totals. The aggregation constraint becomes nonlinear when original quarterly data are taken in logarithms. Nonbinding constraints are typical of balances for which it is simply enforced the equality among demand and supply without fixed values.

The next set of definitions concern methods discussed in these guidelines. Note that related terms are included as they are sometimes used interchangeably in the literature. However, that does not necessarily imply that they can be used interchangeably in every context, just that there may be some relation between them.

Temporal disaggregationDefinition: temporal disaggregation is the conversion of a low frequency time series to a higher frequency time series.Related terms: temporal distribution, benchmarking, interpolation, splining.Example methods: regression based methods (such as Chow-Lin), methods based on ad hoc multivariate models.

InterpolationDefinition: interpolation denotes the generation of values of a time series for time points that have not been sampled within the interval of time of the original time series; for example used for converting a low frequency stock time series to a higher frequency time series of stock data.Related terms: splining, benchmarking, methods based on multivariate models.Example methods: Cubic spline interpolation, any adjustment and regression based methods, methods based on ad hoc multivariate models.

BenchmarkingDefinition: benchmarking is adjusting a high frequency time series to have temporal consistency with lower frequency version of the same variable usually measured from a different data source. This is also known as binding benchmarking.Related terms: temporal disaggregation, temporal distribution, constraining.Example methods: Denton methods, any adjustment and regression-based methods, Growth Rate Preservation (for example Causey and Trager)

ReconciliationDefinition: reconciliation is adjusting multiple time series for contemporaneous consistency.Related terms: balancing, constraining, cross-sectional constraints, raking.Example methods: Multivariate Denton, methods based on multivariate ad hoc models.

ExtrapolationDefinition: extrapolation is predicting values of a time series for future or past time points that have not been sampled and are outside the interval of time of the original

1Introduction


Filippo FM. Moauro, 10/27/17,

In general it is better not to associate a method to the name of authors.

time series. In a temporal disaggregation problem, extrapolation related to estimating the high frequency data for time periods where the low frequency data are not available.Related terms: forecasting, nowcasting, backcasting, prediction.Example methods: ARIMA models, exponential smoothing, any adjustment and regression-based methods, multivariate methods.

CalendarizationDefinition: calendarization is adjusting a time series that is not observed on calendar periods to provide an estimate of the time series on the desired calendar periods.Related terms: temporal disaggregation, temporal distribution, interpolation, benchmarking.Example methods: Any adjustment and regression-based methods, methods based on multivariate ad hoc models.

1.5 MethodsA variety of mathematical and statistical methods have been developed and applied to solve the problem of temporal disaggregation, i.e. disaggregating a low frequency (LF) time series to a high frequency (HF) time series, where some temporal constraints (such as either the sum, the average, or the first or the last value) of the resulting HF series is consistent with the LF series. Some methods for temporal disaggregation also take into consideration accounting constraints that must be satisfied by the temporally disaggregated series. The target variables can be flow, stock, and index series. Typically flow series are associated with the speed of a phenomenon (e.g. sales per month or the number of births per year), and the LF values of flows should naturally correspond to the sums of HF values (e.g. the annual values should correspond to the sums of the sub-annual values). Stock series refer to the level of a phenomenon at a single point in time (e.g. a specific date), that is the annual values of stocks series (e.g. inventories) should pertain to one single sub-annual value, usually the first or the last value of the LF period. For the purpose of benchmarking and interpolation, index series are such that their annual benchmarks pertain to the annual averages of a sub-annual series. Thus, the annual values of index series correspond to the average of the sub-annual values, whether the underlying variable is a flow or a stock.Temporal Disaggregation methods can be performed with or without related indicators. When methods with indicators are applied, the HF series is estimated on the basis of external higher frequency data linked to the relevant variable of interest. If no indicator is available the HF estimates are calculated using some purely mathematical or statistical criterion.The role of benchmarking is to combine (in the best possible way) the available LF and HF information. The benchmarking procedures improve the quality of the resulting HF data by making them consistent with the level of the LF series and coherent with the dynamics of the HF series. The quality of the HF indicators and LF benchmarks must be regularly checked, and the results of this monitoring should be available to the public.

1Introduction


1.6 Principles for temporal disaggregationPrinciples for temporal disaggregation (including benchmarking and reconciliation) are to be applied both at European and member state (MS) level and in several different domains. A general policy for temporal disaggregation describes which issues (not depending on data characteristics) should be decided in a consistent way when performing temporal disaggregation. It is advised to adopt a general policy specifying which methods are to be used, the need for assessment of the benchmarking and reconciliation data quality, the existence of a stable and publically available revision policy, and the need for dissemination of metadata.

The need for domain specific policies may arise due to the fact that each statistical domain can be characterised by data/survey specificities as well as by constraints derived from existing legal acts. When adopting a domain specific policy, it should be ensured that it is fully compliant with the general policy and possibly harmonised at the ESS level. It may also be necessary to take into account the implications that domain specific policies have on other domains at both the national and European level.

Stability of the general and domain specific policies over time is important to foster user confidence, hence the general policies for temporal disaggregation should rarely be revised; when this happens, domain specific policies should be reviewed (and possibly revised) accordingly. When the (general or domain specific) temporal disaggregation policy needs to be changed, it should be communicated well in advance and as far as possible be coordinated at the ESS level.

1Introduction


2. Preliminary analysis2.1: Objective and design of disaggregationMain variable and design of temporal disaggregation (GDP, monthly employment,…) should be clearly defined. One could operate directly or adopt a detailed compilation exercise. This decision strongly depends on available resources regulations and practices of data compilation. Aim of the preliminary analysis is to collect all available data at both low and high frequency of observation, and to analyse at best all the elements useful to achieve disaggregated estimates. Before statistical aspects, these elements come from operative manuals and common practices. This process is carried out when the disaggregation exercise is implemented the first time and it is revised at regular interval of time or in case of extraordinary events.Options The definition of the structure of the entire disaggregation exercise should be in line with constraints coming from current regulations and possibly taking care of changings already planned for the next future. Choice of high frequency indicators should go in favour of official statistics when available. The design of the exercise should take into account of the institutional strategies (e.g. coherence of estimates with methods of compilation adopted at low frequency of observation, coherence with the system of short term statistics and/or minimum revision errors) and best practices of estimation (use of most appropriate methods, goodness of fit and diagnostics).

AlternativesA) Sound definition of target variables and careful design of the disaggregation

exercise in line with all the constraints coming from regulations, institutional strategies and best practices. Preference given to accuracy of estimation and therefore to an indirect approach. Acquisition of high frequency indicators among a wide range of available data, including both official and unofficial statistics measuring similar things of target variables, but giving preference to official statistics. Definition of a regular revision policy.

B) Definition of both target variables and design of disaggregation according to few but well predefined rules. Preference to constraints coming from regulations (like schemes of transmission programs) and institutional strategies with respect to a wider design. Realistic definition of indicators among available data with preference to official statistics also used for compilation of low frequency data.

C) Absence of disaggregation strategy

2Preliminary analysis


2.2: Analysis of indicatorsFeatures A clear set of indicators able to support the temporal disaggregation exercise should be defined. Selected indicators should closely approximate the expected short term movement of the target variable. Other relevant information on available indicators concern: regularity, timeliness, punctuality, appropriate frequency and absence of risk that chosen series are soon suppressed. Better if the indicator is available by elementary component (by sector, product, gender,…) in order to achieve accurate estimations. Depending on the target required, high frequency indicators could be either seasonal adjusted or/and not.

Graphical analysis and descriptive statisticsGraphs concern two main pictures: the evolution of the low frequency series and that of the indicators. Very often indicators are aggregated at the same frequency of the target and plotted together to have an idea of co-movement in levels and growth rates. Also a graph of the ratio between the target series and the indicator at low frequency level is recommended to understand if the two series drift apart or evolve according to similar growth rates. The graph of the indicators alone provides an idea of smoothness of the series, absence of outliers, trend – cycle and seasonal features and absence of excessive erratic behaviours.Graphical analysis could be complemented by tables comparing the evolution of low and high frequency data, simple descriptive statistics and both measures of cross–correlation and residual analysis of basic regressions at the lower frequency. In presence of long time series, integration and co-integration tests carried out at the lower frequency are further elements of analysis. Graphical analysis is carried out both at the stage of the disaggregation design and for evaluating current estimations. Options All the features of the indicators should be analysed in order to guarantee a timely and regular implementation of the disaggregation exercise.A detailed graphical analysis can be really helpful to select right indicators and to discard useless ones. A good practice is to prepare automatic reports with table, charts and synthesis of main descriptive statistics useful also in current estimation.Analysis and selection of indicators could be simplified from the knowledge of strategies and methods of compilation adopted at low frequency.Alternatives



A) All the features of the indicators are considered in the analysis, including a complete report with graphs, tables and descriptive statistics able to compare the dynamic of the indicator with the low frequency variable.

B) Analysis limited to matching internal strategies or regulations. Indicators limited to official statistics. Limited but effective graphical analysis. Acquisition of strategies from good practices coming from more experienced contexts.

C) Limit to acquire few indicators without a proper investigation of their consistency and without graphical analysis.



2.3: Detail of compilationDescriptionWhen a simplified detail of compilation is preferred, indicators should be appropriately chosen in order to be representative of the few sub – components of the target variable. Therefore a larger use of composite indicator could be necessary. Alternatively when a larger detail of compilation is chosen, the exercise implies a larger number of disaggregations through the selection at least of one indicator for each sub – component of the target. Obviously, the detail of compilation should, at least, be in line with requests coming from regulations.Leading criteria are the quality of resulting disaggregation in terms of both accuracy of estimates and adherence with the indicator dynamics. The price is an increase of complexity of the entire exercise. Therefore the compilation at high frequency is generally carried out at a lower detail than that available at low frequency of observation. Detail of compilation should also take into consideration the different source of the indicators adopted for different components (e.g. use of production and turnover indexes for disaggregating respectively value added of industry and services). To avoid as much as possible disaggregations without indicators. Only in specific cases or according to strategic reasons the detail of compilation should be wider than the coverage provided by high frequency indicators. Options When there are not available component indicators and the target variable is not relevant, it could be used a direct approach.If the quality of indicator is less relevant, a low detail of compilation is recommended and a larger use of composite indicator is required. While, if the system of indicators is rich and of good quality, a larger split of elementary components can be adopted. Sometimes the availability of the indicators could even lead the right definition of the detail. Using an indirect approach allows the best use of the information provided by the elementary component series. Alternatives

A) Definition of compilation detail in wide sense, according to availability and quality of data. Right calibration between level of accuracy and reasonable simplified split. Limit at the minimum the choice of a detail which implies disaggregations without indicators.

B) Leading the compilation detail taking care of regulations.C) Uncertain compilation detail or use of a not appropriate direct approach.



3. Methodological aspects3.1: PrefacePreliminary aspectsTemporal disaggregation is an indirect estimation problem usually involving more than one time series, a target low frequency variable and a set of high frequency indicators. For this reason, if there is any cause and effect relationship among the two variables the most appropriate representation is a multivariate one. Solutions provided by univariate forms imply that one of the two variables is exogenous with respect to the other, whereas in reality they join the same environment and are affected by same influences.Popular methods are pro-rata and smoothing adjustment, Denton’s variants, and static regression methods, largely used by statistical agencies to produce official statistics. Other methods concern more recent developments of dynamic regressions and multivariate approaches. Traditional methods are already implemented in several software environments, whereas more recent methods are unknown to main users. In any case, the availability of more than one method is recommended to approach any disaggregation exercise in a production context.

3.2: Available methodsPro-rata adjustment: It simply consists in distributing each low frequency observation according to the high frequency values relative to that period. Main disadvantage is the so called step problem, i.e. the discontinuity which for example emerges in an annual-quarterly disaggregation between the estimate of last quarter of year t-1 and first quarter of year t.

Denton’s variants: The problem is approached by constrained minimization of a quadratic form relative to the differences between disaggregated estimates and the indicator. The penalty function can be specified in terms of either arithmetic or proportionate differences between original indicators and desired disaggregations.

Other mathematical or time series methods: that do not involve the use of related series but rely upon purely mathematical criteria, like those proposed by Boot et al. (1967) and Jacobs (1994), or time series models to derive a smooth path for the unobserved series generally based on ARIMA representation of the series to be disaggregated. Popular static regression methods: Among this category is included the method by Chow and Lin which extends the generalized least squares approach to temporal disaggregation, proposing a univariate regression of low frequency target data on high frequency indicators. The method provides also an optimal solution for extrapolation.

3Methodological aspects


The distinction of the Chow – Lin method from those proposed by Fernandez and Litterman is based on the structure given to residuals: in the former the residuals are distributed according to an AR(1) process, in the Fernandez case as I(1) and in Litterman as ARIMA(1,1,0).Dagum and Cholette: A univariate regression of a the high frequency indicator on available low frequency constraints, deterministic effects and autocorrelated errors. The model, also adaptable to a multiplicative form, nests most common disaggregation methods. It adopts the Kalman filter for its statistical treatment and presents also a generalization to multivariate systems.Dynamic regression methods: This approach, developed by Proietti, generalizes static regressions to autoregressive distributed lag ADL models. The order is limited to ADL(1,1) models, it nests static regressions, it is treated by the Kalman filter and allows for nonlinear disaggregation of data transformed into logarithm.Multivariate approaches: Here low and high frequency indicators are taken simultaneously in a mixed-frequency multivariate model subject to temporal aggregation constraints. Several forms can be considered: vector autoregressions and error correction forms, structural time series and dynamic factor models. Also for this class is used the Kalman filter for statistical treatment.

Options The choice of the disaggregation method should be in line with the aim of the problem and the kind of data available for its solution. In some circumstances naïve methods are very practical and of easy use, while sophisticated methods provide solutions unnecessary complicated. For example pro-rata adjustment could be fine for the treatment of sparse time series with values close to zero and irregular path. Setting-up more than one solution for approaching a wide exercise is recommended.When Denton methods are used solutions and remarks proposed by the IMF QNA manual should be taken into account for correct initialization, extrapolation and preference for the proportional approach.Mathematical methods are often chosen for their simplicity. However most of them have a statistical method as a counterpart (see the first difference Denton’s method well represented by Fernandez). Therefore the use of a statistical approach should be preferred for well-known advantages coming from the possibility of computing checking diagnostics and goodness of fit measures. Within regression methods non-stationary residual models (Fernandez, Litterman, ADL models in first differences, ….) are better suited for non-stationary and non-cointegrated series, whereas stationary models (Chow and Lin and ADL in levels) adapt better to stationary or co-integrated series. The use of the constant is suggested but it loses its importance for models well initialized. When more than one indicator is used, an accurate analysis of collinearity should be carried out. When no indicator is available



these methods are still feasible considering simple regressions of deterministic effects (constant, linear trend, …).Disaggregation exercise with short time series are better approached by simple methods (Denton, Fernandez, …) since less affected by revisions due to parameter uncertainty. Despite their theoretical advantages, disaggregation by ADL methods, nonlinear disaggregation or even multivariate approaches present the limit of complexity, both for implementation and analysis of results. Moreover, some forms are not able to well treat seasonal series. For these reasons they are suggested only for specific problems and their use for massive production processes should be carefully considered.

AlternativesA) Equipment of a large set of methods for any temporal disaggregation need.

Preference to statistical methods better if well supported by efficient algorithms and a set of diagnostics for the analysis of results.

B) Use of basic static regression methods and Denton.C) Pro-rata adjustment, Denton method not amended for initialization and

extrapolation problems



3.3: Related problemsComputational and statistical treatmentTemporal disaggregation requires the implementation of specific algorithms which is straightforward for pro-rata adjustment but more complicated for both adjusting procedures (Denton, …) and parametric statistical approaches. Within adjustment methods it is crucial the definition of the constrained penalty function, whereas for statistical methods optimization often extends to routine algorithms for parameter estimation. Taking into account production needs, selected methods should be well implemented and developed by fast and accurate algorithms. Reports for diagnostic checking, graphical and correlation analysis of results are other required features.

OptionsIn adjustment procedures the penalty function is calibrated according to desired solutions. For instance, in the Denton approach, disaggregated values are alternatively obtained by minimizing the sum of squares of arithmetic or proportionate differences between the original indicator and the disaggregated estimates.In statistical methods the penalty function is also required and it can be either a residual sum of squares, or a likelihood function (given by residual sum of squares, a constant and a determinant term all taken with opposite sign). Optimization of the likelihood function provides several advantages for both parameter and disaggregation estimates. Furthermore, both model and variable selection finds a sound theoretical background.Two-step estimation methods are often adopted, even if they should be avoided since providing sub-optimal solutions.In regression methods derivation of the best linear unbiased estimator is provided by standard generalized least squares algorithms. However statistical treatment can be carried out also using the Kalman filter, which presents several advantages with respect to the former approach: first it allows complete control of initial conditions for disaggregated estimates; second it naturally allows the computation of a set of innovations to be used for diagnostic checking; third it provides the exact solution to the treatment of models formulated in logarithms. Multivariate methods make use of the Kalman filter, but also require the use of a numerical optimization algorithm for parameter estimation. The plug-in for temporal disaggregation wthin JDemetra+ is a valid tool for implemented methods. However it should be further developed to include at least ADL methods, the nonlinear disaggregation for data in logarithms and some facilities for its use in production.

Alternatives



A) Algorithms well implemented for an efficient use in production processes providing also a report of diagnostic statistics, correlations and graphical analysis. Optimization based on the likelihood function for adopted methods. Implementation of options for efficient initialization of disaggregated values

B) Efficient basic algorithms complemented by few but effective statistics and graphs.

C) Use of not verified algorithms and absence of any kind of control on resulting disaggregations.



ExtrapolationThe extrapolation problem finds several solutions in real world applications. From naïve solutions giving primary importance to coherence of extrapolated values with short term indicators, to mathematical or optimal solutions in statistical sense. The choice strongly depends on current practices and confidence to projected data. In regression methods both temporal disaggregation and extrapolation are treated simultaneously and find an optimal solution in statistical sense. For this reason they appear more appealing than simple adjustment methods. However, it is not rare the case that extrapolations derived by regressions are less satisfactory than those of simplest methods.

OptionsFor exercises where the indicator fits well to the target, differences among disaggregation methods are relatively insignificant in terms of extrapolation.The simplest solution is given by extrapolating most recent values according to growth rates of relevant indicators. For example in an annual-quarterly disaggregation exercise one can refer to quarterly or annual growth rates of the indicator. The statistical method which better reproduces this practice is Fernandez as well as proportional Denton among mathematical methods.Differences in the methodology becomes relevant under not perfect fit of the indicator to the target and this occurrence strongly affects extrapolations. Therefore it is useful to define a strategy of model selection based on revision histories. At this purpose the analysis of innovations is recommended.

AlternativesA) Extrapolation by same methodology adopted for temporal disaggregation.

Preliminary evaluation of their quality by real time analysis and use of innovations.

B) General coherence of the extrapolation method with temporal disaggregation.C) Extrapolation by naïve methods



3.4: Model selection and choice of indicatorsTHIS SECTION COULD BE DEVELOPED BUT AFTER DISCUSSION WITHIN THE TF.

Preliminaries Choice of best model is important and for this reason it can be developed a section providing guidelines and tool.The choice of the most appropriate method is even more crucial for users who adopt disaggregation methods also for extrapolating. The analysis is circular, starting from a wide set of data and options, operating the selection and starting again the process if after estimation results are not satisfactory. This process is carried out when the disaggregation exercise is implemented the first time, taken fixed and revised at regular interval of time (e.g. every five years).

Information criteriaStandard likelihood testsMeasures of correlation

-



4. Benchmarking4.1 Choice of benchmarking methodDescriptionBenchmarking is a specific case of temporal distribution, namely where the high frequency (HF) indicator series and the low frequency (LF) benchmark series describe the same phenomenon. For instance, we could have quarterly and annual series of production of mining companies. When the quarterly and annual series are based on different data sources one should not expect the quarterly sums to be equal to the annual figures. In general annual figures are observed later and are subject to better scrutiny. A benchmarking procedure restores temporal consistency between the HF and LF series, by adjusting the HF series while preserving as much as possible the extra information contained in the HF series. Exactly what this HF information is, and how it is preserved, is where benchmarking methods differ.

RequirementsThe single most important principle applied by most benchmarking methods is called Movement Preservation. This means that the benchmarking method aims to preserve either the proportional or additive first differences of the indicator, or mix of these. As a rule, a benchmarking method should not introduce artificial changes in the properties of the indicator. An important example of this is the so called step-problem. If a benchmarking method achieves temporal consistency without looking at the adjustments made to the changes between subperiods of adjacent years, an artificial step can be introduced in the benchmarked series. Another example is time-symmetry: a benchmarking method should yield the same results when performed on a set of benchmark and indictor series when time is reversed. If a benchmarking method does not have this property, it can be shown that the timing of events (like the onset of a crisis) can be changed by the benchmarking procedure. A benchmarking method should also include good diagnostics. It must produce diagnostics from which one can see whether the assumptions the method is based on are valid.

OptionsBenchmarking methods without indicators have no place in the context of benchmarking. Most popular benchmarking methods are: the pro-rata benchmarking method and movement preservation methods such as the proportional or additive Denton method and its variants, the Chow-Lin regression based method and its variants and the Cholette-Dagum method with first-order autoregressive (AR) error. The pro-rata method is a very simple approach. It just multiplies each indicator value with the same constant to achieve consistency with the benchmark value. An additive version exists, which distributes an equal part of the difference between the aggregated indicator and the benchmark. Although its simplicity might appeal, it is a very troubled approach as it introduces large steps between subperiods of adjacent years, as the entire difference between period-to-period changes of the aggregated indicator and the benchmark is allocated to the subperiod-to-subperiod change between years. The

4Benchmarking


subperiod-to-subperiod changes within years are not changed at all. For assessing the performance of this method Benchmark/Indicator (BI) ratios can be calculated, but given the simplicity of the method these are not very informative. Denton methods and its variants obtain the final HF estimates by minimising a quadratic loss function that involves the adjustments made to subsequent period-to-period changes and that is subject to temporal aggregation constraints. It results in a smooth adaptation of the period-to-period changes of the indicator to match those of the benchmark series. The Denton method does not introduce steps between years and is time-symmetric. For assessing the performance of the method, the BI ratio’s or BI differences can be used. As the Denton method aims to make smooth adjustments to these, a simple visual inspection will point out the situations where it cannot make smooth adjustments. The BI ratios also allow numerical analysis, for instance one could calculate the contribution to the loss function per benchmark period, in order to quickly find problem situations. Cholette and Dagum (1994) proposed a benchmarking method based on the generalized least-squares regression model that takes into account the presence of bias in the indicator and the presence of autocorrelation and heteroschedasticity errors in the original data. These characteristics make the Cholette-Dagum approach a very flexible procedure. The (additive/proportional) Denton method can be seen as a particular (approximated) case of the (additive/proportional) Cholette-Dagum regression model, in the limit where the autoregressive parameter approaches 1. In the limit where the autoregressive parameter approaches 0, the Cholette-Dagum resembles the pro-rata method, which is why it is common advise to avoid this limit, as step problems will be introduced. The model is also time-symmetric. Being a statistical model, this method yields information about its performance in the form of regression residuals. Chow and Lin (1971) give a least-squares optimal solution on the basis of a linear regression model involving a set of HF indicator series and a the benchmark series. Having multiple indicators is irrelevant to the setting of benchmarking, however a constant extra indicator can be used to correct for bias between the indicator and benchmark series. Different hypotheses related to the structure of the error in the regression model are the base of the methods variants. The original Chow-Lin method with a single indicator is a limiting case of the additive Dagum-Cholette model with the autoregressive parameter close to 1. However, in the Chow-Lin model the autoregressive parameter is estimated from the indicators, instead of being chosen by the user. This model does not introduce step problems as long as the autoregressive parameter is close to 1 and is time-symmetric. Since it is a regression based model, this method yields excellent information about the model fit and outliers in the form of residuals. The Causey-Trager approach to benchmarking is based on directly minimizing the changes to the proportional first differences of the indicator series, a principle called Growth Rate Preservation. Because of this principle many see it as the most ideal benchmarking method. It’s formulation however, involves a loss function which is nonlinear and even singular, and the model has no solution in a closed form. It must be solved by iterative algorithms, which make it impractical and therefore it is hardly used. Recent advances in estimation techniques have revived interest in this approach. However, it has another less known drawback, namely that is not time-symmetric. This can be seen as follows. A step up from level 100 to 150 is a proportional change of +50 %. The same step down is a decrease of –33 %. This implies that the loss function

4Benchmarking


associated with the GRP principle weighs the same change differently according to whether a change is upward or downward. Equivalently a change forward in time, will always be weighed differently from the same change backward in time and the results of benchmarking are different when the order of the series is reversed. A consequence of this is that the benchmarking procedure may change the timing of relevant events, like the start of an economic crisis. As upward benchmarking adjustments are more expensive than downward adjustments, a large upward benchmarking adjustment that coincides with the start of a crisis might move the start to a later subperiod. This is a highly undesirable property of a benchmarking method. The choice between alternative models should further be based on the available indicator, the usual diagnostics of the LF regression and the validation of the results, in particular the conformity between the dynamics of the indicator and the HF estimated series. Benchmarking methods with movement preservation (Denton, Cholette-Dagum and Chow-Lin) all minimize the impact of revisions on the historical movements of the series.For an exhaustive description on these methods see:

E. B. Dagum and P. A. Cholette. Benchmarking, Temporal Distribution, and Reconciliation Methods for Time Series. Lecture Notes in Statistics. Springer-Verlag, New York, 2006;

IMF (2014) Quarterly National Accounts Manual - Concepts, Data Sources, and Compilation, ch.6, Benchmarking and Reconciliation

Alternatives1

A) The Cholette-Dagum model and the Chow-Lin model provide the most elaborate modelling options and diagnostics and should be preferred. B) The Denton method is an acceptable alternative, provided that an analysis of the BI-ratios is used for diagnostics. C) The pro-rata method is not an appropriate method for benchmarking, because of the step problem. The Causey-Trager GRP approach is not time-symmetric and may lead to to undesirable changes in the timing of relevant events.

1 A) Best alternative; B) Acceptable; C) To be avoided

4Benchmarking


5. Reconciliation and multivariate benchmarking

5.1 Choice of reconciliation methodDescriptionBenchmarking methods listed before consider one target series at a time and do not take into account accounting relationships between other series. Then benchmarked series do not automatically form a consistent set of accounts (accounting constraints; e.g. the temporal disaggregated estimates of GDP from the production side may differ from the temporal disaggregated estimates of GDP from the expenditure side, even though the annual data are consistent). The reconciliation procedure is the tool to obtain consistency in high frequency data that are subject to low frequency aggregation constraints. The most popular reconciliation methods are the multivariate proportional Denton methods (or multivariate Cholette methods) and the family of the two-stage procedures. The multivariate proportional Denton method is a multivariate extension of the univariate proportional Denton method to include all the HF series in the system where the constrained minimization problem is increased to include the constraints. When the dimension of the system is too large, it may become difficult to apply the multivariate Denton approach using standard algorithms. In this case a two-step reconciliation procedure could be used to approximate the results of the optimal multivariate Denton method. The two-step procedure is essentially based on the application of a proportional method for each individual series at the first step, and a least squares adjustment of the system of benchmarked series one year at a time as the second step (Quenneville and Rancourt, 2005). It is also possible to apply a simultaneous Benchmarking and Reconciliation, for example using the Simultaneous Modified Denton (Di Fonzo and Marini, 2015). The choice of the procedure may depend on several factors (in particular form the number of the series to be reconciled). An extension of Multivariate Denton Mehtod for Benchmarking is proposed for large data sets by implementing a state-of-art optimizition solver adapted to the economic relationships in Dutch national accounts (Bikker et.al., 2010). When the sample sizes are too small but the large number of series is to be benchmarked and reconciled according to a complex hieararchical scheme, a state-space model might be used, as discussed in Tiller and Pfeffermann (2011). This method is especially useful for situations when the time series data is combined with cross-sectional data and when the incomplete information is to be treated by Small Area Estimation. See also US Bureau of Labor Statistics (BLS) methodology for producing monthly employment and unemployment estimates where benchmarking is implemented together with seasonal adjustment by the state-space models. Some possible alternative approaches include e.g. multivariate extension of the Chow-Lin model, the Multivariate Random Walk Model and its extensions, the dynamic factor models and some semiparametric approaches such as spline methods (Mazzi and Proietti, 2015).

5Reconciliation and multivariate benchmarking


For an exhaustive description on these methods see: E. B. Dagum and P. A. Cholette. Benchmarking, Temporal Distribution, and

Reconciliation Methods for Time Series. Lecture Notes in Statistics. Springer-Verlag, New York, 2006;

IMF (2014) Quarterly National Accounts Manual - Concepts, Data Sources, and Compilation, chp.6 Benchmarking and Reconciliation.

Di Fonzo T., Marini M. (2011) Simultaneous and two-step reconciliation of systems of time series: methodological and practical issues, Journal of the Royal Statistical Society: Series C (Applied Statistics), 60, 143-164.

Di Fonzo, T. and M. Marini (2015), Reconciliation of systems of time series according to a growth rates preservation principle, Statistical Methods & Applications, Vol.24 (4), pp. 651-669.

Durbin, J. and Quenneville, B. (1997). Benchmarking by State Space Models. International Statistical Review, 65, 23-48.

Pfeffermann, D. and Tiller, R. (2011). State-Space Modeling with Correlated Measurements with Applications to Small Area Estimation Under Benchmarking Constraints. Southampton Statistical Science Research Institute, University of Southampton. Methodology Working Paper M03/11.

Bikker, R., Daalmans, J. and Mushkudiani, N. (2010). A multivariate Denton method for benchmarking large data sets. Statistics Netherlands, Discussion Paper.

Mazzi, G.L. and Proietti, T. (2015). Multivariate Temporal Disaggregation. Handbook on Rapid Estimates, Ch. 8, Eurostat (Nov 13, 2015).

Options Multivariate Denton method and its variants. Two-step procedure and its variants State-space models. Semiparametric models or some other relevant observation-driven methods. Combinations of some of the models mentioned above.

Alternatives2

A) A version of the multivariate proportional Denton method for benchmarking (such as the extended multivariate Denton method) should be used for deriving a system of time series subject to, for example, both annual and quarterly constraints. When the dimension of the system is too large to be solved efficiently in a single step, the previously mentioned two-step procedure may be used in a second step to approximate the optimal results.B) Apply Simultaneous methods by e.g. State-space models.C) Any other methods


5Reconciliation and multivariate benchmarking


6. Specific Issues6.1 Choice of the softwareDescriptionThere are many in-house software packages implemented by NSOs for benchmarking and reconciliation based on free or commercial software such as R packages (tempdisagg). MATLAB tool (Di Fonzo, Marini 2011), SAS (Latendresse et al. 2007, Bérubé and Fortier 2009) routines, and dedicated plug-in JAVA packages as JDemetra+ (National Bank of Belgium), ECOTRIM (Barcellan, 2002) and his extension Jecotrim. To choose among them the user should take into account several aspects: versioning, maintenance and support, compatibility with these Guidelines, documentation, cost, open-source architecture, completeness, usability, use for mass production, computational efficiency, etc. The Software should be updated following a clear software engineering management. Using the same software release is essential for coherence and transparency. Methods and tool versions currently used in data production should be clearly communicated to users. Before migrating, the software and the impact on data should be assessed in the specific IT environment where it will be used.

References: Bérubé, J. and Fortier, S. (2009). PROC TSRAKING: An in-house SAS procedure for

balancing time series. In ASA Proceedings of the Business and Economic Section. American Statistical Association

Di Fonzo, T. and Marini M. (2011) “Simultaneous and Two-Step Reconciliation of Systems of Time Series: Methodological and Practical Issues.” Journal of the Royal Statistical Society. Series C (Applied Statistics), vol. 60, no. 2, pp. 143–164.

Latendresse, E., Djona, M., and Fortier, S. (2007). Benchmarking sub-annual series to annual totals – from concepts to SAS procedure and SAS K12089 Chapter: 10 page: 253 date: February 14, 2012 Restoring Accounting Constraints in Time Series 253 enterprise guide custom task. In Proceedings of the SAS Global Forum 2007 Conference. SAS Institute Inc.

National Bank of Belgium (2016): JDemetra+, https://github.com/jdemetra/jdemetra-app

Sax C. and Steiner R. (2013) Temporal Disaggregation of Time Series, http://journal.r-project.org/archive/2013-2/sax-steiner.pdf.

Options Using the official release of software tool for implementation the recommended

methods (JDemetra+). Using other software packages officially approved at ESS level having capability of

implementing the recommended methods Using old releases the previously mentioned software tools.. Usingsome other commercial or non-commercial software tool (such as SAS, R,

optimization tools like Cplex, Gurobi etc.) with capability to apply the recommended methods.

6Specific issues


6Specific issues


6.2 End of SeriesDescriptionIn common practice, at the end of the series, LF benchmarks (e.g. year) become available several HF periods (e.g. months) after the current LF period is completed. Then no LF benchmark is available for the current year, hence the missing value. This is the extrapolation case. The proportional Denton method takes into account this situation carrying forward the HF Benchmark-Index ratio for the last HF period of the last benchmark year, given implicitly a forecast for the next LF Benchmark-Index ratio. The enhanced proportional Denton method for extrapolation requires an explicit forecast for the annual BI ratio of the following year. This forecast depends on then recent behaviour of the BI ratio: if the annual BI ratio fluctuates stationary around its mean, the best forecast of the next year’s BI ratio is the long-term average BI value. This approach is very close to the solution offered by the proportional Cholette-Dagum method with AR error;If the annual BI ratio shows a systematic upward or downward trend, the best forecast of the next year’s BI ratio is a trend extrapolation in the next year. The proportional Cholette-Dagum benchmarking method with first-order AR error is a way to evaluate extrapolations of target series when the indicator is an unbiased measurement of the LF variable. Also, the Chow-Lin method and its variants give a natural solution to the extrapolation problem, thanks to their regression model structure. Alternative approaches are essentially based on available external information on the behaviour of the annual Benchmark-Index ratio for the last year with no benchmark. This external information can also be obtain by a forecast for the last annual benchmark value based on the times series approach. However, a sufficient number of observations (minimum 10 years) is required to fit time-series models and calculate forecasts with an acceptable level of confidence. A good practice is to check the effects of new and revised benchmarks on the Benchmark-Index ratio.

Options Use some regression-based methods. Forecast the unavailable benchmark, using the best available external

information or methods according with the current framework and may improve the benchmarking of the current sub-annual values. The forecast of the sub-annual series for the sub-annual periods, covered by the forecasted benchmark, is also needed.

Often benchmarking is needed just at the end of the current period, then for the sub-period data production it is possible to apply a reconciliation without a LF constraints.

Alternatives3

A) Use the best regression based method or use external information on forward benchmark values or forecast by time series methods the LF series for benchmarking the target series and reconcile (balance) using the accounting constraints.3 A) Best alternative; B) Acceptable; C) To be avoided

6Specific issues


B) Extrapolate without LF constraints. C) Any other method.

6Specific issues


6.3 Temporal disaggregation in flash estimatesDescriptionRapid estimates refers to a timely numerical evaluation of economic variables, using the real-time data flow, such that the present, the recent past or the near future can be described and/or forecast. More specifically, Rapid estimates or nowcasting allows estimating the latest or the current movements of a variable of interest. Nowcasting is done right before or after the end of the reference period. For example, if the last available information of the interest variable refers to the period t − 1, its nowcasting for the period t is done a few days before the end of the period t or few days after the beginning of the period t + 1. Nowcasting is usually based on an incomplete information set with respect to the one used for the normal data compilation process, and in case of an incomplete coverage, statistical modelling can be used to fill the gaps. The information set can be incomplete from a geographical or a sectorial point of view, and in temporal terms: e.g. for a quarterly variable, just two months out of three of the variables of the information set are available. In theory only data which is characterised by revisions can be the object of flash estimates or nowcasting but also data not subject to revisions can be forecasted. Nowcasting can be performed by a large variety of multivariate methods such as: Regression methods, multivariate time series methods (e.g. VAR), static and dynamic factors models, principal components methods, etc. Flash estimates aim to provide an early picture of the situation in a particular sector or in the whole economy based on one or several variables of interest, possibly related by accounting and/or aggregation constraints. Benchmarking regression based methods or two-step procedures can be used to extrapolate (or “nowcast”) low-frequency benchmarks on the basis of available high-frequency indicators and perform balancing.

OptionsSuggested benchmarking and reconciliation methods are: the proportional Denton, the proportional Cholette-Dagum with first-order autoregressive error, and the regression-based Chow-Lin method. The performance of these methods depend on several factors and results may vary in relation to the characteristics of the available data. In general the Cholette-Dagum method seems to provide most accurate extrapolations when the indicator and the annual benchmarks move along the same trend. However, the Denton and Chow-Lin methods could have a better performance when the quarterly indicator temporarily deviates from the target series (Marini 2016). A two-step procedure might also be applied.

Alternatives4

A) Apply one of the recommended method using the best available high frequency indicator and perform the optimal forecast for the latest year. Choose the best model for the error term for the regression model for Cholette-Dagum with first-order autoregressive error. Apply balance procedure on the best optimal forecast of the interest variable.B) Apply one of the recommended method using the available high frequency indicator. C) Any other relevant methods.


6Specific issues


6.4 Outliers identification and treatmentDescriptionBenchmarking and reconciliation procedures are usually applied at the end of the data production process. The quality of the data produced is therefore critical to obtain a good quality benchmarked target series, where the revisions have limited effects. Outliers in the series can appear in several forms (additive, temporary changes, level shifts, changes in seasonality etc.) and can occur for several reasons. Even if they cause, in general, several problems in the data compilation process (especially in seasonal adjustment), outliers give some information about some specific event. Outliers at the end of the series are a special case. Major economic changes, errors or problems in the statistical compilation process are initially classified as additive outliers at the end of the series. Benchmarking and reconciliation procedures can sometimes smooth the effects of outliers that may occur especially at the end of the series, but this reduction effect closely depends on the amplitude of the outliers. In general, Benchmarking and Reconciliation procedures do not solve or mitigate the effects of problems that may have affected the quality of the target series. If a stable annual Benchmark-Index ratio shows a structural break in the last HF period, which is expected to continue in the future, then the best forecast of the next year’s Benchmark-Index ratio is the previous LF value. In general, the nature of the outliers, its temporary or permanent effects, can be properly seen after certain period of time. Hence,it is difficult to describe a general best practice. Moreover, the identification of outliers and their treatment may heavily depend on the specific statistical domain and on at what step of the data production process the disaggregation/benchmarking/reconciliation procedure is to be applied. This issue should be dealt with a case- to- case approach.

OptionsCarefully investigate the nature of the problem arising in the target series (inadequacy of the sample design, problem in data collection, serious structural changes in the domain or the economy) and according to the source of the problem try a solution to improve the quality of the target series, or do not act as if the problem has a transitory nature.

Alternatives5

A) Apply the best procedure to improve the quality of the target series, or take into account the outlier during the temporal disaggregation procedure.B) Do nothing and act as the outliers do not affect the data.C) Do not disseminate the data.


6Specific issues


6.5 Treatment of short seriesDescriptionMost popular benchmarking methods are based on a statistical regression model for the unknown values to be estimated, e.g. Cholette and Dagum (1994) or Chow and Lin (1971) and their variants. When only short time series are available, these methods are not recommended since the involved parameters can be poorly estimated, which would seriously affecti the quality of the benchmarked series.

OptionsWhen a short time series is available one of the recommended mathematical method with no HF indicator or one variant of the Denton method might be used to obtain the target variable.

Alternatives6

A) Apply one of the recommended mathematical methods with no HF indicator to obtain the target variable.B). Apply one of the recommended Denton methods to obtain the target variable.C) Apply any other relevant methods.


6Specific issues


6.6 Benchmarking and reconciliation in chain linked seriesDescriptionChanges over time in the economic values can be factored into price and volume changes. Then constant prices focus on the volume growth only. ‘Constant prices of the base year’ with base updated about every five years was the common procedure till recent years. In the chain-linking procedure the idea is to update the base year at a higher frequency (generally on annual basis) and link the short term movements. The period to period changes of volumes (the links in the chain) are calculated using the prices as weights of the previous year. The changes between consecutive periods are then linked together to form chained volume measures that show changes in a time series framework. Annually chained linked Laspayres-type indices are used in several domains, in particular QNA. When considering quarterly series, there are three linking methods: one-quarter overlap, annual overlap and over-the year. The introduction of chain-linking directly affects the benchmarking process because in contrast to constant price data, chain-linked volume measures are non-additive, therefore, the chain-linked volume indices of an aggregate cannot be calculated as a weighted average of the chain-linked indices of its components. The non-additivity affects the compilation process when constructing the quarterly indicator. E.g. chain-linked Member States’ data cannot be directly aggregated; while previous year prices have to be used to produce a previous year’s prices indicator. The input of the temporal disaggregation techniques should be, in principle, a time series, therefore it is necessary to chain-link the indicator to obtain a series in the same base with an appropriate length. Moreover the reconciliation, being based on the horizontal additivity hypothesis, will be problematic in the compilation process, since accounting identities are satisfied for unadjusted volumes in average prices of the previous year. An additional element is the interrelations between chain-linking and seasonal adjustment.The three techniques of chain-linking have different impacts on the time series characteristics. Both the annual-overlap and the over-the-year technique can influence the seasonal pattern of the chain-linked volume series. Since the impact of the latter approach can be substantial in case of large substitution effects, this method is not recommended. The one-quarter-overlap technique produces smooth transitions for the changes in price weights. When seasonally adjusted chain-linked volume measures are benchmarked in order to obtain consistency to the non-seasonally adjusted chain-linked LF series, particular attention must be given, since benchmarking might introduce an artificial seasonal pattern into the target series. Therefore, a technique preserving short-term movements is recommended. Best practices depend on the specific framework. For example, the difference between QNA and ANA chain-linked series can be affected by the seasonality of the HF series. Whereas this could not be important if benchmarking is done as final stage of producing seasonally adjusted chain-linked QNA volumes, it could become relevant when chain-linked series based by the one-quarter-overlap approach are benchmarked.

OptionsPerform seasonal adjustment on annually chain-linked Laspeyres(-type) series and decide between direct and indirect adjustment. Benchmark if a direct seasonally adjustment on each individual annually chain-linked series is performed. When deriving

6Specific issues


indirectly seasonally adjusted chain-linked Laspeyres(-type) aggregate series by aggregating seasonally adjusted chain-linked Laspeyres(-type) component series, the following steps defined in ESS Guidelines for Seasonal Adjustement pg. 43.If required, force by benchmark the annual total of the seasonally adjusted chained-linked (or seasonally and calendar adjusted) data to be equal to the annual total of the unadjusted chained-linked (or calendar adjusted) data.Since benchmarking is applied in order to obtain time-consistent data, chain-linked volume measures are usually benchmarked at the last stage of the process, i.e. after chain-linking and seasonal adjustment. If the one-quarter-overlap or the over-the-year method is used, benchmarking before seasonal adjustment may be conducted in addition to it. In either case, the impact benchmarking may have on the seasonal pattern is the critical issue. When conducted before seasonal adjustment, benchmarking might reduce the quality of the results which can be achieved by seasonal adjustment.

Alternatives7

A) Apply one of the recommended method using the best disposable low frequency indicator at the end of the data process according to the chain linked method adopted. If required, force by benchmark the annual total of the seasonally adjusted chained-linked (or seasonally and calendar adjusted) data to be equal to the annual total of the unadjusted chained-linked (or calendar adjusted) data.B) Apply one of the recommended method using the best disposable low frequency indicator at the stage of the process according to current data framework . Do not impose the annual total of the seasonally adjusted chained-linked (or seasonally and calendar adjusted) data to be equal to the annual total of the unadjusted chained-linked (or calendar adjusted) data.C) Any other methods .


6Specific issues


6.7 Benchmarking and reconciliation in National AccountsDescriptionQNA variables must be temporally consistent with annual benchmarks. Statistical methods for deriving quarterly estimates of variables are divided into three categories: (i) methods when only annual estimates are available, (ii) methods that impute values for a variable by modelling the relationship between (annualized) preliminary quarterly estimates of a QNA variable, or a quarterly indicator, and the corresponding annual benchmarks and (iii) methods that impute values for a QNA variable by modelling the relationship between (annualized) preliminary quarterly estimates of a QNA variable, or a quarterly indicator, and the corresponding annual benchmarks. i. These methods are used when only annual data are available. Quarterly estimates

are derived either by a weighted disaggregation of the available annual data according to some purely mathematical criterion (smoothing methods as the recommended Boot, Feibes and Lisman) or by using time series models. In either case, the objective is to provide sufficiently smooth quarterly estimates that are consistent with the annual data. Smoothing methods are based on a quarterly indicator that has some relationship with a QNA variable to be estimated. The annual data must have no zero nor negative values. It is necessary to make a forecast when there are no annual data in order to apply smoothing methods in the latest year. In this case the best possible informed forecast is preferred to a forecast by default, such as by assuming that the growth observed in the previous year continues at the same rate in the current year.

ii. Methods that use a statistical model to estimate a QNA variable entail using regression analysis to fit the model to the annual benchmark data (dependent variable) and the annualized quarterly indicator data (independent variable). The model is then used to impute a QNA variable using the quarterly indicator. Some methods use a benchmarking procedure to ensure temporal consistency in a two-step process, but other methods impute estimates of a QNA variable that are temporally consistent with the annual benchmarks in a one-step process. Possible benchmarking methods are the pro-rata adjustment and methods belonging to the Denton family (recommended).

iii. Regression analysis to estimates QNA variable fit the model to the annual benchmark data (dependent variable) and the annualized quarterly indicator data (independent variable), then the model is used to impute a QNA variable using the quarterly indicator. The suggested procedure is the Chow-Lin method and its variants. The indicator must be good proxies of the variables to be estimated. Moreover, those variables must be associated with the economy of the country for which QNA are compiled. except those regarding international trade. The Chow-Lin method produces estimates that are temporally consistent with the LF benchmark values in one step and the estimates for the quarters of the latest year, for which no annual benchmark data exist, are directly obtained from the quarterly regression model..

Options

6Specific issues


A detailed discussion on benchmarking and reconciliation in NA can be found on chapter 5 of the Handbook of QNA 2013.

6Specific issues


6.8 Benchmarking and reconciliation in Labour MarketDescriptionThe Labour Force Survey (LFS) is the main source for LM statistics, while other sources are the Structural Business Statistics(SBS). LBS is a supplying side, quarterly based analysis; individuals are the statistical units and the methodology is harmonized. SBS is a demanding side, generally monthly based analysis; firms are the statistical units and definitions and methodology can vary across MS according to the national laws for the employment.Eurostat disseminates annual, quarterly and monthly unemployment rates on the basis of the MS survey results. For the monthly unemployment rate, MS can choose their best production procedure, while Eurostat evaluates indicators for MS that are unable to produce their own. In any case the monthly estimates must to be consistent with the quarterly.

OptionsFor monthly estimations there are three alternative production strategies: direct production of monthly data, Temporal disaggregation, three month moving averages. Since the LFS sample is uniformly distributed over the quarter, the monthly LF data can be directly produced. If the sample is not robust enough, the benchmarking and reconciliation method can be used on provisional data to obtain monthly data that satisfy temporal/accounting constraint with the quarterly data. When the sample is robust enough, a quarterly estimation can be obtained by the average of the monthly ones. Temporal disaggregation can be used to obtain monthly data from quarterly LFS data using HF indicator from SBS (for employment) or Administrative data (for unemployment). This procedure can be used for both seasonally adjusted and unadjusted data. Reconciliation is needed to match the temporal/accounting constraints. A source integration is also needed when forecasting monthly data for the current quarter. Sometimes a moving average of the last three months of the LFS can be disseminated. The data show problems in comparison with a short period and this procedure is not recommended.

Alternatives8

A) Apply the best benchmarking and reconciliation methods on provisional data to obtain monthly data that satisfy temporal/accounting constraint with the quarterly data.B) Temporal disaggregation from quarterly LFS data using HF indicator from SBS (for employment) or Administrative data (for unemployment. This procedure can be used for both seasonally adjusted and unadjusted data. Reconciliation is needed to match the temporal/accounting constraints. C) Moving average of the last three months and any other methods


6Specific issues


6.9 Advanced and recent methodBenchmarking and reconciliation can be seen as constrained minimization problems of some mathematical function preserving the movements in the sub-annual values. The Denton (1971) movement preservation principles is the base of the more used benchmarking method. However the need of more sophisticated method especially in a multivariate framework pushed the research to find other suitable procedure. Main contribution are Di Fonzo and Marini (2011, 2012a, 2015) where an extension the PFD criterion to the reconciliation of a system of time series subject to both temporal and contemporaneous constraints is presented.

6Specific issues


7. Presentation7.1 General revision policy and release calendarDescriptionRevisions can be defined as any change in a value of a public released statistic. They can occur either when new observations (one additional month or quarter) become available and some past values are modified or when the current and/or some past values are modified. Then, revisions occur in order to incorporate new, improved information. Usually, not all the data required to produce timely statistics are available at the current end of the series; therefore, the missing values are estimated. Since it is generally the case that the estimated values do not correspond precisely to the late incoming data, the original figures are revised. In addition, benchmarking also gives rise to changes because it produces the adjustment of (generally) higher frequency data to take account of more complete lower frequency results, which become available only later. Since high frequency indicators are benchmarked to the corresponding low frequency ones (and low frequency indicators are sometimes also subject to revisions), high frequency ones are revised accordingly in order to ensure a full consistency. If the benchmarking process takes place with some delay or if no benchmarking at all is performed, temporary or permanent discrepancies between the low and high frequency versions of the same indicators, can appear. A precondition for a benchmarking revision policy is a release and revision policy for the lower frequency statistics, especially release and revision calendars are needed for these statistics. It is important that the revision policy is as coherent and transparent as possible and that it should not mislead the interpretation of the economic picture.

Options Revise data in accordance with a well-defined and publicly available revision policy

and release calendar. Revise data between two consecutive scheduled releases of the release calendar. Perform revisions on an irregular basis and/or do not revise at all.

Alternatives9

A) Revisions are published in accordance with a coherent, transparent and officially published revision policy and release calendar, which is aligned with the revision policy and the release calendar for the unadjusted data. B) Revisions are published in accordance with a coherent, transparent and officially published revision policy and release calendar.C) No revision, absence of a clear and public revision policy, absence of a public release calendar, or policies leading to the publication of misleading information especially for the current period.


7Presentation


7.2 Accuracy of Benchmarking and ReconciliationDescriptionBenchmarking and Reconciliation procedures are generally applied at the end of the data production process. The quality of the data produced is therefore critical to obtain a good quality benchmarked target series, where the revisions have limited effects. The quality of the benchmarked series can be measured by the distance between the original series and the benchmarked series in terms of RMSE or in terms of growth of rate preservation. When statistical methods are used, in terms of estimation accuracy, measured by the usual diagnostic procedure results, of the estimates. Benchmarking and Reconciliation procedures can sometimes smooth the effects of outliers that may occur especially at the end of the series, but this reduction effect closely depends on the amplitude of the outliers. In general, Benchmarking and Reconciliation procedures do not solve or mitigate the effects of problems that may have affected the quality of the target series.

OptionsVerify the estimation accuracy, measured by the usual diagnostic procedure results, of the estimates involved by regression based methods. Evaluate the RMSE and the growth rate preservation. Carefully investigate the nature of the problem arising in the target series (inadequacy of the sample design, problem in data collection, serious structural changes) and according to the source of the problem try a solution to improve the quality of the target series, or to not act if the problem has a transitory nature.

Alternatives10

A) Verify the estimation accuracy, measured by the usual diagnostic procedure results, of the estimates involved by regression based methods. Evaluate the RMSE and the growth rate preservation. Apply the best procedure to improve the quality of the target seriesB) Do nothingC) Do not disseminate the data.

.

7.3 Metadata (?)


7Presentation


ReferencesBarcellan, R. (2002), ECOTRIM: a program for temporal disaggregation of time series,

Workshop on Quarterly National Accounts, Eurostat, Theme 2 Economy and finance: pp.79-95.

Barcellan, R. and R. Buono (2002), Temporal Disaggregation Techniques – ECOTRIM Interface (Version 1.01), User Manual, Eurostat.

Bérubé, J. and S. Fortier (2009), PROC TSRAKING: An in-house SAS procedure for balancing time series. In ASA Proceedings of the Business and Economic Section. American Statistical Association.

Bikker, R., Daalmans, J. and Mushkudiani, N. (2010). A multivariate Denton method for benchmarking large data sets. Statistics Netherlands, Discussion Paper.

Boot, J.C.G., W. Feibes, and J.H.C. Lisman (1967), Further Methods of Derivation of Quarterly Figures from Annual Data, Applied Statistics, Vol. 16 (1), pp. 65–75.

Cholette, P.A. and E.B. Dagum (1994), Benchmarking Time Series with Autocorrelated Survey Errors, International Statistical Review, Vol. 62(3), 365–377.

G. C. Chow and A.-L. Lin.(1971), Best linear unbiased interpolation, distribution, and extrapolation of time series by related series. The Review of Economics and Statistics, 53(4):372–375.

Dagum, E.B. and P.A. Cholette (2006), Benchmarking, Temporal Disaggregation, and Reconciliation Methods for Time Series, Springer edition.

Denton, F. (1971), Adjustment of Monthly or Quarterly Series to Annual Totals: An Approach based on Quadratic Minimization, Journal of the American Statistical Association, Vol. 66, pp. 99–102.

Durbin, J. and Quenneville, B. (1997). Benchmarking by State Space Models. International Statistical Review, 65, 23-48.

ESS (2015) Guidelines on Seasonal Adjustment.ESS (2013) Handbook on Quarterly National AccountsESS (2013) Guidelines on Revision Policy. IMF (2014), Quarterly National Accounts Manual - Concepts, Data Sources, and

Compilation, ch. 6 Benchmarking and Reconciliation.Di Fonzo T. and Marini M. (2011), Simultaneous and two-step reconciliation of systems

of time series: methodological and practical issues, Journal of the Royal Statistical Society: Series C (Applied Statistics), 60, 143-164.

Di Fonzo, T. and M. Marini (2012a), Benchmarking Time Series according to a Growth Rates Preservation Principle, Journal of Economic and Social Measurement, Vol. 37(3), pp. 225–252.

Di Fonzo, T. and M. Marini (2012b), On the Extrapolation with the Denton Proportional Benchmarking Method, IMF Working Paper Series, WP/12/169.

Di Fonzo, T. and M. Marini (2015), Reconciliation of systems of time series according to a growth rates preservation principle, Statistical Methods & Applications, Vol.24 (4), pp. 651-669.


7Presentation

Durbin, J. and B. Quenneville (1997), Benchmarking by State Space Models, International Statistical Review, Vol. 65(1), pp. 23–48.

Fernández, R. B. (1981), A Methodological Note on the Estimation of Time Series, The Review of Economics and Statistics , Vol. 63(3), pp. 471–476.

Latendresse, E., Djona, M. and Fortier, S. (2007). Benchmarking sub-annual series to annual totals – from concepts to SAS procedure and SAS K12089 Chapter: 10 page: 253 date: February 14, 2012 Restoring Accounting Constraints in Time Series 253 enterprise guide custom task. In Proceedings of the SAS Global Forum 2007 Conference. SAS Institute Inc.

Litterman, R. B. (1983), A Random walk, Markov model for the Distribution of Time Series, Journal of Business and Economic Statistics, Vol. 1(2), pp. 169–173.

Marini M. (2016), Nowcasting Annual National Accounts with Quarterly Indicators: An Assessment of Widely Used Benchmarking Methods, IMF Working Paper 16/71.

Mazzi, G.L. and Proietti, T. (2015). Multivariate Temporal Disaggregation. Handbook on Rapid Estimates, Ch. 8, Eurostat (Nov 13, 2015).

National Bank of Belgium (2016): JDemetra+, https://github.com/jdemetra/jdemetra-app.

Pfeffermann, D. and Tiller, R. (2011). State-Space Modeling with Correlated Measurements with Applications to Small Area Estimation Under Benchmarking Constraints. Southampton Statistical Science Research Institute, University of Southampton. Methodology Working Paper M03/11.

Quenneville, B. and S. Fortier (2012), Restoring Accounting Constraints in Time Series – Methods and Software for a Statistical Agency, in Economic Time Series Modeling and Seasonality, ed. T.S. McElroy, Chapman and Hall, pp. 231–253.

Quenneville, B. and E. Rancourt (2005), Simple Methods to Restore the Additivity of a System of Time Serie, presented at the workshop Frontiers in Benchmarking Techniques and their Application to Official Statistics, Eurostat, Luxembourg, April 2005.

Sax C. and Steiner R. (2013) Temporal Disaggregation of Time Series, http://journal.r-project.org/archive/2013-2/sax-steiner.pdf.


7Presentation

https://github.com/jdemetra/jdemetra-app

https://github.com/jdemetra/jdemetra-app

ec.europa.eu · web viewthe benchmarking procedures improve the quality of the resulting hf data by...

Documents