consistency of concepts and applied methods in business statistics
DESCRIPTION
Consistency of Concepts and Applied Methods in Business Statistics. Improving Consistency in the ESS: Target Populations, Frames, Reference Periods, Classifications and their Applications. Q2014, Vienna, 3rd June 2014. presented by Boris Lorenc. Content. - PowerPoint PPT PresentationTRANSCRIPT
|slide 1
Consistency of Concepts and Applied Methods in Business Statistics
Improving Consistency in the ESS: Target Populations, Frames,
Reference Periods, Classifications and their Applications
Q2014, Vienna, 3rd June 2014
presented by Boris Lorenc
|slide 2Content
1. Approaches to evaluating inconsistencies– Definition– Top-down (framework) and bottom-up (inventory) approaches
2. Inventory– Methodology– Main results
3. Proposals– Structure and areas– Example– Principles
|slide 3Content
Part 1: Approaches to evaluating inconsistencies
|slide 4Consistency: the concept
DefinitionConsistency means agreement in a set of concepts (their referents), as reflected in complete metadata, pertaining to two or more produced statistics that leads to the statistics being coherent and comparable.
Different types of consistency can be defined:
1. horizontal - consistency of produced statistics between two or more statistical domains in a participating country or between two or more statistical domains on the EU level,
2. vertical - consistency of produced statistics within the same statistical domain between participating countries, or their joint consistency with the corresponding statistics produced on the EU level,
Knowledge of the metadata is needed Relates to the other quality dimensions
– relevance, accuracy and reliability, timeliness, coherence, accessibility– user input needed
|slide 5Approaches to evaluating inconsistencies
Top-down– From a framework
Bottom-up– From observed issues, e.g. through inventory, that indicate need for
improvement of consistency
Perhaps optimal to work from both perspectives, planning so that their respective actions and results converge
|slide 6Components of a framework
Context– ‘Vision 2020’: from stovepipes to integrated systems for production of statistics– Current European initiatives: FRIBS, ESBR, .....– Similar processes on country level (NZ, AU, NL, CA,...)– Integration of statistical production processes– Some general characteristics:
dedicated efforts through extended periods (5-7... years) integrated data storage BR central appropriate mixes of survey and administrative data integrated production, based on a common set of data user input in development of systems (within and outside the agency) lessons learned, backtracking...
– Consistency one among several quality components, all assessed against cost too
|slide 7Content
Part 2: Inventory
|slide 8
Questionnaires covered: Coverage of target populations, Extensions of coverage, Sampling frames, Reference periods, Breakdowns, and Size classes (some of the areas asked of only statistical domains) (vide Deliverables 2.6 and 3.6 of WP2)
Sent to the Business Register (BR) and 19 subject-matter domains of business statistics (SDs) in 31 EU and EFTA countries (not in every country to all)
Field period: March-May 2013 High response rate: 27 BR responses and in total 466 SD responses
received and taken into the analysis (vide Deliverables 2.7 and 3.7 of WP2)
Inventory: Questionnaire responses
|slide 9
Major thematic areas– A. Target populations and frame coverage
– B. Business register maintenance
– C. Relations between business register and the subject-matter domains of business statistics
– D. Temporal aspects
– E. Reference periods
– F. Sampling methods and sample coordination
– G. Classifications
– H. Breakdowns
– I. Size classes
Inventory: Introduction
|slide 10
Undercoverage– Restrictions in administrative sources which feed into BR or other sampling frames, the
“threshold” issue (businesses with specific properties, e.g. in specific employment size or turnover value intervals, etc, do not enter into administrative sources) Restrictions “by design” (e.g. certain NACE activities left out, due to conflicts of regulations or
due to established practices in the participating countries)
– Temporal restrictions, part I: non-representation of newly established units (in a large country, a separate study indicated an undercoverage in statistics produced by a SD of 22% due to this reason)
– Temporal restrictions, part II: dynamically changing properties of businesses registered with a time lag on the frame
– Undercoverage of market activities due to insufficient clarity of the concepts used, that is, inability to distinguish between market and non-market activities
Inventory: Frame coverage
|slide 11
Overcoverage– Due to continued existence of units that have ceased with their activity
– Due to lag in update of business properties, which would exclude them from target population (e.g. have decreased their turnover to below a certain value)
– Due to inability to distinguish between market and non-market activities
Trade-offs involved– Administrative data may be advantageous to data quality, process quality or
production economy, but disadvantageous to timeliness if they are only available relatively late
Inventory: Frame coverage (cont’d)
|slide 12
Considerable variation in BR practices of maintenance and update– externally available sources of information
– internal sources and practices
Methods to assign values of register variables– Coding of units for: NACE, employment,
employees, turnover, institutional sector codes
Metadata: existence and management
‘Frozen’ vs. ‘live’ frames
Inventory: Business Register maintenance
|slide 13
Use of the BR by SDs hampered by– Prohibited access of SDs external to NSIs to BR due to national legislations
– Lack of sufficient quality (completeness) of the BR: timeliness (a time lag leading to both undercoverage and overcoverage) and coverage (of activities, size classes, etc)
– Target populations of some SDs not identifiable in the BR (e.g. records of transactions, R&D activities)
– Perceived unsuitability of the BR to be used as a frame for conducting censuses
Use of ‘frozen’ vs. ‘live’ frames– Unclear basis for decision
– How is consistency addressed when ‘live’
Updates from BR to SDs after sample selection– Practices vary largely
Feedback from SDs to the BR– Practices vary largely
Inventory: Relations between BR and SDs
Methodology almost completely missing; differing practices can have effects on produced statistics (accuracy, etc)
|slide 14
A: not specifically covered, but indications of considerable variation
B: a time leg of mostly up to 3 months; updates of the BR mostly annual– Considerable delay before update
C: Frozen frames most often created annually, but also more frequently– Current within a year or month; timeliness of BR seen as high
D: occurs throughout the year, but more often around the new year (November – February)– Due to freshness of frozen BR
Inventory: Temporal aspects
|slide 15
Focused on reporting periods for accounting that in businesses are differing from the calendar year– Varies between the countries from a couple of per cents to almost a quarter of the
businesses
– Rule often applied that reported data are assigned to the calendar year in which the reporting period ends
– Can lead to some estimation issues, especially so if recent change is to be estimated
– Better adjustment methods needed, but not trivial to develop
Inventory: Reference periods
|slide 16
Sampling coordination may improve consistency– Common reference periods
– Common auxiliary information
Current situation (from the inventory): “same time”
Integrate same/different periodicity/-ies?
Inventory: Sampling - methods and coordination
FDI ICT INN IFA JVS OFA PRO R&D SB-I SB-IX SB-IV SB-II I SB-II SES ST-B ST-A ST-C ST-D
FDI 1 1 2 1 5 1 1 2 1 1 1 1 1 2 1 2 1 25ICT 0 1 2 0 3 2 5 2 3 3 3 2 3 3 3 3 38INN 1 5 3 3 1 4 8 6 3 3 4 5 3 3 3 3 3 61IFA 3 2 3 1 2 3 5 9 1 7 8 8 3 5 5 5 5 75JVS 1 2 1 0 2 2 1 2 2 2 2 2 4 4 4 4 35LCS 1 2 2 1 3 0 2 2 2 2 2 2 2 3 4 4 4 38OFA 4 0 0 2 0 0 0 1 0 1 1 1 0 1 1 1 1 14PRO 1 2 1 2 0 2 6 2 3 3 6 2 3 5 2 2 42R&D 1 3 6 3 4 1 4 4 4 4 4 5 3 4 3 4 4 61SB-I 3 4 10 3 1 6 6 4 15 16 16 5 5 5 7 5 111SB-IX 1 1 0 0 0 0 0 2 1 1 1 0 0 0 0 0 7SB-IV 2 4 8 3 1 5 6 14 3 14 14 5 5 5 5 5 99SB-III 3 4 10 3 1 6 6 16 4 14 15 5 5 5 7 6 110SB-II 2 4 9 3 1 5 6 15 4 13 14 5 5 5 5 5 101SES 1 2 1 2 0 2 2 2 2 2 2 2 2 3 3 3 31ST-B 1 2 2 4 0 4 2 4 2 5 4 4 3 10 9 9 65ST-A 1 3 2 3 0 5 2 4 2 3 3 4 3 11 9 10 65ST-C 0 1 2 3 0 3 1 4 1 3 5 3 1 7 8 8 50ST-D 1 2 2 2 0 4 2 4 2 4 4 4 2 9 9 10 61
27 44 12 60 42 13 59 55 101 41 86 91 96 45 77 79 83 78 1089
|slide 17Content
Part 3: Proposals
|slide 18Proposals
Set of proposals with the goals towards:
– Strengthening business register’s role in statistics production (1-10)
– Achieving consistent NACE coverage, breakdowns and size classes (11-16)
– Strengthening methodology for achieving consistency (17-24)
– Developing integrated systems for economic statistics production (25-28)
Vary from broad to more specific
To be treated as a whole
|slide 19
1Identify and implement actions leading to BR becoming the backbone of integrated statistics production.
Rationale:This proposal is central for a consistent system of integrated economic statistics
Strengthening business register’s role in statistics production
|slide 20
2-5Work out a methodology for a BR’s set-up and maintenance, including relations between the BR and the subject-matter domains of business statistics.
Develop detailed guidelines for application of NACE coding for the BR purposes
Develop and implement methods to assess gaps in coverage in the BR
Develop manual(s) to support implementation of standards developed by Proposal 2 – Proposal 4.
Strengthening business register’s role in statistics production
|slide 21Principles
Consistency to be optimised, rather than maximised
Consistency best achieved “by design”
Consistency improved by making BR the backbone of business and economic statistics production
The concept might be influenced by changes in data landscape
Improving consistency well aligned with increased standardisation and automatic processing
|slide 22
Please address queries to:
Contact data:Boris Lorenc
Thank you…
http://www.cros-portal.eu/content/public-documents