Download - Quality Assurance in Statistics
Quality Assurance in Statistics: Producer Perspectives in the Light of
International Practices
S.S. Colombage
50th Open Forum: Ensuring Quality of Survey Data Centre for Poverty Analysis
April 29, 2014
Outline
• Rationale for Quality Assurance
• International standards for statistics
• Sri Lanka’s Case
• Concluding Remarks
Rationale for Quality Assurance
• Main statistical agencies – Census and Statistics Department (CSD)
• Household income & expenditure surveys, labour force surveys • Macro-level data: National accounts, prices & wages
– Central Bank of Sri Lanka (CBSL) • Macro-level data: Balance of payments, financial sector, fiscal
sector • Consumer Finances Surveys - abandoned?
• Decision makers in public & private sectors heavily depend on statistics
• Doubts about GDP, inflation, poverty & unemployment data.
Sri Lanka: Dramatic poverty reduction vs. moderate economic growth?
Sources: Department of Census & Statistics Central Bank of Sri Lanka
Sri Lanka: Dramatic poverty reduction vs. stagnant income share of the poorest?
Sources: Department of Census & Statistics Central Bank of Sri Lanka
What is statistical quality?
Old notion: quality = accuracy
Current version: quality = accuracy +
• Relevance
• Timeliness
• Accessibility
• Clarity
• Coherence
• Comparability
• Efficiency etc.
Improving statistical quality means enhancing the fitness of data to meet the expectations of the users.
International Codes of Practices
• At international level, increased attention on statistical quality management since the mid-1980s
• Common quality management tools developed by international agencies & national statistics agencies – UN Statistical Commission – Expert Group: Generic Quality
Assurance Framework (NQAF), 2010 – Quality Assurance Framework of the European Statistical
System – IMF: Data Quality Assessment Framework (DQAF) – Statistics Canada: Quality Assurance Framework – Code of Good Practice in Statistics for Latin America & the
Caribbean
NQAF 4: Assuring professional independence
• Statistical agencies should develop, produce and disseminate statistics without any political or other interference from government agencies, private sector entities or persons.
• The French law on statistics states that "the conception, production and dissemination of official statistics are carried out in full professional independence".
• In Sri Lanka, no mention about autonomy of CSD, which comes under the purview of Ministry of Finance & Planning
NQAF 16: Assuring timeliness & punctuality
• Statistical agencies should minimize delays in making data available – Timeliness: How fast the data are released
– Punctuality: Whether the data are released on the dates promised.
• Sri Lanka CSD: Advance Data Release Calendar (ADRC) available on the website
• Delays – e.g. Final report of Population Census (2011) still not released
NQAF 19: Managing metadata
• Statistical agencies should provide information covering the underlying concepts, variables and classifications used, the methodology of data collection and processing, and indications of the quality of the statistical information - to enable the user to understand all attributes of the statistics.
• Metadata = Data about data • SL CSD: Provision of metadata are inadequate. • Statistics Canada
– Integrated Meta Data Base (IMDB): repository of information on each of nearly 400 active surveys
– IMDB is available to outside users of SC’s website. Each data release on the site includes hyperlinks to metadata from IMDB
• Australian National Health Information Knowledgebase: An electronic storage site for health metadata
• United States Health Health Information Knowledgebase
Quality Assessment & Reporting
• Purpose: To evaluate & prevent problems that may arise during statistical processes.
• Assess product quality in terms of relevance, accuracy & reliability, timeliness & punctuality, coherence & compatibility, accessibility & clarity
• Each of the above dimensions is assessed by using quality indicators. E.g. Accuracy: “How complete is the sample frame?”
• Quality assessments could be conducted by internal or external teams
• Statistical assessment systems are well established in several countries including Canada, Netherlands & South Korea
Sri Lanka: Positive initiatives
• LankaDatta maintained by CSD – National data repository of Sri Lanka, which holds of
over 350 data files from different surveys – LankaDatta adheres to international standards in
archiving microdata such as the Dublin Core Metadata Initiative
• General Data Dissemination System (GDDS) of IMF – Both CSD & CBSL have been participants since 2000 – Information/data are submitted to IMF under Article
IV
Sri Lanka: Shortcomings
• A National Quality Assurance Framework (NQAF) for statistics lacking in Sri Lanka
• Census & Statistics Department (CSD) & Central Bank (CBSL) are silent on quality assurance
• QA is explicit in websites of statistical agencies of some countries – e.g. Australia, Canada
• CSD & CBSL are not very receptive to user feedbacks
Concluding Remarks
• Quality assurance is not prominent in the Census & Statistics Dept., Central Bank & other statistical agencies in Sri Lanka
• Hence, credibility of data are at stake • Recommendations:
– National quality assurance framework for statistics following UN guidelines be adopted
– Quality culture needs to be inculcated – User feedbacks need to be accommodated in
statistical processes – Professional independence of statistical agencies vital