quality management of statistical processes using quality gates narrisa gilbert australian bureau of...
TRANSCRIPT
![Page 1: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/1.jpg)
Quality Management of Statistical Processes Using
Quality Gates
Narrisa Gilbert
Australian Bureau of Statistics
May 2011
![Page 2: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/2.jpg)
Overview
• Background• What are Quality Gates• Attributes of Quality Gates• Six Components of Quality Gates • Outcomes of Quality Gates• Further information
![Page 3: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/3.jpg)
Different perspectives on quality
![Page 4: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/4.jpg)
ABS approach to quality management
• What?– Reinforcing a high degree of credibility for the ABS
and its outputs– Maintaining the relevance of ABS outputs
• How?– Effective relationships with respondents– Processes that produce high quality outputs– Regular review and evaluation of statistical activities– Staff skilled and motivated to assure the quality of
ABS outputs
![Page 5: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/5.jpg)
Background• The Australian Bureau of Statistics produces a
lot of statistics
• On occasion the quality (accuracy) of data has been called into question at the last minute (right before release to the public)
• Sometimes no issues are identified• Sometimes errors are discovered
• Sometimes errors in data are identified after release
![Page 6: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/6.jpg)
What are quality gates?
"checkpoints or decision points at various strategic places in a statistical
process at which the quality of the process at or up to that point is
explicitly assessed"
![Page 7: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/7.jpg)
Attributes of Quality Gates
• Planned in advance of issues occurring
• Designed to facilitate the detection, discussion and resolution of issues and problems with processes earlier
• Constantly reviewed and revised
![Page 8: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/8.jpg)
Attributes of Quality Gates
• Collaborative
• Communicate openly the progress/issues associated with the collections
• Provide a model of accountability and responsibility for survey processes
![Page 9: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/9.jpg)
Attributes of Quality Gates
• Knowledge Management
• Create a store of corporate knowledge
• Document and monitor issues and actions throughout the collection cycle
![Page 10: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/10.jpg)
Six components of quality gates
Placement Quality measures Roles Tolerance Actions Evaluation
![Page 11: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/11.jpg)
1. Placement
Assigning priorities Related to risk profile of collection cycle Want to detect errors 'upstream‘
What can go wrong?
When can this problem occur?
What impact can it have?
![Page 12: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/12.jpg)
Placement of Quality Gates
• For example
Within
Handover
UNECE (United Nations Economic Commission for Europe) Secretariat 2009, Generic Statistical Business Process Model, Version 4, April 2009, UNECE, Geneva.
![Page 13: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/13.jpg)
Areas of risk changes to processes, systems,
methodologies
data transformations
overlap / coordination / integration with other areas
knowledge management
![Page 14: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/14.jpg)
2. Quality measures Choose good indicators of potential
problems Prioritise! Definitions May need to drill down Time series useful
How would we know if this problem occurred?
![Page 15: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/15.jpg)
3. Roles Owner area Sign-off person Operational person / Gate Keeper Stakeholders Gate definition Gate assessment / sign-off
Who is responsible?
Who will this affect?
![Page 16: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/16.jpg)
4. Tolerance Driven by user requirements Consider size of natural variation, sampling
errors, level of detail, importance of outputs Need to form expectations Historical data may help Predetermined!
What is an acceptable level of quality?
![Page 17: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/17.jpg)
5. Actions reflect extent of the problem reflect consequences "traffic lights" concept
What will we do (if there is a problem)?
Who needs to be informed?
![Page 18: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/18.jpg)
6. Evaluation evaluation of processes evaluation of quality gates
What has this information told us about our quality?
How can we improve in the future?
![Page 19: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/19.jpg)
Outcomes from Quality Gates
Business processes well-defined and known by staff
Responsibilities for processes and quality clearly defined
Statistical risk explicitly identified & assessed
![Page 20: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/20.jpg)
Outcomes from Quality Gates
Explicit sign-off occurs at Quality Gates
Quality Measures clearly defined and understood
• Errors identified earlier (and fixed)
![Page 21: Quality Management of Statistical Processes Using Quality Gates Narrisa Gilbert Australian Bureau of Statistics May 2011](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519b92455034667578b48f2/html5/thumbnails/21.jpg)
Further information
• Quality Management of Statistical Processes Using Quality Gates, Dec 2010, (cat.no. 1540.0) on the ABS website.
• http://www.abs.gov.au/ausstats/[email protected]/mf/1540.0