1 jeffrey c. worthington chair, energy and environmental division american society for quality oei...
TRANSCRIPT
![Page 1: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/1.jpg)
1
Jeffrey C. WorthingtonChair, Energy and Environmental DivisionAmerican Society for Quality
OEI Director of QualityOffice of Environmental InformationU.S Environmental Protection Agency
Lorena Romero CedenoProgram AnalystOffice of Environmental InformationU.S Environmental Protection Agency
ASQ 32nd AnnualNational Energy & Environmental ConferenceSan Antonio, TexasSeptember 19-20, 2005
![Page 2: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/2.jpg)
Jeffrey Worthington- BIO
Director of Quality for the USEPA Office of Environmental Information. Jeff has served as the Director of Quality Assurance for the USEPA Office of Research and Development (ORD) National Risk Management Research Laboratory (NRMRL) and as the Director of Quality Assurance for TechLaw, Inc. He is an American Society for Quality (ASQ) Certified Quality Manager and ASQ Certified Quality Auditor. Jeff, Senior ASQ member, founding member of the Education Division, Chairs the ASQ Energy and Environment Division and participates on the ASQ Division Affairs Council. He is a founding member and serves on the Board of Directors for the recently established International Association for Information and Data Quality (IAIDQ). Jeff is a member of the Editorial Board of Quality Assurance, Science, and the Law and previously served as Editorial Board member for: the Journal of Environmental Forensics, Environmental Laboratory magazine, and Environmental Testing and Analysis magazine.
He has been with the Federal Government since 1994. Jeff supported environmental engineering quality at NRMRL, joining a team authoring the combined quality and management system for EPA’s Environmental Technology Verification program. He co-lead the EPA team developing EPA’s Information Quality Guidelines. Jeff co-authored a peer review journal paper and received the USEPA Science and Technological Achievement Award, Level III for equating EPA policies and procedures to U.S. Supreme Court Sound Science Criteria (2002). Jeff has spoken at numerous national and regional conferences on the subjects of quality management, audit management, information quality planning and assessment, data authenticity, data quality, and data integrity.
![Page 3: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/3.jpg)
3
Lorena Romero Cedeno - BIO
Lorena R. Romero-Cedeño is the Manager of Quality for the Office of Planning, Resources and Outreach for the Office of Environmental Information. She graduated in 1999 from Colorado State University and currently is finalizing two master degrees in Community and Regional Planning and Latin American Studies from the University of New Mexico. Lorena has a passion for the integrity of environmental data being translated into Spanish targeted to the Spanish speaking public. She is currently conducting research on environmental data for her master thesis and holds two years of experience teaching Spanish Language at the University of New Mexico.
![Page 4: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/4.jpg)
4
DISCLAIMER
The opinions expressed in this technical presentation are those of the author and do not necessarily reflect the views of the US EPA.
![Page 5: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/5.jpg)
SPONSORED BY MICKEY………..
WHY MICKEY?
Statue theme in 2005 He’s cool!!! He is a performer… must
know something about performance measurements
He is a genius Wears the best clothes Never ages Is very popular Is everywhere Has a lot of friends Certified quality inspector
(usually wears white gloves)
![Page 6: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/6.jpg)
MICKEY FACTS
Cherry Blossoms
Age
Oscar
First words
First cartoon character to ___
Quotes
![Page 7: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/7.jpg)
![Page 8: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/8.jpg)
![Page 9: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/9.jpg)
![Page 10: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/10.jpg)
10
Selected Performance Measurement
Systems Used inEPA and Federal Government
GPRA – Government Performance and Results Act
FMFIA – Federal Managers Financial Integrity Act (management controls)
EVM – Earned Value Management CPIC – Capital Planning and Investment
Coordination (especially information technology)
Balanced Scorecard
![Page 11: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/11.jpg)
11
GPRA – Government Performance and Results Act Mandated by OMB Effort to identify and track
project/program/process outcomes (vs. outputs) Managed at a high level in the organization In EPA, may include:
QA/QC procedures Data quality reviews Data limitations Error estimate
Each office negotiates their own list of measures with OCFO and OMB
Annual performance plan submittals – www.omb.gov
![Page 12: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/12.jpg)
12
Outcomes vs. Outputs
Outputs = productivity or efficiency metrics such as the number of reports written per month
Outcomes = the impact that the output has on success of the process organization
![Page 13: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/13.jpg)
13
FMFIA – Federal Managers Financial Integrity Act (management controls) FMFIA Act of 1982 Agency weaknesses vs. Annual Management Accomplishments
and Challenges report Office’s self-identify adequacy of
“management controls” OIG, GAO, and OMB overview comments
may be part of overview Action plan to make corrections and
identify schedule and measures to correct
![Page 14: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/14.jpg)
14
EVM -Earned Value Management
Standard - ANSI/EIA-748-A Standard for Earned Value Management Systems
EV = value of completed work expressed in terms of the budget assigned to that work
Objective measure of work accomplished Based on budgeted value of the work ‘What you got for what it costs you.”
Compliance standard criteria grouped in 5 areas Planning, scheduling, and budgeting Organization Analysis and management reports Revisions and data maintenance Accounting considerations
Performance measurement Establish a baseline Monitor in time units Review both cost variance and schedule variance from baseline
![Page 15: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/15.jpg)
15
Office of Environmental Information Report- Earned Value Management: Fiscal Year 2004Lessons Learned December 2004
Applied to assist IT project managers consistent with the ANSI/EIA-748 standard and in line with requirements of the OMB Exhibit 300.
3 categories of LESSONS LEARNED Refinement of EVM methods Increasing consistency of project reporting Facilitating management analysis of EVM
data
![Page 16: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/16.jpg)
16
LESSON LEARNED 1Refinement of EVM methods Use EVM across all phases of mixed life-cycle
projects Separate milestones and associated resources
for system life-cycle phase Separate milestones and associated costs by
contractor wherever possible Keep milestones from getting too large in
duration, cost, or scope Attempt to limit milestones to a single fiscal
year (or less) Establish objective measures for determining
earned value
![Page 17: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/17.jpg)
17
LESSON LEARNED 2Increasing consistency of project reporting
Use standard template for
reporting
Institute standard reporting cycles
![Page 18: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/18.jpg)
18
LESSON LEARNED 3Facilitating management analysis of EVM data
Provide both numerical and graphical representations of EVM data
Use color-coded standardized scoring system
![Page 19: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/19.jpg)
19
Balanced Scorecard
A Management System (not only a measurement system) that enables organizations to clarify their vision and strategy and translate them into action.
Provides feed back for both internal processes and external outcomes in order to continuously improve strategic performance and results.
Balanced Scorecard Institute www.balancedscorecard.org
![Page 20: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/20.jpg)
20
Balanced Scorecard – view organization from 4 perspectives
Learning and growth perspective
Business process perspective
Customer perspective
Financial perspective
![Page 21: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/21.jpg)
21
Balanced ScorecardBusiness Process Perspective
Products and services conforming to customer requirements
Mission-oriented processes AND support (repetitive) processes (easier to benchmark repetitive
processes)
![Page 22: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/22.jpg)
22
Balanced ScorecardFinancial Perspective
Timely and accurate funding data
Cost-benefits
![Page 23: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/23.jpg)
23
Balanced ScorecardCustomer Perspective
Customer focus
Customer satisfactions
Matching products and services to customer groups
![Page 24: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/24.jpg)
24
Balanced ScorecardLearning and Growth Perspective
Employee training
Organizational culture toward personal and professional growth
Keeping knowledge workers in a continuous learning mode
![Page 25: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/25.jpg)
![Page 26: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/26.jpg)
![Page 27: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/27.jpg)
27
BRM – OMB’s Business Reference Model
From OMB’s Federal Enterprise Architecture Program Management Office (FEAPMO)
Identifying and categorizing an organization’s lines of business and sub-functions to relate them to the enterprise architecture
![Page 28: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/28.jpg)
28
PRM – OMB’s Performance Reference Model
From OMB’s Federal Enterprise Architecture Program Management Office (FEAPMO)
A standardized approach to IT performance Define measurement indicators Establish relationship inputs and outcomes Sets baselines and targets for improvement Closely related to the BRM
![Page 29: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/29.jpg)
29
GENERALIZATION – for various performance measurement systems
Relationship to organization’s vision and mission may need to be verified
Need to verify measures: Are real Are meaningful Are not redundant Include quality (limitations on use)
Are we measuring this item because it is easy to measure? Is the goal convenient or challenging? Applicability - What does measuring this particular thing tell
you about the organization’s performance? Completeness – Are enough measures being tracked to
characterize a broader sense of organizational performance?
![Page 30: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/30.jpg)
30
How to Speak the Language of Senior ManagementStephen George Quality Progress May 2003
OBSERVATIONS The language of senior management is
very different than the language of quality managers
To have their suggestions heard and accepted, quality professionals need to learn management’s financial vocabulary
There are seven steps you can take to improve communication with management
![Page 31: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/31.jpg)
31
Speaking the Language of Senior Management
The Seven Steps PROVE – quality professionals have to prove the need and
then prove the value of proposed improvement processes COACH – quality managers must help senior management
move goals, objectives, and strategies into action, …. To move from strategies to measures to projects
INFLUENCE – involve senior management in quality processes, participate as trainers, incporated quality in reviews
PARTNER – with the CEO if possible, or with any other senior manager, focus on the area of greatest need, ask for guidance from senior management
PILOT – initiate a project in an area of value to the organization
BENCHMARK – with senior managers in other organizations ALIGN – align the quality system language with the senior
management language, everyone should be speaking the same language
![Page 32: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/32.jpg)
32
Balanced scorecard for a quality system – HOW??
Look for ways for the quality system to demonstrate an improvement in each of the four perspectives
Try to use a balanced scorecard for the quality system itself – can you really do this?
Performance measures of the organization vs. performance measures for individual processes (e.g., quality systems) – does this mean there is a new customer to consider in the balanced scorecard, the internal customer?
How much you are spending internally on your own process, for example
When do the management system and the quality system come into full alignment?
![Page 33: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/33.jpg)
33
Balanced scorecard for a quality system
OUTPUTS
OUTCOMES
![Page 34: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/34.jpg)
34
Balanced scorecard for a quality system
Learning and growth perspective
Business process perspective
Customer perspective
Financial perspective
![Page 35: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/35.jpg)
35
Mickey says: - arithmetic is counting to twenty without taking off our shoes
Quality Mickey says:
Performance measurement of quality is measuring both outputs and outcomes
![Page 36: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/36.jpg)
36
HOW CAN AN INTERESTED QUALITY MANAGER TRAIN THEMSELVES?
www.balancedscorecard.com
![Page 37: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/37.jpg)
37
WHAT ELSE IS GOING ON AT EPA FOR QUALITY NOW?
OEI is re-developing its 5 year QMP OEI has delegated authority for
directives for Agency information policy, including quality policy
![Page 38: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/38.jpg)
THE END
![Page 40: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/40.jpg)
40
![Page 41: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/41.jpg)
41
![Page 42: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/42.jpg)
42
![Page 43: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/43.jpg)
43
![Page 44: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/44.jpg)
44
![Page 45: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/45.jpg)
45
![Page 46: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/46.jpg)
46
![Page 47: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/47.jpg)
47
![Page 48: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/48.jpg)
48
![Page 49: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/49.jpg)
49
![Page 50: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/50.jpg)
50
![Page 51: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/51.jpg)
51
![Page 52: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/52.jpg)
52
OVERVIEW Trends in information quality What is a Body of Knowledge? Who owns a BOK? Basic structure Conclusion Resources
![Page 53: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/53.jpg)
53
Trends in information qualityMORE IS BETTER More information is better? Faster information is better? More privacy is better? More access is better?
![Page 54: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/54.jpg)
54
Trends in information quality
WHAT ARE THE TRENDS? Information technology build-out phase is
nearing completion Increased access Increased transparency Recognition that info is a resource Recognition that IT is not the strategic
advantage (IT Doesn’t Matter)
Increased need for security Increased need for control (?)
![Page 55: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/55.jpg)
55
What is a Body of Knowledge (BOK)?
… an aggregate of what is known and understood within a field of endeavor due to familiarity gained through experience or association www.findmehere.com
![Page 56: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/56.jpg)
56
What is the purpose of a BOK?
To provide a means for people with shared
interest to better communicate
To provide formal organization and
recognition to a knowledge area
To recognize and improve knowledge
To serve as the basis for a test or
certification
![Page 57: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/57.jpg)
57
International Association for Information and Data Quality www.iaidq.org
•IAIDQ purpose – to create a world-wide community of people who: •Understand the critical roles data and information play•Recognize the consequences of poor quality data and information•Wish to help organizations enjoy the benefits of improved data and information
•IAIDQ mission•Increase the awareness of the impact of poor quality data and information.•Help leaders understand that the high losses can be dramatically reduced.•Provide a network for members to exchange tips and techniques for quality improvement.•Provide opportunities to learn critical skills for making quality information and data a reality.
•Membership types•Profession members•Academic members•Lay members•Student members
![Page 58: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/58.jpg)
58
Usability Professionals’ Association - http://www.upassoc.org/
What is usability?
The degree to which something – software, hardware, or anything else – Is easy to use and a good fit for the people who use it.
![Page 59: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/59.jpg)
59
Usability Body of Knowledge
Methods User interface design principles and guidelines Organizational integration of usability
(including managing usability teams, integrating usability into software development, introducing usability to organizations and clients and making a business case for usability)
Roles, skills, and job categories for usability professionals
Definitions of usability terms Related fields and disciplines
![Page 60: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/60.jpg)
60
What is usability?
… a quality or characteristic of a product … whether a product is efficient, effective,
and satisfying for those who use it …the name for a group of techniques
developed by usability professionals to help create usable products
… a shorthand term for a process or approach to creating those products, also called user-centered design
![Page 61: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/61.jpg)
61
Project Management Body of Knowledge -
Project Management Institute
http://www.pmi.org/info/pp_pmbok2000welcome.asp
![Page 62: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/62.jpg)
62
![Page 63: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/63.jpg)
63
Software Engineering Body of Knowledge www.swebok.org
Software configuration management
Software construction Software design Software engineering
architecture Software engineering
management Software engineering
process Software evolution and
maintenance Software quality analysis Software quality analysis Software requirements
analysis Software testing
ASSOCIATED DISCIPLINES Cognitive sciences and human
factors Computer engineering Computer science Management and management
science Mathematics Project management Systems engineering
![Page 64: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/64.jpg)
64
![Page 65: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/65.jpg)
65
EPA Body of Knowledge Areas for Quality
What you need to know to work or support quality in the EPA.
Quality Management Systems Quality Project Planning Data Quality Objective Planning Quality Training Science measures and associated metadata Environmental methodologies Concepts – hierarchial quality systems,
graded approach….Did I leave anything out?
![Page 66: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/66.jpg)
66
Your examples….?
_________________________ _________________________ _________________________ _________________________ _________________________ _________________________
![Page 67: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/67.jpg)
67
Who owns a BOK? Certification program
ASQ Certified Quality Engineer (CQE) has a CQE BOK, also for CQA, CQM, and CSQE
All are owned by ASQ Professional organizations Government bodies Academic groups
![Page 68: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/68.jpg)
68
How to structure a BOK for information and data quality?
Focus on………… The basics – the basic principles The what of information and data –
identify and describe them The how of information and data – the
processes The who – what a information and data
quality practitioner is and how they do their work
![Page 69: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/69.jpg)
69
Basic structure Overarching principles for information and data quality Communication 1: Basic terminology Communication 2: Alignment with the organization Information and data quality features, measures, and
acceptance criteria Preliminary review of an organization’s information
and data quality Managing and planning for information and data
quality Implementing information and data quality Assessing information and data quality Tracking and reporting information and data quality Roles, skills, and job categories for information and
data quality professionals
![Page 70: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/70.jpg)
70
What are the quality principles for information and data?
general quality management principles apply to information, data, and information and data quality in the same way they apply to all basic management processes
information has unique characteristics the bear on your ability to manage the quality
information is a resource of the enterprise and the quality should be managed as any other resource
information quality includes the quality of content, format, and functionality
information quality includes: information features (including functionalities) freedom from defect customer service effectiveness and efficiency
you must be able to measure the information quality, in order to understand it completely, manage it, and to continually improve quality
you must be able to track quality costs (failures, scrap and rework, etc.) in order to effectively manage the resources used to ensure information quality
![Page 71: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/71.jpg)
71
What is unique about information?
…as a resource? It cannot be used up It can be copied It can be shared across large distances It can be moved easily It is difficult to keep secure It may become dated Age may not affect its quality
![Page 72: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/72.jpg)
72
What is unique about information?
…as a product? The same information can be
provided/sold to more than one person
It can be delivered cheaply Your product can be easily shared by
the person who purchased it
![Page 73: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/73.jpg)
73
Communication 1:Basic terminology
Terminology needs for information and data quality Recognition that terminology is discipline-specific Recognition that there may be conflicting views on
the definitions for certain terms (i.e., information, data)
Multiple definitions may be needed for the same term
More complex definitions may be needed (data quality vs. data entry quality vs. data content quality vs. data content transfer quality)
Need to include senior management terminology and align quality terminology with senior management terminology
![Page 74: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/74.jpg)
74
Communication 2:Alignment to the organization
Methodology for tracking to performance measures for the organization
Best practices for communicating information and data quality to senior management
Models for relationship of information and data to the organization’s product
![Page 75: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/75.jpg)
75
Information and data quality features, measures, and acceptance criteria
Standardized list of features and functionalities
Standardized measures and measurement units for the features and functionalities
Suggested acceptance criteria Relationship between concepts and
measures clearly mapped (access vs. usefulness vs. usability vs. integrity vs. security vs. transparency vs. objectivity vs. accuracy etc.)
![Page 76: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/76.jpg)
76
Preliminary review of an organization’s information and data quality
Techniques to identify priorities for measurement in existing systems.
Process to use existing available data about data to determine quality.
Best practices in automated information and data quality measurement for existing data systems.
Methods to review current quality measure in comparison to organization’s mission and goals.
![Page 77: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/77.jpg)
77
Managing and planning for information and data quality
Identification of standard types of products that should be subject to management and planning
For each standard type of information product and data, a standard set of planning criteria
Standard techniques for identifying the level of planning needed
General planning guidance document. Process to identify goals for general information product and
data quality projects. Process to identify acceptable measures to determine
conformance with the goals. Processes to crosswalk management/planning to other
project management initiatives in an organization Processes to crosswalk information product and data quality
objectives to the organization’s performance measurement.
![Page 78: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/78.jpg)
78
Implementing information and data quality
Identification of standard implementation phases
Best Practices for implementing information and data quality initiatives.
Example standard operating procedures (SOPs).
![Page 79: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/79.jpg)
79
Assessing information and data quality
Identification of hierarchy of assessment. (system vs. program vs. project vs. data system vs. data set vs. data field).
Standard procedures for planning, implementing, and reporting assessment information.
Suggested procedures for resolving corrective actions for existing data deficiencies.
Types of assessments – based on existing BOKs, self-assessment, third-party, conformance,
![Page 80: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/80.jpg)
80
Tracking and reporting information and data quality
You can’t manage information and data quality without having the information to manage……….
You must be able to track and report your information and data quality in terms of the organization’s performance goals……. (see COMMUNICATION 2 in the BOK)
![Page 81: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/81.jpg)
81
Tracking and reporting information and data quality
Methodology to identify what to report
Standard reporting formats for information and data quality
Routine reporting to middle and senior management
Techniques to trend data and information quality to track improvement
![Page 82: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/82.jpg)
82
Roles, skills, and job categories for information and data quality professionals
Information and data quality managers
Information and data quality assessors
![Page 83: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/83.jpg)
83
Conclusion•We are already developing a body of
knowledge in this area.
•Structure to our BOK will help us
communicate with each other.
•It will help us better:
• plan for information and data quality
•measure information and data quality
•establish organizational performance
•align with organizational goal.
•The job of quality professionals will be
better planned and understood.
![Page 84: 1 Jeffrey C. Worthington Chair, Energy and Environmental Division American Society for Quality OEI Director of Quality Office of Environmental Information](https://reader035.vdocuments.us/reader035/viewer/2022062716/56649e005503460f94ae89bd/html5/thumbnails/84.jpg)
84
Resources for information quality & data quality & body
of knowledge Larry English, www.infoimpact.com Dr. Tom Redman,
www.dataqualitysolutions.com Michael Bracket Dr. Wang, MIT Data Quality Group American Society for Quality (ASQ)
Information Integrity Group International Organization for Information
and Data Quality (IAIDQ.org) Project Management Institute