exploring the cavern of data governance - aairaair.org.au/app/webroot/media/sigs/sig forum...
TRANSCRIPT
EXPLORING THE CAVERN OF DATA GOVERNANCE AUGUST 2013
Planning and Information Office | SIBI
Darren Dadley | Business Intelligence, Program Director
Data Management Overview
2
SIBI Program Methodology
Definitions: Data Management & Data Governance
The exercise of authority and control (planning,
monitoring, and enforcement) over the management of
data assets. (*)
3
Data Management
The planning, execution and oversight of policies, practices
and projects that acquire, control, protect, deliver, and
enhance the value of data and information assets. (*)
Data Governance
(*) DAMA International 2009
4
Data Governance Challenges – Key reasons for Failure (*)
Data Governance Overview
Data Governance Challenges
Failure to Execute
Lack of knowledge and Understanding by Senior Management (i.e. skills requirements, strategic
outcomes, process improvement) leads to a
failure to execute.
Lack of Ownership
Ownership, responsibility and accountability not
assigned.
Lack of Awareness
Executives and key stakeholders of data
management capabilities have a lack
of knowledge and awareness of DG.
Lack of Accountability
Accountability not assigned to each
process
Task is overwhelming
DG is too big for any one person to accomplish.
Adequate resources are not assigned.
(*) Adapted from 2011 Baseline Consulting Group, Inc.
- Training
- Education
- Communications
- Workshops
- Assign sponsor
- DG Forums
- Personal
development
plans
- KPIs
- Education
- Best practices
- Bench marking
- Leverage other
successes
- RACI
- Data stewards
- Personal
development plans
- KPIs
- Pilot projects
- Series of
manageable
projects
- Identify key areas
of concern
- Split the tasks
- Identify and assign
resources
Data Governance Strategy
5
What is Data Governance
for the University
Develop processes
Identify a key initiative
as a Pilot
Define KPIs as measures for success
Educate and engage
stakeholders
Document improvements and processes
Communicate success
SUCCESSFUL DATA
GOVERNANCE
Managing Expectations
• Develop DG
vision
statement in
line with
University’s
strategic vision
• Define DG
• Scope DG with
context of
University
• Define Data
Governance
Framework
• Define DG
organisation
• Define roles and
responsibilities
(RACI)
• Select Data
Management
Pilot area
• Workshop to
identify and
develop KPIs
• Determine
accountability
for KPIs
• Identify KPI
benefits and
ROI
• Define Pilot
group
• Develop training
plan
• Develop
communications
and engagement
plan
• Educate
stakeholders
about DG
• Review and
define process
maps
• Establish SOPs
(standard
operating
procedures)
• Develop review
process
• Communicate
success to key
stakeholders and
broader audience
(email, bulletin,
newsletters)
Data Governance vs. Data Management
6
Data Governance (Organisation and Activities)
Strategy
Organisation and roles
Deliverables and standards
Projects and services
Issues management
Creating guiding principles
Data asset valuation
Data Management (Execution)
Data profiling
Data quality monitoring
Data cleansing
Semantic rules
Data enrichment
Business rules creation &
maintenance
Enterprise data modeling
Metadata definition
Business glossary definition
Data archival
Backup and Recovery
Authentication
• Provide Guidance
• Create & Implement
Deliverables
• Provide Feedback
• Track Progress
Data Management Overview
7
(DMBOK) Data Management Functions •Analysis •Measurement •Improvement
•Architecture •Integration •Control •Delivery
•Acquisition & Storage •Backup & Recovery •Content Management •Retrieval •Retention
•Architecture •Implementation •Training & Support •Monitoring and Tuning
•Acquisition •Recovery •Tuning •Retention •Purging
•External Codes & Internal Codes •Customer Data •Product Data •Dimension Management
•Enterprise Data Modelling •Value Chain Analysis
•Data Modelling •Database Design •SDLC •Implementation
•Standards •Classification •Administration •Authentication •Auditing
Data Architecture Management
Data Development
Data Security Management
Data Warehousing &
Business Intelligence
Management
Document & Content
Management
Data Quality Management
Reference & Master Data Management
Meta Data Management
Database Operations
Management
Data
Governance
Data Management Overview
› Data Governance
› Data Security Management – Data Visibility
› Data Quality and Data Profiling
› Master Data Management
› Metadata Management & Business Glossary
8
Current focus for SIBI
University of Sydney
Data Management
Data Governance
SIBI
Data Management Overview
9
DMBOK – 7 Environmental Elements
People
Process Technology
• Organisation & Culture
• Roles & Responsibilities
• Goals & Principles
• Activities
• Deliverables
• Practices & Techniques
• Technology
Provide a consistent way to describe and strategically plan each function Technology
Roles &
Responsibilities
Goals & Principles
Organisation &
Culture
Strategy
Deliv
era
ble
s
Acti
vit
ies
Pra
cti
ces &
Tech
niq
ues
Un
ive
rsit
y o
f S
yd
ney D
ata
Ma
nag
em
en
t F
ram
ew
ork
Data Management Overview
10
DMBOK – 7 Environmental Elements
› Goals & Principles – The directional business goals of each function and the fundamental principles that guide performance
of each function.
› Activities - Each function is further decomposed into lower level activities (tasks and steps)
› Deliverables - The information and physical databases and documents created as interim and final outputs of each function.
Some are considered essential, some are generally recommended, and others are optional depending on circumstances.
› Roles and Responsibilities - The business and IT roles involved in performing and supervising the function and the specific
responsibilities of each role in that function. Many roles will participate in multiple functions.
› Practices & Procedures - Common and popular methods and techniques used to perform the processes and produce the
deliverables. Risks and issues management.
› Technology - Categories of supporting technology (primarily software tools), standards and protocols, product selection
criteria and common learning curves..
› Organisation and Culture - These issues might include:
- Reporting Structures, Teamwork and Group Dynamics
- Budgeting and Related Resource Allocation Issues
- Authority & Empowerment
- Shared Values, Beliefs, Expectations & Attitudes
- Change Management Recommendations
- User engagement: communications / training / education
Data Governance Overview
11
Data Governance – University Organisation & Culture
• Support the DGC, by implementing and refining the data ownership, data stewardship and data custodian roles throughout the University. • Provide Subject Matter Expert (SME) knowledge and support to the data governance strategy
• Own the data governance strategy • Promote, endorse and approve the development and enhancement of the data governance management framework
Data Owners Management
Group (DOMG)
Data Modellers Database
Administrators
Data Stewards
Data Integration
Specialists
Data Quality
Specialists
Supported by:
Information / Data
Architect
Data Governance Committee
(DGC)
Organisation
• Operating model
• Arbiters & escalations points
• Data Governance organisation members
• Roles & Responsibilities
• Terms of Reference
• Data ownership and responsibility
Deans of Faculties and Directors of
Professional services Units, e.g.
Finance, Research, HR, ICT
Directors, Heads of department,
Managers of functional areas
12
University Principles and Goals (recommended)
Data Management
Principles
Trusted
Valued
Shared
Re-used
Managed
Governed
Data Management Overview
Trusted. We trust in our information. Access to and use of data
will promote trust and confidence through adherence to relevant
Data Governance Policies and procedures, privacy, confidentiality
and security requirements.
Valued. Data is valued as a strategic resource and an asset. As a
result, data and information will be of high quality, accurate,
relevant, timely and support confident business decisions.
Shared. Information and data is accessible, transparent and
available to be shared as part of the University’s sharing of
information obligations to; the community, staff, students,
researchers and alumni.
Re-Used. Data and information should be obtained from a single
authoritative source. Data and information is collected in a
consistent manner and is available to be used for different
purposes with confidence.
Managed. Data and information is managed throughout its
lifecycle and is compliant. Information Management Procedures
and practices are standardised and applied across the University
and apply to all involved in the data management lifecycle.
Governed. Data and information is governed in accordance with
the roles and responsibilities as defined in the University’s Data
Governance Framework, the University’s strategic goals and in
compliance with the requirements of Law.
13
Deliverables, Activities, Practices & Techniques
Data Management Overview
Data Management Overview
14
DMBOK Functions •Analysis •Measurement •Improvement
•Architecture •Integration •Control •Delivery
•Acquisition & Storage •Backup & Recovery •Content Management •Retrieval •Retention
•Architecture •Implementation •Training & Support •Monitoring and Tuning
•Acquisition •Recovery •Tuning •Retention •Purging
•External Codes & Internal Codes •Customer Data •Product Data •Dimension Management
•Enterprise Data Modelling •Value Chain Analysis
•Data Modelling •Database Design •SDLC •Implementation
Data Architecture Management
Data Development
Data Security Management
Data Warehousing &
Business Intelligence
Management
Document & Content
Management
Data Quality Management
Reference & Master Data Management
Meta Data Management
Database Operations
Management
•Standards •Classification •Administration •Authentication •Auditing
Data
Governance
Data Quality Management
15
Definition
Planning, implementation and control activities that apply quality
management techniques to measure, assess, improve and ensure the
fitness of data for use.*
*Source: DAMA-MBOK 2009
Communication
Pri
nci
ple
s
Organisation & Culture
Roles and Responsibilities
Data Quality Management Framework – HR Pilot
Accuracy
Completeness
Integrity
Timeliness
Validity
Consistency
Issues Log
Risk Matrix
Critical success factors
Authority & Empowerment
Information Compliance
Data Privacy
Govt. Legislation
Internal Audit
Roles
Forums
Data Custodian
Data Owner
Sponsor Data Steward
SIBI Program Board
BOG
Expectations & Attitudes
Pilot group structure
Change Management
Technology: Data Profiling (Informatica), Data cleansing (IDQ-Informatica)
University of Sydney Vision
Goals
*** Develop vision for Data
Quality Mgmt. and for Pilot
with HR data. (workshop)
User Engagement
Education Comms
Deliverables
Activities
Practices & Techniques
External Audit
Data Quality Management
17
Data Quality Dimensions
• Does the data accurately represent reality or a verifiable source?
Accuracy
• Is all necessary data present? Completeness
• Are all data elements consistently defined and understood?
Consistency
• Is the structure of data and relationships among entities and attributes maintained consistently?
Integrity
• Is data available when needed? Timeliness
• Do data values fall within acceptable ranges defined by the business?
Validity
Data Quality Methodology - Roadmap
18
2. Define DQ Requirements
Activities
Deliverables
Technology
3. Profile, Analyse & Assess DQ
IDE – Informatica Data Profiling tool
Baseline
Updated Issue Log
Scorecard Report
IDQ – Informatica Data Quality tool
Recommend Actions
Actions:
- Training / education / comms
- Business Processes
Improvement (SOPs)
- Data Validation (data entry
process)
Data Issue Log
Enables data profiling and analysis with the flexibility to filter and drill down on specific records for better detection of problems.
4/5.Define DQ metrics & Business rules
Enables architects and developers to discover and access all data sources, to improve the process of analyzing, profiling, validating, and cleansing data.
1. Promote DQ
Awareness 6. Test &
validate DQ Requirem.
7. Set & evaluate DQ service levels
10. Clean & correct DQ
defects
11. Design and implement DQM
procedures (SOPs)
Control
Activities
8. Continuously measure and monitor DQ
9. Manage DQ issues 12. Monitor operational DQM procedures and performance
Identify known data
issues
Extract & provide
data
Activities for DQ Pilot Activities for DQ methodology
Vision statement
Data files
DQM Framework
RACI
19
Next Steps
Data Management Overview