mastering data management - dama indiana 22o… · mastering data management mark cheaney regional...
TRANSCRIPT
Mastering Data Management
Mark Cheaney
Regional Sales Manager, DataFlux
Today, the amount of technical information doubles every two yearsevery two years
It is forecast to double every three daysevery three days
There are over 31 Billion searches on Google every month
LOADING
Source: Did You Know 3.0 (Fisch, McLeod, Brenman)
In 2006, this number was 2.7 Billion
Source: Did You Know 3.0 (Fisch, McLeod, Brenman)
Source: Did You Know 3.0 (Fisch, McLeod, Brenman)
Source: Did You Know 3.0 (Fisch, McLeod, Brenman)
Times Are Changing…
1 out of 4 workers have been in their job less than one year.
1 out of 2 less than five years
Top 10 in-demand jobs today didn’t exist in 2004
Are We Keeping Up?
Over 80 US banks failed in 2009
The US government has taken majority ownership of General Motors,
Freddie Mac, Fannie Mae, AIG…
Société Générale lost $7.5B in a 2008 derivatives trading fiasco
Valparaiso, Indiana had an $8M budget shortfall in 2007
US health care now 17% of personal income
What Does This Have to Do with Data?
Mastering Data
Management
Is Data Managed Across Your Enterprise?
Mastering Data Management
Is Data a Trusted Business Asset?
Sales Force
Automation
Database Marketing
IT-driven
projects
Duplicate,
inconsistent data
Inability to adapt to
business changes
Data Warehouse
ERP
CRM
Line of business
influences IT projects
Little cross-functional
collaboration
High cost to maintain
multiple applications
IT and business
groups collaborate
Enterprise view of
certain domains
Data is a
corporate asset
Customer MDM
Product MDM
Business
requirements drive
IT projects
Repeatable, automated
business processes
Personalized customer
relationships and
optimized operations
MDM
Business Process
Automation
Data Governance Maturity Model
How Do We Master Data?
Establish the people and policies for
data governance
Focus data management on business process
improvement
Standardize on a data management
technology platform
Data Governance –
People and Policies
ITBusiness
Data Governance: IT and Business Collaboration
Executive Sponsorship
Data Governance
Council
Data Steering
(business experts)
Data Management
Data Administration
Data Architecture
Security and Privacy
LOB Data
Governance
Data Stewards
58%
No
83
No
Management Support Collaboration
Data Governance – Executive Support
Little to No Support
Noticeable or Better Support
Little Collaboration
Collaboration
Originally published in “A Data Governance Manifesto” by Jill Dyché. Used with permission from Baseline Consulting.
Accountable Consulted Informed
Data Governance – Regimes
SalesCustomer
ServiceFinance Marketing
HumanResources
Data GovernanceCouncil
Procurement
Campaign Management
Hiring
Order Management
Billing
Trouble Ticket Tracking
Core BusinessProcesses
Data Governance – Policy
Creation, documentation (including business vocabulary), approval
process and maintenance of data standards for form, function, meaning
and versioning
Quality and stewardship for data elements, business rules, hierarchies,
taxonomies and content tagging
Creation and maintenance of enterprise data model and enterprise data
services
Metrics, monitoring and evaluation of standards
Business Process
Improvement
Manage Data for Business
Traditional Data Management Approach
Data Source Data Source Data Source
Data Rule Data Rule Data Rule Data Rule Data Rule Data Rule
Data Rule Data Rule Data Rule
Data Domain
Trusted, Integrated Data
Emerging Data Management Approach –Mastering Data for Business
Business Domain
Data Source
Data Rule Data Rule
Data Rule
Data Source
Data Rule Data Rule
Data Rule
Data Source
Data Rule
Data Rule Data Rule
Trusted, Integrated Data
Business Policy
Business Info Business Info
Business Info
Business Policy
Business Info Business Info
Business Info
Business Policy
Business Info Business Info
Business Info
Data Management
Platform
DataFlux UnityPlatform
Business Process Automation
Reporting and Dashboards
Design and Development Environment
Business Vocabulary/Data Definitions
Data Access
Business Rule and Event Processing and Monitoring
Data Archiving
Data Privacy and Security
Metadata Management
Search and Navigation
Identity Resolution
Business Rule Creation and Management
Data Enrichment
Metadata Discovery
Hierarchy and Reference Data Definition
Unstructured Data Discovery
Verification, Normalization, Standardization, Transformation
Data Exploration and Profiling
Data Services and SOA
Data Synchronization
ETL/ELT
Business Process Integration
Merging and Clustering
Business Rules Execution
Grid Computing
Data Federation
Business Data Services
Domain Data Models
Master Data History/Auditing and Exception Reporting
Entity Definition/Management and Search
Best Record Selection
How Do We Master Data?
Establish the people and policies
for data governance
Focus data management on business
process improvement
Standardize on a data management
technology platform
5-steps to Improving Data Management
DataFlux Approach
Data Profiling
Identify data quality issues
Determine if data fits requirements
Identify business process issues
Real-Life Profiling Exercises
A financial services company knew of 3 genders: M, F, and blank. They did not know about X and C.
A home care products company discovered shipments slated for 16’x16’ pallets. The IS manager wondered what kind of truck they would go on.
Prior to a VA audit, a cross-check of medical billings by a healthcare provider showed it was performing open heart surgeries in ambulances.
Consumer products mfr. learned a product of theirs was railroad boxcars.
Analyze - ProfilingTable, Column, & Relationship Metrics
Pattern RecognitionVisualization
Metadata Analysis
Data ProfilingUncover Problematic or Inconsistent Data
View detailed information on the accuracy, completeness, consistency, structure, uniqueness and validity of data
Create and share reports to build consensus on data quality and data governance efforts
Data Quality
Correct identified data quality issues
Normalize inconsistent data
Correct address information
Data contentMissing & Invalid data.
Data domain outliers.
Illogical combinations of data
Data structure and storage
Uniqueness
Referential integrity
Migration/integration
Normalization inconsistencies.
Duplicate or lost data
StandardsAmbiguous Business Rules
Multiple Formats for Same Data
Elements
Different Meanings for the Same
Code Value.
Multiple Codes Values with the
Same Meaning
Field Overuse: used for unintended
purpose.
Data in Filler
Types of Data Quality Problems
Data Quality & Deployment Styles
Data Integration
Identify and eliminate duplicates
Identify and link households
Move data from source to target
Data Integration
SFA ERP
DataWarehouseCall Center
Apex Equipment | Pittsburgh PA
Apex LLC | Pittsburgh, Penn
Apex Construction | Pittsburgh PA
Apex Equip & Const | Pitt PA
Apex Equipment & Construction, LLC | Pittsburgh PA 15233
Data Integration
Data Quality
Data Model
Business Services
Stewardship Console
Business User Interface
Data Governance
Identity Management
Reporting
Data Profiling
Metadata Discovery
Business Rule Definition
Entity Definition
Data Enrichment
Make data more useful
Add postal information to improve customer outreach
Append product codes to speed procurement and materials management efforts
Data EnrichmentValidate and verify
Data validation and verification ensures data accuracy
Test data against other data sources (internal or external) known to be correct or current
Product code verification (industry-standard codes, UPC, ISDN)
Address verification (ZIP codes, geocoding)
Validated dataInput
940 Cary Pkw
Cary NC
27503
940 NW Cary Pkwy Ste 201
Cary
NC
27513-4355
County: Wake
Census Tract: 452.2
Data EnrichmentValidate and verify
Data EnrichmentValidate and verify
Data Monitoring
Data integrity checks & balances.
Business rule development by business analysts.
Data Stewards empowered through dashboard monitoring.
Data MonitoringMaintain High-Quality Data Over Time
Ensure clean data stays clean
Validate data against your business rules
Automatically identify invalid data
About DataFlux
Recognized as a leading provider of data quality, data integration, and MDM solutions
Provides a unique single platform to analyze, improve and control enterprise data
Over 1,200 customers worldwide
Offices in the US, the UK, France and Germany
Founded in 1997
Acquired by SAS, the world’s largest privately owned software company, in 2000
Operates as a wholly-owned subsidiary
Questions
DataFlux Midwest ManagerMark [email protected]
For more information, visit:www.dataflux.com