data systems integration & business value pt. 2: cloud
TRANSCRIPT
Copyright 2013 by Data Blueprint
Data Systems Integration & Business Value Part 2: Cloud-based IntegrationAll organizations are prepared to benefit from aspects of the cloud. These benefits accrue when cloud-hosted datasets share three attributes. They must be of: 1. Higher quality data than those data residing outside of the cloud;2. Lower volume (1/5 the size of data collections) than similar
collections residing outside of the cloud; and3. Increased share-ability than data residing outside the cloud.Increases in capacity utilization, improved IT flexibility and responsiveness, as well as the forecast decreases in cost accruing to cloud-based computing are all possible after these first three conditions have been met. Necessary investments in data engineering can help organizations to save even more money by reducing the amount of resources required to perform their duties and increasing the effectiveness of their duties & decision-making. This webinar will show you how to recognize the opportunities, ‘size up’ the required investment, and properly supervise your efforts to take advantage of the opportunities presented by the cloud.
Date: August 13, 2013Time: 2:00 PM ET/11:00 AM PTPresenter: Peter Aiken, Ph.D.
1
Copyright 2013 by Data Blueprint
Executive Editor at DATAVERSITY.net
2
Shannon Kempe
Copyright 2013 by Data Blueprint
Commonly Asked Questions
1) Will I get copies of the slides after the event?
2) Is this being recorded so I can view it afterwards?
3
Copyright 2013 by Data Blueprint
Get Social With Us!
Live Twitter FeedJoin the conversation!
Follow us:
@datablueprint @paiken
Ask questions and submit your comments: #dataed
Like Us on Facebookwww.facebook.com/datablueprint
Post questions and comments
Find industry news, insightful content
and event updates.
Join the GroupData Management & Business
IntelligenceAsk questions, gain insights and
collaborate with fellow data management professionals
4
Copyright 2013 by Data Blueprint 5
Peter Aiken, PhD• 25+ years of experience in data
management• Multiple international awards &
recognition• Founder, Data Blueprint (datablueprint.com)
• Associate Professor of IS, VCU (vcu.edu)
• President, DAMA International (dama.org)
• 8 books and dozens of articles• Experienced w/ 500+ data
management practices in 20 countries• Multi-year immersions with
organizations as diverse as the US DoD, Nokia, Deutsche Bank, Wells Fargo, and the Commonwealth of Virginia
2
The Case for theChief Data O!cerRecasting the C-Suite to LeverageYour Most Valuable Asset
Peter Aiken andMichael Gorman
Data Systems Integration & Business Value Part 2: Cloud-based Integration
Presented by Peter Aiken, Ph.D.10124 W. Broad Street, Suite C
Glen Allen, Virginia 23060804.521.4056
Copyright 2013 by Data Blueprint
Anticipated Business Value of Cloud-based Integration
7
• Increased Automation and Storage Capacity– Virtually unlimited capacity & flexible storage– Easy to upgrade & Up-to-date software – Automated file synching & backups
• Affordability– Pay as you go– Usage is scaled to fit needs
• Agility, Scalability and Flexibility– Access from anywhere & collaborate– Data is always current
• Free up IT Hours & Staff– Cloud provider takes care of maintenance
• Ease of Use– Easy to use & automated
Copyright 2013 by Data Blueprint
Prerequisites to Cloud-based Integration• Organizational investments in the cloud will be useless from
a data perspective unless:– Data governance, architecture, quality, development practices are
sufficiently mature– You must understand your data architecture and strategy in order to
evaluate various cloud options– Data must be reengineered to be
• Less• Better quality• More shareable
– for the cloud– Failure to do these will
result in more business value for the cloud vendors/service providers and less for your organization
8
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
9
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&A
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
10
Data Program Coordination
Feedback
DataDevelopment
Copyright 2013 by Data Blueprint
StandardData
Five Integrated DM Practice AreasOrganizational Strategies
Goals
BusinessData
Business Value
Application Models & Designs
Implementation
Direction
Guidance
11
OrganizationalData Integration
DataStewardship
Data SupportOperations
Data Asset Use
IntegratedModels
Leverage data in organizational activities
Data management processes andinfrastructure
Combining multipleassets to produceextra value
Organizational-entity subject area data
integration
Provide reliable data access
Achieve sharing of data within a business area
Copyright 2013 by Data Blueprint
Five Integrated DM Practice AreasManage data coherently.
Share data across boundaries.
Assign responsibilities for data.Engineer data delivery systems.
Maintain data availability.
Data Program Coordination
Organizational Data Integration
Data Stewardship Data Development
Data Support Operations
12
Copyright 2013 by Data Blueprint
Hierarchy of Data Management Practices (after Maslow)
• 5 Data management practices areas / data management basics ...
• ... are necessary but insufficient prerequisites to organizational data leveraging applications that is self actualizing data or advanced data practices Basic Data Management Practices
– Data Program Management– Organizational Data Integration– Data Stewardship– Data Development– Data Support Operations
http://3.bp.blogspot.com/-ptl-9mAieuQ/T-idBt1YFmI/AAAAAAAABgw/Ib-nVkMmMEQ/s1600/maslows_hierarchy_of_needs.png
Advanced Data Practices• MDM• Mining• Big Data• Analytics• Warehousing• SOAClo
ud
• Data Management Body of Knowledge (DMBOK)– Published by DAMA International, the
professional association for Data Managers (40 chapters worldwide)
– Organized around primary data management functions focused around data delivery to the organization and several environmental elements
• Certified Data Management Professional (CDMP)– Series of 3 exams by DAMA International and
ICCP– Membership in a distinct group of
fellow professionals– Recognition for specialized knowledge in a
choice of 17 specialty areas– For more information, please visit:
• www.dama.org, www.iccp.org
Copyright 2013 by Data Blueprint
DAMA DM BoK & CDMP
14
Copyright 2013 by Data Blueprint
Series Context• Certain systems are more data
focused than others. Usually their primary focus is on accomplishing integration of disparate data. In these cases, failure is most often attributable to the adoption of a single technological pillar (silver bullet). The three webinars in the Data Systems Integration and Business Value series are designed to illustrate that good systems development more often depends on at least three DM disciplines (pie wedges) in order to provide a solid foundation.
• Data Systems Integration & Business Value – Pt. 1: Metadata Practices– Pt. 2: Cloud-based Integration– Pt. 3: Warehousing, et al.
15
Uses
Copyright 2013 by Data Blueprint
Part 1: Metadata Take Aways• Metadata unlocks the value of data, and therefore requires
management attention [Gartner 2011]
• Metadata is the language of data governance• Metadata defines the essence of integration challenges
SourcesMetadata Governance
Metadata Engineering
Metadata Delivery
Metadata Practices
MetadataStorage
16
Specialized Team Skills
Data Management functions necessary but insufficient for metadata-basedintegration
Copyright 2013 by Data Blueprint
Data Management Body of
Knowledge
17
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
18
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
19
Copyright 2013 by Data Blueprint
Data Management Body of
Knowledge
20
Data Management functionsnecessary but insufficientfor cloud-basedintegration
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
21
Data Governance
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Data Governance
22
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Organizational Data Governance Purpose Statement
• What does data governance mean to my organization?
– Getting some individuals (whose opinions matter)
– To form a body (needs a formal purpose/authority)
– Who will advocate/evangelize for (not dictate, enforce, rule)
– Increasing scope and rigor of
– Data-centric development practices
23
Top Operations
Job
Copyright 2013 by Data Blueprint
Data Governance is a Gateway for IT Projects
24
Top Job
TopFinance
Job
Top Information Technology
Job
Top Marketing
Job
• Data assets are better foundational building block for IT projects• CDO coordinates IT investment priorities with Top IT Job• CDO determines when proposed IT projects are "ready"
Data Governance Organization
ChiefData
Officer
Copyright 2013 by Data Blueprint
25
Data Architecture Management
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Data Architecture Management
26
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Architectural Answers
(Adapted from [Allen & Boynton 1991])
Computers
Human resources
Communication facilities
Software
Managementresponsibilities
Policies,directives,and rules
Data
27
• Where do they go?• When are they needed?• What standards
should be adopted?• What vendors
should be chosen? • What rules should govern
the decisions? • What policies should guide
the process? • How and why do the components interact?• Why and how will the changes be implemented?• What should be managed organization-wide and what should
be managed locally?
Zachman Framework 3.0 - the Enterprise OntologyClassificationNames
ModelNames
*Horizontal integration lines are shown for example purposes only and are not a complete set. Composite, integrative rela-tionships connecting every cell horizontally potentially exist.
AudiencePerspectives
EnterpriseNames
ClassificationNames
AudiencePerspectives
C o m p o s i t e I n t e g r a t i o n s
Alignment
Transformations
C o m p o s i t e I n t e g r a t i o n s
Alignment
Transformations
C o m p o s i t e I n t e g r a t i o n s C o m p o s i t e I n t e g r a t i o n s
Alignment
Transformations
Alignment
Transformations
A l i g n m e n t
A l i g n m e n t
How Where Who WhenWhat Why
ProcessFlows
DistributionNetworks
ResponsibilityAssignments
TimingCycles
InventorySets
MotivationIntentions
OperationsInstances
(Implementations)
TheEnterprise
TheEnterprise
EnterprisePerspective
(Users)
ExecutivePerspective(Business Context
Planners)
Business MgmtPerspective(Business Concept
Owners)
ArchitectPerspective(Business LogicDesigners)
EngineerPerspective(Business Physics
Builders)
TechnicianPerspective
(Business ComponentImplementers)
ScopeContexts
(Scope Identification Lists)
BusinessConcepts
(Business Definition Models)
SystemLogic(System
Representation Models)
TechnologyPhysics(Technology
Specification Models)
ToolComponents(Tool Configuration
Models)
e.g. e.g. e.g. e.g. e.g. e.g.
e.g. e.g. e.g. e.g. e.g. e.g.
e.g. e.g. e.g. e.g. e.g. e.g.
e.g. e.g. e.g. e.g. e.g. e.g.e.g.: primitive e.g.: composite model:
model:
Forecast SalesPlan ProductionSell ProductsTake OrdersTrain EmployeesAssign TerritoriesDevelop MarketsMaintain FacilitiesRepair ProductsRecord Transctns
Material Supply NtwkProduct Dist. NtwkVoice Comm. NtwkData Comm. Ntwk Manu. Process NtwkOffice� Wrk� Flow� Ntwk
Parts Dist. NtwkPersonnel Dist. Ntwketc., etc.
General MgmtProduct MgmtEngineering DesignManu. EngineeringAccountingFinanceTransportationDistributionMarketingSales
Product CycleMarket CyclePlanning CycleOrder CycleEmployee CycleMaint. CycleProduction CycleSales CycleEconomic CycleAccounting Cycle
ProductsProduct TypesWarehouses
Parts BinsCustomersTerritoriesOrdersEmployeesVehiclesAccounts
New MarketsRevenue GrowthExpns ReductionCust ConvenienceCustomer Satis.Regulatory Comp.New CapitalSocial ContributionIncreased YieldIncreased Qualitye.g. e.g. e.g. e.g. e.g. e.g.
Operations TransformsOperations In/Outputs
Operations LocationsOperations Connections
Operations RolesOperations Work Products
Operations IntervalsOperations Moments
Operations EntitiesOperations Relationships
Operations EndsOperations Means
ProcessInstantiations
DistributionInstantiations
ResponsibilityInstantiations
TimingInstantiations
Inventory Instantiations
MotivationInstantiations
List: Timing Types
Business IntervalBusiness Moment
List: Responsibility Types
Business RoleBusiness Work Product
List: Distribution Types
Business LocationBusiness Connection
List: Process Types
Business TransformBusiness Input/Output
System TransformSystem Input /Output
System LocationSystem Connection
System RoleSystem Work Product
System IntervalSystem Moment
Technology TransformTechnology Input /Output
Technology LocationTechnology Connection
Technology RoleTechnology Work Product
Technology IntervalTechnology Moment
Tool TransformTool Input /Output
Tool LocationTool Connection
Tool RoleTool Work Product
Tool IntervalTool Moment
List: Inventory Types
Business EntityBusiness Relationship
System EntitySystem Relationship
Technology EntityTechnology Relationship
Tool EntityTool Relationship
List: Motivation Types
Business EndBusiness Means
System EndSystem Means
Technology EndTechnology Means
Tool EndTool Means
Timing IdentificationResponsibility IdentificationDistribution IdentificationProcess Identification
Timing DefinitionResponsibility DefinitionDistribution DefinitionProcess Definition
Process Representation Distribution Representation Responsibility Representation Timing Representation
Process Specification Distribution Specification Responsibility Specification Timing Specification
Inventory Identification
Inventory Definition
Inventory Representation
Inventory Specification
Inventory Configuration Process Configuration Distribution Configuration Responsibility Configuration Timing Configuration
Motivation Identification
Motivation Definition
Motivation Representation
Motivation Specification
Motivation Configuration
Copyright 2013 by Data Blueprint
28
Copyright 2008-2011 John A. Zachman
Copyright 2013 by Data Blueprint
What is an information architecture?• A structure of data-based information
assets supporting implementation of organizational strategy (or strategies)
• Most organizations have data assets that are not supportive of strategies - i.e., information architectures that are not helpful
• The really important question is: how can organizations more effectively use their information architectures to support strategy implementation?
29
ClassificationNames
ModelNames
*Horizontal integration lines are shown for example purposes only and are not a complete set. Composite, integrative rela-tionships connecting every cell horizontally potentially exist.
AudiencePerspectives
EnterpriseNames
ClassificationNames
AudiencePerspectives
C o m p o s i t e I n t e g r a t i o n s
Alignment
Transformations
C o m p o s i t e I n t e g r a t i o n s
Alignment
Transformations
C o m p o s i t e I n t e g r a t i o n s C o m p o s i t e I n t e g r a t i o n s
Alignment
Transformations
Alignment
Transformations
A l i g n m e n t
A l i g n m e n t
How Where Who WhenWhat Why
ProcessFlows
DistributionNetworks
ResponsibilityAssignments
TimingCycles
InventorySets
MotivationIntentions
OperationsInstances
(Implementations)
TheEnterprise
TheEnterprise
EnterprisePerspective
(Users)
ExecutivePerspective(Business Context
Planners)
Business MgmtPerspective(Business Concept
Owners)
ArchitectPerspective(Business LogicDesigners)
EngineerPerspective(Business Physics
Builders)
TechnicianPerspective
(Business ComponentImplementers)
ScopeContexts
(Scope Identification Lists)
BusinessConcepts
(Business Definition Models)
SystemLogic(System
Representation Models)
TechnologyPhysics(Technology
Specification Models)
ToolComponents(Tool Configuration
Models)
e.g. e.g. e.g. e.g. e.g. e.g.
e.g. e.g. e.g. e.g. e.g. e.g.
e.g. e.g. e.g. e.g. e.g. e.g.
e.g. e.g. e.g. e.g. e.g. e.g.e.g.: primitive e.g.: composite model:
model:
Forecast SalesPlan ProductionSell ProductsTake OrdersTrain EmployeesAssign TerritoriesDevelop MarketsMaintain FacilitiesRepair ProductsRecord Transctns
Material Supply NtwkProduct Dist. NtwkVoice Comm. NtwkData Comm. Ntwk Manu. Process NtwkOffice� Wrk� Flow� Ntwk
Parts Dist. NtwkPersonnel Dist. Ntwketc., etc.
General MgmtProduct MgmtEngineering DesignManu. EngineeringAccountingFinanceTransportationDistributionMarketingSales
Product CycleMarket CyclePlanning CycleOrder CycleEmployee CycleMaint. CycleProduction CycleSales CycleEconomic CycleAccounting Cycle
ProductsProduct TypesWarehouses
Parts BinsCustomersTerritoriesOrdersEmployeesVehiclesAccounts
New MarketsRevenue GrowthExpns ReductionCust ConvenienceCustomer Satis.Regulatory Comp.New CapitalSocial ContributionIncreased YieldIncreased Qualitye.g. e.g. e.g. e.g. e.g. e.g.
Operations TransformsOperations In/Outputs
Operations LocationsOperations Connections
Operations RolesOperations Work Products
Operations IntervalsOperations Moments
Operations EntitiesOperations Relationships
Operations EndsOperations Means
ProcessInstantiations
DistributionInstantiations
ResponsibilityInstantiations
TimingInstantiations
Inventory Instantiations
MotivationInstantiations
List: Timing Types
Business IntervalBusiness Moment
List: Responsibility Types
Business RoleBusiness Work Product
List: Distribution Types
Business LocationBusiness Connection
List: Process Types
Business TransformBusiness Input/Output
System TransformSystem Input /Output
System LocationSystem Connection
System RoleSystem Work Product
System IntervalSystem Moment
Technology TransformTechnology Input /Output
Technology LocationTechnology Connection
Technology RoleTechnology Work Product
Technology IntervalTechnology Moment
Tool TransformTool Input /Output
Tool LocationTool Connection
Tool RoleTool Work Product
Tool IntervalTool Moment
List: Inventory Types
Business EntityBusiness Relationship
System EntitySystem Relationship
Technology EntityTechnology Relationship
Tool EntityTool Relationship
List: Motivation Types
Business EndBusiness Means
System EndSystem Means
Technology EndTechnology Means
Tool EndTool Means
Timing IdentificationResponsibility IdentificationDistribution IdentificationProcess Identification
Timing DefinitionResponsibility DefinitionDistribution DefinitionProcess Definition
Process Representation Distribution Representation Responsibility Representation Timing Representation
Process Specification Distribution Specification Responsibility Specification Timing Specification
Inventory Identification
Inventory Definition
Inventory Representation
Inventory Specification
Inventory Configuration Process Configuration Distribution Configuration Responsibility Configuration Timing Configuration
Motivation Identification
Motivation Definition
Motivation Representation
Motivation Specification
Motivation Configuration
! ! ! !
Copyright 2013 by Data Blueprint 30
Organizational Needs
become instantiated and integrated into an Data/Information
Architecture
Informa(on)System)Requirements
authorizes and articulates sa
tisfy
spe
cific
org
aniz
atio
nal n
eeds
Data Architectures produce and are made up of information models that are developed in response to organizational needs
Copyright 2013 by Data Blueprint
Data Architecture – Better Definition
31
• All organizations have information architectures– Some are better understood and
documented (and therefore more useful to the organization) than others.
• Common vocabulary expressing integrated requirements ensuring that data assets are stored, arranged, managed, and used in systems in support of organizational strategy [Aiken 2010]
Copyright 2013 by Data Blueprint
32
Data Development
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Data Modeling/Data Development
33
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
34
#dataed
Program F
Program E
Program DProgram G
Program H
Program I
Applicationdomain 2Application
domain 3
Data Development Focus
Copyright 2013 by Data Blueprint
35
#dataed
Data Development has greater Business Value
Copyright 2013 by Data Blueprint
36
Conceptual Logical Physical
Validated
Not Validated
Every change can be mapped to a transformation in this framework!
Data Development Evolution Framework
Copyright 2013 by Data Blueprint
37
DataQualityManagement
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Data Quality Engineering
38
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Definitions• Quality Data
– Fit for use meets the requirements of its authors, users, and administrators (adapted from Martin Eppler)
– Synonymous with information quality, since poor data quality results in inaccurate information and poor business performance
• Data Quality Management– Planning, implementation and control activities that apply quality
management techniques to measure, assess, improve, and ensure data quality
– Entails the "establishment and deployment of roles, responsibilities concerning the acquisition, maintenance, dissemination, and disposition of data" http://www2.sas.com/proceedings/sugi29/098-29.pdf
✓ Critical supporting process from change management✓ Continuous process for defining acceptable levels of data quality to meet business
needs and for ensuring that data quality meets these levels• Data Quality Engineering
– Recognition that data quality solutions cannot not managed but must be engineered– Engineering is the application of scientific, economic, social, and practical knowledge in
order to design, build, and maintain solutions to data quality challenges– Engineering concepts are generally not known and understood within IT or business!
39
Spinach/Popeye story from http://it.toolbox.com/blogs/infosphere/spinach-how-a-data-quality-mistake-created-a-myth-and-a-cartoon-character-10166
Copyright 2013 by Data Blueprint
Quality Dimensions
40
Copyright 2013 by Data Blueprint
Startingpointfor newsystemdevelopment
data performance metadata
data architecture
dataarchitecture and
data models
shared data updated data
correcteddata
architecturerefinements
facts &meanings
Metadata &Data Storage
Starting pointfor existingsystems
Metadata Refinement• Correct Structural Defects• Update Implementation
Metadata Creation• Define Data Architecture• Define Data Model Structures
Metadata Structuring• Implement Data Model Views• Populate Data Model Views
Data Refinement• Correct Data Value Defects• Re-store Data Values
Data Manipulation• Manipulate Data• Updata Data
Data Utilization• Inspect Data• Present Data
Data Creation• Create Data• Verify Data Values
Data Assessment• Assess Data Values• Assess Metadata
Extended data life cycle model with metadata sources and uses
41
Copyright 2013 by Data Blueprint
DQE Context & Engineering Concepts • Can rules be implemented stating that no data can be
corrected unless the source of the error has been discovered and addressed?
• All data must be 100% perfect?
• Pareto – 80/20 rule– Not all data
is of equal Importance
• Scientific, economic, social, and practical knowledge
42
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
43
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
44
Copyright 2013 by Data Blueprint
http://visual.ly/amazing-journey-data-cloud
45
Copyright 2013 by Data Blueprint
http://visual.ly/amazing-journey-data-cloud
46
Copyright 2013 by Data Blueprint
http://visual.ly/amazing-journey-data-cloud
47
Copyright 2013 by Data Blueprint
Gartner Five-phase Hype Cycle
http://www.gartner.com/technology/research/methodologies/hype-cycle.jsp48
Technology Trigger: A potential technology breakthrough kicks things off. Early proof-of-concept stories and media interest trigger significant publicity. Often no usable products exist and commercial viability is unproven.
Trough of Disillusionment: Interest wanes as experiments and implementations fail to deliver. Producers of the technology shake out or fail. Investments continue only if the surviving providers improve their products to the satisfaction of early adopters.
Peak of Inflated Expectations: Early publicity produces a number of success stories—often accompanied by scores of failures. Some companies take action; many do not.
Slope of Enlightenment: More instances of how the technology can benefit the enterprise start to crystallize and become more widely understood. Second- and third-generation products appear from technology providers. More enterprises fund pilots; conservative companies remain cautious.
Plateau of Productivity: Mainstream adoption starts to take off. Criteria for assessing provider viability are more clearly defined. The technology’s broad market applicability and relevance are clearly paying off.
Copyright 2013 by Data Blueprint
Gartner Cloud Hype Cycle “While clearly maturing, cloud
computing continues to be the most hyped subject
in IT today.”
49
Copyright 2013 by Data Blueprint
50
• Cloud computing is location-independent computing, whereby shared servers provide resources, software, and data to computers and other devices on demand, as with the electricity grid. • Cloud computing is a natural evolution of the
widespread adoption of virtualization, service-oriented architecture and utility computing. • Details are abstracted from consumers, who no
longer have need for expertise in, or control over, the technology infrastructure "in the cloud" that supports them.
Cloud Computing
Copyright 2013 by Data Blueprint
Five Essential Characteristics of Data Cloud Infrastructure
• Gartner defines "cloud computing" as the set of disciplines, technologies, and business models used to deliver IT capabilities (software, platforms, hardware) as an on-demand, scalable, elastic service.
• Five essential characteristics of cloud computing:
– It uses shared infrastructure
– It provides on-demand self-service
– It is elastic and scalable
– It is priced by consumption
– It is dynamic and virtualized
51
Copyright 2013 by Data Blueprint
52
Cloud Scalability
Copyright 2013 by Data Blueprint
Cloud Rendering
53
Copyright 2013 by Data Blueprint
Cisco's Ladder to the Cloud
54
Copyright 2013 by Data Blueprint
Cloud Options
55
Copyright 2013 by Data Blueprint
Solving the Big Data Puzzle
56
http://damfoundation.org/2012/06/whats-the-big-deal-about-big-data/
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline
57
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline
58
Copyright 2013 by Data Blueprint
Benefits
59
Copyright 2013 by Data Blueprint
Benefits
60
Copyright 2013 by Data Blueprint
Anticipated Benefits
61
0% 13% 25% 38% 50%
Improve data quality
Reduce installation and maintenance efforts
Reduce implementation efforts
Eliminate manual processes
Reduce time require to collect and prepare data
Apply data governance policies
Copyright 2013 by Data Blueprint
Similar Opportunity
• IT Infrastructure. Your submission should include funding for the timely execution of agency plans to consolidate data centers developed in FY 2010 (reference FY 2011 passback guidance). In coordination with the data center consolidations, agencies should evaluate the potential to adopt cloud computing solutions by analyzing computing alternatives for IT investments in FY 2012. Agencies will be expected to adopt cloud computing solutions where they represent the best value at an acceptable level of risk.
• Adopt Light Technologies and Shared Solutions. We are reducing our data center footprint by 40 percent by 2015 and shifting the agency default approach to IT to a cloud-first policy as part of the 2012 budget process. Consolidating more than 2,000 government data centers will save money, increase security and improve performance.
62
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
63
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
64
Copyright 2013 by Data Blueprint
Data in the cloud should have three attributes that data outside the cloud should not have. It should be:
65
Sharable-er
Cleaner
Smaller
Copyright 2013 by Data Blueprint
Aspirational Data in the Cloud
66
Copyright 2013 by Data Blueprint
Effective Cloud Transformation
• Transformation into cloud computing cannot be done in a manner that benefits organizations unless data is re-architected – formally with two goals: – Maximizing effective, organization-wide data sharing; and – Minimizing organizational data ROT.
• Resulting data volume reduction should be 1/5 what is currently is – A significant economic motivator.
• All existing organizations have data collections that possess unique strengths and weaknesses– Strengths that should be leveraged– Weaknesses must be addressed
• Neither of these can be accomplished without formal data rearchitecting prior to cloud loading.
• There are very few who work in the area for a living but my team has achieved some remarkable successes.
67
Copyright 2013 by Data Blueprint
Transform
68
Problems with forklifting 1. no basis for
decisions made2. no inclusion of
architecture/engineering concepts
3. no idea that these concepts are missing from the process
LessCleanerMore shareable ... data
Getting into the Cloud
Copyright 2013 by Data Blueprint
Data Leverage
• Permits organizations to better manage their sole non-depleteable, non-degrading, durable, strategic asset - data– within the organization, and – with organizational data exchange partners
• Leverage – Obtained by implementation of data-centric technologies, processes, and human skill
sets– Increased by elimination of data ROT (redundant, obsolete, or trivial)
• The bigger the organization, the greater potential leverage exists
• Treating data more asset-like simultaneously 1. lowers organizational IT costs and 2. increases organizational knowledge worker productivity
69
Less ROT
Technologies
Process
People
Copyright 2013 by Data Blueprint
The Cloud as a Data Quality Tool
Enterprise PortalData DeliveryData Analysis
Quality
Technology
Continuous ImprovementData BaseliningStatistical Data ControlCost of Quality ModelEmpowerment
Data ReductionPattern AnalysisMathematical AnalysisSchema Validation
ReusabilityLogic & Logic ProgrammingRelational DB TechnologyData Migration TechnologiesStatistical Programming Languages
70
Copyright 2013 by Data Blueprint
Fixing Data in the Cloud Using A Glovebox
71
Copyright 2013 by Data Blueprint
72
Conceptual Logical Physical
Validated
Not Validated
Every change can be mapped to a transformation in this framework!
Data Development Evolution Framework
Copyright 2013 by Data Blueprint
73
Data Reengineering for More Shareable Data
As-is To-be
TechnologyIndependent/Logical
TechnologyDependent/Physical
abstraction
Other logical as-is data architecture components
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
74
Copyright 2013 by Data Blueprint
1. Data Management: Contextual Overview2. Necessary Data Management Functions
(Prerequisites)- Data Governance- Data Architecture- Data Development- Data Quality
3. Understanding Cloud-based Technologies
4. Cloud-based Benefits5. Cloud-based Integration
- Cleaner- Smaller- Shareable
6. Take Aways, References and Q&ATweeting now:
#dataed
Outline: Cloud-based Integration
75
Copyright 2013 by Data Blueprint
Part 2: Take Aways• Data governance, architecture,
quality, development maturity are necessary but insufficient prerequisites to successful data cloud implementation
• A variety of cloud options will influence cloud and data architectures in general– You must understand your architecture
and strategy in order to evaluate the options
• Data must be reengineered to be – Less– Better quality– More shareable – for the cloud
• Failure to do these will result in more business value for the cloud vendors/service providers and less for your organization
Copyright 2013 by Data Blueprint
Questions?
It’s your turn! Use the chat feature or Twitter (#dataed) to submit
your questions to Peter now.
77
+ =
Data Systems Integration & Business Value Pt. 3: WarehousingSeptember 10, 2013 @ 2:00 PM ET/11:00 AM PT
Show me the Money: Monetizing Data ManagementOctober 8, 2013 @ 2:00 PM ET/11:00 AM PT
Sign up here: www.datablueprint.com/webinar-schedule or www.dataversity.net
Copyright 2013 by Data Blueprint
Upcoming Events
78