methodology for sustainable mdm and cdi success · api, from applications other das and business...

Methodology for sustainable MDM and CDI success

Kalyan ViswanathanPractice Director, MDM Practice - Tata Consultancy Services

Global Consulting Practice (GCP)Master Data Management

Agenda

• Some Definitions - SOA and MDM

• Transitioning from Legacy to SOA

• Some Definitions - Data Governance

• Methodology for sustainable success with MDM and CDI

• The TCS MDM / CDI Framework


Some Definitions

• What is a Services oriented Architecture ?

• What is Master Data ?

• What is Master Data Management ?

• What is Data Governance ?

• What is Data Stewardship ?

• What is the relationship between Master Data Management and Data Governance ?


What is a Service Oriented Architecture (SOA) ?

A Service Oriented Architecture involves :

1. The Building blocks of - Services and Business Processes• Monolithic Applications decompose into Services and Business

Processes

2. A Robust Interoperability Architecture

3. Maximization of Re-use of services that are loosely coupled

4. Elegant Enablement of Composite Business Process across heterogeneous application and Service domains

5. Standard agreements and protocols

6. The exploitation of the Convergence of XML, Web Services, and EAI – resulting in the potential for unprecedented levels of integration (Meta Group)


What are the “promised” benefits of SOA ?

1. Agility of IT to respond to Business change

2. Re-use of legacy code without a total rip and replace

3. Flexibility of IT systems – easy adoption of new sales channels and B2B interfaces

4. Enhanced Productivity of the IT organization

5. Information integration and standardization (eg. Single customer view)

6. Potential to build new kinds of applications and business processes – faster and cheaper


What are the “Challenges” of implementing SOA ?

1. Introduces new ways of thinking about IT systems – almost a cultural paradigm shift

2. Unclear as to what the first steps and the sequence of activities towards an SOA architecture are

3. Re-use has to be engineered carefully – It does not occur by accident

4. Difficult to create business cases for stand alone SOA projects –Ownership and Funding

5. SOA standards still developing

6. Service Levels, Performance and Scalability


What is Master Data ?

Master Data – includes the core business entities which constitutes the enterprise’s data assets such as Customers, Products, Accounts, Suppliers, Parts, Employees, Locations, Factories, Reference Data, Hierarchies, and so on that meet the following three criteria:

Definitions – Criteria for Master Data

1. Master Data is relatively “slowly changing” data – that is updated infrequently (each subject area may differ in frequency)

2. Master Data is “referenced” by many different transactional scenarios (reference data is a subset / category of master data)

3. Master Data generally tends to be widely “replicated” in many applications and used by many business processes (some master data may be totally private, and is still master data)


Data

Master Data

Transactional Data

Analytical Data

Describes relatively static dimensions of a CustomerIncludes Customer Identity, Profile, Demographics, Relationships with other customers (both individual and corporate), Credit History, Account Relationships, Privacy, Channel Preferences etc.

Describes a Customer’s Operational State with respect to his/her relationships Eg. Balances, Principal, Premiums, Interest accrued, Claims, Invoices, withdrawal, payment, Deposit, Transfers, Sales status, Service tickets Represents business activity at a point in time

Describes a Customer’s performance with a historical or futuristic viewEg. Trends, Forecasts, Sales history, Buying patternsAlso includes derived information such as profitability, segmentation, propensity to buy, Lifetime Value, Risk exposure

What is Master Data as distinct from other data ?Eg. Customer Master Data


What is Master Data – Contd. ?

Characteristics –

• Master Data Semantics i.e. Entity and Attribute definitions have to be reconciled, between different departments and business functions

• Master Data ownership is often distributed and sometimes the same attribute has multiple owners, while others have no owners

• Master Data has a definitive scope / list for any organization

• Master Data occurs as dimensions in analytical environments

• From 15% to 40% of any Legacy Application deals with Master Data


What is Master Data Management (MDM) ?

Master Data Management is the combination of

1. Governance2. Processes and 3. Technologies

needed to

define, create, store, maintain, secure, and make available

a Consolidated, Consistent, Contextual, Complete and Accurate view of master data

across multiple Business Processes, external business partners, and application systems.


Data Access Services provide : • Virtualization of Data - Enterprise-wide data abstraction layer• Integrated views of data from multiple sources

– Relational databases, Applications, Files, Data Warehouses, Data Marts, ODS & MDM environments

• Ubiquitous Access - Re-useable Data Services for data consistency• Data Access Services are consumed by other Services and Applications• Data Access Services are closely related to Enterprise Data Management, which organizes and

manages data over its life cycle

Data Access Services Layer

Operational and Analytical Applications : Transactional Systems Portals, Reporting Apps, Workflow, Imaging et al.

Legacy Data Sources : DB & file Data Marts & ADSMaster Data Hubs ODS & Warehouses

Business Services Layer

DATAFARM

Met

aD

ata

Data Access Services and Business Services


Products Customers Dealers Contracts Channels Transactions

Master DataSystem

TransactionalSystems

Operational Data Stores

(ODS)

Data Warehouse

(DWH)

Data Marts (DM)

Data Class Master Master Transaction

Master Transaction

Master Transaction

Analytic

Master Transaction

Analytic

History Yes NoNone

Limited Yes Yes

Integration Yes No YesLimited

Yes Yes

Data Currency Real Time Real Time

Close to Real time (+1

Day)

Sometimes Daily

Weekly Monthly

Sometimes Daily Weekly Monthly

Data Scope Fully Integrated Application

Neutral

Local Application

Specific

Integrated (Limited to a few

applications)

Fully Integrated Application

Neutral

Analytical, Derived and Summarized, Application specific

Data Creation

Yes Yes No Limited to Derived Data

Limited to Derived Data

The Information Value Chain – Looking at the Data Farm

Master Data moves left to right through the Information Value chain over its life cycle


Attributes Business Services Data Access Services

Core concern Business Logic composition Data Access virtualization

Encapsulates Re-usable Business Logic, Business Rules Data Definitions & Sources, Data Integrity constraints

Characteristics Large Grained, Functionally oriented Fine Grained, Data Oriented

Invocation Patterns

From the Web, through EAI layer, direct API, from Applications

Other DAS and Business Services, from Applications, from the Web, through EAI Layer

Design Method Top down, Process Driven Bottom up, Data Driven

Modeling Method Business Process Models Canonical Data Models

User Interfaces Will frequently invoke Business Services for executing Business Logic

May occasionally directly invoke a Data Access Service for direct data access

Relationship Business Services will invoke Data Access Services

Data Access Services will rarely invoke business services – but may invoke other Data Access Services

Degree of Re-use Re-used within similar Business Processes eg. Open Account

Re-used across widely varying Business Processes – Maximal Re-use

Data Access Services versus Business Services


Architecting the Service Mosaic

Large grained Business Service

Fine grained Data Service

Increasing Degree of Reuse

How should a service be defined in order to maximize its Re-use potential and its compos-ability characteristics ?

Increasing Value of Reuse


Agenda







The Key Challenge on the SOA Journey

Legacy SoA

Monolithic, Large sized Legacy Applications somehow have to “unbundle”or “dis-aggregate” into an orderly set of Business and Data Services, in the new Architecture. How do we accomplish this – while also ensuring Business Continuity and Service levels ?


Service Identification – Ensuring Re-usability and Compos-ability

1. How should a service be identified and defined in order to maximize its Re-use potential and its compos-ability characteristics ?

2. How do “combine” fragments of shared business logic currently scattered in many Legacy Applications to form a useful Service ?

3. How do we ensure that Business Rules that are scattered in different Applications are consolidated, rationalized and encapsulated into Re-usable services ?

4. Should we start with an Enterprise wide Services Architecture first ?

5. How do we consolidate Data that is widely replicated in many Applications, so as to encapsulate them into Re-usable Data Services ?

6. How do we ensure that Services are “tested” for future use, in service composition scenarios not yet identified ?


Master Data Services – A natural place to start

Legacy SoA

The Re-organization of Master Data in an enterprise that consolidates widely replicated instances of Master Data, reconciles and creates a single version of the Master Data Record, along with a sufficient set of Master Data Access Services, is an important milestone in the SOA Road Map. It represents a critical “unbundling” step of the Legacy Applications

Customer Product

Location Account


Legacy Transition – A Method

Legacy Applications undergo the following sequence of transitions in the wake of their Master Data being Re-architected :

1. The Master Data from Legacy Applications are extracted and consolidated into a separate Master Data environment.

2. Interfaces are constructed to keep the Legacy Master Data in synch. with the new Master Data environment

3. New Master Data Access Services are created that deliver Master Data to “Requesting” Applications

4. Gradually, Master Data Access in the Legacy Applications are transitioned to the new Master Data Environment

5. Eventually Master Data Creation and Maintenance is also transitioned into the new Master Data Environment

6. Finally, the physical instance of Master Data within the Legacy Applications are altogether removed, representing a near surgical “Cut”of the Legacy environment (15 to 40%)


Agenda







What is Data Governance ?

Data Governance creates a culture where creating and maintaining high quality data is a core discipline of the organization.

Data Governance is the formalized discipline of ensuring accountability for the management of an Enterprise’s Core Information Assets.

Data Governance includes

1. A defined Process

2. An Organization Structure

3. Well defined Roles and Responsibilities

4. Clear Rules of Engagement and Escalation


What is the Relationship between Master Data Management and Data Governance ?

The scope of Data Governance goes well beyond Master Data. It includes

Structured Data

1. Master Data2. Transactional Data3. Analytical Data4. Meta Data

Unstructured Data

1. Documents2. Content3. Knowledge


Products Customers Dealers Contracts Channels Transactions

Master Data

System

Transactional

Systems

Operational Data Stores

(ODS)

Data Warehouse (DWH)

Data Marts (DM)

Data Class

Master Master Transaction

Master Transaction

Master Transaction

Analytic

Master Transaction

Analytic

History Yes No None Limited

YesYes

Integration Yes No YesLimited Yes Yes

Data Currency

Real Time Real TimeClose to Real time (+1

Day)

Sometimes Daily

Weekly Monthly

Sometimes Daily

Weekly Monthly

Data Scope


Neutral

Local Application

Specific

Integrated (Limited to a

few applications)


Neutral

Analytical, Derived and Summarized, Application

specific

Data Creation

Yes Yes No Limited to Derived

Data

Limited to Derived Data

Data Warehouse and BI versus MDM oriented Data Governance

Data Warehouse and Business Intelligence focused Data Governance Initiatives lack a Data Life-cycle view, and get exclusively focused on preparing Data for Data warehouses, Data Marts and Reporting environments.

Master Data Management focused Data Governance Initiatives, are concerned with the entire life cycle of the Master Data, starting from how they originate and are used in the various Operational Systems as well as the analytical environments, however they tend to ignore the transactional data.

Data Warehouse and Business Intelligence focused Data Governance Initiatives

Master Data Management focused Data Governance Initiatives


Agenda






The Case for a Strategic MDM Vision


Master Data Management represents an Architectural paradigm shift

Rationale

• Master Data is highly shared across many business processes and applications• At the heart of MDM is a radical idea that the governance of Master Data must

be de-coupled from Transactional, Operational and Analytical systems• The way Master data is delivered to various applications, must be through well

defined and highly re-used Master Data Access Services

Implication

• The introduction of an MDM Solution is disruptive to the existing Architecture• The way existing applications own, manage and share Master data will

fundamentally change in the wake of MDM initiatives• The Architectural style of Off the shelf Applications (i.e. Products) must also

adapt to his emerging paradigm shift


De-coupling Master Data from Transactional Systems is the end state

Rationale

• The concept of Master Data Management goes beyond merely consolidating master data and providing appropriate data governance

• It even goes beyond synchronizing master data across many different applications, from a central Master Data Hub

• In a Services oriented Architecture, it is possible to deliver Master Data to various consuming applications, through Data Access services

Implication

• Application design must undergo a radical transformation. • Instead of making the Master Data as part of the Application design, it must

allow for the possibility that Master data may be accessed through services• Off the shelf products must also begin to allow for externally provided Master

Data Services (as an alternative to their internal Master Data tables)• Managing referential integrity will be more complex in this environment


Master Data Management is a key milestone in SOA adoption

Rationale

• At the heart of Service Oriented Architecture is the concept of separation of data from process.

• As SOA adoption scales beyond the initial Pilot exercises, it will encounter a critical cross road i.e. the fact that Master Data is scattered in many different applications

• Top down process oriented approaches to SOA adoption do not adequately address the problem of highly replicated, and scattered data

Implication

• Bottom up Data oriented approaches to SOA adoption will be recognized as key milestones in the SOA Road map

• MDM initiatives and SOA initiatives must be closely coordinated and converged

• Master Data services will be building blocks for creating more coarse grained Business oriented services


Master Data Management initiatives are enterprise programs

Rationale

• Master Data Management initiatives are not “yet another project”. They deliver critical Enterprise infrastructure

• They impact many different applications and business processes including most concurrent projects within the organization

• It these implications are not adequately accounted for, the real potential value of Master data initiatives will not be realized

Implication

• An enterprise level vision and road map must be established for Master Data Management initiatives

• The governance of Master Data Initiatives must be co-ordinated at the enterprise level

• The Executive level governance structures must provide the right steering signals to align concurrent initiatives with the MDM initiatives


Master Data Management must address all Master Data subject areas

Rationale

• Some subject areas are more obvious and critical to the business than others. They may not get recognized in the initial stages of an MDM program.

• An enterprise level vision and road map for Master Data Management must encompass all Master Data subject areas

• These subject areas are typically the key Master Data concepts within an enterprise such as “Customer”, “Product”, “Location”, “Contract”, “Geography”, “Employee”, ‘Asset” and so on.

Implication

• An enterprise level MDM program, will have many tracks, with each track pertaining to a subject area

• There may be different business sponsors for these tracks • The MDM Solution pattern for each subject area, may also vary by subject

area


Customer Data Integration (CDI) is an important track within MDM

Rationale

• The emerging CDI space is a critical subject area within the Master Data Management space

• CDI has gained momentum, simply due to the fact that the Customer Data is critical to most businesses

• Customer Master Data is also scattered widely amongst many operational and analytical systems

Implication

• Enterprise wide MDM initiatives can benefit from the early success of CDI initiatives

• When CDI initiatives do not have an Enterprise wide MDM context, the potential value of MDM may not be realized

• When CDI initiatives are created as stand alone initiatives, they will not yield re-usable patterns across other subject areas


Enterprise Master Data Models are critical for Data consolidation

Rationale

• Data Modeling allows an enterprise to establish unique definitions and address issues of semantic dissonance, across its various organizational components

• Data concepts such as “Customer” and “Party” acquire different definitions over time, especially in different contexts.

• MDM initiatives are occasions to address the fundamental definitions that a business is based upon a well defined business concept and context

Implication

• The notion that we are buying a “Product” so why focus on Data modeling which is common in the contexts of ERP applications, must be counteracted

• While an “MDM” product vendor can provide a “Starter” Data model, these must be carefully assessed and customized

• MDM by its very nature is “Application neutral”. So Application specific data models, must be avoided in MDM initiatives


Data ownership and governance are critical for Data Synchronization

Rationale

• Data ownership implies that there is a single business owner, for each business attribute, in each Master Data subject area

• Historically data ownership has been confined to “Application” boundaries. The resolution of data semantics across applications has been relegated to the domain of “Integration”

• Enterprise wide Data ownership and Governance is not well understood or developed, today. But it is a critical milestone in MDM

Implication

• It is obvious that the SVP of Sales and Marketing is a good candidate to own and govern ‘Customer’ Master Data.

• However, he has to own and govern ‘Customer’ Master Data on behalf of the entire enterprise, not just Sales and Marketing. This is a different paradigm.

• New Enterprise wide roles and responsibilities are required in the wake of Master Data Management initiatives

The Case for tactical MDM Delivery


MDM initiatives impact many business areas

Rationale

• Master Data Management results in improved data quality, semantic consistency and can impact many business areas.

• For example, a typical CDI initiative can impact a variety of areas related to the Customer subject area – such as Sales, Marketing, Servicing, Business Intelligence, Risk Management, Privacy, Compliance and so on

• The real business value of MDM initiatives is the aggregate of the business benefits in each impacted area

Implication

• Building a comprehensive business case for MDM requires understanding its potential business value across these varied business areas

• No one Business Executive is going to have MDM on the top of his or her agenda

• Selling the concept of MDM to the business requires constant evangelization, and advocacy across a broad range of business executives


MDM Initiatives must deliver business value for sustainability

Rationale

• The creation of a Centralized Master data hub cannot be an end in itself. It has to be leveraged appropriately to deliver measurable business value

• The business value of an MDM initiative must be forecasted before the initiative begins, and assessed after it ends

• When business value is clearly demonstrated, subsequent iterations of the MDM Road map can be funded more easily

Implication

• The business value must be articulated clearly through a well developed business case

• The business case must be built on the basis of critical business measures that will be impacted by the MDM initiative

• Business executives must be educated on the fact that MDM initiatives deliver critical infrastructure, which must be accompanied by business strategies and solutions


The business benefits of MDM are intuitive yet difficult to quantify

Rationale

• MDM initiatives only deliver the ‘foundational infrastructure’ for potential business value.

• For example, the fact that we now have a single customer view in the customer data hub, does not guarantee that the Cross Sell ratio will go up.

• The single view of the customer has to be leveraged through ‘effective’ cross sell business strategies

Implication

• As an enabling infrastructure, it is difficult to directly attribute business value to an MDM initiative.

• Selling the concept and acquiring funding for an enterprise wide MDM initiative is going to be challenging

• It will always be easier to sell an enterprise on the concept of a tactical MDM initiative, (as part of a larger business project) with limited scope and budget.


MDM Solution delivery must be aligned with tactical business projects

Rationale

• Often there are in-flight, funded projects that are currently being addressed through custom development, or off the shelf products

• These efforts may not have considered an MDM based approach at all• If an MDM Architectural vision is not aligned tactically with ongoing funded

initiatives, it may never get off the ground• By aligning an MDM Road map with tactical Business initiatives, the enterprise

can begin the MDM journey

Implication

• Some central agency must own the MDM architectural vision and arbitrate it across numerous in flight and upcoming initiatives

• Tactical business projects must be assessed for convergence with the MDM architectural vision

• There must be a well articulated MDM Architectural vision, that has been established at the enterprise level


MDM Initiatives must have many iterations within each subject area

Rationale

• MDM initiatives impact multiple business areas. The delivery of business value to a specific business area or function could be constituted as a specific MDM iteration

• For example, delivery of Customer Master data to the Sales function, and the attendant benefits could be a single iteration in the MDM Road Map

• The delivery of business value to a specific business area is easier to manage in a shorter time frame. Shorter time to value builds confidence in the initiative

Implication

• The MDM vision for an enterprise must encompass the notion of an “MDM adoption iteration”

• The business case and the solution footprint of each iteration must be carefully constructed in order to build growing consensus for the initiative

• There are no tactical MDM projects – only MDM adoption iterations that build upon a Strategic Architectural vision

The Case for creative balance between Strategic Vision and Tactical Alignment


Lack of Strategic Focus will yield disconnected MDM infrastructures

Rationale

• Without a strategic MDM Vision, it is possible for most large enterprises to end up with multiple MDM infrastructures, which are not appropriately integrated

• The design of MDM solutions will not be established to evolve through multiple MDM delivery iterations

• Tactical MDM design patterns that do not scale up to a coherent MDM architecture are predictable

Implication

• A strategic MDM Vision that has the buy in with the key enterprise stakeholders, on both the Business and IT organizations is critical to success along the MDM journey

• A mechanism to co-ordinate the project portfolio with the MDM initiatives is also critical to MDM success


Lack of tactical alignment will result in challenges in getting started

Rationale

• Without tactical alignment, the MDM journey will remain a vision without a sponsor

• A big bang approach without tactical short term value delivery will be difficult to get funded

Implication

• The MDM Vision must be reconciled with the current funded project work • A beach head project must be identified that can become the ‘pilot’ to get the

MDM program started• The business value of tactical MDM iterations must be clearly demonstrated


MDM initiatives must balance strategic focus with tactical alignment

Rationale

• Without an appropriate balance between strategic vision and tactical alignment, MDM initiatives can suffer for want of adequate sponsorship and visibility

• Every MDM initiative during its life cycle will lean either towards the strategic vision or the tactical alignment

• Maintaining this balance is a key challenge with steering MDM initiatives

Implication

• Enterprises must have the appropriate ownership and governance structures to achieve this balance with MDM initiatives

• Executive steering signals must drive organizational behavior • Senior executives must be sensitized to the nature of this balance


Enterprise Architecture must provide the Strategic guidance for MDM

Rationale

• Enterprise Architecture has the broadest view of the IT portfolio in the organization

• Without such a broad view of the IT portfolio, MDM initiatives can become narrowly focused

• Enterprise Architecture must provide the leadership and guidance, to establish an MDM road map

Implication

• There must be a strong Enterprise Architecture team • EA must educate and evangelize the concept of MDM and the nature of the

MDM journey with the senior IT and Business executives• Enterprise Architecture must guide individual projects, product selection

exercises, and technology choices to align them with the MDM vision


Enterprise Program Management must align Projects with MDM

Rationale

• Without a program management function governing and overseeing the current in flight projects, there is a risk of individual projects making decisions that are not in alignment with the over all MDM vision

• Individual project design and architecture decisions made without the overall MDM vision in mind can be counter productive and cause inefficiencies

• It is not uncommon to have multiple MDM like initiatives sprout up, each with a narrow focus and little convergence

Implication

• The enterprise program management function, must work closely with Enterprise Architecture to provide Architectural convergence across the portfolio of projects currently in flight

• Clear mechanisms for Architectural reviews, compliance assurance and exception management must be established


Agenda






TCS Approach to MDM


Master Data Management is a journey, not a project. MDM Solutions redefine an Enterprise’s approach to data management, across both their operational and analytical environments

Envision

Building anMDM consensus

EnvisionTo-Be BusinessConcept model

EnvisionBusiness Value

Assessment

StrategizeMaster/Meta

Data Architecture

Strategize

Data Quality

StrategizeTechnology

Selection

StrategizeData Governance

Framework

Build & Deployment

ArchitectSolution Architecture

&Design

ArchitectDetailed Analysis

& Planning

Envision

Program Road Map

Support & Evolution

Enabling the complete MDM Adoption cycle

Deliver Deliver

Enabling the MDM / CDI Adoption Cycle

Account Data

Management

Customer Data

Integration

Geo Spatial Data Management

Product Information Managemen

t

Reference Data

Management

Location Data

Management

Manage Your Master

Data

Strategize M

DM

Del

iver

M

DM

Envision MDM

ArchitectMDM

Thank You & Q & A