lean integration:lean integration: translating an ... integration at misrc[1...lean integration...

Lean Integration:Lean Integration:Translating an Innovative Agile Approach into Business Valuepp

MIS Research Center SeminarMarch 25 , 2011

John SchmidtVP, Global Integration Services

1

InformaticaThe #1 independent leader in Data Integration

• 2009 Revenue: $500 million $500

$550

• 5-year AverageGrowth Rate: 18% per year

• Customers: 3,931$400

$450

$500

• 84 of Fortune 100

• 87%+ of Dow Jones

• Government organizations $350

$400

gin 20 countries

• Partners: 400+

• Major SI, ISV, OEM and $250

$300

On-Demand Leaders

• Employees: 1,755

• Direct in 28 Countries $150

$200

2005 2006 2007 2008 2009

2

2005 2006 2007 2008 2009

Our Singular MissionEnabling The Information EconomyEnabling The Information Economy

We enable organizations to reduce IT costs and gain a competitive advantage in today’sand gain a competitive advantage in today s global information economy by empowering them to access, integrate and trust all theirthem to access, integrate and trust all their

information assets.

3

The Role of IT in Business

“CEO’s, CFO’s and CIO’s all recognize there is zero separation between business strategy and IT execution today. There is pretty much nothing you can do, from cutting costs or growth without the enablement of IT in one way shape or form ”the enablement of IT in one way, shape or form.

Steve Schuckenbrock, President, Dell Large Enterprise Business

4

Reference: Financial Times Special Report, Wednesday, October 27, 2010

The Data Driven EnterpriseThe Data Driven Enterprise

Partner Trading Network(B2B)

Cloud Computing TraditionalEnterprise (B2B)Enterprise

SEPA

HIPAA

NACHA

5

Problem: Data chaos results from unnecessary complexityunnecessary complexity

Root Cause - Integrations manufactured as custom point solutions

6

Craftsman Approach – Custom Development

Manufacturing WorldSkilled manual workersManufacture by hand

Custom products (works of art)

Integration WorldSkilled IT Specialists

Hand codingCustom integration points

7

Custom products (works of art) Custom integration points

Assembly Line Approach – This is ICCs now

BenefitsConsistent Quality

Lower Cost

Integration WorldDedicated skilled labor

Standard ToolsPeriodic Large Releases

Manufacturing WorldDedicated skilled labor

Standard ToolsMass production

8

Periodic Large ReleasesMass production

Modern Factory Approach – Automated Flow of Materials and Information

BenefitsLow cost AND high qualityExactly the way you want it

Flow of Materials

Flow of Information

M f t i W ld I t ti W ldManufacturing WorldAutomation – new roles

JIT WorkflowMass-Customization

Integration WorldTemplate-driven self-service solutions

Agile/iterative DevelopmentConfigurable re-useable objects

9

g j

It is Critical That You Have an Expert and S t i bl I t ti C bilitSustainable Integration Capability

Efficiency Spectrum

Operational EfficiencyLow High

y p

Hand codingIntegration

Tools (EAI,...)

IntegrationTechnology

Integration Platform

Integration Factory Platform

Projects& Ad hoc

integration

ProgramManagement

ManagementPractices

Competency Center

LeanIntegration

Fast(quick & dirty)

ImprovedQuality

CostEfficiency

Faster, Better AND Cheaper

BusinessBenefits

1990’s 2000’s 2010’s

10

Point-in-time Methods Sustaining Methods

Cultural Change is easy – isn’t it?

11

Transform complexity and chaos to an Agile, Data-driven Enterprise

Complexity Agility

Competitive Weapon

ata

No Compromise

Complexity Agility

Shared Resource

ion

of D

a

Controlled cost

Perc

epti Business

EmpowermentAND

IT Control

Unmanaged silos

StandardizationAND

Innovation

Technology Hand coding IntegrationTools

Integration Platform

Integration Factory Platform

Methodology

Independent to Tactical to Managed to

Project Program CompetencyCenter

LeanIntegration

12

Transformationsp

Interdependent Strategicg

Cultural

Analyst Perspective

“Taking a production line approach to integrations makes a lot of sense…they should be more widely considered and i l t d ”implemented.”

Philip Howard, Research Director - Data ManagementBloor Research, January 7, 2009Bloor Research, January 7, 2009

13

Lean Integration Principles

Eli i t W t Continuous ImprovementEliminate Waste Continuous Improvement

Automate Processes Empower the Team

Build Quality InPlan for Change

14

Optimize the Whole

g

1. Eliminate WasteExample: Value Added Ratio – Before Lean

Scenario: Simple Change Request to the Data Warehouse

5 min 5 min 2 hours 1 hour 1 hour 50 min

1 k 2 k 2 k 1 k 2 k

Value-Add

NonValue-Add 1 week 2 weeks 2 weeks 1 week 2 weeks

15

1. Eliminate WasteExample: Most non-value added activities eliminated

5 min 5 min 2 hours 1 hour 1 hour 50 minValue-Add

4 hours 8 hours 0 weeks 4 hours 16 hours

NonValue-Add

Lead Time Reduction = = = 97%Original Lead Time – Current Lead Time 1344 - 32

Lead Time Reduced from 2 months to <1.5 days

16

Lead Time Reduction 97% Original Lead Time 1344

1. Eliminate WasteKeeping data in production that no-one uses is waste

Archiving reduces datafile size (GB)

1 200

1,400

Without archiving With archiving …and truncating temp tables

400

600

800

1,000

1,200

0

200

400

Current Year 1 Year 2 Year 3

Data that is not used still costs a lot:• Disk capacity (production, test, DW)• Backup costs (tapes, labor, storage)• Business slow down by end of period jobs

Projected savings by eliminating data waste

$12,907,437

• Business slow-down by end-of-period jobs• Application upgrade costs• Impact on real-time transactions• Compliance costs

$1,603,277

$4,925,769$6,378,391

Year 1 Year 2* Year 3 3-year Total

17

• Data quality analysis and resolution timeYear 1 Year 2 Year 3 3 year Total

Take Away: Eliminating Waste is not about cutting out fatg

“Don’t think of lean as requiring a diet to lose weight; think of it as focusing the people, process, technology, and architecture that you possess on the projects that have the most impact for the business.”

Mike Gualtieri, John Rymer, Jeffrey Hammond, Top Five Changes For Application Development

Forrester, January 4, 2010

18

2. Automate ProcessCreate assembly lines for rapid customization

• Generating custom wizards for templates to maximize productivity for end users– developers and/or analysts

IT Designs Templates and Patterns – Users create custom solutions fast

• Ability to build your own wizards based on patterns without coding

• Common data integration and event processing patterns

2. Design Template1. Understand and Define Common Patterns

3. End Users Quickly Configure and Customize in Assembly Line-Fashion

Examples:• Slowly changing dimension in

warehousing• Data file interchange with supply

chain partnersTi b d t l ti

Mapping Template Example: Slowly Changing Dimension Wizard

• Time-based event correlation

19

Huge Productivity Gains for End Users

Take Away: Automation is a key to simplificationsimplification

“Factory IT couples lean management techniques and process improvements with advances in cloud computing and software development to simplify thecomputing and software development to simplify the operating environment and improve productivity and cost performance.”

Reshaping IT management for turbulent times,Roger Roberts,Hugo Sarrazin, Johnson Sikes, McKinsey

Quarterly, November, 2010

20

3. Empower the TeamProblem: IT is the bottleneck for data requestsSolution: Give the business self-service capabilities

Business Self-Service

Enables business self-service, improving

business agility

S

IT maintains governance and oversight

IT personnelSupport

21

Take Away: Self-service improves business agility AND gives IT better governance

“Lean is about ensuring IT is more closely aligned to the Business.”

Stephen Pritchard Financial TimesStephen Pritchard, Financial Times

22

Source: Financial Times Special Report, Wednesday, October 27, 2010

4. Continuously ImproveProblem: How can groups improve themselves?Solution: Use visual management of the entire integration lifecycle.Solution: Use visual management of the entire integration lifecycle.

Data Quality MetricsData Volumes by ProjectDevelopment Activity

Object reuse Project X Development Project Y Development

23

Take Away: Don’t use Lean to cut staff

“Best-practice executives view lean as aBest practice executives view lean as a performance improvement strategy rather than merely a cost-cutting exercise.”

Alexander Peters, Forrester Research

24

Source: Financial Times Special Report, Wednesday, October 27, 2010

5. Build Quality InProblem: Quality issues accumulate without business involvementSolution: Involve business in fixing highest priority issuesSolution: Involve business in fixing highest priority issues

Line of business managermanager

DataData Steward

Data Quality

ITDeveloper

25

Take Away: Metadata is an enabler for IT automation

“Metadata allow systems to adapt more quickly to changing requirements by pushing power into the g g q y p g pdata, and out onto users.”

Brian Foote and Joseph Yoder Big Ball of MudBrian Foote and Joseph Yoder, Big Ball of Mud, Addison-Wesley Software Patterns Series, 2000

26

Reference: Financial Times Special Report, Wednesday, October 27, 2010

6. Plan for ChangeInsulate consuming applications from change

Applications PortalsAnalytics

S i S iS i Service ServiceService

Service

ServiceServiceServiceService

Service

ServiceService Service

ServiceService ServiceService

ServiceService Service

ServiceService

CUSTOMER ORDER PRODUCT INVOICE

27

Lean IntegrationProven Value Proposition: Better, Cheaper & Faster

Build Quality

I

ContinuouslyImprove

Optimize The

Wh l

PlanFor

Ch

Focus on Customer Value and

p , p

AutomateProcesses

Empower the Team InImprove WholeChangeEliminate

Waste

Processes the Team

Implemented data lit t i d

Cloud Services The InformaticaPlatform

Mapping ArchitectFor Visio

PCAE - Metadata Manager

Saved $1.4M in first 6

Data Quality

Centrally managed

V9 Data ServicesDatabase Archiving

Business analysts 4 month payback from After re-working their Reduced impact quality metrics and monitoring system;

Saved est.$3M in development costs

and avoided $20M in AML regulatory fines

months because of improved customer master data quality; Reduced SKU’s by

40% by reducing old or obsolete parts

virtual views speed up time to add products to

DW portfolio from 1700 hours to 40

hours

use Informatica Cloud Services to self-serve data for their own analysis, cutting lead-times to information by

90%

p ystorage reduction,

and rapid retrieval of archived claims for improved customer

service

ETL processes to always use design patterns, achieved 8x efficiencies on

all new projects and re-work on old

projects

analysis time between 85-95%, reduced time for business analyst inquiries by 50%, reduced end-user support time by

65%

28

over 65%

Informatica Confidential and Proprietary

Manufacturing Production Lines

Bread factoryBottling LineBottling Line

Car Assembly Line

29

Leading Practice: Integration Systems are Managed Like First Class Business SystemsLike First-Class Business Systems

Business SystemsIntegration Systems

Business Systems

Enterprise Resource Planning Systems

Sales & Marketing Systems

Customer Product and Order M t

Human Resource Systems

Finance & A ti

IntegrationHub

TransformationHub

Application Integration

DataWarehouse

Business Analytics

Extract Transform Load

Data Services Enterprise Service Bus

CloudServices

CustomerSystems Management

SystemsAccounting

Systems

Distribution and Inventory Systems

External Systems (Software as a

Service)

Engineering & Manufacturing

Systems

IntegrationPortal Process

Master Data Management

Managed File Transfer

Information Security

Enterprise Messaging

Business to BusinessData Quality

Integration Systems

IntegrationHub

TransformationHub

Application Integration

Integration Repository

Portal Process Management

InformationPortal

Directory &Single Sign-on

Metadata Manager

Business Glossary

Process State Engine

ActivityMonitoring

Integration RepositoryPortal Process

Management

Content Management Service Registry Workflow

Search Complex Event Processing

Data Stewardship

30

Informatica Platform Mapping

Integration SystemsIntegration

HubTransformation

HubApplication Integration

D t E t t Cl d

UltraMessaging

CEPDataWarehouse

Business Analytics

Master Data

Extract Transform Load

Data Services

Managed File

Enterprise Service Bus

Information

CloudServices

CEP

B2B DataExchangeB2B DataExchangeB2B DataExchangeB2B DataExchange

Integration R it

Portal Process M t

Management Transfer Security

Enterprise Messaging

Business to BusinessData Quality

ILM

CloudServices

Repository Management

InformationPortal

Directory &Single Sign-on

Metadata Manager

Business Glossary

Process State Engine

ActivityMonitoring

PowerCenter

ILM

PowerCenterPowerCenterPowerCenterPowerCenter

Content Management Service Registry Workflow

Search Complex Event ProcessingData Stewardship

Data QualityData Quality

31

MDM

Informatica Factory Platform Products in Eight Growth Categories

Data Quality Assessment

Informatica Cloud PlatformData Loader

Data Synchronization

Data Replication

Data Quality Global Address

Verification

IdentityResolutionData ArchiveRulePoint PowerExchange MDMAdvanced XMLLBM Streaming

Data Explorer

Data QualityData Transformation

Data

Resolution

Data Services

PowerCenter Data Subset

Data Privacy

RuleCast

Real-Time

gg

UME Persistence

UME Queueing

32

Data ExplorerExchange Data Services Data PrivacyAlert ManagerUME Queueing

Production Line for Application Modernization


Data Loader


Data Replication


VerificationInformatica Cloud Platform

IdentityResolutionData ArchiveRulePoint PowerExchange MDMAdvanced XMLLBM Streaming

Data Explorer


Data

Resolution

Data Services


Data Privacy

RuleCast

Real-Time

gg

UME Persistence

UME Queueing

33


Production Line for Supply Chain Integration


Data Loader


Data Replication



IdentityResolutionData Archive RulePoint PowerExchange MDMAdvanced XMLLBM Streaming

Data Explorer


Data

Resolution

Data Services


Data Privacy

RuleCast

Real-Time

gg

UME Persistence

UME Queueing

34


Production Line for Channel Integration (ESB)


Data Loader


Data Replication




Data Explorer


Data

Resolution

Data Services


Data Privacy

RuleCast

Real-Time

gg

UME Persistence

UME Queueing

35

Data ExplorerExchange Data Services Data PrivacyAlert Manager UME Queueing

Production Line for Business Intelligence COE

Data Loader


Data Replication

Informatica Cloud PlatformData Quality Assessment


Verification


Data Explorer


Data

Resolution

Data Services


Data Privacy

RuleCast

Real-Time

gg

UME Persistence

UME Queueing

36

Data ExplorerExchange Data Services Data PrivacyAlert Manager UME Queueing

Good technology alone does not guarantee success

“Deploying technology, by itself, is a poor strategy for improving the implementation of interfaces. An effective strategy for p gyintegration is established by using these critical success factors: • Establish an ICC. • Develop an integration "city plan" consisting of: p g y p g

• Policies that implement the necessary governance • A set of best practices that is continually refined by measuring

the result of applying the best practices in integration projects pp y g p g p j• A technology architecture that establishes standards for the

technologies to be used during application integration • Have integration competency staff drive the selection of the g y

integration technologies that are to be deployed. “

The Seven Critical Success Factors for Application Integration,

37

pp g ,Jess Thompson, Gartner, November 10, 2010

Lean methods in a nutshell

38

Lean methods – a more complete list

39

As-Is Value Stream MapDOIT Corporation: Value Stream Map (AS-IS) for Change Request Process

Monday, December 20, 2010

GMNA Applications Team(Irina)

Data Warehouse Team

Change Request

Confirmation Request

Telephone Tag

Status Request

Status Update

Automated Workflow/Tracking (Cust satisfaction)

Status Request

Semi-Weekly Review

CR Review Committee

Production CR ApprovedChanges

Clarify Requirements

Requirements Clarification

Status Request

Status Update

Data Dictionary to clarify rqmnts

(13 days)

Bypass Council for simple CR’s

(26 d )

Notify Customer(Cust Satisfaction)

(5 days)

Status Update

Integration Team Manager

Assign Resource

Test Team Manager

Test SchedulingData Warehouse Team

SubmissionAdd CRTo List

g

ForwardCR Request

Change Management Board

CR Approval

Daily ETL Batch Run

Test Results Distribution

Charge Request

Bypass Committee for

simple changes (8 days) Design

DocumentApproved Designs

(26 days)

Automated

Architecture Review Council

Design Approval

Development Team

1

Requirements Review

Infrastructure Team

Production Deployment

Test Team

Test Execution

Development Team

Testing Handoff

Development Team

Design & Development

CR’sP1 12

Automatic Daily ETL Batch Run

Production Execution

CR RequestTo Developer Approved CR

& Design DocsDesign Docs& Schedule

Test Team

Test Case Development

CR Approval& Schedule

Automated Regression

Testing(21 days - requires

investment)

P1x12P2x35

P3x124

30 Minutes

8.8 Days

15 Minutes

8.5 Days

90 Minutes

12.8 Days

180 Minutes

1 Day

15 Minutes

26 Days

180 Minutes

13.3 Days

Work Time = 510 Minutes (8.5 hrs or 0.35 Days)

Lead Time = 75.6 Days0.3 Days

Value Ratio: Work Time / Lead Time = 0.5%Notes:

40

(1) Lead Time includes 5 delay in customer notification(2) Lead Time could be reduced to 24 days with just process changes and using existing tools(3) Lead Time could be reduced to 3 days with a capital investment for automated testing

Integration Factory Organizational Structure

41

Helpful resources: Best practice referencesAvailable at amazon.com or integrationfactory.com

• Integration Competency Center (2005): A permanent cross-functional team operatingpermanent cross functional team operating as a shared service function supporting multiple organizational units and sustaining integration in a coordinated manner. g

• Lean Integration (2010): A teachable, sustainable management system that emphasizes creating value for customers, gcontinuous improvement, and eliminating waste as a sustainable data management and integration practice.

42

CIO Magazine, October 2010Staff Pick for IT Management & LeadershipStaff Pick for IT Management & Leadership

44

Sharing Knowledge Among Users

The go-to destination to buy and sell proven The go-to destination to buy and sell proven data integration, data quality, and data management solutions (Blocks).

G t t d! R d ti t k t ithGet promoted! Reduce your time to market with pre-built solutions

Get ahead of the competition! Increase yourGet ahead of the competition! Increase your sales reach with Informatica Marketplace

Maximize your Informatica investment withMaximize your Informatica investment with Informatica Marketplace - www.informaticamarketplace.com

45

Integration Maturity Assessment Survey

Integration Maturity Assessment

60 0

Integration Maturity Assessment

60 0

30.0

40.0

50.0

60.0

MAX

- Av

erag

e Lean

Sustaining

Lean

Sustaining30.0

40.0

50.0

60.0

MAX

- Av

erag

e

0.0

10.0

20.0

agem

ent

hodolo

gy

agemen

tch

itectu

re

Process

agement

System

s

MIN

- M

Program

Project

Program

Project0.0

10.0

20.0

agem

ent

hodolo

gy

agemen

tch

itectu

re

Process

agement

System

s

MIN

- M

Financia

l Man

agInt

egratio

n Meth

oMeta

Data Man

agInf

ormati

on Arch

Busines

s PMod

eling M

anag

Integ

ration Sy

As of September 1, 2010Fina

ncial M

anag

Integ

ration M

etho

MetaData

Manag

Inform

ation

ArchBus

iness P

Modelin

g Mana

gInt

egratio

n Sy

46

http://vip.informatica.com/?elqPURLPage=7960

Impact: IT Costs are Increasing

47

Takeaways

• Lean is Easy – don’t try to be “perfect”

• Learn a Lean technique and start using it

• Invest in Factory tools• Invest in Factory tools

48

Reference Models are an essential element of an effective Business FrameworkBusiness Framework

How does a framework help?

• Faster projects

Solutions defined in terms of• Higher quality solutions

• Decreased cost through reuse of technology assets

Tighter integration of

Solutions defined in terms of

• Tighter integration of business and IT – more

flexible and agile organization

C B i

IFW FOUNDATION MODELS

FSDM FSFM FSWM

IFW PROCESS & INTEGRATION MODELS

IFW DATA MODELS

• Common Business Language

• Up-to-date Risk and Compliance requirements

i t t d th h t

IBM’s IFW

IFW PROCESS MODELS

FS-BOM

FS-IDM

BDWM

BSTs

IFW ODS

Banking

Solutions defined in terms of integrated throughout

50

Case Study: Wachovia’s Business Architecture

51

Information Architecture Framework

Layer 4 – Enterprise ViewOverarching consistent context for Vi

ews

Reference Models

Information ModelsBusiness GlossaryCanonical Model

Process ModelsBusiness Event Model

Operational Workflow Model

Overarching consistent context for executives & information stewards

Layer 3 - Business ViewDomain models for business owners and project sponsors 3R 3M

4

Bus

ines

s V

Business Context DiagramFunction/Information Matrix

Master Data List4

Metadata(federated

repository of relevant models)Physical Data Models

Logical Data ModelsEntity Relationship Diagram

Transformation ModelsI t f S ifi ti

Interaction ModelsLogical Data Map

Sequence diagrams

grat

ion

View

s

owners and project sponsors

Layer 2 – Solution ViewArchitecture models for specific

systems and solutions 2R 2M

3R 3M

Layer 1 Technology View models)yFile/Table PropertiesStorage Information

Interface SpecificationsTransformation rulesProtocol Definitions In

teg

1R 1M

Layer 1 –Technology ViewTechnical models for developers,

engineers and operations staff

Data at Rest Data in Motion

A layered approach is needed to manage data complexity, provide multiple stakeholder views and maintain traceability to business requirements

52

stakeholder views, and maintain traceability to business requirements.

Data Governance Methodology

One-Time Effort Repeat for every program

2Define

Governance Framework

6Assess

Baseline & Data Quality

10Implement Projects


1Organize

Governance Committee

5Scope

Program

3Develop

Enterprise Models

Data Quality

8Plan Migration

Roadmap


7

9Develop Program Models

4Assign

OrganizationalRoles

7Develop Target

Architecture

Entry Points1. Enterprise Initiative (Large-scale program with defined ROI – e.g. Merger or major business transformation)2. Top-level Directive (C-level or higher – e.g. one view of customer/business, control IP, data security, etc.)3. Scale-up DQ and MDM Projects (Leverage successful solutions that demonstrate the value for a specific

53

p j ( g pbusiness area and scale them up to cross-functional enterprise-wide initiatives)

lean integration:lean integration: translating an ... integration at misrc[1...lean integration...

Documents