asif pradhan_307 asug presentation

8/3/2019 Asif Pradhan_307 ASUG Presentation

1/63

]

CHARAN MARWAH[ASUG INSTALLATION MEMBER

MEMBER SINCE: 2003

RAECHAL MARTI N[ASUG INSTALLATION MEMBER

MEMBER SINCE: 2004

SYLVIE GAUTHIER[ASUG INSTALLATION MEMBER

MEMBER SINCE: 1999

Creating Trustworthy Master Data Using SAPBusinessObjects Data Services

Asif Pradhan


2/63

Real Experience. Real Advantage.

[

2

Learning Points

Master Data and its challenges

SAP BusinessObjects Data Services and SAP NetWeaverMaster Data Management (MDM)

Steps to create trustworthy data using SAP BusinessObjectsData Services

Profiling

Address Cleansing Data Cleansing

Matching


3/63


[

3

Heart of Every Organization - Data

Data - All organizat ions, par tners, individuals and systemsdepend on i t


4/63


[

4

What is Master Data?

It is the practice of defining and maintainingconsistent definitions of data, then sharing themvia integration techniques across multiple ITsystems within an enterprise.

TDWI MDM research

Every company has master data

Master data supports critical businessprocesses across the enterprise

Master data is a strategic corporate asset


5/63


[

5

Master Data is Crucial to Your BusinessBut Every Department has a Different Version of it

CallCenter

Jane Smith4418 N. Str.

Chicago, IL60611

Part: 2574

SRM

Part: 8975

VENDOR:ABC123

ERP

Jane Peters199, 3rd StreetPalo Alto, CA

Part: B7521

Logistics

VENDOR:XYZ456

Master data is dataabout your customers,products, suppliers etc.

Trading partner data

introduces yet anotherversion of data

Inaccurate data leadsto $30+ billion cost onsupply chains


6/63


[

6

Master Data Management - Common Questions

Which employee should we assign to?

Do I have the right product?

Who is my best supplier?

Who is my customer?

=

ERP

Jane Peters

199, 3rd StPalo Alto, CA

Call

Center

Jane Smith

4418 N. Str.Chicago, IL

60611


7/63


[

7

Master Data Communication - Data Consolidation andFlow Control

Distributor CompanyManufacturer

Data travels in multiple paths.

This way you receive the same record multiple times.

It is not easy to re-consolidate the records into one record.

How do I handle complex data flows?


8/63


[

8

External Driven Master Data Updates - PeriodicalData Synchronization

YourCompany Data Storage Sites

YourYourCompany Data Storage SitesCompany Data Storage Sites

SAP

Neurother St rasse

vs.

Dietmar H opp Allee

Business Objects

vs.

SAP BusinessObjects

Real-world Updates

Data travels in multiple paths.

This way you receive the same record multiple times.

It is not easy to re-consolidate the records into one record.

How do I keep track of external data changes?


9/63


[

9

Master Data Creation and Maintenance - EnsuringData Quality

Bob Bilder

Constreet 2

Bob Builder

Con Street 2

Bob BuilderRenovation 6

My name is Bob Builder.

I would like to

I cant find you in the system.

But I can create a new record to help you.

?

Records are created due to unavailable or inaccurate information.

The quality of the record content is often not ensured.

This often causes disruption in business process flows.

How do I ensure data quality from the beginning?


10/63


[

10

Enterprise Master Data Pain Points

Reduced Sales Effect iveness Lack of consolidated view of the customer across

channels Customer transactions are compromised due to

limited view of the facts

Sub-opt im al Procurement Decisions Lack of transparency into suppliers and products High costs due to insufficient supplier selection and

rationalization

Failed Market ing Plans

Failure to capitalize on market opportunity whenintroducing new products

Lack of real-time collaboration with front office, thushindering customer services


11/63


[

11

Enterprise Master Data Pain Points contd

Ineffect ive Business Decisions Lack of consistent information to support innovation

and growth Compromised business dynamics with negative

impact on managing resources effectively

Not Act ively Addressing Data Quality Garbage in, garbage out

Distrust of data in the system

Overwhelming loading exceptions - lack of

transformation and normalization of data


12/63


[

12

Impact of Poorly Managed Master Data on Processes

Quality of master data dramatically impacts transactional and analyticalprocesses

What is worse, youll only notice it after the damage is already done


13/63


[

13

Results of Poorly Managed Data - Negative impact onbusinesses in all industries

40% of orders blocked due to master data problems.EMEA High Tech Company

90% of upper level management feel they dont have thenecessary information for critical business decisions; 56%of them are afraid they are making poor decisionbecause of it.

Economist

Higher performing companies are 50% more likely touse analytical information strategically.

Competing on Analytics,Thomas Davenport

Through 2010, Global 1000 enterprises will incur morecosts, due to poor-quality data, than the benefits theywill gain from implementations of customer relationshipmanagement, enterprise risk management and businessintelligence applications.Key Issues for Data Management and Integration Initiatives2009, Gartner


14/63


[

14

Typical Causes of Loading Messy Data

Data content errors

Missing data

Invalid data

Data domain outliers

Data inconsistencies

Multiple formats for samedata elements

Different meanings for the

same code value Multiple code values with the

same meaning

Field overuse: used forunintended purpose.

Data in filler

Errors in migration (ETL)

Normalization inconsistencies

Duplicate or lost data

Data structure problems

Referential integrity problems

e.g. your ORDER DETAILStable contains part IDs thatdo not exist in the PARTS

table.

X

?

X

?

A critical component of your master data project is understanding your source systems and

recognizing any errors, inconsistencies or omissions in the data.


15/63


[

15

Load integrated datainto t he right place with

Data Services (DataIntegrator)

Make data t rustwort hywith Data Services

(Data Quality)

Ensure data remainstr ustwort hy with

Master Data Management

Master data is the ultimate goal ofEnterprise Information Management

SAP BusinessObjects Data Services and SAP NetWeaverMaster Data Management (MDM)

SAP BusinessObjects Data Services +SAP NetW eaver MDM


16/63


[

16

SAP BusinessObjects Data Services

Integrate and Improve the Quality of All Data

Market-leading, unified solution forenterpr ise-class data integration and

data quality

Single, easy-to-use user interface to build, testand deploy projects

Connect, transform and make the informationavailable from virtually any sources

Comprehensive data quality solution forcleansing and enriching all types of data

Embed data quality steps directly into the ETLprocess to build a data warehouse

Having all this functionality as part of one application makes it easy to select fromthe different data transforms provided the software and run data through themquickly and efficiently.

Richard WestPresident, Peachtree Data Inc.


17/63


[

17

Oracle

IBM DB2

Microsoft SQLServer

Sybase ASE & IQ

Informix

Teradata

Netezza

HP NeoView

MySQL

ODBC

JD Edwards

Oracle EBS

PeopleSoft

Siebel

Salesforce.com

SAP NetWeaverBW & BWA

SAP ERP & R/3

ABAP

BAPI

IDoc

Text delimited

Text fixed width

EBCDIC

XML

Cobol

Excel

HTTP

JMS SOAP

(Web Services)

ADABAS

ISAM

VSAM

Enscribe

IMS/DB

RMS

Both direct andchange data

SAP BusinessObjects Data ServicesData Integrator: Enterprise-Wide Data Access

Support for Structured and Unstructured Data Broad connectivity to databases, applications, legacy and file formats

Any text file type

32 languages

Databases Applications Files/Transport Mainframe(with Partners)

Unstructured Data


18/63


19/63


[

19

SAP BusinessObjects Data Services capabilities

ProfileProfile

AccessAccess

CleanseCleanse ValidateValidate

DeliverDeliver


20/63


[

20

SAP NetWeaver Master Data Management (MDM)Deliver a Single View of Business Information

Deliver trusted product, customer, supplier masterdata and global data synchronization

Pre-integrated with SAP BusinessObjects Data Services

for market leading data quality and data integration Pre-packaged IT and business usage scenarios

Compose cross-application processes in SOA withconsistent master data

Consolidate, harmonize, and centrallymanage master data of SAP and

heterogeneous environments

Prior to using SAP NetWeaver MDM and SAP BusinessObjects IM solutions, ourheterogeneous IT landscape made it difficult for us to view our total product line, what ourcustomers were buying, and how we could serve those customers better.

Joe YoungSenior Manager, IT, Lexmark International, Inc.


21/63


[

21

IDENTIFY SUPPLIERTAKE ORDER MANAGE CUSTOMERVERIFY AVAILABILITY

Who is mycustomer?

Do I have the rightproduct?

Who is my bestsupplier?

Which employeeshould we assign to?

SAP NetWeaver MDM - Manage master data in yourindustry-specific business processes

MDM


22/63


[

22

The Power of our Combined Product Offering -SAP NetWeaver MDM and SAP BusinessObjects Data Services


23/63


[

23

Broad sources / targets support

Bulk Data Movement

DQCapabilities MDM DI

Hierarchy Management

Governance and Stewardship

Workflow Management

Flexible Data Modeling

Authoring Master Data

Match / Merge

Universal Data Cleanse

Address Cleansing

Data Enhancement

Complex Transformations

Data Services

Comparing Data Services (DQ and DI focus) with MDM


24/63


[

24

Comparing Data Services with MDM


25/63


[

25

Creating Trustworthy Data with SAP BusinessObjectsData Services

Measure and analyzedata through dataassessment and continuousmonitoring

Cleanse and enhancecustomer and operationaldata anywhere across theenterprise

Match and consolidatedata at multiple levelswithin a single pass forindividuals, households, orcorporations


26/63


[

26

Creating Trustworthy Data Profiling


27/63


[

27

Data Profiling/Assessment measure and analyze

Inspecting data

Measuring the data defects

Analyzing the cause and impact of

those defects

Reporting the results of theanalysis to key stakeholders


28/63


[

28

1. Analysis of data beyondviewing

Frequency distribution

Distinct values

Null values Minimum/Maximum values

Data Patterns (e.g. Xxx

Xxxx99, 99-Xxx)

2. Comparison of valuesbetween data sets todetermine fit

3. Can drill down to viewspecific records

Data profiling - Analyzes Contents, Quality, Structure andRelationships


29/63


[

29

Creating Trustworthy Data Cleansing


30/63


[

30

Creating Trustworthy Data Address CleansingWorldwide Address Coverage

City Level Validation

Street and House Level Validation

Statistics

Country specific = 36

All world (countries & territories) > 240


31/63


[

31

Creating Trustworthy Data Address CleansingWorldwide Address Coverage


32/63


[

32

Standardization Example: United States Postal Service (USPS)address preferences

ADVANCE MOVERS

1500 E MAIN AVE STE 201

SPRIN GFIELD VA 22162-1010

Recipient Line

Address Line

Last Line


33/63


[

33

Address components

Line 1 Jack Schneider

Line 2 Business Objects, an SAP companyLine 3 332 Front St. South

Line 4 La Crosse, WI 54601-4023

Name Data (non-address data)

Firm Data

Postfix

PostcodeRegionPrimary TypePrimary Name

Locality

Primary Range


34/63


[

34

Prepare your input data

Before you start address cleansing, you must decide which kindof address line format you will input.

Both the USA Regulatory Address Cleanse transform and the

Global Address Cleanse transform accept input data in thesame way.

Address line formats:

Discrete

Multiline

Multiline Hybrid


35/63


[

35

Discrete Format

In discrete record format, data appears consistently in theexact same field arrangement in every record.

RECORD 1 RECORD 2

Firm = Micro Elektronic Ges.m.b.H. Address1 = Harborview

Address1 = Baumschulengasse 250 Locality1 = La Crosse

Lastline = 1010 Wein Region1 = WI

Country = Germany Postcode = 54601

Country = USA


36/63


[

36

Mult iline Format

In multiline format, data is not consistently located in the samearrangement in all records. That is, data items "float" amongfields.

RECORD 1 RECORD 2Multiline1 = Oxford Publishing Multiline1 = Gran Via de Carles S.A.

Mult iline2 = Wetherby House Mult iline2 =

Mult iline3 = 20 Oakly Rd. Mult iline3 = Ibarra Campillo 16-3-A

Multiline4 Multiline4 = 48010 Bilbao

Multiline5 = Harlow Multiline5 = SpainMultiline6 = Essex

Multiline7 = CM19 5AE

Multiline8 = United Kingdom


37/63


[

37

Multiline Hybrid Format

In multiline hybrid format, data is in both discrete and multilinefloating format.

RECORD 1 RECORD 2

Multiline1 = Jonas Lancaster Multiline1 =

Mult iline1 = Woodworks, Ltd. Mult iline2 = Akzo ChemicalsMultiline2 = Wetherby House Multiline3 =

Multiline3 = 20 Oakly Road Multiline4 = Moreelsepk 24

Multiline5 = Multiline5 =

Locality2 = Harlow Multiline6 =

Locality1 = Multiline7 =Region1 = Essex Lastline = 3511 EP Utrecht

Postcode = CM19 5AE Country = Netherlands

Country = United Kingdom


38/63


[

38

Creating Trustworthy Data Data CleansingData Quality Data Flow


39/63


[

39

SAP BusinessObjects Data ServicesData Cleanse Transform

Data CleanseDictionary

Parsing Rules


40/63


[

40

Creating Trustworthy Data Data CleansingParse Data


41/63


[

41

Creating Trustworthy Data Data CleansingStandardize Data

Assign gender and prenames

Create personalized greetings

Create separate data for each person


42/63


[

42

Creating Trustworthy Data Data CleansingPrepare Records for Matching


43/63


[

43

Creating Trustworthy Data Data CleansingParse, Standardize, and CorrectInput r ecord

Maggie.kline@future_electronics.com

Maggie Smith-Kline phd

FUTURE Electronics 5/23/03

101 6th ave

manhattan

ny

10012

001124367

Output record

Salutation: Ms.

First name: Maggie

Last name: Smith-Kline

Postname: Ph. D.

Match standards: Margaret, Magdalena,Magnolia

Company name: Future Electronics

Address 1: 101 Avenue of the Am ericas

City: New York

State: NYZIP+4: 10013-1933

Email: maggie.kline@future_electronics.com

SSN: 001-12-4367

Date: May 23, 2003
mailto:Maggie.kline@future_electronics.commailto:maggie.kline@future_electronics.commailto:maggie.kline@future_electronics.commailto:Maggie.kline@future_electronics.com


44/63


[

44

SAP BusinessObjects Data Services - Data Cleansing

Data Input Standardized Output


45/63


[

45

Creating Trustworthy Data Enhancement


46/63


[

46

Data Enhancement

Completes records with directoryinformation

by appending name, address,code data and more

Provides geocoding capabilities

for geographic anddemographic marketinginitiatives

Provides geo-spatial assignment

of customer addresses for taxjurisdictions, insurance ratingterritories, and insurancehazards, etc.


47/63


[

47

Data Enhancement

Directory information

Geocoding information

Geo-spatial information

Margaret Smith-Kline, Ph.D.Future Electronics101 Avenue of the AmericasNew York, NY 10013(222) 922-9922

Centroid Latitude: 40.723195

Centroid Longitude: -74.004977

Address Latitude: 40.723175Address Longitude: -74.004970

FIPS State Code: 36 New YorkFIPS County Code: 061 New York

FIPS Place Code: 51000 New York

MCD Code: 44919BSA Code: 35620Metro Code: 5600Section Code: 0051001012

-1933


48/63


[

48

Cleansing and Enhancement Capabilities in Data Services

Mult i-line Input Record Output Record

Data Parsed intoIndividual Components

[email protected]

Michael Schmidt phd

HYPERtech, Inc.

5/23/03330 east 63 avenue # 10b

Manhattan ny

10021

001124367

Salutation: Mr.

First name: Michael

Last name: Schmidt

Postname: Ph. D.Match standards: Michael, Mike, Mick

Gender: Strong Male

Company name: HyperTech, Inc.

Address 1: 330 E 63rd St Ste 10B

City: New York

State: NYZIP+4: 10065-7706

Email: [email protected]

SSN: 001-12-4367

Date: May 23, 2003

Corrections

Enhancements,Casing and

Standardization

Extract, cleanse, normalize, correct, and validate data using predefined extractors Configure and execute rules to identify records with data quality concerns
mailto:[email protected]:[email protected]


49/63


[

49

Creating Trustworthy Data Matching


50/63


[

50

Creating Trustworthy Data MatchingBreak Groups

Break Groups provide the ability to control the number ofrecord comparisons in the match process. This is importantfor a couple of reasons:

Speed

Optimal processing time is achieved with many small breakgroups; however, valid matches may not be identified if breakgroups are too small.

Match quality

Optimal match quality is achieved with fewer and larger breakgroups; however, larger break groups require more comparisonsand processing time.


51/63


[

51

Creating Break keys All records Each record belongs to a

postal code Set the break key on the first

three digits of the postal code

Records that contain 809 asthe first three digits, form thebreak group A.

Records that contain 981 asthe first three digits form thebreak group B.

Records in A1 are comparedto records in A2, but never torecords in B1 or B2.

Records in B1 are comparedto records in B2, but never torecords in A1 or A2.

Break keys and matching

80920 80909

9814498146

A

B

A1 A2


52/63


[

52

Break group records


53/63


[

53

Creating Trustworthy Data Matching and Consolidating

Unlocks the relationshipsbetween distinctly differentsets of data by:

Householding data

Creating a panoramicsingle best record

Providing identity

resolution to uncovernon-obvious relationshipsfor fraud detection


54/63


[

54

Creating Trustworthy Data Matching and Consolidating

Input records Consolidated record

M. Bernard Martin1, place des Saisons92083 Paris-la-Dfense CedexFrance

B MARTINAXA France AssuranceTour AXA1, place des Saisons92083 CourbevoieFRANCE

M. MARTINAXA1, place des Saisons92083 Paris-la-Dfense+33 (0)1 47 74 10 0123/7/2003

Nom: M. Bernard MARTIN

Societe: AXA France AssuranceDate dachat: 23 Juillet 2003Adresse: 1, place des SaisonsCourbevoie92083 Paris-la-DfenseCode Postal: 92083Telephone: +33 (0)1 47 74 10 01

Adresse e-mail:[email protected]
mailto:[email protected]:[email protected]


55/63


[

55

Creating Trustworthy Data Matching Records

Duplicate records often exist in one or more source systems. The goal of

matching is to determine whether records refer to the same entity. Thisinvolves evaluating how well the individual fields, or attributes, of records

match each other.

SAP BusinessObjects Data Servicesemploys powerful matching algor ithmsto account for data entry errors,

character transposition, and other dataerrors to match records.

These three records have beendeemed matching records basedoff of the business rules you define

in the matching process with SAPBusinessObjects Data Services


56/63


[

56

Creating Trustworthy Data Populate missing values

Once matches have been identified, data from these match groupscan be salvaged and posted to form a single best record or postedto update all matching records.

Master records or fields can be defined with theconsolidation process to align with your business rules.For example, you could decide to post the mostcurrent phone number to all three records.


57/63


[

57

Mr Michael Schmidt Ph.D.HyperTech Inc* 330 East 63 Avenue Suite 10BNew York NY [email protected] 23, 2003; E3 Stamping Machine

Name: Mr. Michael Schmidt Ph.D.

Company name: HyperTech Inc

SSN: 001-12-4367

Address: 330 E 63 rd St Ste 1000City , State, ZIP: New York , NY 10065-7706

Latitude: 40.722970

Longitude: -74.005035

Phone: (222) 922-9922

Email: [email protected]

Purchase history:

5/23/03; E3 Stamper, $1,300,000

10/21/04; A1 Injector, $520,000

6/30/05; C2 Fabricator, $23,000,000

Input r ecord var iat ions

Complete/Consolidated master record

* Mike SchmidtHypertech Corp330 S 63rd St # 1000Manhattan, NY [email protected]; Fabrication Facility class C

Mr. Mick SchmidtHype_Tech330 E 63rd Road* New York NY 10065001-12-4367(222) 922-992210/21/04; Victory Injection Molder

Data Quality: Matching & Merging - Using Cleansed Datato Identify Potential Duplicate Records
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


58/63


[

58

Process Flow for Integrating Master Data - LeveragingSAP BusinessObjects Data Services

Using SAP BusinessObjects ETL andData Quality capabilities

Sequence of t his process

Extract master data from remote systems using SAP

BusinessObjects Data Services

Cleanse data, match duplicates and deliver to MDM ImportManager

Map data using MDM Import Manager and send to MDM Server

Persist data in MDM repository

Access data using MDM Data Manager for post-processing (orSAP NetW eaver Portal)

11

22

33

44

55

MDM Server

MDMImport

Manager

MDM Data Manager

55

44

33

11

MDM

SAP BusinessObjects

Data Services

22

Third party source systems

Import


59/63


[

59

Integrating SAP BusinessObjects Data Services withSAP NetWeaver MDM

Prior to loading data into SAP MDM:Extract master data from client systems

using predefined extractorsAnalyze the quality of the extracted dataCleanse and correct master data

Identify duplicate recordsGenerate best record based on duplicate

records and survivorship rulesUse the SAP MDM Import Manager to load

the best record and link the duplicaterecords based on the matching ID

ERP ORCL SEBL

ABC12

Margret

Smith-Klein

ABC12

MargretSmith-

Klein

12345

Maggie

Smith

12345

MaggieSmith

678DE

Peg

Klein

678DE

PegKlein

ABC12MargretSmith-

Klein

ABC12

MargretSmith-

Klein

12345MaggieSmith

12345

MaggieSmith

678DEPegKlein

678DE

PegKlein

BestRecordMargaret

Smith-Klein

BestRecordMargaretSmith-Klein

MDMID 789

Match ID 2Margaret

Smith-Klein

MDMID 789Match ID 2

Margaret

Smith-Klein

ERP

ABC12

Match

ID 2

ERP

ABC12

Match

ID 2

ORCL

12345

Match

ID 2

ORCL

12345

Match

ID 2

SEBL

678DE

Match

ID 2

SEBL

678DE

Match

ID 2

= =

= + +

1 2 3 4

FindDuplicates

Generate Best Record

Import into MDM based onMatch ID

MDMID 789 -> ERP ABC12MDMID 789 -> ORCL 12345MDMID 789 -> SEBL 678DE


60/63


[

60

Added Value of Data Services with MDM

MASTER DATACONSOLIDATION / INITIAL

LOAD:Customers want an efficient,reusable mechanism to load

pristine data into MDM

Data Services enables cleansing, matching& consolidation for initial and delta loadsto MDM

Data Services supports rule-based auto-merge and survivorship

CONTROLLED DATAGOVERNANCE AND DATA

QUALITY REPORTING:Customers want to prevent

duplicate entries from enteringtheir systems, at the source

Data Services provides data cleansing andmatching services for data maintenanceon MDM

Business rules are checked within Portal iViewsbefore records are committed

PERIODIC CLEANSE:Customers recognize that aperiodic comprehensive cleanse

of their data may be necessary, tocatch hidden incomplete entriesand to update all with the most

recent address information

Data Services supports periodic cleanse,match and consolidate of the MDM repository

Export, process and re-import the MDMrepository via the enrichment adapter


61/63


[

61

1. Data

READINESS

2. Data

INTEGRATION

& CLEANSING

3. Data

CONSOLIDATION

Data Quality

MDG

MDM

Data Quality

MDM

Data Quality Data Quality

People & Process Maturity

Value

Data Integrator Data Integrator Data Integrator

Trustworthy data with SAP BusinessObjects Data Services

Paths to Master Data

4. DataGOVERNANCE

Conclusion:Data Services is the foundation for trustworthy master data.


62/63


[

62

Key Learnings

Data is spread throughout systems and applications in the enterprise

Data quality issues are everywhere in the information supply chain

SAP BusinessObjects Data Services should be used to create trustworthy

master data

Make sure you understand your data well by profiling your source data

Standardize, cleanse, and enrich data before loading into target applications

Avoid duplicate data - match and consolidate to get one golden record


63/63

[

[

] Thank you for par t icipat ing.

SESSION CODE:

307

Please remember to complete and return yourevaluation form following this session.

For ongoing education on this area of focus, visit the Year-Round

Community page at www.asug.com/yrc
http://www.asug.com/yrchttp://www.asug.com/yrc

asif pradhan_307 asug presentation

Documents