uccnet: one version of the truth - ittoolboxhosteddocs.ittoolbox.com/jo021805.pdfsynchronization...

UCCnet: One Version of the Truth

A Data Synchronization White Paper

Copyright © EXTOL International, Inc. 2004. All rights reserved.

UCCnet: One Version of the Truth A Data Synchronization White Paper

Part 1. Introduction ………...………………………………………………………………………….. 3 Part 2. Data Synchronization Fundamentals …………………………..…………………………… 3 Part 3. Defining “One Version of the Truth” ………………………………………………………… 4 Part 4. Bridging the Data Synchronization Gap ……………………………………………………. 5 Part 5. Why Does Data Synchronization Require a Local Catalog?.…………………………….. 7 Part 6. Managing Systems of Record ………………………………………...…………………….. 9 Part 7. Data Synchronization Architecture ……………………………….…..……………………..10 Part 8. Integration Use Cases ………………………………………………………………………..11 Part 9. Choosing an Integration Strategy ..………….……………………….……………………. 16 Part 10. Choosing a Data Synchronization Solution ...…………………………………………… 16 Part 11. UCCnet Integration Solutions from EXTOL ...…………………………………………… 18

EXTOL International, Inc. 795 Franklin Avenue

Franklin Lakes, NJ 07417 (201) 847-1200 www.extol.com

Introduction

As part of the Global Data Synchronization Network (GDSN), UCCnet defines standards that enable trading partners to exchange data about products, business locations, pricing, and other entities referenced in electronic commerce systems. By synchronizing data before engaging in electronic business transactions – in effect, establishing “One Version of the Truth” between trading partners for fundamental trade information, businesses can reduce shipment errors, returns, charge-backs, out-of-stock conditions, error corrections, and other problems that cost time and money and erode margins.

But before businesses can engage in external data synchronization, complete, accurate source data must be present on the supply side. This raises a second data synchronization challenge: how to establish data consistency across enterprise application systems that create and maintain the source data, within supply-side companies. In effect, suppliers must first establish “One Version of the Truth” across their back- and front-office applications, and only then engage in synchronization with their customers.

A study conducted by A.T. Kearney concluded that data synchronization investments return an average of one-tenth of 1% of annual revenues on both the supply and demand side. So, for example, a $500M company might save $500,000 annually, by engaging in data synchronization. On top of that, by implementing e-collaboration processes such as EDI or CPFR on top of a synchronized base of data, the study projected an additional annual return on investment of between 2% and 3%. At this level, a $500M company might experience returns on the order of $10M a year, or more. Such benefits are achievable only if the foundation for data synchronization – complete, accurate,

internally consistent source data – is put in place first.

Data Synchronization Fundamentals Figure 1 illustrates the basic ingredients of data synchronization. These include, first of all, standards that establish a common way to identify and describe trade items, business locations and other data used in electronic commerce. The Global Standards Management Process (GSMP) sponsored by GS1 (formerly EAN/UCC) has defined a standard for trade item identification, the Global Trade Item Number (GTIN®), and another for location identification, namely, the Global Location Number (GLN), as well as standards for describing trade items, locations, and other electronic commerce data.

To support data synchronization, information is represented using the EAN.UCC XML message standard. Trading partners synchronize data by sending and receiving XML messages across an Internet-based network infrastructure (the “GDSN”) that consists of a central Global Data Synchronization Registry (GDS Registry) and a collection of “Data Pools” (UCCnet, WWRE, ECCnet, and Transora are examples) which mediate communications between end points and the GDSN. Companies interact with UCCnet and other data pools using a “synchronization services” protocol to register, publish, query, and perform other operations on trade data. At the communication level, the AS2 Communication standard provides transport services to send messages into and across the GDSN.

The infrastructure depicted in Figure 1 enables suppliers to not only register trade items, but also to describe in great

Figure 1: Global Data Synchronization Network (GDSN) Infrastructure

GDS Registry

RecipientData Pool

SourceData Pool

SupplierSupplier CustomerCustomerCustomer

AggregateAugmentValidateSynchronize

MaintainSynchronization

statusSubscribe to itemsRequest publications

Get published item dataConfirm / reject items

Transmit item descriptions

Register items Locate items

CRMCRMOrderOrderMgmtMgmt

WhseWhseMgmtMgmtERPERP

ProductProductMasterMaster

detail the items that they sell, to publish information about items to their customers, to withdraw items with they become obsolete and also to retrieve confirmations and notifications from trading partners about items in which they have interest. On the demand side, buyers can subscribe to items of interest, request trade item publications from suppliers, and respond to supplier publications, among other things. Defining “One Version of the Truth” As the sources of commercial goods, supply-side companies are responsible for ensuring the accuracy and consistency of product data introduced to the GDSN. Given the variety of data locations, formats, and change events involved in designing, producing, packaging, selling, and moving products, establishing a reliable basis for synchronization – “One Version of the Truth” – can be a daunting challenge. As a supply-side company, how can you ensure that you really do have “One Version of the Truth”, before you engage in data synchronization with trading partners? There are three factors to consider:

1. First, your source data must be complete and accurate. In most companies, a significant data inventory and rationalization effort must be undertaken before synchronization can occur, particularly for trade item data, which can be voluminous. Physical measurements of product dimensions, weights, packaging and other item attributes may be needed, not just to verify recorded information, but also to supply data that is not maintained in your application systems.

2. Your data must be valid according to semantic integrity rules defined by the EAN.UCC XML and GSMP content standards. These rules govern what information must be present and what values are allowed (whether enumerated, referenced, or declared) for each message type and in each synchronization context. This requirement goes beyond the syntactic transformation of data from internal database and file representations to the XML used for communication with UCCnet.

3. Finally, your source data must be internally consistent before you


synchronize with trading partners. In most companies, there are multiple, different sources of trade item data stored in different application systems – Engineering, Manufacturing, Sales, Distribution, etc. The fact that these sources are inconsistent doesn’t necessarily mean that some versions are wrong. Data describing a new engineering change or a packaging change, for example, may not be reflected in the description of a currently sold product. Which version or which combination of versions is the right source for data synchronization is an important question to answer before you share data with your trading partners.

The main goal of external data synchronization is to ensure that data used in e-commerce and other e-collaboration applications is consistent between supply-side and demand-side partners. In order to achieve that goal, the processes that maintain source data in ERP, Warehouse Management, CRM, and other application systems must somehow integrate with each other and with e-collaboration processes. So an internal change driven by a new product introduction or a modification to an

existing product, package, or price drives corresponding data synchronization changes, both internal and external, before the updated information is used in an EDI transaction. Bridging the Data Synchronization Gap

Figure 2 illustrates how data synchronization bridges the gap between systems that create and maintain data and e-collaboration applications that use the data. Bridging the gap requires integration in two directions. On one hand, we need to integrate data synchronization processes with e-collaboration processes, so that synchronized trade information is used in external business-to-business (B2B) interactions. On the other hand, we need to integrate data synchronization processes with Product Data Management and other enterprise processes that create and maintain trade item data, to ensure that the information we synchronize is complete, valid, and consistent. Figure 3 drills down on the data synchronization circle in Figure 2, showing how synchronization processes integrate with Product Data Management and e-collaboration applications:

Product DataManagement•Engineering•Manufacturing•Packaging•Warehouse Mgmt•Marketing•Sales•Order Management•Shipping

Item DataSynchronization•Standard identifiers•Standard descriptions•Data rationalization•Item Registration•Item Publication•Internal trade itemdata synchronization

e-Collaboration•EDI•VMI•CPFR•Outsourcing•Private labeling

External Synchronization

Internal Synchronization

Figure 2: Bridging the gap between product data management and e-collaboration


The icons in the bottom row represent the key external synchronization processes for supply-side companies, including item registration, cataloging (providing additional descriptive information about trade items), publishing information to buy side partners, notification, and receiving partner confirmations.

The local trade item catalog in the center provides a complete, valid, and consistent store of data for use in the data synchronization process. It also provides a basis for integration to and from back-end application data and processes. We’ll examine the local catalog later, in greater detail.

The numbered arrows at the top represent the integration points with product data management processes on the one hand and e-collaboration processes on the other.

There are four integration points depicted in Figure 3:

1. The synchronization process begins at the top left ( ), where product data is extracted, transformed, and aggregated from existing application sources into a staging area, and then validated and imported into the local catalog.

2. Once the local trade item catalog is loaded, we need to ensure that, as changes occur to back-end application source data, consistency is maintained with corresponding data in the local catalog. This integration process is represented by the curved arrow on the left ( ).

3. After external synchronization occurs through the process sequence shown at the bottom, customer responses, which reflect demand-side decisions that govern subsequent e-collaboration activities, must be synchronized with the catalog, and then with enterprise applications.

TradeItemsTradeTradeItemsItemsTradeItemsTradeTradeItemsItems

Import

Augment /Maintain /Analyze

LocalTrade Item

Catalog

LocalTrade Item

Catalog

Approve

IntegrateIntegrateIntegrate

ManageManageManage

Integrate withLifecycle

Processes

Integrate withLifecycle

Processes

StagedImportData

StagedImportData

StagedImportData

StagedImportData

Applica-TionData

Applica-TionData

Applica-TionData

Applica-TionData

Extract / Transform / LoadLocal Catalog Data

Extract / Transform / LoadLocal Catalog Data

IntegratePartner

Responses

IntegratePartner

Responses

Inventory Trade Item DataInventory Trade Item Data

SynchronizeSynchronizeSynchronizeRegisterRegister ConfirmConfirmCatalogCatalog NotifyNotify

Data Pool Synchronization ServicesData Pool Synchronization ServicesData Pool Synchronization Services

Integrate with e-Commerce /Collaborative Commerce

Integrate with e-Commerce /Collaborative Commerce

PlanPlanPlan

Approve Approve Approve

PublishPublish

Figure 3: Integrating data synchronization processes


This integration process is depicted by the curved arrow on the right ( ).

4. The final arrow at the upper right ( ) represents the integration of synchronized data with EDI and other e-collaboration applications.

Collectively, these integration points address all of the integration requirements for data synchronization in a supply side company. The first integration point, which produces an initial set of data from existing application sources, is required in all cases. The second integration point, which maintains the currency of local catalog data, is required in any case where trade items, packaging, pricing, or location data have a chance of changing (the vast majority of situations). The third integration point, which connects applications with customer responses to data publications, is required in all cases. And the fourth integration point, which ensures use of synchronized data in subsequent e-collaboration processes, is also required in all cases.

Somehow, every supply-side company must implement all of the integration points depicted in Figure 3. Otherwise, the data it incorporates in e-collaboration activities (e.g., EDI) has no chance of being correct or complete. The only choice in this matter is how to implement integration. It can be implemented through manual procedures, through custom programming, or by employing integration technology. How a company chooses to implement these integration points is a critically important element of its data synchronization strategy.

Why Does Data Synchronization Require a Local Catalog? It’s clear from the discussion above that some integration requirements surface because of the need to create and maintain a local catalog that is separate and distinct from related application

stores. But if a supplier already has all the information needed for data synchronization in its product master or other application databases, why does it need a separate catalog for data synchronization? There are five main reasons:

1. Data aggregation: In a typical enterprise, the source data required for data synchronization usually exists in multiple locations, even when a product master is in use. Aggregating data from all sources into a single location – the local trade item catalog – ensures that data synchronization will occur using One Version of the Truth. The local catalog provides a single, consistent, consolidated, aggregated description of the trade item information needed for data synchronization.

2. Data augmentation: Few enterprises, if any, maintain all of the data attributes needed for data synchronization in their back-end application systems. In the typical case, a supply-side company will be able to identify about half of the attributes that they need from internal sources. Other data may be derivable from some combination of internal and external sources. But there will almost always be missing data. The local catalog provides a place to put data that doesn’t exist in other internal data stores.

3. Data validity: Data synchronization depends on the application and enforcement of standards for item and location identification and description. The EAN.UCC XML and GSMP standards define most of the rules that govern data validity for synchronization purposes. Existing application data sources do not comply with these validity rules, so they must be transformed and scrubbed before synchronization can occur. By implementing a schema that enforces synchronization-specific


domain integrity and referential integrity rules, the local catalog enforces data validity.

4. Data isolation: Only a part of the product, pricing, and other data maintained in a typical supply-side company is used in e-collaboration processes, and even then, only at the right time. Your company may want to share some product, pricing, or packaging changes with partners, while keeping other data – a proposed engineering change, for example – private. After the engineering change is promoted to manufacturing, you may decide to share the updated product information with your partners.

5. So another function of the local trade

item catalog is to isolate data that is ready for synchronization from data in other lifecycle stages, to prevent inconsistencies due to ongoing changes in back-end applications.

6. Data synchronization workflow management: As item registration,

cataloging, publishing, and other data synchronization processes ensue, information about the synchronization state of each trade item must be maintained somewhere, to enable enforcement of rules governing synchronization processes. For example, an item cannot be cataloged or published unless it is first registered. The local trade item catalog gives us a place to store synchronization status and history information for every trade item.

At a more abstract level, the local trade item catalog implements a system of record for data synchronization, that is independent of the multiple locations, versions, formats and content standards

present in various back-end applications. We examine the system-of-record concept in the next section. But the basic idea is that, by providing a data store and associated management application that is specifically designed to support external data synchronization, the local catalog establishes One

LocalTrade Item

Catalog

LocalLocalTrade ItemTrade Item

CatalogCatalog

Back EndApplications:• ERP• Mfng• Eng / PIM• CRM• Sales• WMS• Distrib’n• Other

Back EndBack EndApplications:Applications:•• ERPERP•• MfngMfng•• Eng / PIMEng / PIM•• CRMCRM•• SalesSales•• WMSWMS•• Distrib’nDistrib’n•• OtherOther

ApplicationData

ApplicationData

ApplicationData

ApplicationData

CustomerCustomerCustomer

Firewall

Firewall

AggregatedData

DataPoolDataDataPoolPool

Data / ProcessData / ProcessIntegrationIntegration

Trade ItemTrade ItemDataData

InventoryInventory

Missing Data

ChangingData

Invalid Data

State Management Data

Figure 4: Why a local catalog is essential for data synchronization


Version of the Truth for trade item information, to the outside world.

Managing Systems of Record By itself, of course, the local catalog cannot guarantee that the One Version of the Truth you exchange with partners is consistent with the information maintained in your enterprise applications. As we’ve already observed, ensuring such consistency requires integration between the data synchronization application and enterprise application systems that maintain product information. And in order to reap the benefits of data synchronization, we know that EDI and other e-collaboration applications also must be integrated with synchronized data, as illustrated in Figure 3.

The key challenge to consistency between application source data and the local catalog is data redundancy. Most of the information in the local item catalog is present in some form in one or more enterprise application systems. Other data is present only in one place or another. For example, data captured early in the product life cycle, such as the

proposed engineering change mentioned earlier, may not be appropriate for synchronization with partners. Conversely, some data required for the data synchronization process itself, like synchronization state data, may not be present in any existing application and must be maintained in the catalog.

Typically, the majority of synchronization source data is maintained by some combination of ERP, Warehouse Management, CRM, and other enterprise applications. These are the systems of record for the information that they each maintain. The key point is that different trade item attributes may be maintained in different systems of record. For example, core product metrics may be maintained in a manufacturing system. Packaging data may be maintained in a warehouse management or distribution system. Pricing data may be maintained in a CRM system. All of this data is needed for data synchronization. So the challenge really is to maintain consistency between the local trade item catalog and multiple systems of record for source data, based on which attributes we need for data synchronization. Another way to say


LocalTrade Item

Catalog


CatalogCatalog

CustomerCustomerCustomer

Firewall

Firewall

ApplicationData

ApplicationData

ApplicationData

ApplicationData



Trade Item DataMaintained in

Application Systems

Trade Item DataRequired for

Synchronization

System of Recordfor Data

Synchronization

No Default System of Record(more than one version of the truth)

System(s) of Recordfor Product Lifecycle

Management

Lifecycle ProcessIntegration

Figure 5: Reconciling systems of record for trade item data


this is that we need to be able to reconcile copies of trade item data that are maintained in more than one place. Figure 5 illustrates this challenge conceptually. The shaded intersection

between enterprise application data on the left and synchronization data on the right represents trade item attributes that occur in both places. For each trade item attribute that resides in this intersection, you must decide what the system of record will be. In most cases, the logical decision will be to continue to maintain this data in existing enterprise application systems. But if those systems are incapable of enforcing validity for data synchronization purposes, you might decide, instead, to maintain some attributes in the local catalog. In either case, we need some means of updating secondary stores as changes occur in systems of record, so that consistency is maintained across multiple attribute copies.

Data Synchronization Architecture The diagram in Figure 6, below, shows the fundamental architecture of a data synchronization solution that is capable

of maintaining consistency across systems of record: Synchronization Management: The

Synchronization Manager

implements bi-directional message-based communication between the supplier organization and a data pool (e.g., UCCnet). This includes marshalling and validation of messages, communications handshaking for supply-side synchronization commands, synchronization state management, communication event notification, and synchronization logging.

Catalog Management: In the middle, the Catalog Manager implements business user functions for trade item maintenance and synchronization management, governed by appropriate access controls. This includes interactive data viewing and editing (if authorized), validation, initiation and approval of synchronization tasks, and processing of synchronization queries and responses.


Data Synchronization Application

Firewall

Firewall

Data / ProcessData / ProcessIntegrationIntegration

CatalogCatalogManagementManagement

SynchronizationSynchronizationManagementManagement

AuditLog

AuditAuditLogLog

Integration ServicesIntegration Services

LocalTrade Item

Catalog


CatalogCatalog

IntegrationIntegrationProcessesProcesses

...

IntegrationIntegrationProcessesProcesses

...

BusinessProfessional

ITProfessional

AS2AS2CommsComms

IntegrationToolset

Data SyncApplication

ApplicationData

ApplicationData

ApplicationData

ApplicationData



Figure 6: Data synchronization application architecture


Data / process Integration: On the left and bottom middle, we see the components needed for integration of the data synchronization application with enterprise applications and business processes. This includes a set of integration services, depicted by the gray rectangle at the bottom, plus functions for data import/export, event notification, and system administration. Also, an integration tool set is required in order to specify custom integration processes that connect enterprise applications and the data synchronization environment.

Integration services and tools must be present as part of any data synchronization solution because of the unique combination of business processes, applications, data, and business rules present in every enterprise. These factors govern the details of integration between enterprise systems and the data synchronization application environment, and rule out the possibility of off-the-shelf, “plug-in” integration approaches.

Integration Use Cases We’ve seen that achieving “One Version of the Truth” requires integration between the data synchronization application environment (especially the local trade item catalog) and enterprise applications that create and maintain product data. The primary context for understanding how this integration works is the product lifecycle. As business events occur – new products are introduced, product-packaging changes, new pricing terms are negotiated – data synchronization activities must occur to support them. For example, introduction of a packaging change requires that the packaging description in the local catalog be updated from some back-end system of record (application integration), and that customers who carry the associated

product be notified of the change (data synchronization).

Implementing integration for data synchronization requires an understanding of the product lifecycle process context – chiefly, which business events drive data synchronization content changes and notifications, and what process steps must be executed when each event type occurs. It also requires a thorough understanding of the data in both product data management and data synchronization systems of record, to ensure that data transformations and validity rules are applied correctly in each context.

The figures in this section illustrate the most important integration use cases for data synchronization:

Initially loading enterprise application data into the local trade item catalog

Periodically updating the local catalog with accumulated application system changes

Triggering an immediate local catalog update when an application data change occurs

Triggering application updates and business processes when synchronization events occur

In each case, supply-side companies can choose from a variety of integration implementation approaches. For example, a business professional could manually re-key information from existing applications into the local catalog to implement the initial data load. Or s/he could write program code to extract, transform, and validate data from each source application and update the local catalog. Or s/he could employ general-purpose integration technology – the approach we will examine in this document. The key insights here are (1) you must address integration somehow, and (2) the approach you take to integration will determine, to a significant degree, the costs and benefits you


realize from your data synchronization investment. Figure 7 shows the main components of an integration process that loads data from back-end systems of record into the local trade item catalog. For now, we’ll focus on the functional requirements for this and the other integration use cases, rather than the implementation details.

The main components of the initial catalog load process are:

1. Manual process launch: The timing of the initial load process is determined primarily by the readiness of the source data, not by a business event or schedule. So all we need in this case is a way to launch the load process manually.

2. Load process script: There are just two steps in our load process use case: (1) extracting / transforming / loading data from a system of record into a staging database; and (2) notifying a business user when the process completes. The purpose of

the staging database is to provide a “halfway house” for source data, so that data from multiple systems of record can be aggregated and reconciled before it is imported into the local catalog.

3. Transformation rules: The focus of the initial catalog load process is a transformation step that reads source data from a system of record, transforms it to comply with local

integrity and validity rules, and updates the staging database.

4. Source and target “schema” definitions: In order to interpret application data and produce appropriate result data in the staging database, the transformation step requires “schemas” (document definitions) that describe the content and type information for both sources and targets. Ideally, schemas can be generated from existing metadata, as in this case, from the source and target DBMS catalogs.

5. Resource adapters: Adapters are general-purpose interfaces to persistent data, communications


WhseWhseMgmtMgmt

InputDocu-ment

OutputDocu-ment

Adapter Adapter

IntegrationProcess

Transformation

Schema

eMaileMail

ERPERP


StagingStagingDatabaseDatabase

LocalLocalCatalogCatalog

Launch

Meta data

Enterpriseproduct data

Standard-complianttrade item data



ITProfessional

(import)Target Schema

Figure 7: Initial loading of application data into the local catalog


services, database management services, and other resources needed by integration processes. In the initial load process, database or file adapters provide access to source and target databases for the transformation step.

6. Import utility: Upon receiving the email notification that the staging database has been populated by the initial load process, a business user or system administrator invokes an import utility that validates the data and, if valid, loads it into the local catalog.

Figure 8 shows a process that looks very similar to Figure 7, but implements a different function, namely, updating the catalog periodically as source data changes occur in back-end systems of record.

Implementing a catalog update process is important in any supply-side company where changes occur in data synchronization source data. Examples of such changes include:

Introduction of new products (including seasonal introductions)

Changes to product attributes (colors, styles, dimensions, weights, etc.)

Changes to product packaging (including case and pallet configurations)

Changes to product pricing (including customer-specific terms)

Withdrawal of obsolete products

Once data that describes such changes is captured in a system of record, corresponding data in the local data synchronization catalog must also change, and, from there, the changes must be synchronized externally with customers. Figure 8 illustrates the components of a process that automates this kind of change process, based on

periodic extraction of change data from back-end systems of record.

The main differences between this process and the initial data load process shown in Figure 7 are:

Launch / Schedule

InputDocu-ment

OutputDocu-ment

Adapter Adapter

IntegrationProcess

Transformation

Source SchemaStagingStaging

DatabaseDatabase


Meta data




ITProfessional

Flat FileFlat FileExtractExtract





(time stamped)(import)Target Schema

(pre-extracted)

eMaileMail

Figure 8: Periodically updating the local catalog


1. Scheduled process launch: On the assumption that product data changes continually in back-end applications, the update process runs on a cyclic schedule. The update frequency may depend on source data volatility, batch window availability, seasonal product introduction cycles, or other factors.

2. Adapter configuration: Although the data transformation step is essentially identical to the one depicted in Figure 7, the source adapter in this case must be configured to access only data that has changed since the last time the update process ran. This can be accomplished by either (1) querying or filtering source data using time stamp criteria; or (2) periodically creating a flat file extract that contains only changed data.

Whether the update process also automates the external synchronization of changes with customers depends on the business process management policies implemented by the supplier. In some cases, approval of the external synchronization step may occur prior to exporting change data to the update process, so the need for a separate synchronization approval step is absent. In other cases, a separate

synchronization approval step may be desirable to ensure that any exceptions raised by the update are reviewed, prior to synchronization. Note that, due to the similarity between our initial catalog load process (Figure 7) and catalog update process (Figure 8), there are opportunities here to reuse integration components – transformation rules and schema definitions, for example. Figure 9 shows an alternative catalog update mechanism, using triggers to automate the detection of changes and invocation of the update process in near-real time. The term “near-real time” means that source data updates and triggered catalog updates occur asynchronously, but with the appearance of simultaneity. The advantage of this approach is that local catalog data remains consistent with source data in back-end systems of record, on a continual basis.

The main differences between this process and the processes examined above are:

1. Triggered process launch: In order to maintain near-real time consistency between application source data and local catalog data, neither manual nor scheduled process launching suffices. Instead, processes are

InputDocu-ment

OutputDocu-ment

Adapter Adapter

IntegrationProcess

Transformation

Schema

StagingStagingDatabaseDatabase


Trigger

EventEventMonitorMonitor

Meta data




ITProfessional





(import)Target Schema

eMaileMail

Figure 9: Triggered catalog updating in near-real time


launched automatically by an event monitor, which is notified by a database stored procedure that triggers on application data changes.

2. Process selection: Based on notification data passed by the stored procedure, the event monitor launches an update process that is designed to transform and update the type of data maintained by a given system of record.

3. Transformation granularity: Each process instance transforms a single input data instance, rather than a batch of source data changes.

One final difference between the triggered and periodic update processes is that immediate synchronization of changes through the data pool is not practical, in the triggered case. The reason is that synchronization latency – the time it takes for the data pool to process synchronization commands and return responses – is unpredictable. Rather than flood the data pool with a high volume of triggered synchronization requests, it makes more sense to queue synchronization requests locally, and process them on demand, or on a cyclical schedule.

Figure 10 shows the final application integration use case, in which the processing of inbound customer responses retrieved from the data pool triggers changes to back end application data. When a customer confirms interest in an item you have published, for example, the receipt of that confirmation is an important business event. Although it falls short of an order transaction, it is potentially a buy signal, and should trigger appropriate human and automated responses.

Exactly what business response is appropriate will depend on the policies and processes established by each supplier. Perhaps we might want to update data about the customer in our CRM data base, or update an order management system to allow orders for that product to be accepted from the customer, or notify a salesperson or customer service agent. Or all of the above. At the process automation level, in addition to transforming inbound responses and related catalog data into application inputs and notifying relevant business professionals, we might want to launch application programs or scripts, to exploit existing business logic.

Figure 10: Triggering application updates from synchronization events

OutputDocu-ment

InputDocu-ment

ProgramProgram ScriptScript

Adapter Adapter

IntegrationProcess

Transformation

DeploymentConfigurationSchema

eMaileMail


Trigger

EventEventMonitorMonitor

Meta data

Customer responsesand other significant

event data



ITProfessional

Enterprisedata updates





The main differences between this process and the processes examined earlier are:

1. Trigger source: Instead of triggering based on update events in back-end application databases, this process triggers on an inbound communication event.

2. Transformation direction: Instead of transforming application source data into equivalent catalog content, this process transforms inbound confirmation data (and possibly related data from the local catalog) into corresponding application update content.

As with the other use cases examined above, the same integration components and reuse principles apply in this case, as well.

Choosing an Integration Strategy As observed earlier, integration between the data synchronization environment and enterprise applications is fundamental to achieving data consistency – “One Version of the Truth” – and realizing full benefit from your data synchronization investment. The question is not whether, but how you implement this integration in your company.

Here are some questions to consider before you choose an integration implementation approach:

Data volume: Is the number of items to be entered, maintained, and synchronized manageable without automation?

Data quality: Do you need to automate interfaces to ensure that the data you need to synchronize is correct and consistent with internal systems, or do you have the necessary procedures and checks in place to do this manually?

Process latency: Is the need for visible, rapid response to key business events sufficient to justify automation?

Training and expertise: For data that is entered manually, do your business professionals understand the source application data and its relationships to data synchronization attributes? For programmed integration, do you have the necessary trained staff to implement all required process and application interfaces, and maintain them, as changes occur?

Maintainability: For programmed integration, how will you maintain your code when changes occur to business processes, source and target database schemas, data integrity rules, commercial and in-house applications, runtime platforms, etc.?

Environment dependence: For programmed integration, how will you manage the impact if hardware, operating systems, DBMS, communications, or other factors change?

Business goals: To what degree is your integration strategy driven by corporate goals to reduce costs, improve customer service, improve quality, or gain competitive advantage? Which business goals and objectives are achievable without automated integration, and which are best realized through a systematic integration approach?

Choosing a Data Synchronization Solution What first appears to be a very simple idea – synchronizing product, location, pricing, and other trade information with customers – is actually a set of core business processes that nest between systems that maintain enterprise data


and systems that use that data in electronic commerce and other e-collaboration applications. Although the simplicity of file transfer and spreadsheet-based offerings may be enticing, the reality is that integration is the key to unlocking the full business benefits of data synchronization.

Here’s a list of capabilities to consider as a starting point, when choosing a data synchronization solution for your company:

Local Trade Item Catalog: Full local catalog schema support

for EAN.UCC data synchronization message standard

Ability to manage and operate on large numbers of trade items with good response times

Extensible to support maintenance of company-standard product identifiers on trade items

Extensible to support additional customer-defined attributes

Support for multiple test and production catalogs

Support for migration of data from test to production

Support for source data loading from multiple data sources

Validating importer for loading data from external sources

Catalog Management: Access controls for all application

functions and local catalog contents

Concurrent, multi-user application support

Role-based restrictions on data access and modification (e.g., based on data ownership)

Complete user interface support for local entry, validation, viewing,

and editing of trade item, location, pricing, and other data

Interactive controls for registering, cataloging, publishing, and other data synchronization functions

Supervisory approval interface for all synchronization activities

Business-oriented user interface with visual cues for process sequence, data completeness and validity

Configurable user interface labels for customized terminology

Global constraints to prevent modification of local catalog attributes maintained in other systems of record

Instrumentation for notification and automated response to catalog management events

Synchronization Management: Certified compliance with current

UCCnet Synchronization Services release

Black box automation of all synchronization details (marshalling, communications, retry, error reporting, etc.)

Configurable synchronization frequency

Instrumentation for notification and automated response to synchronization events

Integration Support: Specification of complete

integration processes without programming

Automation of conditional, multi-step business processes

Scheduler for automated process invocation based on calendar or elapsed time


Event manager for automated process invocation based on distributed event notifications

Ability to transform data from any source syntax (file, database, XML, etc.) to any target syntax

Ability to access data remotely, on other computers and operating system platforms

Email notification support

Ability to invoke external applications and scripts / bat files

Platform-independent architecture to preserve integration requirements.

Metadata importers for generation of source and target document (“schema”) definitions

Offline integration modeling and testing

UCCnet Integration Solutions from EXTOL EXTOL International, a leading provider of B2B Integration solutions to mid- market companies, offers UCCnet solutions that address a broad range of synchronization strategies and integration requirements. From small suppliers with limited synchronization needs to large manufacturers with volatile product lines and complex lifecycle management processes, EXTOL offers a solution with the right combination of application functionality, integration power, and ease of use. EXTOL’s flagship offering for UCCnet is EXTOL Business Integrator® for UCCnet™ Services (“EBU”), a comprehensive UCCnet integration solution that includes the following features and functions:

Figure 11: EXTOL Business Integrator® for UCCnet™ Services

UCCnetDataPool

UCCnetUCCnetDataDataPoolPool

EXTOL Business Integrator for UCCnet™ ServicesEXTOL Business Integrator for UCCnet™ ServicesData / ProcessData / Process

IntegrationIntegration

CommunicationsCommunications

Business Event Business Event ManagementManagementand Routingand Routing

Portability LayerPortability Layer

Business Process Business Process Automation and Automation and

ManagementManagementContent Content

TransformationTransformationParty Party

Provisioning and Provisioning and GovernanceGovernance

AdaptersAdaptersAdaptersAdapters

LocalTrade Item

Catalog


CatalogCatalog


ITProfessional

EXTOL Integration

Toolset

EXTOL Integration

Toolset

Trade Item DataImport / Export

External Dataand Process

Synchronization

CatalogManagement

InternalItem Data

Synchronization

DataSynchronization

Management



CustomCustomIntegrationIntegrationProcessesProcesses

ApplicationData

ApplicationData

ApplicationData

ApplicationData

CatalogCatalogManagementManagement

UCCnetUCCnetSynchronizationSynchronization


A complete, local UCCnet trade item catalog designed to stage and manage all UCCnet-compliant trade item data required for synchronization with grocery, hardlines, and other retail partners.

A catalog import utility for

populating the local UCCnet trade item catalog with data extracted from lifecycle management and other enterprise applications.

An interactive catalog

management application that enables business users to maintain trade items and manage UCCnet synchronization workflow. Full UCCnet validation and workflow management are built-in.

An automated UCCnet synchronization manager that manages all UCCnet interactions transparently, using the most current versions of EAN/UCC XML messaging and UCCnet Synchronization Services.

A complete integration broker

platform and modeling environment that enables integration with ERP, CRM, PLM, EDI, and other application systems, without programming.

The power of EXTOL’s UCCnet technology derives from its integration foundation. At the heart of every EXTOL UCCnet solution is EXTOL Business Integrator®, a platform-independent integration broker that enables rapid integration with each company’s unique array of business applications and processes. Companies can exploit this integration technology to extend existing trade item management processes and systems, making them more powerful and effective, rather than implementing UCCnet as a separate appendage that

introduces data conflicts and process redundancies, and consumes scarce human resources. Beyond providing technology for customer integration, EXTOL has exploited its own integration technology in the construction of the EBU application. For example, the UCCnet synchronization “black box” uses the underlying integration platform to transform bi-directionally between local catalog data and UCCnet message payloads, enabling EXTOL to respond faster to future UCCnet changes. And EXTOL’s UCCnet application is instrumented extensively for customer integration, with support for event notifications, process triggers, and customer interfaces to local catalog data. EXTOL offers its UCCnet technology either as a stand-alone application or as a UCCnet integration solution, so companies can purchase and use only the functionality they need. Customers with a limited number of trade items can use the application by itself to populate the local catalog and rapidly achieve compliance. Customers with a large number of trade items and more complex integration needs can use the application with the companion integration platform, to achieve both compliance and deep business integration. As integration needs evolve, companies can acquire additional integration capabilities, without sacrificing their investments in UCCnet data or EXTOL software. This includes the ability to license EXTOL Business Integrator to address business needs beyond UCCnet.

uccnet: one version of the truth - ittoolboxhosteddocs.ittoolbox.com/jo021805.pdfsynchronization...

Documents