universal data element framework (udef) overview geia – dhs workshop phoenix – april 11-14, 2005...

Universal Data Element Framework (UDEF)

Overview

GEIA – DHS WorkshopPhoenix – April 11-14, 2005

Ron SchuldtRon SchuldtApril 13, 2005April 13, 2005

UDEF BackgroundUDEF Background

CALS ISG - Developed UDEFin the late 80s – early 90sIndustryRon Schuldt (Chair) - Lockheed Martin Barbara Barman (Vice Chair) - RaytheonRob Bryant - DynCorpRuey Chen - David Taylor Research CenterBob Hodges - Texas InstrumentsNeal McNamara - Analysis & Technology Inc.Bud Orlando - TRWMadelyn van der Bokke - ASECGeorge Walther - Lockheed Martin

GovernmentNorma Kornwebel - PM JCALSDinah Beres - NAWCSteve Waterbury - NASA

EIA - Applied UDEFin the mid 90s and 2002IndustryRon Schuldt (Chair) - Lockheed Martin Rick Lang - Texas InstrumentsPam Stanfield - Lockheed MartinGary O’Hara - Hughes Space and CommTony DiPerna - Ericsson CommunicationsKen McTee - Texas InstrumentsCindy Hauer - Mevatec CorpFred Bahrs - CMstat CorporationDoug Drury - ITT Federal ServicesLee LeClair - Texas Instruments

GovernmentDeborah Cornelius - US Army Missile CmdC. H. VanLandingham - NOAA Nat’l Wea Svc

Today AFEI (formerly CALS ISG) holds the Intellectual Property Rights to the UDEF

UDEF Recent BackgroundUDEF Recent Background

UDEF Global Host Request For Information (RFI) by AIA and AFEI – UDEF Global Host Request For Information (RFI) by AIA and AFEI – Oct 2003Oct 2003• Sought candidate global non-profits interested in hosting the Sought candidate global non-profits interested in hosting the

UDEF tree structuresUDEF tree structures• Responsible for managing extensions to the trees and for Responsible for managing extensions to the trees and for

conducting global UDEF training and promoting adoption of conducting global UDEF training and promoting adoption of UDEFUDEF

• RFI response submitted jointly by CompTIA, OAGi and RFI response submitted jointly by CompTIA, OAGi and Contivo – later backed away for variety of reasons but all Contivo – later backed away for variety of reasons but all three are still interestedthree are still interested

• Draft Agreement and Performance Specification developed Draft Agreement and Performance Specification developed

Once Global Host is Selected (Agreement Signed)Once Global Host is Selected (Agreement Signed)• Pilot will be conducted to demonstrate that UDEF helps Pilot will be conducted to demonstrate that UDEF helps

reduce costs of developing interfacesreduce costs of developing interfaces• Once pilot is completed successfully and the Performance Once pilot is completed successfully and the Performance

Spec requirements are satisfied, then the non-profit will be Spec requirements are satisfied, then the non-profit will be given approval to formally launch the UDEF Global Hostgiven approval to formally launch the UDEF Global Host

Problem – Global PerspectiveProblem – Global PerspectiveEach organization is attempting to set its own semantics standardEach organization is attempting to set its own semantics standard

Each must interface with organizations they do not controlEach must interface with organizations they do not control

The problem is the lack of common semantics and The problem is the lack of common semantics and schema between organizationsschema between organizations

DoD

NASA

DHS

Ship Retail

Trans

Aero

ChemElec

Banks

Other

Organization

Problem – Supply Chain PerspectiveProblem – Supply Chain Perspective

Cost of a single interface ranges from $10K to $1M depending on scope and complexity

~ N (N-1) mapping problem & expense

ContractorContractorContractorContractor

Tier-1Tier-1SupplierSupplier

Tier-1Tier-1SupplierSupplier

GovernmentGovernmentCustomerCustomer

GovernmentGovernmentCustomerCustomer

LogisticsLogisticsProviderProvider

LogisticsLogisticsProviderProvider

IndustryMembers

BusinessPartners

Supply Chain Perspective

excessive cost,

complexityand time

PublicPublicExchangeExchange

PublicPublicExchangeExchange

OEMOEMOEMOEM

CommercialCommercialCustomerCustomer

CommercialCommercialCustomerCustomer

n-Tiern-TierSupplierSupplier

n-Tiern-TierSupplierSupplier

• Today’s reality… point-to-point solutions introduce excessive cost, complexity and time

• Examples of gaps and/or inefficiencies:

- Increasing number of unique portals adding cost to suppliers

- Multiple, redundant, incompatible “portal systems”

- Incompatibilities in information exchange contribute to delay, rework, and error

• Excessive cost, complexity and time impeding supply chain agility

Problem – Internal PerspectiveProblem – Internal Perspective

<PARTNUMBER>111-222-333</PARTNUMBER>

<partNumber>111-222-333</partNumber>

<PartNumber>111-222-333</PartNumber>

<partnumber>111-222-333</partnumber>

Though semantically equal, the following are 4 different XML tag names

XBRL HR-XML

OAGIS

Other XML Standards

Legacy Data

Conflicting semantic overlaps between back-office systems

The Impact on IntegrationThe Impact on Integration

To interface two or more applications either within the enterprise or between enterprises ----

• Middleware vendors build and sell “adaptors” for interfaces between major commercial applications (i.e., SAP - PeopleSoft)

- As the version of a given application changes it typically drives the need for changes to the adaptors

• Application interface developers (data architects) spend considerable time and effort analyzing the semantics of data that need to be exchanged between systems

- Semantics analysis requires up to 35-40% of the requirements and design-time phase labor cost of building a given interface

- To minimize the semantics analysis effort typically requires support (availability) from the “experts” of each application since most applications are poorly documented

The Goal

Current Point-to-Point Approach --- n(n-1)

Global Semantics Standard Approach --- 2n

$$

Savings

GlobalSemanticsStandard

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Reduce Requirements and Design-Time Phase Semantics Analysis Time and Cost

Derived XML Schema RequirementsXML Schema “Structure” RequirementsXML Schema “Structure” Requirements• All new XML Schema used or built by an organization must conform to W3C All new XML Schema used or built by an organization must conform to W3C

XML and XML Schema specificationsXML and XML Schema specifications» http://w3.org/TR/2004/REC-xml11-20040204/ http://w3.org/TR/2004/REC-xml11-20040204/ » http://w3.org/XML/Schema#dev http://w3.org/XML/Schema#dev

• Adopt global naming convention standard – ISO/IEC 11179-5Adopt global naming convention standard – ISO/IEC 11179-5» http://isotc.iso.ch/livelink/livelink/fetch/2000/2489/Ittf_Home/PubliclyAvailableStandards.htm

• Adopt United Nations XML schema naming and design rulesAdopt United Nations XML schema naming and design rules» http://www.disa.org/cefact-groups/atg/downloads/index.cfm http://www.disa.org/cefact-groups/atg/downloads/index.cfm

XML Schema “Payload” RequirementsXML Schema “Payload” Requirements• Adopt global set of basic data representation terms and their definitions – Adopt global set of basic data representation terms and their definitions –

e.g., Amount, Date, Name, Quantitye.g., Amount, Date, Name, Quantity» Tables 8-1 and 8-3 in United Nations Core Components Technical Specification (CCTS) – ISO Tables 8-1 and 8-3 in United Nations Core Components Technical Specification (CCTS) – ISO

15000-515000-5

• Adopt global and industry XML schema “payload” standards whenever Adopt global and industry XML schema “payload” standards whenever possible before building organization unique XML schema – see example possible before building organization unique XML schema – see example list on next slidelist on next slide

Example “Payload” StandardsExample “Payload” Standards• OAGIS – Open Applications Group http://www.openapplications.org/

» Participants - ERP and middleware vendors and end users» Example payloads – RFQ, purchase order, invoice, bill of material

• HL7 - Health Care http://www.hl7.org/ » Participants – health care providers across the globe» Example payload – health records

• ACORD – XML for the Insurance Industry http://www.acord.org/ » Participants – insurance providers across the globe» Example payload – insurance claim

• XBRL – Business Reporting - Accounting http://www.xbrl.org/ » Participants – accounting firms across the globe» Example payload – financial reports to SEC

• EIA-836 – Configuration Management Data Exchange and Interoperability http://63.249.145.5/836/default.htm

» Participants – DoD and aerospace and defense industry (AIA and GEIA)» Example payload – engineering change

Derived XML Schema Requirements

XML Schema “Infrastructure Support” RequirementsXML Schema “Infrastructure Support” Requirements

• Organization adopted and internally developed XML schema must be easily Organization adopted and internally developed XML schema must be easily discovered for reuse – requires a centralized registry/repositorydiscovered for reuse – requires a centralized registry/repository

• Use Universal Data Element Framework (UDEF) as means to index Use Universal Data Element Framework (UDEF) as means to index the points of intersection between various XML schema “payloads” the points of intersection between various XML schema “payloads” and XML instances AND to improve data interoperability. UDEF and XML instances AND to improve data interoperability. UDEF directly supports and enhances an organization’s metadata directly supports and enhances an organization’s metadata management strategy.management strategy.

XML Schema “Standards” Architecture

EIA-836*XBRLACORD OthersPLCSOAGIS HL7

ISO/IEC 11179-5, ebXML CCTS, UN Naming and Design Rules

Cross Standard Semantics and Metadata Alignment – UDEF*, RDF, OWL

W3C – XML, XML Schema

….

Organization Specific Implementation Conventions (subsets & extensions)

* Not quite ready for full adoption

XML Schema “Structure” Standards

XML Schema “Payload” Standards

XML Schema “Infrastructure Support” Standards

UDEF – Similar to Dewey Decimal SystemUDEF – Similar to Dewey Decimal System

10 Main Classes

000 Generalities100 Philosophy200 Religion300 Social Sciences400 Languages500 Science600 Applied science700 Arts800 Literature900 History

500 Natural sciences510 Mathematics520 Astronomy530 Physics540 Chemistry550 Earth sciences560 Paleontology570 Life sciences580 Botany590 Zoology

530 Physics531 Solid mechanics532 Fluid mechanics533 Gas mechanics534 Sound535 Light536 Heat537 Electricity538 Magnetism539 Modern physics

537.5 Electronics537.534 Radio waves537.5342 Long waves537.5343 Short waves

1. Divided human knowledge into ten major Classes2. Subdivided these classes into ten Divisions

3. Division subdivided into ten Sections.4. These sections subdivided, etc

How to classify a book on Radio Short Waves

The Dewey Decimal Classification or DDC was invented in 1876 by a young American librarian, Melvil Dewey.

UDEF – ScopeUDEF – Scope

Transactions – Structured DataTransactions – Structured Data• Purchase OrdersPurchase Orders• Purchase Order ChangesPurchase Order Changes• Purchase Order Purchase Order

AcknowledgementsAcknowledgements• Purchase Order Change Purchase Order Change

AcknowledgementsAcknowledgements• InvoicesInvoices• Remittance AdviceRemittance Advice• Request for QuoteRequest for Quote• Request for Quote ResponseRequest for Quote Response• Shipping ScheduleShipping Schedule• Etc.Etc.

Heavily influenced by finance and Heavily influenced by finance and procurement functions and ERP type procurement functions and ERP type applicationsapplicationsRelevant open standards – X12, Relevant open standards – X12, EDIFACT, ebXML, OAGIS, RosettaNetEDIFACT, ebXML, OAGIS, RosettaNet

Collaboration – Unstructured DataCollaboration – Unstructured Data• Mission RequirementsMission Requirements• Concept of OperationsConcept of Operations• SpecificationsSpecifications• Product DesignsProduct Designs• Engineering Change ProposalsEngineering Change Proposals• Trade-off StudiesTrade-off Studies• Test ReportsTest Reports• Meeting MinutesMeeting Minutes• PlansPlans• SchedulesSchedules• PresentationsPresentations• Etc.Etc.

Heavily influenced by engineering, Heavily influenced by engineering, manufacturing and program manufacturing and program management functions and management functions and PDM/Document Management type PDM/Document Management type applicationsapplicationsRelevant open standards – STEP, Relevant open standards – STEP, PLCS, EIA-836PLCS, EIA-836

UDEF transcends the transaction and collaboration worlds

UDEF Built on Standards

Data Element Name

Object Class Term

0...n qualifiers +1 or more required

Object Class

+

Example UDEF-Based Data Element Names

Document Abstract Text

Enterprise Name

Product Price Amount

Product Scheduled Delivery Date

Engineering Design Process Cost Amount

UDEF ObjectClass List• Entity• Document• Enterprise• Place• Program• Product• Process• Person• Asset• Law-Rule• Environment• Condition• Liability• Animal• Plant• Mineral

Property Term

0..n qualifiers +1 required Property

ebXMLProperty List• Amount• Code• Date• Date Time• Graphic• Identifier• Indicator• Measure• Name• Percent• Picture• Quantity• Rate• Text• Time• Value• Sound• Video

An Instantiation of ISO 11179-5 Naming Convention and Supports ebXML

UDEF names follow the rules of English – qualifiers precede the word they modify

ISO 11179-5 Naming Convention

ISO/IEC 11179 – Data Element Concept

DataElementConcept

DataElement

ValueDomain

ObjectObjectClassClass

PropertyProperty RepresentationRepresentation

CoreData

Element

ApplicationData

Element

UDEF Maps Data Element Concepts

Creating UDEF IDsUDEF Trees

16 Object Class Trees 18 Property Trees

Entity Asset Document Amount Code… …

Order

ChangeWork Technical

t

Purchase

20 1

a b c d

Type Defined…Region …

41…

1 33 66

…

Purchase Order Document_Type Code has UDEF ID = d.t.2_33.4

See http://www.udef.org/

Example Mappings

PDM Sys APart No

OAGIS 7.1ItemX

X12 (EDI)Product/Service ID

STEP AP 203Product ID

PDM Sys BPart Num

RosettaNetProprietaryProductIdentifier

EDIFACTItem Number

xCBLPartID

9_9.35.8

UDEF Universal Identifier

Product(9)_Manufacturer(9).Assigned (35).Identifier(8)

N (N-1) mapping effort instead becomes a 2N mapping effort

Organizations cannot avoid multiple data standards** Need global semantics standard **

Mapping Concepts to UDEF

1. Identify the applicable UDEF property word that characterizes the dominant attribute (property) of the data element concept. For example, Name, Identifier, Date, etc.

2. Identify the dominant UDEF object word that the dominant property (selected in step 1) is describing. For example, Person_Name, Product_Identifier, Document_Date, etc.

3. By reviewing the UDEF tree for the selected property identified in step 1, identify applicable qualifiers that are necessary to unambiguously describe the property word term. For example, Last Name

4. By reviewing the UDEF tree for the selected object identified in step 2, identify applicable qualifiers that are necessary to unambiguously describe the object word term. For example, Customer Person

5. Concatenate the object term and the property term to create a UDEF naming convention compliant name where it is recognized that the name may seem artificially long. For example, Customer Person_Last Name

6. Derive a structured ID based on the UDEF taxonomy that carries the UDEF inherited indexing scheme. For example <CustomerPersonLastName GUID=“as.5_5.10”>

DHS Typical Example

1. Identify the applicable UDEF property word that characterizes the dominant attribute (property) of the data element concept. For example, Name, Identifier, Date, etc.

Data Element Concept Name

United States Passport Number

UDEF ID Object Type or Role Object Class Property Type Property


2. Identify the dominant UDEF object word that the dominant property (selected in step 1) is describing. For example, Person_Name, Product_Identifier, Document_Date, etc.

Identifier

DHS Typical Example




Identifier

3. By reviewing the UDEF tree for the selected property identified in step 1, identify applicable qualifiers that are necessary to unambiguously describe the property word term. For example, Last Name

Document

DHS Typical Example




IdentifierDocument

4. By reviewing the UDEF tree for the selected object identified in step 2, identify applicable qualifiers that are necessary to unambiguously describe the object word term. For example, Customer Person

DHS Typical Example



State DepartmentGovernment Assigned


IdentifierDocument

5. Concatenate the object term and the property term to create a UDEF naming convention compliant name where it is recognized that the name may seem artificially long. For example, Customer Person_Last Name

United StatesPassport

DHS Typical Example





IdentifierDocument

6. Derive a structured ID based on the UDEF taxonomy that carries the UDEF inherited indexing scheme. For example <CustomerPersonLastName GUID=“as.5_5.10”>

a.ck.2_2.13.35.8

DHS Typical Example




United StatesPassport

UDEF Use with XML

UDEF Name Purchase Order Document_Identifier

UDEF ID = d.t.2_8

• Optional UDEF IDs applied only when aligning ANY two systems for the first time (design time) – stored as an alias within each system

- Non-intrusive – does not require change to data element names

• Leverage the UDEF Web Service to generate a gap analysis report

http://xml.eidx.org:8080/udef/servlet/UdefReport1

System B (OAGIS 7.x Format)

System A (xCBL 3.x Format)

Two different systems with need toexchange purchase order data

• UDEF derived intelligent ID is the “key” to systems integration

• Mapping to UDEF is a rules-based approach to determine semantic equivalence

Possible DHS-DoJ UDEF Example

Person Identifier 5_8

Person Last Name 5_5.10

Person First Name 5_4.10

Person Birth Date 5_51.6

Person Birth City Name 5_2.10.10

Suspect Id aw.5_3.13.35.8

Suspect Name aw.5_5.10 & aw.5_4.10

Suspect Date of Birth aw.5_51.6

Suspect City of Birth aw.5_2.10.10

Criminal Id av.5_3.13.35.8

Criminal Name av.5_5.10 & av.5_4.10

Criminal Date of Birth av.5_51.6

Criminal City of Birth av.5_2.10.10

Local Police – Suspect Role Local Police – Criminal Role

UDEF-Based Common Vocabulary

A Metadata Managed Architecture

EAI

Transformation Engines

Interfaces to Back-Office

Systems

• Data Dictionary

• Mapping Matrices

• Std XML Schema

UDEF-Indexed Metadata Registry/Repository

InterfaceDevelopers

Run Time

Data ModelersAnd Apps Developers

Design Time

Internet

UDEFChange Board

Global UDEF Registry

Vendors with Canonical Models

Software Vendors

with UDEF IDAPIs Web

Public

Extend Matrices

UseMatrices

Std Schema

UDEF-Indexed Metadata Registries

Build/Extend Schema

Centralized metadata registry/repository

• Enables reuse to reduce costs

• Encourages standardization

Sample Scenarios and Benefits

Sample ScenariosSample Scenarios Sample BenefitsSample Benefits

Application developer uses metadata Application developer uses metadata repository and UDEF IDs to discover a data repository and UDEF IDs to discover a data model containing 60% of the data for the model containing 60% of the data for the new applicationnew application

Data model reuse – time and cost reductionData model reuse – time and cost reduction

Helps enable a corporate-wide common Helps enable a corporate-wide common vocabularyvocabulary

Interface developer uses metadata Interface developer uses metadata repository and UDEF IDs to discover 35% repository and UDEF IDs to discover 35% of fields already mapped between two of fields already mapped between two applicationsapplications

Interface reuse – time and cost reductionInterface reuse – time and cost reduction

CEO wants to know state-by-state CEO wants to know state-by-state breakdown how many dollars in last fiscal breakdown how many dollars in last fiscal year went to suppliers for a single programyear went to suppliers for a single program

Provide answer in 4 hours rather than 5 Provide answer in 4 hours rather than 5 daysdays

Interface developer uses automated tools Interface developer uses automated tools that have been UDEF enabled to help build that have been UDEF enabled to help build Java code that directly feeds the run-time Java code that directly feeds the run-time EAI translation toolEAI translation tool

Reduce time and cost to automatically Reduce time and cost to automatically develop Java code from 4 hours to 1 hourdevelop Java code from 4 hours to 1 hour

A new supplier uses RosettaNet rather than A new supplier uses RosettaNet rather than the industry standard xCBL. The semantic the industry standard xCBL. The semantic mapping based on UDEF is provided to the mapping based on UDEF is provided to the suppliersupplier

Reduce the supplier’s time and cost to Reduce the supplier’s time and cost to complete the mapping.complete the mapping.

DoD End-to-End Procurement Mapping• AIA effort by Raytheon, Lockheed Martin and Northrop Grumman -

based on actual DoD procurement of a Raytheon spare part

• Mapped all fields in five documents plus Central Contractor Registration to both the UDEF and to OAGIS 8.0 XML standard

– RFQ, Response to RFQ, Purchase Order, DD250, Invoice

• DD250 includes the 16 fields required to support UID

UDEF Example Mappings

• UDEF names are unambiguous UDEF names are unambiguous

• UDEF names cross-referenced and indexed by UDEF UDEF names cross-referenced and indexed by UDEF IDs to a table would allow interfaces to be built without IDs to a table would allow interfaces to be built without being dependent on availability of the “system expert”being dependent on availability of the “system expert”

DPASPriority Purchase Order Document Government Assigned Priority Code PurchaseOrder/Header/Priority

SysName UDEF Name OAGIS XML Path Name

DateOfOrder Purchase Order Document Date PurchaseOrder/Header/DocumentDateTime

Examples From DoD Purchase Order Mapped to UDEF and OAGIS 8.0

IssuedByDODAAC Buyer Enterprise Defense Logistics Agency Assigned Identifier PurchaseOrder/Header/Parties/

SoldToParty/PartyID/ID

UDEF ID

d.t.2_1.4.31.4

d.t.2_6

q.3_6.35.8

Summary – Key PointsSummary – Key Points

• Adopt Key Global XML Schema Standards – especially …Adopt Key Global XML Schema Standards – especially …

- ISO/IEC 11179-5 “Naming and Identification Principles for Data ISO/IEC 11179-5 “Naming and Identification Principles for Data Elements”Elements”

- ISO 15000-5 Core Components Technical Specification (especially ISO 15000-5 Core Components Technical Specification (especially Tables 8-1 and 8-3)Tables 8-1 and 8-3)

• Work together to promote and adopt the UDEF – an Instantiation Work together to promote and adopt the UDEF – an Instantiation of ISO/IEC 11179-5 and ISO 15000-5 CCTS Table 8-1 and 8-3of ISO/IEC 11179-5 and ISO 15000-5 CCTS Table 8-1 and 8-3

For Additional Information

ISO/IEC 11179 – Specification and standardization of data elements

http://isotc.iso.ch/livelink/livelink/fetch/2000/2489/Ittf_Home/PubliclyAvailableStandards.htm

UDEF.ORG Web Sitehttp://www.udef.org/

Ron Schuldt – 303-977-1414 or [email protected]

universal data element framework (udef) overview geia – dhs workshop phoenix – april 11-14, 2005...

Documents