Data Exchange Data Exchange Through XMLThrough XML

IPMA Forum 2003IPMA Forum 2003May 21, 2003May 21, 2003

St Martins College St Martins College Lacey, WashingtonLacey, Washington

Presentation by:Presentation by:Guy Outred – Windsor Solutions, Inc.Guy Outred – Windsor Solutions, Inc.Louis Sweeney – Ross and AssociatesLouis Sweeney – Ross and Associates

Tom ClarkeTom Clarke – – Office of Administration of the Office of Administration of the CourtsCourts

Data Exchange Thru XML

• Guy Outred – Guy Outred – Windsor SolutionsWindsor Solutions– Challenges and History of Data Exchange– What is XML? How will it improve

exchange?

• Louis Sweeney – Louis Sweeney – Ross & AssociatesRoss & Associates– The National Environmental Information Exchange The National Environmental Information Exchange

Network (NEIEN)Network (NEIEN)

• Tom Clarke – Tom Clarke – Office of Administration of the Office of Administration of the CourtsCourts– Justice Information Network (JIN)Justice Information Network (JIN)

www.windsorsolutions.com

Data Exchange Thru XML

From a government perspective

•Challenges and History of Data Exchange•What is XML? •How will it improve data exchange?

www.windsorsolutions.com

Data Exchange Thru XML

From a government perspective

•Challenges and History of Data Exchange•What is XML? •How will it improve data exchange?

Processes Span Many Processes Span Many

OrganizationsOrganizations

G2G ExamplesG2G Examples

• Environmental dataEnvironmental data• Judicial dataJudicial data• Procurement dataProcurement data• Financial dataFinancial data• Business and tax information (e.g., Business and tax information (e.g.,

UBI)UBI)• Demographics / GIS dataDemographics / GIS data• Federal agency inter-changeFederal agency inter-change

B2G ExamplesB2G Examples

• Tax returnsTax returns

• Environmental reportingEnvironmental reporting

• Productivity informationProductivity information

• Financial reportingFinancial reporting

• Real estate historyReal estate history

5 Steps for Data 5 Steps for Data ExchangeExchange

1.1. Define and agree what data to exchangeDefine and agree what data to exchange

2.2. Specify the agreed definition to ensure quality of Specify the agreed definition to ensure quality of exchange; i.e., Data Structure + Data Rulesexchange; i.e., Data Structure + Data Rules

3.3. Develop an extraction and transformation process to Develop an extraction and transformation process to procure the dataprocure the data

4.4. Develop a transformation (and loading) process to use Develop a transformation (and loading) process to use the datathe data

5.5. Develop a repeatable process to facilitate exchangeDevelop a repeatable process to facilitate exchange

History of Data Exchange History of Data Exchange TechnologiesTechnologies

19751975– Electronic Data Interchange (EDI)Electronic Data Interchange (EDI)

• First standard way to communicate over a First standard way to communicate over a networknetwork

• Complex and costly, only major implementationsComplex and costly, only major implementations

• 80s and 90s80s and 90s– Universal conduits attemptedUniversal conduits attempted

• CORBA, DCOM, Unix RPC, Java Remote Method CORBA, DCOM, Unix RPC, Java Remote Method InvocationInvocation

• Inadequate momentum gained Inadequate momentum gained

Historical Lesson Historical Lesson LearnedLearned

Standards and their Adoption is a two Standards and their Adoption is a two way street.way street.

Web to the RescueWeb to the Rescue• A low level transport standard was bornA low level transport standard was born

– The web has been adopted using TCP/IP and HTTPThe web has been adopted using TCP/IP and HTTP– Has become a universal business standardHas become a universal business standard– Messaging and data encapsulation missingMessaging and data encapsulation missing

• Feb 1998 draft XML 1.0 by W3CFeb 1998 draft XML 1.0 by W3C– Provided the platform-independent data Provided the platform-independent data

descriptiondescription• Dec 1999 SOAP revealedDec 1999 SOAP revealed

– Provided a protocol to process data exchangeProvided a protocol to process data exchange• Fall 2000 WSDL announcedFall 2000 WSDL announced

– Standard to invoke and service web data exchangeStandard to invoke and service web data exchange• Cross-vendor support unprecedentedCross-vendor support unprecedented

What about flat files?What about flat files?

• The cheap (non-EDI) way to share dataThe cheap (non-EDI) way to share data• Flat files provide the data StructureFlat files provide the data Structure• The Rules can be captured by:The Rules can be captured by:

– A specification document of the rulesA specification document of the rules– A parser application used after receiptA parser application used after receipt– A parser application for use during file A parser application for use during file

generationgeneration

• Simple, technology independent, but Simple, technology independent, but messymessy

www.windsorsolutions.com

Data Exchange Thru XML

From a government perspective

•Challenges and History of Data Exchange•What is XML? •How will it improve data exchange?

XML in a NutshellXML in a NutshellXML supports Step 2: Specify the agreed definition to XML supports Step 2: Specify the agreed definition to

ensure quality exchangeensure quality exchange

An XML Schema defines the Structure and the Rules

An XML Document contains the Data (and references the Schema)

Create an XML document, open it with IE and it validates itself

Reusable Good Information = Structure + Rules + DataReusable Good Information = Structure + Rules + Data

Underlying TechnologiesUnderlying Technologies XML Is the GlueXML Is the Glue

Program the Web

XML

Browse the Web

HTML

TCP/IP

Connect the Web

Technology

Innovation

Connectivity PresentationConnecting Applications

FTP, E-mailWeb Pages Web Services

• eXtensibleeXtensible Markup Language Markup Language • XML is designed to represent and XML is designed to represent and

transfer structured datatransfer structured data– In HTML: In HTML: <p>Jan 15, 2000 </p><p>Jan 15, 2000 </p>– In XML: In XML: <OrderDate>Jan 15, <OrderDate>Jan 15, 2000</OrderDate>2000</OrderDate>

• XML does not display or transform dataXML does not display or transform data

XML OverviewXML Overview

• XML is composed of tags and XML is composed of tags and attributesattributes– Tags can be nestedTags can be nested

• Representing entities, entity properties, and Representing entities, entity properties, and entity hierarchyentity hierarchy

<ROOT><Orders OrderID="10643" CustomerID="ALFKI" EmployeeID="6" OrderDate="1997-08-25T00:00:00" RequiredDate="1997-09-22T00:00:00" ShippedDate="1997-09-02T00:00:00" />

</ROOT>

XML OverviewXML OverviewXML SyntaxXML Syntax

• XML schemas describe the structure of XML schemas describe the structure of an XML documentan XML document– XML schemas describe the tag and attribute XML schemas describe the tag and attribute

specificationsspecifications– XML schemas also describe constraintsXML schemas also describe constraints

• Schemas can reference other schema Schemas can reference other schema (inheritance)(inheritance)– Encourages reuse of data definitionsEncourages reuse of data definitions– Enables cross-fertilization of data exchangeEnables cross-fertilization of data exchange

XML SchemasXML Schemas

Step 5: Develop a Step 5: Develop a repeatable process to repeatable process to

facilitate exchangefacilitate exchange

Ubiquitous Communications: Internet

Universal Data Format: XML

Service Interactions: SOAP

Formal Service Descriptions: WSDL

Publish & Find Services: UDDI

• SOAP is the protocol used by Web SOAP is the protocol used by Web Consumers for sending requests and Consumers for sending requests and receiving responses receiving responses

• It enables cross-platform interoperabilityIt enables cross-platform interoperability– OS, object model, programming language OS, object model, programming language

neutralneutral– Hardware independentHardware independent– Protocol independentProtocol independent

• Works over existing Internet Works over existing Internet infrastructureinfrastructure

SOAP SOAP

SOAP Message

SOAP Envelope

SOAP Header

SOAP Body

Message Name & Data

Headers

Headers

XML-encoded SOAP message name & data

<Body> contains SOAP message name

Individual headers

<Header> encloses headers

<Envelope> encloses payload

Protocol binding headers

The complete SOAP message

SOAPSOAPMessage StructureMessage Structure

www.windsorsolutions.com

Data Exchange Thru XML

From a government perspective

•Challenges and History of Data Exchange•What is XML? •How will it improve data exchange?

Example: Washington’s Example: Washington’s TurboWasteTurboWaste

• Regulatory reporting of hazardous waste Regulatory reporting of hazardous waste activitiesactivities

• Old wayOld way– Dept of Ecology provided flat file capability in 1995Dept of Ecology provided flat file capability in 1995– Software distributed to validate data with rulesSoftware distributed to validate data with rules

• New wayNew way– XML Schema provided to reporters, web XML Schema provided to reporters, web

submission abilitysubmission ability– Structure + Rules encapsulatedStructure + Rules encapsulated

• The FutureThe Future– States to exchange monthly data exchange to States to exchange monthly data exchange to

identify missing trans-state hazardous wasteidentify missing trans-state hazardous waste• Faster, better quality data interchangeFaster, better quality data interchange

XML is not a silver bulletXML is not a silver bullet

• Agreement on the definition of data is Agreement on the definition of data is still the same old challengestill the same old challenge

• Standards are still maturingStandards are still maturing• Infrastructure for XML exchange is Infrastructure for XML exchange is

non-trivialnon-trivial• It takes two to tango It takes two to tango

– And government agencies prefer to rumbaAnd government agencies prefer to rumba– Adoption amongst constituents will take Adoption amongst constituents will take

timetime