Michigan DMR Data Exchange With EPA

or “What I Node To Be True”

Bill GeakeMichigan Department of Information Technology

Overview of DMR Data Flow Process

RegulatedWastewater

FacilitiesMichigan

DEQ US EPA

MI NodeClient

PCS

DEQ DataCoding Operators

IDEF GenTran

Legacy CDX

CDXNode

Michigan DEQ

DEQ DMZ DEQ Intranet

NMSDB

E-DMRDB

Facility-to-State DMR Data Flow

RegulatedWastewater

Facilities

DB

State-to-EPA DMR Data Flow

EPA

CDX Node

State of Michigan

DEQ Intranet

NMSDB

MI NodeClient

PCS

Legacy CDX

IDEF GenTran

Michigan Node Client

Michigan Node Client Processing

E-DMRXML File

DMRFlow.DLL

E-DMRToolset

NMS DBXML

ParserXML

Parser

DMR-IDEF-v1.xslt

IDEF MVXML File

EPA

Legacy CDX

IDEF GenTran Processor

PCS

Receipt Acknowledgement

IDEF PCS Report Notification

EPA-to-State Feedback Flow

CDX Node

From State

State of MichiganEPA

EPA-to-State Feedback Flow

CDX Web Site

PCS Update Audit Report

NMS DB

DEQ DataCoding Operator

PCS Reject Resolution Utility

The BIG Picture

PCSState NPDES

Permit Database

80-Column text file

Edit/Update Reports

The BIG Picture

PCSState NPDES

Permit Database

IDEF XML

Edit/Update Reports(PDF Format)

IDEF GenTran

CDX Web Site

The BIG Picture

PCSState NPDES

Permit Database

IDEF XML

Edit/Update Reports(PDF Format)

IDEF GenTran

CDX Web Site

CDX Node

FLOWS are about PROCESSING!

Two Aspects of Data Exchange:Format

Data Container XML Schema

Process Rules surrounding the flow of data More complex than creating a format Can we have XML-based processing

instructions?

The “Ideal” Data Flow Scenario

One Owner/Authority for a given dataset

Each owner publishes to their node Nodes each provide a common

interface for given data flow

The “Ideal” Data Flow Scenario

Data Consumer should initiate Requests from the Data Provider

“Come and get it” approach Only the consumer knows what it

needs and when it needs it

DataProvider

Node

DataConsumer

Node

Want My Data?

No…No…No…

The “Ideal” Data Flow Scenario

Data Provider should need NO knowledge of Data Requestor’s state

Complexity of storing data consumer’s state on the provider side

Not as scalable

DataProvider

Node

DataConsumer

Node

Gimme the Data I Don’t

Have

Nope. Tell me what you want!!!

The “Synchronization” Data Flow Scenario

Push or Pull will work Should all happen in the background Only require user intervention when and

exception occurs Difficult to achieve when synchronizing

heterogeneous systems Crosswalking data…what a mess!

Final Thoughts…

The Ubiquitous “Questions” Slidewith Tacky WordArt