1 wrap technical support system ko mtg – team briefing image matters llc jeff ehman & john...

11

WRAPWRAPTechnical Support SystemTechnical Support SystemKO Mtg – Team BriefingKO Mtg – Team Briefing

Image Matters LLCImage Matters LLC

Jeff EhmanJeff Ehman

&&

John DavidsonJohn Davidson

22

TSS Team BriefingTSS Team Briefing

Outline Outline • Problem and Need DefinitionProblem and Need Definition• BackgroundBackground

* WIGIMS Project Overview* WIGIMS Project Overview

* Data Sharing Overview* Data Sharing Overview

* Metadata and Catalogs for Data* Metadata and Catalogs for Data

SharingSharing• Image Matters – Areas of ExpertiseImage Matters – Areas of Expertise• Technical Approach Options Technical Approach Options

33

TSS Problem StatementTSS Problem Statement Overall statement (from WIGIMS)Overall statement (from WIGIMS)

Display, analysis, and referencing of emissions, Display, analysis, and referencing of emissions, monitoring, and air quality modeling data and monitoring, and air quality modeling data and analyses can be combined in a single master analyses can be combined in a single master reference as they were for the §309 Technical reference as they were for the §309 Technical Support DocumentSupport Document11. While convenient, this . While convenient, this type of monolithic [and static] document limits type of monolithic [and static] document limits the ability of the reader to analyze the the ability of the reader to analyze the meanings and potential ramifications of the meanings and potential ramifications of the data and the associated analyses. Providing data and the associated analyses. Providing transparent access to many types and levels of transparent access to many types and levels of users allows custom displays and analyses of users allows custom displays and analyses of the same unmodified base data to suit the the same unmodified base data to suit the user’s wishes.user’s wishes.

11http://wrapair.org/309/031215Final309TSD.pdfhttp://wrapair.org/309/031215Final309TSD.pdf

44

TSS Needs DefinitionTSS Needs Definition Regulatory NeedsRegulatory Needs

• SIP / TIP documentation generated for user-SIP / TIP documentation generated for user-defined area, accessing the most current and defined area, accessing the most current and most appropriate data bases available.most appropriate data bases available.

Other User NeedsOther User Needs• Providing transparent access for many types Providing transparent access for many types

and levels of users allowing custom displays and levels of users allowing custom displays and analyses of the same unmodified base and analyses of the same unmodified base data to meet the user’s needs.data to meet the user’s needs.

• Discovery, Evaluation, and Access services for Discovery, Evaluation, and Access services for WRAP data to serve all existing & potential WRAP data to serve all existing & potential usersusers

• Participation in the NSDI?Participation in the NSDI?

55

Wrap Integrated Geospatial Wrap Integrated Geospatial Information Management Information Management

StrategyStrategy(WIGIMS) (WIGIMS)

Project OverviewProject Overview

66

WRAP “Data System Nodes”WRAP “Data System Nodes”- The Primary Producers - - The Primary Producers -

EDMS – Emissions dataEDMS – Emissions data VIEWS – Monitoring data and infoVIEWS – Monitoring data and info RMC – Predictive Modeling RMC – Predictive Modeling CoHA – Causes of Haze ProjectCoHA – Causes of Haze Project

NSDI, Data Federations, States, TribesNSDI, Data Federations, States, Tribes

77

WRAP UsersWRAP Users

As defined for the Needs Assessment:As defined for the Needs Assessment:

States and TribesStates and Tribes WRAP Committee, Forum, and/or WRAP Committee, Forum, and/or

Work Group MembersWork Group Members Air Quality Managers, Policy-Makers, Air Quality Managers, Policy-Makers,

or Analysts or Analysts Interested Parties Interested Parties

88

Summary of Needs AssessmentSummary of Needs Assessment

A large number of WRAP-supported users need access to A large number of WRAP-supported users need access to integrated geospatial emissions, monitoring, and integrated geospatial emissions, monitoring, and modeling informationmodeling information. .

Access to integrated ancillary (i.e., contextual) geospatial Access to integrated ancillary (i.e., contextual) geospatial information is also required to improve understanding of information is also required to improve understanding of the 3 primary data types.the 3 primary data types.

Users require a diverse set of spatial and temporal Users require a diverse set of spatial and temporal aggregation levels; aggregation levels; flexibilityflexibility should be maintained by should be maintained by providing for user-defined levels of data aggregation.providing for user-defined levels of data aggregation.

4/15 users: integrating monitoring, emissions and 4/15 users: integrating monitoring, emissions and modeling output was of “high” or “critical” importance to modeling output was of “high” or “critical” importance to their mission and over “50 users/month” (each mission)their mission and over “50 users/month” (each mission)

99

System Architecture OptionsSystem Architecture Options

Consider system with 6 distributed components:Consider system with 6 distributed components:

The three primary WRAP data nodes.The three primary WRAP data nodes.- VIEWS monitoring data- VIEWS monitoring data- EDMS emissions data- EDMS emissions data- RMC modeling data- RMC modeling data

WRAP project information (i.e., CoHA).WRAP project information (i.e., CoHA).

The resources available through DataFed.The resources available through DataFed.

Data resources accessible via the NSDI.Data resources accessible via the NSDI.

1010

Four Basic System ConfigurationsFour Basic System Configurations

The names of these options reflect the system-level approach, The names of these options reflect the system-level approach, as as it relates to location of the data standardization and integration it relates to location of the data standardization and integration efforteffort::Status QuoStatus Quo. The distributed system as it currently exists. . The distributed system as it currently exists. CentralizedCentralized. A single database houses copies of all primary WRAP . A single database houses copies of all primary WRAP data. Several possible variations:data. Several possible variations:DistributedDistributed – provider standardization. – provider standardization.DistributedDistributed – mediated standardization and integration. – mediated standardization and integration.

WIGIMS Implementation OptionsWIGIMS Implementation Options+ Status Quo+ Status Quo+ ESRI-centric Option+ ESRI-centric Option+ GOS-I type Option+ GOS-I type Option+ Mediated Data Integration (CAPITA) Option+ Mediated Data Integration (CAPITA) Option

Recommended Option Recommended Option (Presented later in Recommended Technical Approach (Presented later in Recommended Technical Approach

section)section)

11

VIEWSEDMS RMC CoHA PublicContextual

Data

WRAP User

4.2.1 Status Quo – Integration by Users

12


DataSuggested Option? - ArcIMS at Nodes?

WRAP Users

ArcIMSSite

ArcIMSSite

ArcIMSSite

?

ArcIMSSite

WRAPGeospatial Information Portal

Built on ESRI Technology

Thin Thin Thin Thick Thick

Gazetteer Catalog

ESRIPlace Names

&Attributes

Data & ServiceMetadata

Web BrowsersESRI-based GIS

Software & DSS Apps

?

13


Data

WRAPUsers

WMS

WFS

WCS

WMS

WFS

WMS

WFS

WMS

WCS

WMS

WFS

WCS


WMS WFS WCS


Gazetteer Catalog

Place Names&

Attributes


4.2.3.1 Standardization at WRAP Data Nodes

Web Browsers GIS Software, DSS

14


Data

WRAPUsers

WMS

WFS

WCS


WMS WFS WCS


Gazetteer Catalog

Place Names&

Attributes


4.2.3.2 Standardization by Data Mediator


CAPITA

WMS

WFS

WCS

1616

Core TSS CapabilitiesCore TSS Capabilities

Charts, Graphs, Maps and TablesCharts, Graphs, Maps and Tables SIP/TIP ReportsSIP/TIP Reports Data DiscoveryData Discovery Data Evaluation (web mapping and Data Evaluation (web mapping and

metadata drilldown)metadata drilldown) Data DownloadData Download

1717

Data Sharing Overview Data Sharing Overview

1818

Data Integration and Access NeedsData Integration and Access Needs

Emissions Data – your jurisdiction and that Emissions Data – your jurisdiction and that which surrounds youwhich surrounds you

Monitoring DataMonitoring Data Modeling DataModeling Data Climatologic and Micrometeorologic DataClimatologic and Micrometeorologic Data Other essential dataOther essential data

- Model input data- Model input data- Contextual data- Contextual data

*Need access to the “best available” data**Need access to the “best available” data*

1919

WRAP WRAP Spatial Spatial Data Data

ArtifactsArtifactsand and

SourcesSources

ArtifactArtifact Data TypeData Type Data SourceData Source Emissions DataEmissions Data Point, line, polygon;Point, line, polygon;

ASCII, NIF, IDA, ASCII, NIF, IDA, shapefile?, XML? GML?shapefile?, XML? GML?

EDMSEDMS

Monitoring Data Monitoring Data (IMPROVE et al.)(IMPROVE et al.)

Point;Point;

ASCII, XML, MS Excel, ASCII, XML, MS Excel, shapefile?shapefile?

Via VIEWSVia VIEWS

Modeled Air Quality ConditionsModeled Air Quality Conditions Raster;Raster;netCDF, shapefile?netCDF, shapefile?

RMCRMC

Analytical ProductsAnalytical Products Graphics, geodata;Graphics, geodata;.gif and .png images, MS-.gif and .png images, MS-Access, ASCII, shapefiles?, Access, ASCII, shapefiles?, XML? XML?

CoHA, AoH, NEARCoHA, AoH, NEAR

Contextual Data (Geospatial Info)Contextual Data (Geospatial Info) Vector and Raster; Vector and Raster; shapefiles, Grids, shapefiles, Grids, geodatabases, .gif, .png, , geodatabases, .gif, .png, , etc. etc. converging to GML converging to GML and simple referenced and simple referenced imagesimages

NSDI, CGDI, Mexico SDI NSDI, CGDI, Mexico SDI

2020

Data IntegrationData Integration Vertical data integrationVertical data integration consists of two primary tasks, consists of two primary tasks,

standardizing the projection (including the datum and associated standardizing the projection (including the datum and associated spheroid), and selecting data sets with the same fundamental spheroid), and selecting data sets with the same fundamental scale.scale.

Horizontal integrationHorizontal integration – vector data with line or polygon features – vector data with line or polygon features normally involves discrepancies where adjacent dataset meet -- normally involves discrepancies where adjacent dataset meet -- such as offsets, discontinuities, and differences in feature density.such as offsets, discontinuities, and differences in feature density.

Attribute integrationAttribute integration is necessary for horizontally-integrated is necessary for horizontally-integrated databases and consistent portrayaldatabases and consistent portrayal

Air Quality data adds another dimension – time Air Quality data adds another dimension – time

TTime integrationime integration of different geodata sets with spatial information of different geodata sets with spatial information with time series encoded in 1) attribute values, 2) as values of with time series encoded in 1) attribute values, 2) as values of separate attributes, or 3) as separate datasets in a data series separate attributes, or 3) as separate datasets in a data series

2121

Levels of Data IntegrationLevels of Data Integration Generally, users need geodata that is integrated to either of Generally, users need geodata that is integrated to either of

two two levelslevels, defined by functional requirement of the , defined by functional requirement of the requisite application:requisite application:

1) 1) visual integrationvisual integration – simple visualization, i.e., looking at – simple visualization, i.e., looking at images of the primary and ancillary data layers in the same images of the primary and ancillary data layers in the same or adjacent frames, and/oror adjacent frames, and/or

2) 2) computational integrationcomputational integration – which allows for quantitative – which allows for quantitative analyses to be performed with two or more primary and/or analyses to be performed with two or more primary and/or ancillary datasets.ancillary datasets.

Visual integration is defined as a fundamentally lower level Visual integration is defined as a fundamentally lower level of data integration than computational integration. of data integration than computational integration.

2222

Data Management and AccessData Management and AccessGuiding PrinciplesGuiding Principles

““Own what you must, access what you need” Own what you must, access what you need”

Data development, production, and maintenance Data development, production, and maintenance take staff time and money, especially if primary take staff time and money, especially if primary acquisition is involvedacquisition is involved

Data is best maintained by stewards who are the Data is best maintained by stewards who are the most familiar with it (i.e., the original data most familiar with it (i.e., the original data producers)producers)

A distributed IT environment supports accessing A distributed IT environment supports accessing what you need, and sharing what you must own what you need, and sharing what you must own with others. with others.

2323

Discover resource (i.e., information or Discover resource (i.e., information or service) from listing in published service) from listing in published catalogcatalog

Evaluate relevant resource through Evaluate relevant resource through examination of examination of metadatametadata &/or web- &/or web-mapping for geospatial datamapping for geospatial data

Link to remote resource through Link to remote resource through description in description in metadatametadata

Provide direct access to distributed Provide direct access to distributed remote data for analytical useremote data for analytical use

Vision of Open AccessVision of Open Access

2424

OpenGIS Consortium (OGC):OpenGIS Consortium (OGC):General Service ModelGeneral Service Model

Bind

Find PublishPublish

Requestor

Bind

Find

Broker

ProviderRequestor

•Interoperability is the capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of

the unique characteristics of those units.

2525

Spatial Data Infrastructure (SDI)Spatial Data Infrastructure (SDI)

A A spatial data infrastructurespatial data infrastructure is an integrated, on-line network is an integrated, on-line network that delivers geospatial data, services and information for that delivers geospatial data, services and information for applications, better business and policy decision-making, applications, better business and policy decision-making, and value-added commercial activities.and value-added commercial activities.

Based on Based on organizational commitmentsorganizational commitments, integration of , integration of spatial informationspatial information, and , and technologies technologies standards, standards, specifications for discovery, visualization, and use of specifications for discovery, visualization, and use of primarily digital geographic informationprimarily digital geographic information

Three relevant national SDIs:Three relevant national SDIs:- - Canadian Geospatial Data InfrastructureCanadian Geospatial Data Infrastructure (CGDI) (CGDI)- US - US National Spatial Data InfrastructureNational Spatial Data Infrastructure (NSDI) (NSDI)

- - Mexico’s Spatial Data Infrastructure is developing.Mexico’s Spatial Data Infrastructure is developing.

2626

CGDI, NSDI, and other National SDIs join to form a global SDICGDI, NSDI, and other National SDIs join to form a global SDI

CGDICGDIU.S.NSDI

AustralianSDI

GlobalGlobalSDISDI

EuropeanGII

OtherSDI

.

Top Level of Global HierarchyTop Level of Global Hierarchy

2727

Same structure as Global SDI, Same structure as Global SDI,

but for the Western Air Quality Communitybut for the Western Air Quality Community

RMCRMCEDMSEDMS

VIEWSVIEWSWRAPWRAP

SDISDI

CoHA CoHA et alet al

OtherSDIs

.

WRAP SDIWRAP SDI

2828

SDI ObjectivesSDI Objectives Reliance on concensus-based open Reliance on concensus-based open

standards, e.g., content, exchange standards, e.g., content, exchange formatformat

Increase accessibilityIncrease accessibility Decrease redundant effortDecrease redundant effort Offer information via the InternetOffer information via the Internet Reliance on partnerships and sharing Reliance on partnerships and sharing

of data at some levelof data at some level

= Air Quality Community Objectives?= Air Quality Community Objectives?

2929

Realizing the VisionRealizing the Vision

State, Tribes, and WRAP cannot anticipate all future needs or State, Tribes, and WRAP cannot anticipate all future needs or applications, but it can participate in an architecture that is applications, but it can participate in an architecture that is

open and flexible!open and flexible!

A geospatial data infrastructure is needed.A geospatial data infrastructure is needed.

The good news: The good news:

- you have been assembling many of the components of a - you have been assembling many of the components of a spatial data infrastructure (SDI) for some time.spatial data infrastructure (SDI) for some time.

- the interoperability specifications and standards required - the interoperability specifications and standards required to support an Air Quality SDI have already been developed to support an Air Quality SDI have already been developed and many are already in use by several data providers and many are already in use by several data providers frequented by WRAP users and for WRAP projects.frequented by WRAP users and for WRAP projects.

- there is a growing support community devoted solely to - there is a growing support community devoted solely to facilitating the successful development of new and facilitating the successful development of new and emerging SDIs emerging SDIs

3030

Metadata and Catalogs Metadata and Catalogs for Data Sharingfor Data Sharing

3131

WRAP TSS: Clearinghouse WRAP TSS: Clearinghouse FunctionFunction

WRAPTSS

RMCVIEWSEDMSOther

Partners

Tribe User

Fed Agency User

State Govt. User

Educational User

Public User

NSDI

3232

Metadata Roll-upMetadata Roll-up

WRAP NodePartner Node

External FederatingNode (e.g. GOS)

…

GOSCatalog

Public Catalog

Resource MetadataCatalog

WRAP Node

Roll-up

Selectively Populates

Roll-up

WRAP Node

•All metadata will be cached in the WRAP portal for local processing and will be searchable through the WRAP portal.

•Search by geographic extent, place keyword, theme, publishing organization, hazard category, progress status, time period, etc

•Data can either be local to the WRAP or distributed (partners)

Metadata & Data

Metadata & Data

3333

WRAP CatalogWRAP Catalog A standards-based application providing A standards-based application providing

inventory, discovery, access and analysesinventory, discovery, access and analyses of Air of Air Quality and Emission (AQ&E) Inventory data.Quality and Emission (AQ&E) Inventory data.

Uses leading Web Service technologies and data Uses leading Web Service technologies and data content, such as OpenGIS® Catalog Service, W3C content, such as OpenGIS® Catalog Service, W3C standards, and standards-based metadata standards, and standards-based metadata content for publishing, discovering and access content for publishing, discovering and access datasets.datasets.

Enables Federal, State, Tribal and Local Enables Federal, State, Tribal and Local stakeholders to lead in key mission areas such as stakeholders to lead in key mission areas such as the inventory of Air Quality and Emission the inventory of Air Quality and Emission Inventory data products and status of planned or Inventory data products and status of planned or active data acquisition/development activities.active data acquisition/development activities.

3434

Catalog Use CasesCatalog Use Cases

View Text Report (Brief, Summary, Full)

By Geographic Area

By Organization

By Hazard Type

By Keyword

DownloadData

SearchMetadata

PublishMetadata

ViewMap

ViewResults

Data Steward

View Map

FollowLink

CatalogService

User

By Data Type

3535

Metadata is the Glue Metadata is the Glue A standard way to produce, inventory, discover share and A standard way to produce, inventory, discover share and

exploit community information.exploit community information. Answers questions like Who? What? Where? When?Answers questions like Who? What? Where? When?

• ““Does a dataset on a specific topic exist (‘what’)?”, Does a dataset on a specific topic exist (‘what’)?”, • ““For a specific place (‘where’)?”, For a specific place (‘where’)?”, • ““For a specific date or period (‘when’)?” and For a specific date or period (‘when’)?” and • ““A point of contact to learn more about or order the dataset A point of contact to learn more about or order the dataset

(‘who’)?”(‘who’)?” Uses of metadataUses of metadata

• data discovery and clearinghouse operationsdata discovery and clearinghouse operations• data exchange and usedata exchange and use• data managementdata management• document planned data acquisitions to promote data document planned data acquisitions to promote data

development partneringdevelopment partnering Key Portal/Clearinghouse Capabilities Key Portal/Clearinghouse Capabilities

• search for data, applications and servicessearch for data, applications and services• access, compile & download data from different sourcesaccess, compile & download data from different sources• seek out data development partnershipsseek out data development partnerships• create and upload metadata about data and resourcescreate and upload metadata about data and resources• join a Community of Interest (e.g., GOS thematic “channels”)join a Community of Interest (e.g., GOS thematic “channels”)

3636

From Data to MetadataFrom Data to Metadata Need to list all types of data handled by the systemNeed to list all types of data handled by the system

The data is eitherThe data is either• Generated by WRAP tools => tools generate metadata Generated by WRAP tools => tools generate metadata • Uploaded by the user (generated using some other tools) => Uploaded by the user (generated using some other tools) =>

metadata is uploaded along with the data, or a form is metadata is uploaded along with the data, or a form is available for user to enter metadataavailable for user to enter metadata

But first, how do we describe that metadata? But first, how do we describe that metadata? • IssueIssue:: choice of metadata schema (Dublin Core, choice of metadata schema (Dublin Core,

FGDC CSDGMFGDC CSDGM, DHS DRM), DHS DRM)• Issue: Issue:

Need to define WRAP metadata profileNeed to define WRAP metadata profile Which of the optional elements are needed for WRAP data? Which of the optional elements are needed for WRAP data? Are the FGDC elements enough to describe WRAP data?Are the FGDC elements enough to describe WRAP data?

http://biology.usgs.gov/fgdc.metadata/version2/

3737

Metadata Generation in WRAPMetadata Generation in WRAP

IssueIssue: : • How is the metadata generated in How is the metadata generated in

WRAP? WRAP? • How does that fit within the workflows How does that fit within the workflows

and data lifecycles?and data lifecycles?• Can the WRAP data nodes generate Can the WRAP data nodes generate

metadata? metadata? • When does metadata need to be When does metadata need to be

generated? generated? • When is it consumed?When is it consumed?

3838

WRAP Metadata RequirementsWRAP Metadata Requirements (Strawman)(Strawman)

Only one metadata file is required for each WRAP datasetOnly one metadata file is required for each WRAP dataset Within each WRAP metadata file, different origins of various Within each WRAP metadata file, different origins of various

source datasets must be identified/distinguishedsource datasets must be identified/distinguished• Any database created by merging information from distinct Any database created by merging information from distinct

sources must be described in sufficient detail to identify the sources must be described in sufficient detail to identify the actual source for each element in the file.actual source for each element in the file.

Metadata follows the FGDC CSDGM (or Dublin Core?)Metadata follows the FGDC CSDGM (or Dublin Core?) Metadata must include Metadata must include

• Description of source material from which the data were Description of source material from which the data were derivedderived

• Methods of derivation, including transformations involved in Methods of derivation, including transformations involved in producing the digital filesproducing the digital files

• Description shall include dates of the source materials and Description shall include dates of the source materials and dates of ancillary information used for updatedates of ancillary information used for update

• Date assigned to a source must reflect the date that the Date assigned to a source must reflect the date that the information corresponds to the ground (if unknown, then date information corresponds to the ground (if unknown, then date of publication must be used and noted)of publication must be used and noted)

3939

Image Matters ExperienceImage Matters Experience

4040

Image Matters ExperienceImage Matters Experience1) Standards and Specification development2) GI Strategy Development

Lake Ontario / St. Lawrence River Study (IJC)CAROL (GLC), NS HLS GI (DHS GMO)GOS II System architecturesWRAP, CAROL

3) System integration and architecture4) Catalog and Metadata services

MapMod (FEMA)5) Interface and Portal development

CAROL, WA Coastal Communities6) Tool and DSS development

MapMod, CAROL Chart Generation Tools

4141

The CAROL PortalThe CAROL Portal(Centralized Air emissions (Centralized Air emissions

Repository On Line)Repository On Line)Impetus for new approach:Impetus for new approach: Great Lakes states and Ontario had been working on integrated Great Lakes states and Ontario had been working on integrated

emissions inventories for over 10 yearsemissions inventories for over 10 years Integrated database resided at EPA behind firewallIntegrated database resided at EPA behind firewall Data difficult to access and maintain by data providersData difficult to access and maintain by data providers Data accessible through RAPIDS, which made it inaccessible for a Data accessible through RAPIDS, which made it inaccessible for a

broad range of interested usersbroad range of interested users

New approach:New approach: Host data at Great Lakes Commission (outside of firewall)Host data at Great Lakes Commission (outside of firewall) Provide simple and extensible thin client interface (lots of Provide simple and extensible thin client interface (lots of

“portlets”)“portlets”) Demonstrates use of non-proprietary componentsDemonstrates use of non-proprietary components Demonstrates “accessing what you need” – ingests WMS feedsDemonstrates “accessing what you need” – ingests WMS feeds Provide accessibility to data using open standard (interoperable) Provide accessibility to data using open standard (interoperable)

web interfaces (i.e., make inventory available as WMS and WFS – web interfaces (i.e., make inventory available as WMS and WFS – images and streaming GML!)images and streaming GML!)

4242

The CAROL PortalThe CAROL Portal

4343


4444


4545

Our Experience with Open Extensible Our Experience with Open Extensible Standards and SpecificationsStandards and Specifications

Contributed to or lead the development of these OGC Contributed to or lead the development of these OGC interoperability specifications: interoperability specifications: • Web Mapping Service (WMS)Web Mapping Service (WMS)• Web Feature Service (WFS)Web Feature Service (WFS)• Web Coverage Service (WCS)Web Coverage Service (WCS)• Catalog Service 2.0 (CSW)Catalog Service 2.0 (CSW)• Open Location Services (OpenLS)Open Location Services (OpenLS)• GazetteerGazetteer• GeocoderGeocoder• Styled Layer Descriptor (SLD)Styled Layer Descriptor (SLD)• Location Organizer Folder (LOF)Location Organizer Folder (LOF)

Developer of the FGDC metadata profiles used by the NFIP Developer of the FGDC metadata profiles used by the NFIP MapMod portal and its stakeholders MapMod portal and its stakeholders • Tracking the emergence of ISO-19115 and 19139 models and Tracking the emergence of ISO-19115 and 19139 models and

schema and implications to existing metadata tools, services schema and implications to existing metadata tools, services and data repositories.and data repositories.

Developer of the MapMod’s “MIP-Geospatial Interoperability Developer of the MapMod’s “MIP-Geospatial Interoperability Framework” components, including…Framework” components, including…

4646

Metadata Tools & ServicesMetadata Tools & Services Catalog Service for the Web (CSW)Catalog Service for the Web (CSW)

• Implements the FGDC CSDGM metadata modelImplements the FGDC CSDGM metadata model JZACS Z39.50 Gateway ServiceJZACS Z39.50 Gateway Service

• Gateway between the CSW component and the NSDI. Gateway between the CSW component and the NSDI. Dynamically translates Z39.50 search requests into HTTP CSW Dynamically translates Z39.50 search requests into HTTP CSW requests and processes the results accordingly.requests and processes the results accordingly.

Metadata ToolsMetadata Tools• Converter – translates metadata into XML.Converter – translates metadata into XML.• Corrector – fixes common/well-known problems with metadataCorrector – fixes common/well-known problems with metadata• Validator – validates conformance of XML metadata records to Validator – validates conformance of XML metadata records to

normative XML schema.normative XML schema.• Loader – upload or batch-loadLoader – upload or batch-load• Manager – used by Data Administrators (actors) to manage Manager – used by Data Administrators (actors) to manage

and audit information held in the Catalog.and audit information held in the Catalog.

4747

Metadata Tools & ServicesMetadata Tools & Services

Converter

Corrector

Validator

Loader

Manager

Catalog Service

Catalog Client-1

Catalog Client-2

Catalog Client-n

3rd-partyCatalog

.txt

.xml

.xml

Errors

.xml

Inte

rnet

States, Tribes, States, Tribes, Regulators, Regulators, Industry, Industry, ScientistsScientists

4848

CTP Data Sharing Demonstration ProjectCTP Data Sharing Demonstration Project Why Data Sharing?Why Data Sharing?

• Reduce wasted money and effort:Reduce wasted money and effort: A key element of FEMA’s strategy is to A key element of FEMA’s strategy is to use quality local use quality local datadata whenever it is available and whenever it is available and cost sharecost share when data is not available when data is not available

• Not just for Flood Maps:Not just for Flood Maps: A core value of FEMA’s approach is to A core value of FEMA’s approach is to maximize the reuse of the maximize the reuse of the data that is collected and produced by the NFIPdata that is collected and produced by the NFIP

• FEMA prefers not to store data it doesn’t createFEMA prefers not to store data it doesn’t create.. There are existing web-based geospatial There are existing web-based geospatial services that meet FEMA specificationsservices that meet FEMA specifications

• Pressure from above:Pressure from above: There are several There are several Federal government rulesFederal government rules relating to geospatial relating to geospatial that FEMA must comply withthat FEMA must comply with

DHS GEA/GMO; Executive Order 12906/13286; OMB Circular A-16; etc.DHS GEA/GMO; Executive Order 12906/13286; OMB Circular A-16; etc.

Who?Who?• FEMA Mapping PartnersFEMA Mapping Partners

IDIQs, CTPs (e.g., North Carolina: NCFMP, Delaware: DNREC) and FEMA RegionsIDIQs, CTPs (e.g., North Carolina: NCFMP, Delaware: DNREC) and FEMA Regions Other Federal AgenciesOther Federal Agencies

• National Spatial Data Infrastructure National Spatial Data Infrastructure State and Local Governments State and Local Governments DHS Geospatial Management Office/Enterprise ArchitectureDHS Geospatial Management Office/Enterprise Architecture FGDC/ NSDI ClearinghouseFGDC/ NSDI Clearinghouse GOS Portal GOS Portal National Map/ NDEP/ NDOPNational Map/ NDEP/ NDOP Private IndustryPrivate Industry General PublicGeneral Public

4949

CTP Data Sharing Demonstration ProjectCTP Data Sharing Demonstration Project What data?What data?

• Geospatial data.Geospatial data. ““Intermediate” Data Submittals Intermediate” Data Submittals

• Terrain, Survey, Hydrology, Hydraulics, Coastal, Alluvial Fan, Terrain, Survey, Hydrology, Hydraulics, Coastal, Alluvial Fan, Scoping, etcScoping, etc

DFIRM Submittals DFIRM Submittals • Preliminary, Post-PreliminaryPreliminary, Post-Preliminary

Effective DFIRM layers Effective DFIRM layers BasemapsBasemaps

• including Orthophoto, Elevation and other FGDC Framework Layersincluding Orthophoto, Elevation and other FGDC Framework Layers Metadata!Metadata!

• Different ways to share data.Different ways to share data. Visualize (view maps)Visualize (view maps) Publish (submit datasets and metadata)Publish (submit datasets and metadata) Discover (find datasets of interest)Discover (find datasets of interest) Access (query, subset, download)Access (query, subset, download)

5050

CTP Demonstration ScenariosCTP Demonstration Scenarios Discover DataDiscover Data

• Find datasets of interest at CTP node and/or at MIP nodeFind datasets of interest at CTP node and/or at MIP node• Browse and drilldown into metadata for datasetsBrowse and drilldown into metadata for datasets• Select datasets to view in mapSelect datasets to view in map

Visualize DataVisualize Data• View maps consisting of Flood Hazard and Basemap layers View maps consisting of Flood Hazard and Basemap layers

from remote serversfrom remote servers• Select layers to viewSelect layers to view• Set styles and apply uniform symbologySet styles and apply uniform symbology

Access DataAccess Data• Query properties of selected featuresQuery properties of selected features• Download data to local systemDownload data to local system

Publish DataPublish Data• Publish metadata to local catalog @ CTPPublish metadata to local catalog @ CTP• Publish metadata to remote catalog @ MIPPublish metadata to remote catalog @ MIP• Submit (upload) data packages (Intermediate data and DFIRM Submit (upload) data packages (Intermediate data and DFIRM

submissions)submissions)

5151

Map

MIP-GIF Web-Based Clients MIP-GIF Web-Based Clients and Services and Services

“MIP”DemoClient

Feature

Coverage

“CTP”

CatalogMetadata

DFIRM / Basema

p

Imagery &

Elevation

Map

Feature

Coverage

Catalog Metadata

DFIRM / Basema

p

Img. &Elevati

on

Map

“MIP”

Userw/ Web Browser

MapMap Other Web

Services

5252

Benefits of the ApproachBenefits of the Approach Alignment with Federal InitiativesAlignment with Federal Initiatives

• DHS GEA/GMO; Executive Order 12906/13286; OMB Circular A-16; etcDHS GEA/GMO; Executive Order 12906/13286; OMB Circular A-16; etc Alignment with MapMod objectivesAlignment with MapMod objectives

• Use local data whenever possible, cost share when its not (save money Use local data whenever possible, cost share when its not (save money and effort)and effort)

• Enable multiple uses of flood mapsEnable multiple uses of flood maps• Don’t store data FEMA doesn’t createDon’t store data FEMA doesn’t create• Comply with Federal architectures, guidelines, policies, directivesComply with Federal architectures, guidelines, policies, directives

NSDI, e-Gov, FEA, GOS, NDEP, NDOP, National MapNSDI, e-Gov, FEA, GOS, NDEP, NDOP, National Map Outreach and collaboration with stakeholdersOutreach and collaboration with stakeholders Technology that “lowers the bar” for implementation and up-takeTechnology that “lowers the bar” for implementation and up-take Leverages Standards-based Commercial-Off-The-Shelf (SCOTS) but works Leverages Standards-based Commercial-Off-The-Shelf (SCOTS) but works

with Open Source and GOTS software too.with Open Source and GOTS software too. A framework of specifications, guidelines, cookbooks and software for data A framework of specifications, guidelines, cookbooks and software for data

sharing with a variety of software and data sourcessharing with a variety of software and data sources• Allows stakeholders to “bite off only as much as they want to chew”Allows stakeholders to “bite off only as much as they want to chew”

5353

Technical ApproachTechnical Approach

5454

Technical ApproachTechnical Approach Recommended Implementation OptionsRecommended Implementation Options

• TSS as the Community Data PortalTSS as the Community Data Portal• Open standards for future extensibilityOpen standards for future extensibility

Develop the Technical Baseline of Agreements, Develop the Technical Baseline of Agreements, Specifications and StandardsSpecifications and Standards• PoliciesPolicies• Datasets and formatsDatasets and formats• ServicesServices

Integrate components, don’t build from scratchIntegrate components, don’t build from scratch• Encapsulate existing implementations and software with Encapsulate existing implementations and software with

open standards (as much as practical)open standards (as much as practical)• Metadata Catalog provides the “glue” for connecting Metadata Catalog provides the “glue” for connecting

stakeholders, data and servicesstakeholders, data and services

5555

Standards and SpecificationsStandards and Specifications•WRAP

•Coordinate Reference Systems•Lambert Conic Conformal•Modeling Grids

•Shapefiles for GIS data transfer (?)•NSDI

•Data Content Standards (Framework Data Standards, (INCITS L1) )

•Data Quality (Accuracy) Standards(National Standard for Spatial Data Accuracy, ASPRS Accuracy Standards for Large-scale Maps)

•Geodata Discovery Service(OGC Catalogue Service, Z39.50 profile)

•FGDC and ISO Metadata Standards(CSGMD, ISO 19115 DIS)

•Web Map, Feature, and Coverage Services(OGC WMS, OGC WFS, OGC WCS)

•Geographic Markup Language(OGC GML)

http://www.incits.org/tc_home/l1.htm

56

Technical Approach – Short-term Option


Data

WRAPUsers

WMS

WFS

WCS

WRAPTechnical Support System

WMS WFS WCS


Gazetteer Catalog

Place Names&

Attributes


Standardization by Data Mediator


ResourceIntegrators &

Interface Providers

WMS

WFS

WCS

57


Data

WRAPUsers

WMS

WFS

WCS

WMS

WFS

WMS

WFS

WMS

WCS

WMS

WFS

WCS

WRAPTechnical Support System

WMS WFS WCS


Gazetteer Catalog

Place Names&

Attributes


Standardization at WRAP Data Nodes


Technical Approach – Long-term Goal

5858

Thank You!Thank You!

201 Loudoun Street, SW201 Loudoun Street, SW Leesburg, VA 20175 Leesburg, VA 20175

andand4899 N. Old SR 374899 N. Old SR 37

Bloomington, IN 47408Bloomington, IN 47408www.imagemattersllc.comwww.imagemattersllc.com

1 wrap technical support system ko mtg – team briefing image matters llc jeff ehman & john...

Documents

data federations

primary data types

wrap data system nodes

data sharing overview

primary wrap data nodes

unmodified base data

data resources accessible

air quality modeling