managing data interoperability with fme
DESCRIPTION
Managing Data Interoperability with FME. Tony Kent Applications Engineer IMGS. IMGS. We deliver innovative spatial solutions For the desktop, web and mobile Built on our partner’s technology - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/1.jpg)
Managing Data Interoperability with FME
Tony KentApplications Engineer
IMGS
![Page 2: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/2.jpg)
IMGSWe deliver innovative spatial solutions For the desktop, web and mobile Built on our partner’s technology
Designed to meet the challenges of Government, Mapping Agencies, and Utility & Communications Customers
![Page 3: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/3.jpg)
Safe SoftwarePowering the flow of spatial data with FME
3
Enabling people to use their spatial data where, when and how they want to
Most Used Spatial Interoperable Solution in Ireland
![Page 4: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/4.jpg)
Why Spatial ETL?Significant proliferation of different spatial data formats and types
Hundreds of formats, with more added each yearMultiple types of data stored in multiple systemsUnique data model requirements for each application
4
![Page 5: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/5.jpg)
Why Spatial ETL?
Traditional approaches to data translation and data model manipulation are not viable
Complex, inefficient and time-consuming
5
![Page 6: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/6.jpg)
Why Spatial ETL?Increasing pressure for access to spatial data
More users, beyond traditional GIS usersExpectations of real-time custom data views, 24x7
6
![Page 7: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/7.jpg)
FME Capabilities
7
The only complete spatial ETL solution
Translate spatial data from one format to another
Transform spatial data into the precise data model you need
Integrate different data types into a single data model
Distribute spatial data to users where, when and how they need it
![Page 8: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/8.jpg)
FME DesktopFlexible and powerful spatial ETL toolsetTranslate, transform and integrate data in hundreds of formatsGraphical authoring environment
8
Step 1 - Extract
Select and add the source dataset(s)
Step 2 - Transform
Add transformers to manipulate the
data as it moves from source to
destination Step 3 – Load
Load the transformed data into a
destination format and source
![Page 9: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/9.jpg)
FME Workbench
9
Use simple point and click to easily define spatial data flows to translate, transform and integrate your data
![Page 10: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/10.jpg)
ExamplesAutomating Ordnance Survey data updates
Pushing NTF data to multiple GIS platformsStripping out unnecessary data Adding custom styling and symbology – CADE.g. Eircom, ESB, Fingal County Council
Publishing data to internal public portals Bulk and transactional updates Fire wall Friendly – use selected portCompletely automated E.g. Dublin City Council
![Page 11: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/11.jpg)
Open Data ChallengeYou want to meet Open data requirements, but your data is organized rather differently
?
![Page 12: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/12.jpg)
?What FME does …
Build data bridges to your SDI
![Page 13: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/13.jpg)
SDI Harmonization Core Concepts
Harmonization: implied requirement for building an SDIDisparate sources must be mapped to a common destination data modelCore to the harmonization workflow is a process called schema mapping.Delivered by services based on open standards
![Page 14: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/14.jpg)
Harmonization PrinciplesTypical stages:1. Evaluation2. Assembly3. Transformation4. Validation5. Publication
Based on the Spatial ETL concept (Extract, Transform and Load), as applied to INSPIRE SDI’s
![Page 15: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/15.jpg)
Metadata – Data about data
Describes data structurestablesgeometry typesdata typesfields
Describes data contentcoordinate systemextentmodification datequality, ownership, etc.
![Page 16: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/16.jpg)
Metadata - Purpose
![Page 17: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/17.jpg)
Key FME Metadata CapabilitiesReading
Writing
Updating
Harvesting
Validating
Integration with web services
![Page 18: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/18.jpg)
Data Transformation - Schema
Reshape source data to match required destination schema
Schema mappingfeature typeattribute namenew attribute creationcode listsconditional value
mappings
![Page 19: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/19.jpg)
Feature Type Mapping in FME Workbench
Attribute Mapping in FME Workbench
Schema Mapping in FME
![Page 20: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/20.jpg)
Value Mapping
FME Data Model Restructuring: Attribute Names & Values
![Page 21: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/21.jpg)
FME SchemaMapper: INSPIRE geographic names
Name mapping
Name & value mapping
FME Workspace
![Page 22: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/22.jpg)
Transformation: GeometryNon-spatial to spatialGeometry extraction (spatial to GML)Representation transform: CAD drawing lines with labels to GIS polygonal features with attributesCoordinate System Reprojection (ED50 to ETRF89)Simple to complex geometry
Source point and polygon data to multiple geometric representations (city as point / area, river as line / area)
Generalization and interpolationHighly granular national and regional datasets often require thinning to be usable on pan-European scales
![Page 23: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/23.jpg)
ValidationSchema validation i.e. INSPIRE (xsds)Data integrity
Unique IDsGeometric integrity (closed polygons)Null values (nullable?)Valid values: ranges and domain codesData gapsBoundsNetwork integrity
Custom validity rules specific to domainValidation automation via FME Server upload
Ensure data quality throughout the data transformation process
![Page 24: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/24.jpg)
Publish workspace to FME ServerStore the workspace in a central repository
Make your FME workspaces available to others –over the web
Register the workspace with one or more services (Data Streaming, Data Download, etc.)
Publication with FME Server
24
![Page 25: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/25.jpg)
Format translationSchema mappingString and list manipulationData validationDatabase load and extractXML,GML,WFS: reading, validation, publicationWeb services: WFS, WMS, integration with othersMetadata supportEnterprise services with FME Server
FME Tools for INSPIRE
![Page 26: Managing Data Interoperability with FME](https://reader031.vdocuments.us/reader031/viewer/2022020501/568162f8550346895dd37296/html5/thumbnails/26.jpg)
FME can provide all the tools to help build support your data sharing needs:
Integrate your data sources
Manage your meta data catalogues
Transform your data to standard schemas
Publish the data in the required formats
Summary