FME World Tour 2014Local Government: FME Server Pilot Project

A Local Government GIS User Group Initiative

Barry Doyle: Roscommon County Council

Jon Hawkins: Waterford City Council

The Problem

• Lots of people looking to get their hands on LA datasets with the aim of using these within their own systems and apps, integrating these with their own datasets, and in building national views, but....

• 34 independent Local Authorities *

• Not all LA’s create and store the same spatial datasets, in accordance with the same schemas, to the same quality standards, in the same formats

• Large overhead on LA in processing data requests.

10th April 2014

The Solution

• A Local Government Spatial Data Infrastructure (LGSDI)

• More than a data warehouse it is a framework of geographic data, metadata, users and tools

• Based on the application of standards

• Enables data to be discovered, accessed, and used in an efficient manner

• Promotes data access and use

10th April 2014

The Question

• Will FME Server, implemented as a shared LA service, provide the technical architecture required to support:

– Multiple source data format reconciliation, integration, and storage

– Transformation of source data to structured schemas

– Data validation

– Data management and distribution

10th April 2014

Local Authority Spatial Datasets

• Approximately 80 spatial datasets created by LA’s though this varies and number continues to grow

• Approximately 20 – 30 can be described as Core

• Some required under Legislation (Development Plans, Road Schedule, etc.)

• 50% created by less than 10 LA’s

• 40% of datasets created are identified as being shared with or required by Public Agencies

• 40% of datasets created are identified as being shared with the Public

10th April 2014

Who is looking for LA Spatially Referenced Data

Other Local Authorities

10th April 2014

..and what do they want? (Sample set)INSPIRE MyPlan LocalGov.ie Irish Water EPA Heritage

Council DAHG

Road Schedule

Local Authority Bridges

Record of Protected Structures

Tree Preservation Orders

Land Use Zoning

High amenity area

Landscape Character Areas

Parks & Open Spaces

Quarries

Scenic Views and Routes

Waste Permits

Landfill Sites

Burial Grounds

Fire Stations

Public Lighting

Waste Water Network

Water Network

Architectural Conservation Areas

Water monitoring stations – investigative

Bathing sites

Water abstraction facilities

Industrial sites - Section 4 and section 16 licenses

Habitat

Mines

Planning Register

Planning Enforcements

Libraries

Polling Stations

Recycling facilities

10th April 2014

Local Authority 1

Agency 1

Local Authority 2 Local Authority 3Metadata

Transform

Validate

Agency 2 Agency 3Transform

Validate

Merge

Current Situation

10th April 2014

Local Authority 1 Local Authority 2 Local Authority N

Metadata

TransformTo LG Schema

Validate

Spatial DataHub Metadata

Transform to LG Schema

Validate

Merge

DiscoveryViewDownload

Sectoral Data Warehouse

Data Download Web Services

10th April 2014

Benefits

• Single point of access to LA data

• Single version of data

• Standardised data

• Quality of data

• Supports data sharing and re-use

• Automation

• Reduced overhead on LA in dealing with data requests

• Reduced overhead on agencies gathering, processing, and integrating datasets

• Supports compliance with INSPIRE requirements

10th April 2014

How we hope to make it work

Step 1: Get / receive the data

LocalAuthority

Option A: Utilise Existing Desktop Licences to push [post] data

Workspace on serverWaiting for data

Local workspace that can be run manually or by using a bat file and windows scheduled taskNo issues with firewalls or need for setting up FTP rules or fixed IP’s

Step 1: Get / receive the data

Local Authority

Option B: Reference datasets directly

Local Authority

Option C: Email with attachment

Output of Step 1: Data

Step 2: Validate and load Scheduled workspaces

Valid Yes

No

PostGISLoad, Merge and

TimeStamp

GenerateError

Report.htm

Daily

Periodically

Workspaces to check data is current, by using testers that check the last upload date was within agreed limits

Step 3: Transform, manipulate, enhance and present

Downloads

REST Services

Notifications

Present and deliver how the data is neededDefine and set standards

Web Services

Self-serve

Distribute

Other things we hope to do

Utility Data

Current Situation: Separate contracts with separate Local Authorities

Local Authority

Local Authority

Local Authority

Hope to investigate: One central contract