transfer of statistical data from the german federal and...

© con terra GmbH, 20111

Mark Döring & Sören Dupke

Transfer of Statistical Data from the German Federal and State

Governments to the European INSPIRE Network

INSPIRE Data harmonization projects

IT.NRW

© con terra GmbH, 20113

Project Summary

Project Partner

> IT.NRW (Representative for the statistics authorities of the federal states)

> con terra GmbH

Prototype

> Transformating statistics data of the federal states from the GENESIS DB into

INSPIRE Annex III (SU, PD)

Documentation of the approach

> Aggreement on how to continue with the statistics data from the federal states

© con terra GmbH, 20114

Spatial ETL – Extract Transform Load

Integrate

Translate

Datenconversion > 300 formats and Datasources

Merging of Information and loading inot

information systems

Transform

Schema mappingand transformation

Distribute

Distribution of Data in specific formats

and schemas

„Get the right data to the right system, in the right schema, at the right time“

© con terra GmbH, 20115

Data Harmonization with Spatial ETL

Extraction of statistics data from Geodata

> Attribute information and spatial reference

> Migration into standard Applications

– Excel, Access, Oracle, MS SharePoint, Informatica 7

Migration of statistics data into SDIs

> INSPIRE (Statistical Units, Population Distribution)

> Migration into standard GIS Applications

– ArcSDE, Geodatabase, Shape Files, ...

© con terra GmbH, 20116

Defining the Transformation process with FME Workbench

Input-M

odel

Schema-M

apping

Output-M

odel

© con terra GmbH7

ArcGIS for INSPIREINSPIRE Solution Pack for

FME

General Approach

ETL

INSPIRE

Datapecifications & Services

INSPIRE

Consumer

European SDI

Internal Information Resources

and Databases

© con terra GmbH8

Initial situation of INSPIRE schema mapping

INSPIRE expert knowledge and domain expertise are required

High complexity of the source data and the INSPIRE destination model

It has to be accounted for local characteristics (quality, history, contents)

- Internal data models

- Meta data

- Data history

- Data quality

- Expertise (Domain)

- INSPIRE data models

- INSPIRE specifications

- INSPIRE legislation

- INSPIRE requirements

- Expertise (Annex 1)

INSPIRE Solution Pack for

© con terra GmbH9

Template Workspaces for all Annex I-Themes

ETL-Process supported by INSPIRE Solution Pack for FME

© con terra GmbH10

ETL-Process supported by INSPIRE Solution Pack for FME

INSPIRE relevant source data requires individual mapping

Individual Mapping

Pre-definedMapping

© con terra GmbH, 201111

Challenge

Complex Source data

> IT.NRW has the semantic knowledge

> Knowledge about INSPIRE Data specifications and data structure at con

terra GmbH

Complex Targetmodel (AU, SU, PD)

> Beta Status of the tools (ISP, AfI)

> Specifications not final (Draft Version, SU, PD)

© con terra GmbH, 201112

Source Data

Gemeinsamen Neuen Statistischen Informations-System (GENESIS)

Available Formats / Protocols

> SOAP Service

> XML Web-Service

> CSV-Files (preprocessed)

© con terra GmbH, 201113

Sourcedata – GENESIS Export

© con terra GmbH, 201114

Shape Datasets

ca. 12.000 Features

Gemeindeschlüssel (ID)

Source Data IT.NRW – Gemeinde (municipality) Geometries Daten

© con terra GmbH15

Population Distribution

© con terra GmbH16

INSPIRE Population Distribution (class diagramm)

© con terra GmbH17

Semantic Mapping

© con terra GmbH18

Mapping Process (GENESIS to INSPIRE)

© con terra GmbH19

Project report

INSPIRE Data Download Service (Prototype)

Project results

© con terra GmbH20

Conclusion

Good knowledge of source data and source schema neccessary

> Content and structure

> Complex target schema

Communication neccessary

> Close cooperation between Geo- und Statistics Department

Fuzzy use of source schemamake the mapping difficult

> Spatial ETL to harmonize the sourcedata

Schema mapping usually possible during a time frame between 5 and

10 days

© con terra GmbH, 201121

ArcGIS

Online

Nutzung von INSPIRE Diensten

ArcGIS Desktop

FME Desktop

Datenhaltung

GDB

European

SDI

INS

PIR

E

GM

L D

ien

ste

ArcGIS Server

Arc

GIS

Se

rve

r

Die

nst

ArcGIS Online

Java Script Client

ETL Prozess

MXD-Dokument

© con terra GmbH, 201122

© con terra GmbH, 201123

© con terra GmbH, 201124

con terra –

Gesellschaft für Angewandte

Informationstechnologie

Martin-Luther-King-Weg 24

48155 Münster

Telefon +49 251 747 45 0

Telefax +49 251 747 45 2111

Hamburg

HannoverMünster

Bonn

Wiesbaden

Leipzig

Zürich

Burgdorf

Nyon

Kranzberg

Thank you for your attention!

Mark Döring

E-Mail: m.doering@conterra.de

Sören Dupke

E-Mail: s.dupke@conterra.de