Social Validation of INSPIRE Annex III Data Structures in EU Habitats

WP3: Data and metadata modelling

IMCS, HSRSMaris Alberts, Ota Čerba, Karel Charvat,

Peteris Bruns, Premysl VohnoutHABITATS Final Review

Luxembourg, April 19 2013

WP3 Main objectives

Define data and metadata models for the following INSPIRE data themes:

16. Sea regions

17. Bio-geographical regions

18. Habitats and Biotopes

19. Species distribution

The results should be in compliance with INSPIRE directive and possible INSPIRE data models (not existing before the project started)

WP3 Specific operational objectives include

– analysis of existing data and metadata models

– definition of platform-neutral conceptual models and metadata profiles

– survey of tools for the interactive construction of data models for existing data sets

– design of data transformation processes

– contribution to INSPIRE coresponding data model development

Contribution to INSPIRE

• To ensure HABITATS data model and meta data profile compliance with corresponding INSPIRE data themes.

• IMCS joined INSPIRE TWG BR-HB-SD

BR 17. Bio-Geographical regions

HB 18.Habitats and Biotopes

SD 19.Species distribution

INSPIRE TWG BR-HB-SD

Exceptional case;

single TWG worked with 3 data themes!

Contribution of IMCS on behalf of HABITATS project in TWG activities:

– Physical TWG meetings: 2010 in Ispra, 2011 Amsterdam, 2012 in Vienna,

– participation in regular team teleconferences,

– contribution to data model development and documentation,

– Data model testing review analysis and final data model improvement according to comments.

HABITATS benefits from participation in INSPIRE TWG

Habitats project conceptual data model was prepared short period before INSPIRE TWG was established

All HABITATS activities related to INSPIRE data themes after establishment of TWG was reported in TWG

Feedback from TWG decisions guaranteed complete harmonization of HABITATS conceptual data models with INSPIRE corresponding Annexes.

Data usage use cases

• Regional data are used regionally

• Global data are used regionally

• Regional data are used cross-regionally (here works INSPIRE)

• Regional data are used globally (here works INSPIRE)

• Global data are used globally (here works INSPIRE)

Regional data used regionally

There is not direct requirement for INSPIRE data modelsLocal data models could be wider

Local data models reflect regional needs and also regional decision processes

If data are not shared outside of region (but in many cases it is necessary), in principle global standards are not needed

Standards are needed in case of more data suppliers, to guarantee data consistence

Global data used regionally

Global data are in some content something like de facto standards

In some cases it is necessary to be possible transform data into such models, which is required by regional decision processes

The global model has to cover regional decision needs (GMES case for example)

Open problems: the transformation happens either on fly or as

pretransformed data snapshot

Language problem in the case 'on fly'

Regional data used cross regionally

In the case of cross border regions data harmonization faces extreme challenges.

In many cases, for example tourism, we need to deal simultaneously with several INSPIRE related data themes. This is much more complex task than single data theme case.

In some applications data model could be broader than corresponding INSPIRE definition.

Open problem – how to manage multi lingual problems

Regional data used globally

• Probably most relevant case for INSPIRE data model

• The idea is to combine several local data sets into single standardized data set

• The regional data has to be transformed (in many cases simplified) into global data model

• Relevant cases are tourism, transport, education, research, environment protection, risk management, strategic decision

• Language problem

Global data used globally

• Global data are either standard or de facto standard.

• It is expected that in the case of public sector data INSPIRE compliance will be guaranteed.

• Concrete application areas may require specific transformation. Transformation could be based on Feature Encoding or Styled Layer Descriptor (SLD)

T3.1 Conceptual data models

Task 3.1 Conceptual data Task 3.1 Conceptual data modelsmodels

•Deliverable 3.1 Conceptual data models

– Contact person / Task leader: Ota Čerba, Karel Charvat (HSRS)

•Subtasks

– Comparative analysis of data and metadata models currently used in EU countries and between project partners.

– Information sources:

• Content from project partners,

• Reports from national and EU level INSPIRE TWG's,

• Other EU projects (eg. Plan4all, HUMBOLDT).

– Comparison of national legislations and definition sets of items for metadata sharing.

– Description of conceptual models for single country to reach common agreement across EU.

Task 3.1 Task 3.1 Results and conclusions

• Collected information about each project partner – data descriptions and documentation, SDI state of the art and data examples necessary for detailed analysis.

• As defined data models are compatible with INSPIRE TWGs draft data models in M9 they are slightly different from final INSPIRE data models.

• HABITATS deliverable 3.1 Conceptual data models (finished M9) was contributed to NSPIRE TWG. In futher project work was used INSPIRE TWG actual data model versions.

T3.2 Metadata profile

Task Task 3.2 Metadata profile3.2 Metadata profile• Deliverable 3.2 Metadata profile

– Contact person / Task leader: Gregorio Urquía(TRAGSATEC)

•Delivery status:

– Draft M9 (Delivered)

– Final M15 (Delivered)

•Subtasks– Comparative analysis of data and metadata models

currently used in EU countries.– Information sources:• Collect content from project partners,• Collect and analyse reports from national and EU level

INSPIRE TWG's,• Comparison of available national legislations and definition

sets of items for metadata sharing.

T3.3 Data models and transformation processes

Task 3.3 Data models and Task 3.3 Data models and transformation processtransformation process

Initially by DOW was planned two seperated deliverables:

- T3.3 Tools for interactive data modelling

- T3.4 Transformation process design (Overview of task)

After M15 consortium aggreed to merge T3.3 and T3.4.

Contact person / Task leader: Peteris Bruns (IMCS), Jachym Cepicky / Karel Charvat (HSRS)

Authors: Jachym Čepický (HSRS), Jan Jezek (HSRS), Karel Charvat (HSRS), Maris Alberts IMCS), Peteris Bruns (IMCS), Ota Čerba (HSRS)

Task 3.3 Data models and Task 3.3 Data models and transformation processtransformation process

Input and information sources:Content from project partners (questionnaires, inputs to deliverable),Reports from EU level, INSPIRE TWG BR-HB-SD,Information from D3.1, D3.2 and INSPIRE TWG BR-HB-SD discussions,Analysis of good practice from other EU projects such as Plan4all, Humboldt etc.

Main tasks:To assist end users to describe their current models using interactive modelling toolsTo develop training materials and provide trainig sessions for users.To designs the main transformation processes required for data sets, based on the requirements activities of Task 2.3

Data transformation trainingsData transformation trainings

Training materials prepared and afterwards used in workshops for data transformation

Basic data transformation training principles are demonstrated on practical examples describing the process of data transformation using simple tools. The following operations were trained:

– Simple value mapping and extraction– Spatial data file manipulations– Geometry manipulations– How to create repeatable transformation scripts for routine or batch

transformation tasks

Advanced transformations were only demonstrated using advanced systems and tools.

Basic transformationsBasic transformationsBasic script for automated data merge of data maintained as multiple files

Basic data merge of data maintained in multiple file structure using free and opensource desktop application

Two solutions for data merging

Basic transformationsBasic transformations

Basic SQL script for geometry extraction from multigeometry, further can be used as one setep in larger data transformation process

Basic geometry extraction from multigeometry using free and opensource desktop application

Two solutions for data extracting

Advanced transformationsAdvanced transformations

Data Specifications 2.0

Habitats and biotopes

Harmonization

FMI Data

Advanced transformation – scheme (1)

Open SHP fileand its scheme

Save finalSHP file

ReclassificationFMI → EUNIS

New datamodel

New data mode (1)Existing FMI data model +

referenceHabitatTypeId: CharacterStringreferenceHabitatTypeScheme: ReferenceHabitatTypeSchemeValuelocalSchemeURI: URIlocalNameValue: CharacterString

geometry: polygonreferenceHabitatTypeId: eunis_valuereferenceHabitatTypeScheme: eunislocalSchemeURI: link_to_FMI_classificationlocalNameValue: FMI_classification_value

Data model mapping (1)

Taxonomy – reclassification(FMI → EUNIS) (1)

0 Pine → G3.42,"4","Middle European [Pinus sylvestris] forests"

1 Oak → G1.87,"4","Medio-European acidophilous [Quercus] forests"

2 Beech-oak → G1.82,"4","Atlantic acidophilous [Fagus] - [Quercus] forests"

3 Oak-beech → G1.82,"4","Atlantic acidophilous [Fagus] - [Quercus] forests"

4 Beech → G1.6,"3","[Fagus] woodland"

5 Fir-beech → G4.6,"3","Mixed [Abies] - [Picea] - [Fagus] woodland"

6 Spruce-beech → G4.6,"3","Mixed [Abies] - [Picea] - [Fagus] woodland"

7 Beech-spruce → G4.6,"3","Mixed [Abies] - [Picea] - [Fagus] woodland"

8 Spruce → G3.1D,"4","Hercynian subalpine [Picea] forests"

9 Dwarp pine → F2.45,"4","Hercynian [Pinus mugo] scrub"

Reclassification (1)

FMI Data

INSPIRE / HabitatsData

Advanced transformation (2)

CQLfilter

Advanced transformation (3) – Ontology

Description

Nomenclatures

Derivedtransformation

rules

Description in ontology (3)

Examples of transformed structures

Results and conclusionsTransformation of data from multiple data models into a unified data model provides an opportunity to compare and use data in a wider area and from multiple data providers. It stipulates the economy in through research, cross-border cooperation and other important areas.We provide successful experimentation with on fly coordinate transformation

Results and conclusionsThe experimentation demonstrate, that in most cases is not realistic on fly data model transformation (it is in relation with other relevant projects like HUMBOLT, Plan4all, EuroGEOSS or implementation of Czech INPSIRE portal or Czech Cadastre). There is need for replication of so called INSPIRE data or there could be used brokerage model like in EUROGEOSS project, which provide harmonization of data. There is also clear need to thing about other approaches then currently WMS, WFS based services

Results and conclusions

There is important to understand, why we are harmonizing data? What is main purpose of data transformation? It is necessary only in cases, that data are shared across regions and countries. If pilots are working standalone, then standardization is not necessary On other side we have recognize, that INSPIRE data models are only some minimal set, covering basic needs. We have expect, that inside of different communities will be necessary to extend this models, to fulfill requirements, of communities. Every used data model has cover needs of users

Results and conclusionsThe conditions for data harmonization of the HABITATS pilots differ due to the facts that:Partners has different needsIn some cases partners were more data users, then data supplierspartners have different levels of development and GI technologies;partners have a different level of expertise in GI technologies;partners have a different understanding of spatial data and models.Different conditions impose certain restrictions and requirements for the harmonization process of data and metadata.

Results and conclusions Habitats contribute to INSPIRE Standardization process. Habitats made recommendation for different technologies, which could be used in certain context of data harmonization Habitats provide training of stakeholders for data harmonization Habitats made demonstration of harmonization data on more pilots Habitats design advanced ontology based concept for data harmonization in relation to solved tasks

Thanks!