xslt for inspire transformation by dutch kadaster in esdin
DESCRIPTION
Slides of presentation given at EuroGeographics KEN workshop on INSPIRE Data Harmonization, Paris oct 8-9, 2013: http://www.eurogeographics.org/event/inspire-ken-schema-transformation-workshop. Describes the XSLT-based INSPIRE Transformation technique used by Dutch Kadaster in EuroGeographics ESDIN INSPIRE project (2009-2011).TRANSCRIPT
XSLT FOR
INSPIRE TRANSFORMATION
In ESDIN by Kadaster
KEN - Paris - Oct 8, 2013
About MeIndependent Open Source Geospatial Professional
Secretary OSGeo Dutch Local Chapter Member of the Dutch OpenGeoGroep
Just van den [email protected] www.justobjects.nl
OPEN SOURCEFOR INSPIRE
MAKES SENSE
Picture from: http://cameronshorter.blogspot.com
3. COMMUNITYENABLER
INSPIRE Projects
2009-2011 Dutch Kadaster - Kademo Lab - EURADIN and ESDINinspire.kademo.nl - Ebrahim Hemmatnia - Ko van Raamsdonk - Haico van der Vegt
2011 Kadaster - INSPIRE Transformations Key Registries Data CP,AD,TN,GN,HY inspire.kademo.nl
2011-2013 PDOK - National GeoPortal with INSPIRE www.pdok.nl
2010-now - cooperation deegree community - Lat/lon, Occam Labs www.deegree.org
Talk 1
XSLT - what, how
ESDIN - INSPIRE transformation Kadaster
Talk 2
formalizing approach from Talk 1
using Stetl framework for INSPIRE
AGENDA
EXTENSIBLE STYLESHEET LANGUAGE (XSL)
Origins : SGML+DSSSL ==> XSL for XML documents
W3C XSL Working Group
First called “XSL”, then split into three standards
1.XSL Transformations (XSLT)
2.XSL Formatting Objects “FOP”
3.XPath - XML Navigation/Query Language
PRINCIPLE
The XSLT processor takes one or more XML source documents, plus one or more XSLT stylesheet modules, and processes them to produce an output document. The XSLT stylesheet contains a collection of template rules: instructions and other directives that guide the processor in the production of the output document.
http://en.wikipedia.org/wiki/XSLT
EXAMPLE
http://en.wikipedia.org/wiki/XSLT
Input XML Document
EXAMPLE
http://en.wikipedia.org/wiki/XSLT
XSLT Stylesheet Document
EXAMPLE
http://en.wikipedia.org/wiki/XSLT
XML Result Document
EXAMPLE
http://en.wikipedia.org/wiki/XSLT
HTML Generation with Alternative XSLT Stylesheet
EXAMPLE
http://en.wikipedia.org/wiki/XSLT
(X)HTML Result Document (alternative Stylesheet)
XSLT PRINCIPLES
Basically Tree Transformation
Input: well-formed XML document
Output: XML document or anything “text” (SQL!)
Declarative, rule-based, functional, recursive
But also: Structured and Procedural (named templates)
XSLT PRO’S
Well-established (sinds 1999)
stable standards
huge amount of knowledge on the web
Optimized XSLT tools in any programming language
XML-based, same base language as source/target docs
Compact, reusable stylesheets (ala subroutines)
Extension points for custom code
XSLT PRO’S CONT.
TEXT
IN THE BEGINNING...
"In the Beginning was the Command Line"
By Neal Stephensonhttp://www.cryptonomicon.com/beginning.html
XSLT CONS
Complex
intellectual challenge: thinking declarative/functional
poorly understood, often messy stylesheets
Poor scalability with large documents
No GUI
INSPIRE: only model transform (no coordinate transform)
IMPLEMENTATION
INSPIRE - WHAT
WFSWMSCSWWPS
.
.Local Data
INSPIRE Data
Web Services
INSPIRE - DATA THEMES
INSPIRE - TRANSFORMATIONS
source: Geonovum.nl
KADASTER - PHASE 1COMBINED TRANSFORMATION
On-the-fly Transformation
Offline Transformation
National Data
(Oracle)
Extract
Transform
Load (ETL)
INSPIRE Data
(PostGIS)
Web
Feature
Service
(WFS)
INSPIRE
GML
OSGeo - Bolsena - 2010
KADASTER - PHASE IIOFFLINE TRANSFORM + DEEGREE
INSPIRE
View Service
(WMS)
Extract
Transform
Load (ETL)
INSPIRE
Feature Store
(BLOB or Hybrid)
INSPIRE
Download Service
(WFS)
deegree 3
INSPIRE Publish
1) WFS-T
2) deegree loader
Configuration Data
- INSPIRE Data Themes (XSD)
- Data Mappings (XML)
- WMS Styling (SLD)
Kadaster Data
(Oracle)
KADASTER - PHASE IIREUSABLE ETL + SERVICES
INSPIRE
View Service
(WMS)
Extract
Transform
Load (ETL)
INSPIRE
Feature Store
(BLOB or Hybrid)
INSPIRE
Download Service
(WFS)
deegree 3
INSPIRE Publish
1) WFS-T
2) deegree loader
Configuration Data
- INSPIRE Data Themes (XSD)
- Data Mappings (XML)
- WMS Styling (SLD)
Kadaster Data
(Oracle)
ReusableSpecific
KADASTER - PHASE IIETL CHAIN
ETL CHAIN - TOOLS
• Extract
• internal tools: e.g. Oracle to Shapefile or XML
• Transform
• GDAL/OGR - Coordinate and create Simple Features (GML)
• XSLT - Model Transformation
• Load
• directly in PostGIS database
• via deegree FeatureStore Loader
• via WFS-T
SOFTWARE STACK
Architecture
PostGIS
WFS
deegree
LocalData
ScriptsGDALXSLT
Unix shell
ETL
INSPIRE Download/View Services
TomcatJava J2EE
Server
Heron Mapping Client
WFS-TFSLoad
WMS
SLD
data flow
API (web service) API
storage
ESDIN Test Framework
Now Stetl!
XSLT STRUCTURE
Local CP GMLto
INSPIRE SpatialDataset
Local CP GMLto
INSPIRE GML
GenerateCP INSPIRE GML
ReusableXSLT ScriptsReusable
XSLT Scripts
Theme CP
Local AU GMLto
INSPIRE SpatialDataset
Local AU GMLto
INSPIRE GML
GenerateAU INSPIRE GML
Theme AU
Local GN GMLto
INSPIRE SpatialDataset
Local GN GMLto
INSPIRE GML
GenerateGN INSPIRE GML
Theme GN
Called by All
Locally Specific XSL
GenericXSL
XSLT Template Call
XSLT - 3 MAIN STEPS/SCRIPTS
1.Generate Spatial Dataset GML Container (specific)
2.Extract data values from local OGR simple feature data (specific)
3. Call XSLT template per Theme Feature type (generic)
XSLT - STEP 1
XSLT - STEP 2
XSLT - STEP 3
XSLT - REUSE
THE “GUI”
validated file contentproject files
commands
run deegree
CONCLUSIONS
1.XSLT for INSPIRE (Model) transformations ? YES
2.XSLT shines in combination with OGR and PostGIS
3.Don’t Develop INSPIRE Transforms in isolation but in context of
your global INSPIRE SDI Architecture
4. Kadaster overall approach in ESDIN worked well
GETTING INVOLVED
INSPIRE @ OSGEO
http://lists.osgeo.org/mailman/listinfo/inspire-data
http://wiki.osgeo.org/wiki/INSPIRE_data_experiments
MAILING LIST
WIKI
LINKS
STETL
PROJECT SITE
DOCU DEMOS AND INFO
http://inspire.kademo.nl
http://inspire-foss.org
http://stetl.org