spatial processing redefined with sap hanasap hana vora is an in-memory query engine which leverages...
TRANSCRIPT
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 1Public
SPATIAL processing redefined with SAP HANA
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 2Public
Disclaimer
This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 3Public
Agenda
• SAP HANA
• SAP HANA Spatial
• SAP & Esri
• Outlook
• Live Demo
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 4Public
SAP HANAA common Database Approach for OLTP and OLAP Using an In-Memory Column Database
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 5Public
SAP HANABasic Technology
In-Memory Computing• Keep Data in memory to speed up data access
• Enterprise server with terabytes of main memory (scale-up)
• Minimize data movement by• column store
• compression
• performing calculations at database level
• Divide and Conquer• Use multi-core architecture of processors, multi-processor servers
• scale-out into a distributed landscape
12x17x
2000x
150x
Register Cache RAM SSD/Flash Hard Disk
1
0,001
1.000.000
1.000
volatile non-volatile
Id Date Name
1 123 12
2 200 32
3 333 18
# Name
12 John
32 Flore
18 Lee
1 2 3 123 200 333 12 32 18
1 123 12 2 200 32 3 333 18
IN-MEMORY, ACID-COMPLIANT, COLUMN BASED
MULTI-CORE/PARALLELIZATION
ADVANCEDCOMPRESSION
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 6Public
Supports any Device
Any appsAny app serverAny apps
Any app serverSAP Business Suite
SAP NetWeaver Application Server for ABAP (SAP Business Warehouse)
SAP Business Suite SAP NetWeaver Application Server for ABAP
(SAP Business Warehouse)
JSONR Open ConnectivityMDXSQL
Other appsLocationsReal timeHADOOPMachineUnstructuredTransaction
SAP HANA platformSQL, SQLScript, and JavaScriptSQL, SQLScript, and JavaScript
Integration ServicesIntegration Services
SpatialSpatial
Business function library
Business function library
SearchSearch Text miningText mining
Predictive analysis library
Predictive analysis library
DatabaseservicesDatabaseservices
Stored procedure and data modelsStored procedure and data models
Planning enginePlanning engine Rules engineRules engine
Application and UI services
Application and UI services
The SAP HANA platform converges database, data processing, and application platform capabilities and provides libraries for predictive, planning, text,
spatial, and business analytics to enable business to operate in real time.
SAP HANA platform More than just a database
Geospatial TechnologiesSAP HANA Spatial
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 8Public
Spatial processing with SAP HANA
SAP HANA Spatial Engine: Native spatial engine as part of column store Colum-wise storage of spatial data Spatial data types (ST_POINT, ST_GEOMETRY) SQL/MM spatial (ISO/IEC 13249) OGC Simple Feature Access - SFA (TF) 1.2.1 Points, line strings, polygons, circular strings … Spatial predicates, functions, formats … Spatial reference systems, units of measure … Access via SQL, SQLScript, CalcEngine, CDS,
Modeler, MDS, … Application development on XS with geo-content and
mapping services ESRI query layers, feature services support Multidimensional type support Clustering …
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 9Public
Spatial Column Scan
MULTICORE + SIMD + RAM
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 10Public
AdvancedGeoSpatialCapabilities
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 11Public
Clustering
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 12Public
ClusteringNative Clustering Algorithms
x
y
Grid
x
y
DBSCAN
x
y
K-Means
• Good for first impression• Easy to use• Extremely fast grid clustering
• Best for non-spherical clusters• Density based• Higher complexity, better insights
• Best for spherical clusters• Centroid based• Higher complexity, better insights
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 13Public
ClusteringSQL Example – Use Cases
Identify for each point the corresponding clusters (similar to window functions)
SELECT ST_ClusterId()
OVER (CLUSTER BY location USING KMEANS CLUSTERS 5 ) AS cluster_id, vm_id, location, revenue
FROM vending_machinesWHERE revenue < 15000 ORDER BY cluster_id, revenue
Clustering with metadata accessor(similar to group by aggregates)
SELECT ST_ClusterId() AS cluster_id, ST_ClusterCentroid() AS centroid, ST_ClusterEnvelope() AS envelope, count(*) AS num_hholds, avg(income) AS avg_clus_income
FROM households WHERE income > 120000 GROUP CLUSTER BY location USING DBSCAN EPS 4 MINPTS 1000 HAVING count(*) >= 300
Cluster_id Centroid Envelope Num_hholds
Avg_clus_income
1 Point(5 4) Polygon((…)) 311 304.123
2 Point(15 78) Polygon((..)) 621 714.234
…
Cluster_id Vm_id Location Revenue
1 1 Point(1 1) 1.311
1 2 Point(1.4 1.2) 1.166
1 3 Point(1.2 1.3) 799
2 4 Point(5.3 5.0) 2.125
2 5 Point(5.7 6) 1.750
3 6 Point(20 20) 1.532
…
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 14Public
GeospatialEcosystem
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 15Public
SAP HANA Spatial Ecosystem
GIS
Maps, Geo-content & Geo-services
Location Intelligence Solutions
GIS Systems Integrator and Consultancy
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 16Public
EsriEsri
ArcGIS Server & Desktop
SAP HANA + Esri – IntegrationTechnical Overview
Esri
Data ExchangeEsri Shapefile
support for ‘offline’ manual data movement
SAP HANA
ArcGIS Server & Desktop
Query Layers“Push-down”
spatial and non-spatial queries
directly into SAP HANA Spatial
engine
Esri Shapefile(.shp)
SAP HANA
Query Layers
ETLBatch data movement
via SAP Data Services, Safe FME, Esri Interoperability Extensions, or other third-party solutions
SAP HANA
ETL
Feature ServicesWeb services
allowing reading, writing, editing, and
symbology of spatial data from
SAP HANA
SAP HANA
Feature Services
Esri
ArcGIS Server & Desktop
ArcGIS Server & Desktop
Geo-ServicesIntegration with Esri
Server & ArcGIS Online for mapping,
content, and advanced geo-service
processing
SAP HANA
XS Engine
Esri
ArcGIS Server & ArcGIS Online
Spatial Data & Processing
Spatial Data & Processing
Spatial Data & Processing
Spatial Data & Processing
Spatial Data & Processing
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 17Public
Esri
Advanced Geoprocessing
Esri advanced functions running
in SAP HANA
SAP HANA
ArcGIS Server & Desktop
Spatial Data & Processing
Esri Embedded Process
Esri
GeodatabaseSAP HANA as a
certified geodatabase
SAP HANA
ArcGIS Server & Desktop
Spatial Data & Processing
Esri ArcSDE
SAP HANA + Esri – IntegrationNext Steps & Vision
Geodatabase• Full geodatabase support for SAP HANA
• topologies• Networks• Versioning• …
• Coming Q4 2016 with ArcGIS v10.4.2
Advanced Geoprocessing• Leveraging SAP HANA technology with AFL SDK
• Hotspot analysis• Stream network• Watershed• Geometric network trace• …
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 18Public
Geographic Data Enrichment
Spatial Analytics
Rich Mapping
ArcGIS For ServerSAP HANA
ArcGIS For Desktop
ArcGIS Online
SAP
ERP
CRM SCM
SAP and EsriBringing Location to the Enterprise
HCP / HEC
SAP HANA
SAP & Esri Portfolio• Enterprise Applications• Database Technology• Cloud Services• Mobile SDK• Analytics
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 19Public
ENTERPRISE BIG DATA
Bridging the Digital Divide
Introducing
SAP HANA Vora
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 19
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 20Public
The Five Megatrends Driving Our Digitized WorldAnd Their Implications for Distributed Big Data Management
Hyper ConnectivityEverybody has
access
Super Computing
Super computers power everywhere
Cloud Computing
The cloud is where we compute
Smart World
Your fridge knows what you want for
dinner
Cyber-Security
High-powered security is now the
norm
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 21Public
SAP HANA VoraWhat’s Inside and What Does It Do?
DemocratizeData Access
Make PrecisionDecisions
SimplifyBig DataOwnership
SAP HANA Vora is an in-memory query engine which leverages and extends the Apache Spark execution framework to provide enriched interactive analytics on Hadoop. Drill Downs on HDFS
Mashup API EnhancementsCompiled Queries
HANA-Spark ControllerUnified LandscapeOpen Programming
Any Hadoop Clusters
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 22Public
YARN
HDFS
Enable Precision DecisionsWith Contextual Insights In Enterprise Systems
Other Apps
Files Files Files
HANA-Spark Controller for improved performance between distributed systems
Gain business coherence with business data, spatial data and big data
Compiled queries enable applications & data analysis to work more efficiently across nodes
Familiar OLAP experience on Hadoop to derive business insights from big data such as drill-down into HDFS data
Compiled Queries
Spark Controller
Drill Downs
SAP HANA in-memory platform
Vora
Spark
Vora
SparkIn-Memory
StoreApplication Services
Database Services
Integration Services
Processing Services
SAP HANA Platform
Vora
SparkHANA Smart Data
Access Spark Controller
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 23Public
Spatial Compliance
Co-Innovation (Customers, ESRI)
Customer-Driven types, functions
Start with OGC
SQL M/M Standard
Import/Export
HANA spatial roadmapNext goals
SQLOptimizer / Statistics
Full Round Earth support
Raster data
New Innovations & Features
Geometry Compression
Spatial temporary analysis
OGC services (WFS, CSW, …)
QGis, MapServer, GeoServer, …
Topology-, Network-support
Routing, GeoCoder, …
3D function support (e.g. ST_Intersects3D), Visual Enterprise, AutoCAD, …
Streaming
Polygon Simplifier
SPS10
multi-dimensional type support
ST_Transform
ST_AsSVGAggr
Spatial preview in hdbstudio
Spatial support in CDS
Spatial support Modeller
Improved support forSQLScript
Improved supportability
SPS07 SPS10SPS08This is the current state of planning and may be changed by SAP at any time.
SPS09
SPS09
Spatial Join
Full Spatial Support in CDS
First Spatial Support SQLScript
GEOCODE INDEX
HERE Partnership
ST_CircularString
ST_UnionAggr
SPS11 next
SPS11
Clustering
ConvexHullAggr, EnvelopeAggr, IntersectionAggr
Scalar type support in SQLScriptand LLANG
Multi dimensional modeling(Cloud for analytics)
Partitioning with spatial types
Metadata Installer
OGC compliance (SFS 1.2.1)
Spatial support in SDA
Grid Join improvements
SFC improvements
SPS12
Performance Optimization(grid join, scans)
Focus on Esri development
Modeler support with SpatialFilter and calcuated columns
Spatial Type support in AFL
Spatial Viewer
Extended Round Earth Support
Research
PoC
Graph and Spatial(networking and topologies)
Developer howtos, Development Series, Blogs, whitepapers, documentation, hands on labs
SPS12
© 2014 SAP SE or an SAP affiliate company. All rights reserved. 24Public
LIVEdemo
© 2014 SAP SE or an SAP affiliate company. All rights reserved.
Thank youSpatial Processing with SAP HANAhttp://hana.sap.com/abouthana/hana-technology/advanced-analytics.html
SAP HANA Spatial Referencehelp.sap.com/hana/SAP_HANA_Spatial_Reference_en.pdf
SAP HANA Academywww.saphana.com/academy
HANA Spatial Team