enabling access to big geospatial data with locationtech and apache projects
TRANSCRIPT
Rob Emanuele
ENABLING ACCESS TO BIG GEOSPATIAL DATA WITH
&
What we’ll be covering…
LocationTech projects that geospatially enable Apache big data frameworks by providing spatial indexing.
Discuss how those four project approach indexing, focusing on the use of space filling curves.
STORING AND PROCESSING GEOSPATIAL DATA @ SCALE
STORING AND PROCESSING GEOSPATIAL DATA @ SCALE
WHAT IS ?
GEOJINNI(FORMERLY SPATIALHADOOP)
SPACE FILLING CURVES
00 01
1011
10
11 00
01
11
10
00
01
Hilbert Index (52) = 11 01 00
Geo +
accessed through
Z curve
Z curve(also XZ)
Geo +
accessed through
GEOWAVE
Hilbert Curve
Range Decomposition
70 -> 75 92 -> 99 116 -> 121
False positives - secondary filtering
Geo +
Rasters +
Z or Hilbert
Data Node
Data Node
Data Node
Name Node
Master
Tablet Server
Tablet Server
Tablet Server
Accumulo
BigTable clone (columnar database)
Records stored on HDFS
Lexicographically sorted table index
partitionid
splitid
splitid
partitionid
Tiered Indexing
Tiered Indexing
Periodicity (time dimension)
1997 1998 1999
Periodicity (arbitrary dimensions)
Time
Elevation
Velocity
Spatial index stored per file on HDFS
Z order (2D and 3D),Hilbert (N-Dimensional)
Z order (2D and 3D)Binned per week for spatiotemporal
N-Dimensional Hilbert witharbitrary binning and tiered indexing
Spatial Indexing
CQL
Future integration work
?
THANK YOU
@lossyrob
gitter.im/geotrellis/geotrellis
github.com/geotrellis/geotrellis
GeoMesa
GeoWave
Tiered Indexing