![Page 1: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/1.jpg)
Cloud-based Geospatial Data services and
analysis Xuezhi Wang
Scientific Data Center
Computer Network Information Center Chinese Academy of Sciences
2014-08-25
![Page 2: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/2.jpg)
Outlines
1 Introduction of Geospatial Data Cloud
2 Cloud-based Geospatial Data Analysis platform
3 Cloud-based Geospatial Data visualization platform
![Page 3: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/3.jpg)
Geospatial Data Cloud
I. Platform overviews II. Geospatial data services III. Geospatial model services IV. Services achievements
![Page 4: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/4.jpg)
Why do we do?
In 2008, USGS opened all geospatial datasets to the public for free.
• Landsat, MODIS is the most popular remote sensing dataset for scientists and researchers. Especially, the Landsat series.
• Due to the low international network speed, many Chinese scientists cannot take advantage of these resources easily.
• Based on the CNIC’s mass storage and high speed network( CSTNET: Chinese Science and Technology Network), we mirrored the geospatial datasets covered China and surrounding area, and provided online data services for
chinese scientists and researchers
![Page 5: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/5.jpg)
Design a new cloud-based geospatial analysis and visualization platform
USGS opened its Landsat datasets. we contacted the USGS staffs through email.
We got an oral permit for Landsat redistribution from USGS. we download the Landsat datasets, and provide public
services for Chinese scientists
Cooperated with National Center for Atmospheric Research (NCAR)
Mirrored the NCAR datasets provide public services for Chinese
Scientists
We got a permit for SRTM dataset redistribution from International Centre for Tropical Agriculture(CIAT). We cache
SRTM global elevation data, and provide online data services.
Accroding to the user requirements, We make some remote sensing products from Landsat dataset and Modis dataset.
such as vegetation index, water cover dataset.
Dowload MODIS dataset covered CN Host the first user conference
Design a new cloud-based data platform
Geospatial data cloud opened for public services Host the second user conference with ESRI China
Platform History
![Page 6: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/6.jpg)
Platform objectives Based on our cloud infrastructure, we hope to construct an 'one-stop'
geospatial data service platform for Chinese scientists and researchers.
An open cloud-based platform meeting the requirements of data search, data acquisition, online storage, scientific models, online calculation and map visualization.
Model
Online Calculation
Search Acquisition
Storage Visualization
http://www.gscloud.cn
![Page 7: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/7.jpg)
General framework
Database cluster Online storage Near-line
storage
Data integration
Automatic Metadata crawling
Automatic data
entities cache
Mass Storage
Cloud environments
Distributed file system
Virtual Machines
Task scheduling
Data query
Data category
Integrated search
Complex search
One-key indexes
Data services
Smart data Order
Data application Favorite data Batch
download
Model services
Model package
Online execution
Web map services
Map publish Map visualization
Map style edit
Model management
User data spaces
Data management
Task management
Map management
Model flow
Map share
Service management
System managem
ent Users
Data
Models
Indexes
Tasks
Services
Resources monitor
![Page 8: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/8.jpg)
Geospatial Data Cloud
I. Platform overviews II. Geospatial data services III. Geospatial model services IV. Services achievements
![Page 9: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/9.jpg)
Online data resources Open geospatial datasets: LANDSAT、MODIS、EO-1、DEM、NCAR、
NOAA and LUCC
More than 350TB data entities, and over 7 million data records.
Covered China and surrounding area
Online services for Chinese scientists and researchers with free of charge
Landsat1-3 MSS Landsat 4-5 MSS Landsat 4-5 TM Landsat 7 SLC-on Landsat7 SLC-off Landsat 8 OLI
TERRA AQUA
TERRA AQUA
GDEM 30m SRTM 90m Ocean elevation
HYP_L1R HYP_L1G
AL1
LANDSAT MODIS MODIS_L1B DEM EO-1
![Page 10: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/10.jpg)
Generated geospatial products Water-cover of China from Landsat TM/ETM in Summer
Slope, Aspect, Topographic position indexes et. of China
Land Surface Temperature, Vegetation indexes (5 days, 15 days, and 30 days) of China, from 2000 – now
ASPECT Water Cover LST
![Page 11: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/11.jpg)
Geospatial data query
Massive geospatial metadata
I need Landsat data in
summer, 2000
I need DEM data of Beijing
Input: geo-location or address
PostgreSQL+PostGIS Cluster
Based on geospatial
storage cluster and search
engine, we provide three
ways for users to access
their preferred data:
1: by address name
2: by geo-location
3: by administrative unit
![Page 12: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/12.jpg)
Platform Screenshots 1. search by address name
![Page 13: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/13.jpg)
Platform Screenshots 2. Search by geographic coordinates
![Page 14: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/14.jpg)
Platform Screenshots 3. Search by polygon features
![Page 15: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/15.jpg)
Platform Screenshots 4. Search by administrative unit
![Page 16: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/16.jpg)
Platform Screenshots The result page that matched the query criteria
Extra attribute
filters
![Page 17: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/17.jpg)
Geospatial Data Cloud
I. Platform overviews II. Geospatial data services III. Geospatial model services IV. Services achievements
![Page 18: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/18.jpg)
Geospatial model services Why do we need this services?
For example: If we need Beijing DEM data, there have two ways. Traditional way: search DEM data, download DEM data,
mosaic DEM data with ArcGIS, and then clip DEM data with Beijing shape mask .
Convenience way: select Beijing administrative unit in the web, submit a clip task to system, and then download Beijing DEM data.
Which is the best way? I prefer the second way.
That’s why we provide online geospatial model services.
![Page 19: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/19.jpg)
We developed an elastic online task processing system with cloud techniques.
Users fill model parameters on the web page and submit tasks to the system;
The tasks were run under cloud virtual machines;
The results were stored in cloud storage.
Task scheduling
Task
Task
Task
Computing resource pool Storage pool
VM VM
VM
virtual machines in Cloud
Overall frameworks
Model page Model page
![Page 20: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/20.jpg)
Online models and tools DEM tools:
DEM clip and mosaic; Reprojection; Aspect, Slope, Topographical index calculation of
specified region LANDSAT tools:
Online gap-fill model for Landsat ETM7 SLC-off dataset
MODIS tools: Online Land Surface Temperature calculation of user
specified time range Online vegetation indexes calculation of user specified
time range
![Page 21: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/21.jpg)
Private cloud storage We provide user with private cloud storage to store their own model inputs and outputs data.
User manage their own data here, including upload, download, rename, delete, create folder and move
![Page 22: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/22.jpg)
Platform Screenshots Online category page for Geospatial tools and models
![Page 23: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/23.jpg)
Platform Screenshots Model parameters page of DEM clip and mosaic tool
User only need to fill task name, select a region in the web map, and choose output folder
![Page 24: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/24.jpg)
Platform Screenshots Task tracking page list user’s submitted tasks
They can tracking task states, stop and start task, adjust task priority here.
![Page 25: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/25.jpg)
Platform Screenshots Task result page
When the task was finished, the results can be download from here, or from user private cloud storage
![Page 26: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/26.jpg)
Geospatial Data Cloud
I. Platform overviews II. Geospatial data services III. Geospatial model services IV. Services achievements
![Page 27: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/27.jpg)
Our platform was wildly used in many research fields in China: earth science, environmental science, ecology, hydrology, meteorology, oceanography, natural disasters, city planning ...
Service target
![Page 28: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/28.jpg)
Based on statistics in November 2013
User characteristics
Currently, over 70,000 users
Doctor
Master Bachelor
Others
University
CAS
Government
Company
Others
![Page 29: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/29.jpg)
Service statistics
Avg. downloaded over: 200 GB/day; online downloaded over: 250 TB;
offline copied over: 137 TB;
![Page 30: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/30.jpg)
Outlines
1 Introduction of Geospatial Data Cloud
2 Cloud-based Geospatial Data Analysis platform
3 Cloud-based Geospatial Data visualization platform
![Page 31: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/31.jpg)
Cloud-based Geospatial Data Analysis Platform
Why do we need this platform? For example: If we want to calculate Chinese monthly
temperature variation from 2000 to 2014, how can I achieve this?
Traditional way: download all data, and calculate with workstations. It may spent several months to download data. This is a very bad thing.
Shortcuts: using cloud computing technology. Apply virtual machines in the cloud; Write code in the virtual machines; read all data from cloud storage; Run program in parallel
Which way is the best? I prefer the second way. That’s why we want to construct this platform.
![Page 32: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/32.jpg)
Overall frameworks
...... Storage resource pool
Cloud manager with OpenStack
Distributed file system
Data pool
Computing resource pool
Public geospatial data
User private geospatial data
virtual machine pool
VM VM VM VM
![Page 33: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/33.jpg)
Currently progress of the platform We have finished the prototype system. The system is
under inner tests for access pressure and stability. In the end of this year, it will be provided for the public access.
![Page 34: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/34.jpg)
Outlines
1 Introduction of Geospatial Data Cloud
2 Cloud-based Geospatial Data Analysis platform
3 Cloud-based Geospatial Data visualization platform
![Page 35: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/35.jpg)
Cloud-based Geospatial Data Visualization Platform
Why do we need this platform? In many cases, users want to publish their geospatial data,
model results, vector map to the web site; They also want to customize the map layer styles through
the web pages; They also want to customize the components and styles of
the webgis pages; To achieve this goal, we construct this platform.
![Page 36: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/36.jpg)
Optimized Web Map Service
Computing resource pool Storage pool
VM VM
VM
Dynamical render map in Cloud HTML5 map client ...
Tiles Storage
Dynamical cache
Map styles editor
Map request
Overall frameworks
![Page 37: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/37.jpg)
Currently progress of the platform Most of system functions were already finished,
under inner tests and bug fixed.
Published services provided:
OGC WMS(web map service), Google TMS , OGC WFS (web feature service)
GeoJson , WorldWind ...
![Page 38: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/38.jpg)
System screenshots I. Render multi-bands raster layer with RGB false-color,
custom band composition
![Page 39: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/39.jpg)
System screenshots II. Render thematic raster layer with unique color theme
![Page 40: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/40.jpg)
System screenshots III. Render raster layer with classified color theme
![Page 41: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/41.jpg)
System screenshots IV. Render raster layer with gradient color theme
![Page 42: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/42.jpg)
System screenshots I. Render vector layer with Simple symbol
![Page 43: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/43.jpg)
System screenshots II. Render vector layer with colorful symbol by unique
attributes
![Page 44: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/44.jpg)
System screenshots III. Render vector layer with colorful symbol by classified
attributes
![Page 45: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/45.jpg)
System screenshots IV. Render vector layer with sized symbols by classified
attributes
![Page 46: Cloud-based Geospatial Data services and analysis](https://reader031.vdocuments.us/reader031/viewer/2022012300/61e15a42574dcf38696255ce/html5/thumbnails/46.jpg)
System screenshots Vector attributes query and edit