Scientific and Multidimensional Raster

Support in ArcGISHong Xu, Kaixi Zhang

Outline

• Multidimensional raster data model

• How to visualize

• How to analyze

• How to manage and disseminate

• Q/A

Visualize

Analyze

Manage

disseminate

Data Model

Scientific Data and Multidimensional Raster

Meteorology

• Air temperature

• Atmospheric moisture

• Wind speed

Oceanography

• Sea temperature

• Salinity

• Ocean currents

• Algal blooms

Weather Forecasting

• Temperature

• Humidity

• Precipitation

• Dew point

Conservation

• Land temperature

• Species ranges

• Vegetation cover

• Land cover

Remote Sensing

• Landsat

• Sentinel

• MODIS

Climate

• Soil moisture

• Sea ice

• Snow cover

Multidimensional Raster

• Contains one or multiple variables in one file

- A variable is cube or cubes

- A variable has a time and/or depth

- Each slice is a 2D array

• Common formats

- NetCDF, GRIB, HDF

Spatial, temporal, and vertical dimensions

141 241 341 441

131 231 331 431

121 221 321 421

111 211 311 411

142 242 342 442

132 232 332 432

122 222 322 422

112 212 312 412

143 243 343 443

133 233 333 433

123 223 323 423

113 213 313 413

t = 1t = 2

t = 3

tim

e

longitude

Multidimensional Raster Data Model in ArcGIS

• Direct access NetCDF, GRIB, and HDF

• Multidimensional mosaic dataset

• Multidimensional Cloud Raster Format (CRF) (2.4)

Local or cloud storage

NetCDF CF convention

GRIB NCEP convention

HDF HDFEOS

time

longitude

Multidimensional

Mosaic Dataset

Multidimensional CRFNetCDF, GRIB, HDF

Sensor/

Rasters

ArcGIS Multidimensional Cloud Raster Format (2.4)Designed for multidimensional raster storage and analysis

• Enhanced from CRF format

- Optimized for fast slicing operation

- Transposed for optimal profiling operation

• Create from an existing multidimensional raster

- Using Copy Raster Geoprocessing tool

- Multidimensional option

- Transpose option

- Build Multidimensional Transpose

time

latitu

de

tim

e

longitude

Visualize Multidimensional Raster

Multidimensional Raster - Visualization

• Use Add Multidimensional Raster Layer tool

- One layer per variable

• Drag drop from Catalog Pane

- Each format has its default behavior

How to Add to ArcGIS Pro

Multidimensional Raster – VisualizationHow to display in ArcGIS Pro

• Time slider

• Range slider

• Temporal Profile

• Rendering

- Raster renderer

- Vector field renderer

- For wind and current

Visualize

Multidimensional

Raster

Analyze multidimensional raster using the new Geoprocessing tools

Analyze multidimensional raster with raster functions

Analyze multidimensional raster using ArcPy

Analysis

Multidimensional Raster Analysis – Using Geoprocessing Tools (2.4)

• Aggregate Multidimensional Raster

- Along a given dimension and interval

- Yearly, monthly, daily etc.

- Output multidimensional CRF

• Generate Multidimensional Anomaly

- A long time series raster

- Output multidimensional CRF

Require ArcGIS Image Analyst or Spatial Analyst license

Input Aggregated

mean

Anomaly

Input

Multidimensional Raster Analysis - Using Raster FunctionsCube based analysis / cube based map algebra

• Support multidimensional raster (except global functions)

• Functions with one input – just works

• Functions with two or more inputs - rules (2.4)

- Variable names must match

- Disable the Match Variables option

Unit Conversion GreaterThan

V1 + V2V1

V2

Multidimensional Raster Analysis - Using ArcPy (2.4)

• Enhanced Raster class for multidimensional raster

- mdRaster1 = arcpy.Raster(“temperature.nc”, True)- mdRaster1.mdinfo- mdRaster1.save(“c:/data/mdraster.crf”)

• Support multidimensional map algebra

- mdRaster1 + mdRaster2

• New arcpy.ia functions- arcpy.ia.Apply- arcpy.ia.Foreach- arcpy.ia.Subset- arcpy.ia.Aggregate- arcpy.ia.Anomaly

• Interop with NumPyArray

- arcpy.RasterToNumPyArray and arcpy.NumPyArrayToRaster

Scripting environment for scientists

(V1 + V1, V2 + V2)

mdRaster1(V1, V2)

mdRaster2(V1,V2)

Multidimensional

Raster Analysis

How to create multidimensional mosaic dataset

How to publish as image services

Manage and Disseminate

Multidimensional Mosaic Dataset

• Mosaic dataset

- Catalog and index a collection of rasters

- Process on-the-fly

• Multidimensional mosaic dataset

- Manage variables and dimensions

- Multidimensional Info

A mosaic dataset that manages multidimensional raster data

HDF, GRIB, netCDF

time series raster data

table

Multidimensional

mosaic dataset

mosaic

raster

Multidimensional Mosaic Dataset - Creation

• Create Mosaic Dataset tool

• Add Rasters to Mosaic Dataset tool

- NetCDF, GRIB, or HDF raster type

- Define variables

- Processing template

- Estimate statistics option

From NetCDF, GRIB, HDF files

Multidimensional Mosaic Dataset - Creation

• Create Mosaic Dataset tool

• Add Rasters to Mosaic Dataset tool

- Landsat, Sentinel raster type

• Build Multidimensional Info (2.4)

From time series imagery

Multidimensional Mosaic Dataset – Processing TemplatesRaster function template (RFT)

• Variable selector Template

- Select one variable

- Create by default during Add Rasters

• Process with one or multiple variables

- Define template properties with group and

tag fields in the mosaic dataset

- Group – dimension

- Tag field - variables

Processing by groupT1

T2

T1

T2

When to Use Mosaic Dataset vs Multidimensional CRF

• Use mosaic dataset

- Aggregate data from multiple files

- Publish service with one or more processing templates

- One service with multiple variables

- One service with multiple processing templates

• Convert to CRF

- Optimize temporal profile option when for long time series data

- Use in cloud storage

Disseminating through ArcGIS Platform as Image ServiceArcGIS Enterprise and ArcGIS Image Server

• Serves data - true pixel values

• Dynamic server side processing

- LERC compression

- Resample method

• WMS/WCS dimension query

- Time=t1&Elevation=z1

- DIM_Time=t1&DIM_depth

Consuming multidimensional image services

• Web App and ArcGIS Pro

- Access single slice

- Time/dimension sliders

- Access with processing template

• Custom applications

- Web AppBuilders

- JavaScript APIs

Support various clients

Create and

Publish

Multidimensional

Mosaic Datasets

Summary

• ArcGIS is a platform for scientists and organizations to work with scientific data

• Leverage raster functions for your multidimensional raster analysis

• Leverage ArcPy API for your scripting workflow

• Mosaic dataset continues to be a model for manage your multidimensional raster data

- Use CRF to improve your temporal profile performance for long time series dataset

• More resources

http://sampleserver6.arcgisonline.com/arcgis/rest/services/ScientificData

https://pro.arcgis.com/en/pro-app/help/data/imagery/an-overview-of-multidimensional-raster-data.htm

What is Coming Next

• New user experience for multidimensional raster

- Multidimensional Tab

• More multidimensional raster analysis tools

- Trend analysis for time series imagery

- Statistics of multidimensional raster

- Subset multidimensional raster tool

- Make multidimensional raster layer

- NetCDF, GRIB, HDF

- Multidimensional CRF, mosaic dataset, and image services

- OPeNDAP

- Open data cube

• Continue to improve multidimensional CRF

• Continue to improve ArcPy API for multidimensional raster

Please Share Your Feedback in the App

Download the Esri

Events app and find

your event

Select the session

you attended

Scroll down to

“Survey”

Log in to access the

survey

Complete the survey

and select “Submit”

See Us Here

• Managing Imagery Using Mosaic

datasets and Image Service

WORKSHOP LOCATION

• SDCC Room 05A

• The Imagery Island

TIME FRAME

• 1:00 – 2:00

• All day