Data InteroperabilityAdvanced Use

Bruce Harold bharold@esri.comDale Lutz Safe Software

This is your world – ask us today about best practicesAutomate Moving Data

No Domain Limitations

ArcGIS Data Interoperability

• Formats –well, yes….

Trends

ArcGIS Data Interoperability

• Formats –well, yes….• Web Integration increasingly defines ETL

Trends

ArcGIS

JSON

XML

GML RSS APIS

ArcGIS Data Interoperability for ArcGIS Pro

• ArcGIS Pro 1.3 improves the ETL Tool authoring experience- ETL Tools may reference or import FMW files- Supports bringing FME-based work into Projects- Supports sharing work as FMW files

• Launch Workbench from the ribbon’s Analysis tab- Ad-hoc workflows starting from a blank workspace- Drag and drop in a FMW file to a blank workspace- Drag and drop in a recognized format file to a blank workspace- Drag and drop in a custom transformer FMX from store.safe.com

Where the extension is at in Pro

DemoArcGIS ProSynchronizing GDB with AGOL

LiDARPoint Cloud Scenarios

• Write- LAS, LASD, ZLAS, LAZ

Convert Point Cloud Formats

• Read- ASPRS LAS- ASTM E57- Microsoft (Mojang) Minecraft- Oracle Spatial Point Cloud- Point Cloud XYZ- RIEGL Laser Scan Database- Terrasolid Terrascan- Z+F LaserControl ZFS

Create LASD, ZLAS from Airborne or Terrestrial Point Clouds

Drone Acquired Point Clouds Are Coming!

DemoConvert Terrascan to ZLAS

Transforming Point Clouds

• Thin, Clip to reduce data• Overlay on Raster• Create a Surface Model• Slice or Profile along a line• Update Extents• Combine Clouds, Merge Components• Set Components Manually• Point-by-point Calculate Components• Point-by-point Filtering• Convert from or to Geometry Features• …

Reprojection

• Ideally LiDAR has a spatial reference- LAS for example requires this to be compliant- But often it is missing or incorrect

• ArcGIS cannot Project LASD datasets- Do this in Workbench during import

• ArcGIS will see the projection in LASD/ZLAS• Common conversion is from ECEF coordinates

Clipping

• Discard points outside a defined area or volume• Cubic Clipping: Clip to a 3D Solid• Data becomes more manageable

Tiling

• Chop the Point Cloud into a Series• Tile by tile size in ground units• Tile by tile count in each axis

Tiler Transformer

Splitting

• Split by Component Value- Classification, Color, Intensity….- Exact values or ranges

Subset Point Cloud

DemoSplit a Point Cloud

Colorization

• Automatically set color components• View in ArcScene or ArcGIS Pro

Inject component values for RGB

DemoColorize point Cloud with Orthophoto

3D GISWorking with 3D Data

Working with 3D Data

• Z-enabled features read and written• Multipatch features are supported• CAD solids become GDB multipatch• Create 3D from 2D data:

- 3D Buffering- Extrusion- By Attribute Value or Expression

Data Interoperability works in 3D

DemoCreate 3D Sketchup Building Model

- 2D Shapefile building outlines- LiDAR elevation values for rooftops- LiDAR ground surface model- Extrude Buildings- Set Appearance- Write Sketchup

BIM and GISWorking with BIM Data

Working with BIM

• Important touch point with AEC world• File size and complexity make BIM data hard to use• BIM exports IFC (too complex) or DWG (too simple)• Key: keep what you need, discard what you don’t

Web Scene Visualization

BIM to GIS: Mount Vernon

• George Washington’s home laser scanned to create high quality BIM in Revit

• Data Interoperability extension used to convert BIM to GIS

• Export Revit data (plus added georeferencing) with FME Revit Exporter

• Import to ArcGIS with Data Interoperability extension• Reconnect attribution• Web Scene delivery• Rich detail of BIM with spatial context of GIS

BIM to GIS Scenarios

• Read IFC or RVZ• Translate Revit floorplan to Geodatabase• Extract/use BIM georeferencing• Complete control over IFC hierarchies• Reduce BIM data volumes• Update BIM from Excel tables• Webinar: GIS and BIM Interoperability - fme.ly/gisbim

•

MinecraftAs Outreach Technology

Working with Minecraft

• A Minecraft world is a point cloud• BlockID and BlockData components• Convert any data to Minecraft• Vector• Raster• LiDAR• BIM• etc.

ExampleBowen Island: GIS + DEM to Minecraft

Workspace Summary

• Read road lines (Shapefile), forest polygons (MapInfo), and raster DEM• Buffer and clip vectors to avoid overlap• Rasterize vector layers

- Make numeric rasters for Z, blockID, blockData

• Convert raster pixels to points- Set Z value and point cloud components

GIS & DEM to Minecraft

Publish as Public Consultation

Minecraft and the real world

• Hypothetical scenarios- Lava flow- Forest fires- Rising sea levels

• Geodesign- Sweden used Minecraft for public urban planning

• Fun: fme.ly/MazeGenerator

Try it, Learn it

• 60-day Trial:• http://www.esri.com/software/arcgis/extensions/datainteroperability/evaluate• Courses – search ‘ETL’ in the Training Catalog:

- Transforming Data Using Extract, Transform, and Load Processes- http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseid=2571- Controlling Data Translations Using Extract, Transform, and Load Processes- http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseid=2737- Data Validation checklist and webinar – fme.ly/validation- FME and ArcGIS Online resources – fme.ly/agol

Why Data Interoperability?

Please take our Survey – Session 218 Offering 213Your feedback allows us to help maintain high standards and to help presenters

Find the session you want to review

Find your event in the Esri Events App

Scroll down to the bottom of the session

Answer survey questions and submit