san diego, california - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/files/123.pdfspatial analytics...
TRANSCRIPT
![Page 1: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/1.jpg)
Technical Workshops |
Esri International User Conference San Diego, California
Integrating Open-Source Statistical Packages with
ArcGIS Mark V. Janikas, Ph. D.
Xing Kang
7-25-12
![Page 2: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/2.jpg)
Outline
• Introduction to Spatial Data Analysis in ArcGIS - Spatial Statistics, Geostatistics and Spatial Analyst - Python: Directly and Indirectly Extendable - Collaborative Motivation
• PySAL (Python Spatial Analysis Library) - Direct
• R - Indirect
• Conclusions and Future Directions
![Page 3: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/3.jpg)
Spatial Analytics in ArcGIS: Past and Present
• Traditional Spatial Analysis - Core tools continue to evolve
• Spatial Analyst - Raster - Map Algebra
• Geostatistics - Raster and Vector - Continuous Data
• Spatial Statistics - Vector - Exhaustive Data - Python
![Page 4: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/4.jpg)
Spatial Analytics in ArcGIS: Moving Forward
• Python - Spatial Analyst
- Raster NumPy - SciPy
- Spatial Statistics and Geostatistics - Data Access Module - Vector NumPy - Spatial Statistics Data Object and Utilities - matplotlib
![Page 5: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/5.jpg)
The Great and Extendable Python
• Direct - Numeric/Scientific Python Modules - http://wiki.python.org/moin/NumericAndScientific - +50 Modules Listed - Check Compatibility… Then Plug and Play
• Indirect - Alternative Languages - No Python Hooks or Module - Compatibility Fails - Python Serves as Active Script and OS - Out of Process
![Page 6: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/6.jpg)
Collaborative Motivation
• ArcGIS Users - Substantive Interest - Methodological Choice - Quantitative State of Mind - Enhancement Requests
- User Conference - Resource Center and Forums - Email… You know who you are…
• What we build and why - What is left? - PySAL and R
![Page 7: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/7.jpg)
Collaborative Motivation: 2nd Side of the Coin
• Value Added in the Other Direction - Direct or indirect linkage should focus on the strengths of
each - A Simple (Humorous?) Example
- The R module “Midas” - Method
- Inputs: X, Y, Z Coordinates, Image Reading at Location - Output: Probability that Gold Exists at each location
- I’m rich!... But what if “they” get there first? - ArcGIS
- Network Analyst: How do I get there? Best route? - Spatial Statistics: Hot-Spots to identify an area to focus on - Geostatistics: Kriging to predict and unknown locations
![Page 8: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/8.jpg)
Introduction to PySAL
• Open Source Python Library for Spatial Analytical Functions
- ASU GeoDa Center for Geospatial Analysis and Computation - Luc Anselin
- PySpace - Sergio Rey
- STARS - BSD License
![Page 9: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/9.jpg)
Anatomy of PySAL
![Page 10: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/10.jpg)
Collaborative Advantages: PySAL and ArcGIS
• PySAL - Advance Spatial Analysis code base with novel functions
- E.g. Regionalization, Spatial Econometrics - Do not have to “reinvent the wheel”
• ArcGIS - GIS User Interface - ~800 GP Tools - Easy-to-use Script Tool Framework - Enriched functionalities from ArcGIS Python scripts
- arcpy, SSDataObject, SSUtilities, SSReport etc. - Multiple input/output data format - Error messages - Pyharness framework for robust testing
![Page 11: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/11.jpg)
Basic PySAL – ArcGIS Interaction Model
PySAL Analytical Function
NumPy
Spatial Weights Input Data
Output Data
SSDataObject
SSUtilities
Environment Settings Projections Field Qualification Z/M Values Bad Records Error/Warning Messages Localization Feature Accounting
NumPy
![Page 12: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/12.jpg)
PySAL – ArcGIS Toolbox Demonstration: Regional Income Distributions
![Page 13: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/13.jpg)
What is R? Why should I use it?
• R (The R Project for Statistical Computing) - Over 60 CRAN sites across 30+ countries - Its Free
- GNU GENERAL PUBLIC LICENSE - Base is powerful
- Statistics, Linear Algebra, Visualization , etc… - Its extendible
- 1800+ Contributed Extensions - splancs, spatstat, spdep, rgdal, maptools, shapefiles
![Page 14: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/14.jpg)
Indirect Integration Strategy
• Python and R: “Decoupled” - Used as the core script tool - Hooks into the Operating System to call R - Post-Processor - “Out of Process”
• RPy/RPy2 - Compatibility
• win32com - Windows only - Works for other programs as well
![Page 15: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/15.jpg)
Basic R – ArcGIS Interaction Model
R
ArcGIS
Enhanced Output
Data
Python
Retrieves Parameters Organizes into R command Executes R command Post-Processing Apply Symbology Apply Projections Report
Input Params
Output Data
![Page 16: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/16.jpg)
R – ArcGIS Toolbox Demonstration: Regional Income Distributions
![Page 17: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/17.jpg)
Conclusions
• PySAL - Advanced spatial analytic techniques - Combined with SSDataObject and Utilities
- Directly compatible - Python Harness Implementation - Spatial Econometrics and Spatial Weights Conversion
- ESDA, Clustering, Spatial Dynamics etc. - BSD
![Page 18: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/18.jpg)
Conclusions
• R - Contains “cutting edge” data analysis techniques from a
wide body of academic and applied fields - Extendible - Indirectly compatible
- Direct via RPy/RPy2 and win32com - GNU - Revolution - esri continues to focus on improving the interaction in the
future
![Page 19: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/19.jpg)
Software Links
• PySAL - https://geodacenter.asu.edu/pysal - http://code.google.com/p/pysal/
• NumPy and SciPy - http://www.numpy.org/
• R - http://www.r-project.org/index.html
![Page 20: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/20.jpg)
- http://www.arcgis.com/home/group.html?owner=ArcGISTeamAnalysis&title=Analysis%20and%20Geoprocessing%20Tool%20Gallery
- Using R in ArcGIS (Version 10) - R Point Clustering Tools for ArcGIS (Version 9.3) - Using R in ArcGIS (Version 11) – Coming Soon! - Using PySAL in ArcGIS (Version 11)
- Either here or at the GeoDa Center
![Page 21: San Diego, California - maps.uky.edumaps.uky.edu/esri-uc/esri_uc_2k12/Files/123.pdfSpatial Analytics in ArcGIS: Past and Present • Traditional Spatial Analysis -Core tools continue](https://reader031.vdocuments.us/reader031/viewer/2022022011/5b085bd67f8b9a5f6d8c5269/html5/thumbnails/21.jpg)
Related Sessions
• Search the Online Agenda - http://events.esri.com/uc/2012/infoWeb/OnlineAgenda/ - Keywords
- Python - Spatial Statistics - Geostatistics - Spatial Analyst - Business Analyst