koop: using 3rd party services in arcgis
TRANSCRIPT
![Page 1: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/1.jpg)
Koop: Using 3rd Party Services in ArcGISDaniel Fenton
@dmfenton
https://github.com/dmfenton
![Page 2: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/2.jpg)
Koop is a bridge
http://arcg.is/2kI8efh
https://www.yelp.com/search
https://washingtondc.craigslist.org/search/apa
![Page 3: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/3.jpg)
Background
• Node.js- JavaScript on the server- Asynchronous i/o- https://nodejs.org
• NPM- Package manager for JavaScript- Npm install- https://npmjs.org- Module.exports = {}
• Geoservices- Open API Specification used by ArcGIS- https://geoservices.github.io
![Page 4: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/4.jpg)
New in Koop 3.0
• Simplified Provider- Route and Controller are Optional- FeatureServer Routes and Handler built in
• Simplified Cache API- Koop-Cache-Memory
• New Plugin Type: Outputs- Handle a set of routes automatically- Can call functions on Koop and Provider model
Output
Core
Cache
Provider
![Page 5: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/5.jpg)
Koop is extensible
• Provider: Translate remote APIs- https://github.com/koopjs/koop-provider-yelp
• Output: Translate GeoJSON into an API standard e.g. Geoservices, SODA, WFS- https://github.com/koopjs/koop-output-geoservices
• Cache: Store and Process GeoJSON- https://github.com/koopjs/koop-cache-memory
• FileSystem: Write files to disk or cloud storage- https://github.com/koopjs/koop-filesystem-s3
• General: extend the Koop API in any way- https://github.com/koopjs/koop-plugin-reverse-geocoder
![Page 6: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/6.jpg)
Providers
• Translate between any remote API and GeoServices• Pass-Through
- Communicate with remote API on every request- Data is too big to gather all at once- And/Or API supports filters and geometry queries- E.g. https://github.com/koopjs/koop-provider-yelp
• Cached- Gather all data at once- Data is small or doesn’t change often- E.g. https://github.com/dmfenton/koop-provider-craigslist
Output
Core
Cache
Provider
![Page 7: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/7.jpg)
Start the server locally
• npm install => install dependencies from npmjs• npm start => activate the server• View on command line with: `curl --silent
http://localhost:8080/sample/FeatureServer/0
![Page 8: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/8.jpg)
Deploy a server
1. Download and install the Now application from Zeit- https://zeit.co/now
2. From the root directory run `now --npm -e KOOP_SERVER_PORT=80`3. That’s it!
![Page 9: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/9.jpg)
Sample ProviderTriMet Bus API to Feature Service
GeoServices Output
Core
Memory Cache
Portland TriMet Bus API
`https://developer.trimet.org/ws/v2/vehicles/onRouteOnly/false/appid/${key}`=>`http://apapters.koopernetes.com/trimet/FeatureServer
![Page 10: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/10.jpg)
How to write a provider
1. Find API docs or Reverse Engineer API via network tab2. Implement `getData` function3. Configure settings4. Launch Server
From: https://washingtondc.craigslist.org/jsonsearch/apa/
To: http://adapters.koopernetes.com/craigslist/washingtondc/apartments/FeatureServer/0
Output
Core
Cache
Provider
![Page 11: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/11.jpg)
getData in detail
• Purpose is to talk to remote API and return geojson• Function takes in a `req` and `callback`
- Req contains parameters `id` and `host- Callback is called with an error or geojson
• Cache time is managed by this function• Example: https://
github.com/dmfenton/koop-craigslist/blob/master/craigslist.js#L7-L20
Output
Core
Cache
Provider
![Page 12: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/12.jpg)
Index.jsDialing in the specifics of your provider
https://gist.github.com/dmfenton/93379d71f4412716b3e508e1c8612cfa Output
Core
Cache
Provider
![Page 13: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/13.jpg)
Advanced ProvidersExtending the routes and functionality of Koop
- Routes- Maps http verbs and paths to handler functions- E.g. https://github.com/koopjs/koop-provider-agol/blob/master/routes.js
- Controller- Handles actions specified in routes- E.g. https://github.com/koopjs/koop-provider-agol/blob/master/controllers/index.js
Output
Core
Cache
Provider
![Page 14: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/14.jpg)
Outputs
• Adds routes to every provider• Works like Express middleware• Can call functions on the provider
- Or Koop• Leverage the cache automatically
Teaching Koop New languages
Output
Core
Cache
Provider
![Page 15: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/15.jpg)
Cache
• Used automatically by Koop- If a TTL is set (Time To Live)
• Avoid rate limiting• Improve response time• Add capabilities not supported by remote API• Use when:
- Data does not change often- Data is too large to request each time
Output
Core
Cache
Provider
![Page 16: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/16.jpg)
Plugins
•Extend Koop’s Core•Can be used by any part of stack•E.g. Queue, Tile Generator, File Exporter
Output
Core
Cache
Provider
![Page 17: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/17.jpg)
Lego Blocks
https://gist.github.com/dmfenton/93379d71f4412716b3e508e1c8612cfa
• Winnow: query geojson without a database- https://github.com/featureserver/winnow
• FeatureServer: given geojson and a route, respond like a feature server would- https://github.com/featureserver/featureserver
![Page 18: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/18.jpg)
Resources
• https://github.com/koopjs/koop• https://github.com/koopjs• https://koopjs.github.io• https://github.com/koopjs/koop-sample-provider
![Page 20: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/20.jpg)
Please Take Our Survey on the Esri Events App!
Select the session you attended
Scroll down to find the survey
Complete Answers and Select “Submit”
Download the Esri Events app and find your event
![Page 21: Koop: Using 3rd Party Services in ArcGIS](https://reader035.vdocuments.us/reader035/viewer/2022062218/58cea91b1a28abb26e8b6a2b/html5/thumbnails/21.jpg)