implementing a simple web application with fme server
DESCRIPTION
Presented by Logan Pugh, City of AustinTRANSCRIPT
CONNECT. TRANSFORM. AUTOMATE.
Implementing a simple web application with FME ServerLogan PughProgrammer Analyst, City of Austin
Tony CastroGIS Analyst, City of Austin
Ben VanderfordGIS Analyst Senior, City of Austin
Who, what, when, where, why and how?
Who: Small GIS shop in a big (and growing) city
What: Provide zoning verifications (among other services) to property owners and developers
When: 50 to 80 requests a month
Where: City of Austin, Texas
Why: Each request used to take 1-2 hours
How: Tedious manual process of verifying zoning, hand-editing a PDF and sending it to customer
What is zoning verification?
Zoning in Austin started in 1930s
Zoning verifications started in 1960s
Developers and property owners need zoning verification to get financing for (re)development
Old Process was Cumbersome: The Fatigue Factor
Lots of research and keying in values Name of applicant and address Parcel ID Index grid Zoning type, definition and ordinance
Print, sign and mail/fax/hand-deliver letter to sales office for customer pickup
Old letter format New letter format
The Tipping Point
With an ever-increasing workload and not-so-increasing budget/staffing, we need to increase efficiencies
Our staff time stretched, a new zoning verification process became a high priority
Along comes FME Server: we had a new option!
Project Requirements
Accept a parcel number in various formats
Perform zoning info drilldown with complex filtering logic
Allow operator to edit/correct info
Generate PDF with electronic signature automatically
Ability to easily be configured for future changes (new users, jurisdictions, etc.)
Spell checking
Challenges
How to automate what is by definition a human-verified process?
How can we own the process but provide access to people outside of our workgroup?
How can we build this without any additional infrastructure?
FME Server Data Streaming Service
Serves up the results of workspaces directly
Can be HTML, JSON, PDF, etc.
Result must be a single file
Allows everything to work within the context of a web browser
FME Server REST API
Simple protocol for programmatically interacting with FME Server
Uses HTTP requests, e.g. GET, PUT, DELETE
Authentication using simple token service
Tokens are per-user, but can be shared for specific applications
FME REST API Developer Playground: http://fmeserver.com/userweb/sharper/playground/
Workspaces
Zoning info drilldown workspace
Given a parcel number and parcel agency, returns the zoning types, ordinances and case numbers intersecting a given parcel
Negative buffer to reduce erroneous results caused by inaccuracies between zoning and parcel data
Filter out non-applicable zoning ordinances
Zoning Info Drilldown workspace
Workspaces, cont.
ZoningOrdinanceDateCalculator custom transformer ZoningOrdinanceFilter custom transformer
Workspaces, cont.
PDF generation workspace
Given various parameters, produces a complete, signed zoning verification letter in PDF format
CSV to JSON workspaces
Editors
Requestors
Agencies
Workspaces, cont.PDFTextBlockCreator custom transformerPDF Builder workspace
PDFRectangleTextCreator custom transformer
PDFImageCreator custom transformer
Workspaces, cont.
HTML form builder workspace
Serves up an HTML file that ties everything together
Uses JavaScript (Dojo Toolkit) and the FME Server REST API to request, display, edit, and submit data
Can be pre-populated with data via parameters submitted to FME Server
Demo
Behind the scenes
Key Transformers FeatureReader: Performs queries against any
FME-supported format. The queries can have both a spatial and a nonspatial component.
One query is issued to the FME-supported format for each feature that enters the transformer. The results of the query are then output.
This means workspaces can read data dynamically
One of the most powerful transformers in FME!
Key Transformers, cont. PythonCaller: Executes a Python script to
manipulate the feature.Useful when implementing logic too complex to handle with other transformers.
Examples:
Fixed-width word-wrapping, justification and special character escaping for text elements on output PDF
Using regular expressions to replace tokens in HTML file with parameter values
Building SQL expressions and sanitizing user input
Key Transformers, cont. JSONTemplater: Populates a JSON document
with FME feature attribute values{ "status" : xs:int(fme:get-attribute("_statuscode")), "message" : fme:get-attribute("_message"), "data" : { "parcel_agency": fme:get-attribute("_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] }}
Key Transformers, cont.
Key Transformers, cont.
{ "status" : xs:int(fme:get-attribute("_statuscode")), "message" : fme:get-attribute("_message"), "data" : { "parcel_agency": fme:get-attribute(“_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] }}
Key Transformers, cont.
{ "status" : xs:int(fme:get-attribute("_statuscode")), "message" : fme:get-attribute("_message"), "data" : { "parcel_agency": fme:get-attribute("_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] }}
Configuration
Enterprise applications require configurability
Avoid putting file paths, user names, passwords, etc. directly in FME workspaces
Allows workspaces to be shared with confidence, placed under source control (e.g. Git), and configured externally
Uses simple text-based configuration files
Configuration, cont.
FME ServerRepositories
Workspace 1
Workspace 2
Workspace n
dev
SecureNetworkFolders
Workspace 1
Workspace 2
Workspace n
test
Workspace 1
Workspace 2
Workspace n
prod
Configuration, cont.
import os, ConfigParser
global config_dict # Make variable accessible to other scripted parametersconfig_dict = None
# Get path to config file located alongside published workspaceconfig_path_file = os.path.join(FME_MacroValues['FME_MF_DIR'], 'config_path.txt')
# Return config key/value pairs as a dictionarywith open(config_path_file, 'r') as f: config_file = f.readline().strip() config_parser = ConfigParser.RawConfigParser() config_parser.read(config_file) config_dict = dict(config_parser.items('config')) return config_file
Use Python scripted parameters to read in configuration data. In first scripted parameter:
Configuration, cont.
# Just reference dictionary keys to get config valuesreturn config_dict['fmerest_token']
Subsequent scripted parameters are easy:
Conclusions
Reduced processing time from 1-2 hours to ~15 minutes
Ability to email generated PDF to customer, saving on paper use and mailing expenses
Laid groundwork for use by other departments and the public
Demonstrates flexibility of FME Server
Future improvements
Make app directly available to public
Tie in with existing zoning profile web application
Accept payments online
Collaboration with external parcel agencies on data alignment for improved accuracy
Thank You!
Questions?
For more information: Logan Pugh
[email protected] City of Austin
Related web app available for public: http://www.austintexas.gov/gis/zoningprofile/