using fme to create open data datasets

Use of FME Workbench in Edmonton’s Open Data Catalogue

Bohdan HrynyshynCity of EdmontonIT Business Analyst, Open Data Team

Edmonton, Alberta - April 12, 2013

What is Open Data?

Certain data to be freely available for everyone to use and republish as they wish, without restrictions or control

Rationale

Data generated or collected by public organizations belong to taxpayers, and where releasing it will not violate any laws or rights to privacy

Component of Open Government Building and maintaining relationships between

government and citizens by sharing ideas and information

Make Local Government More …. Open Accountable Accessible

Why Open Data?

Open Government …..

Encourage a Constant Exchange of Ideas

Want Government to make better decisions for all of us

Openness ……..

Laws and Decisions Open for Discussion Collaboration in policy-making, policy

implementation, and policy enforcement

Accessibility ……..

Budgets may be reviewed by anyone Government does not have all the answers – participation

through various communication tools Public may have necessary knowledge and experience to

deal with issues

Economic opportunities by turning data into apps, web-sites or other useful products

Accountability ……

Public Officials are Accountable Government meetings are open to everyone Civic engagement, stakeholder groups collaboration,

collaborative public management, and dispute resolution with citizens and stakeholders have answerability

Why Open Data Edmonton?

make accessible, data about Edmonton’s municipal services to Edmonton's citizens, media, and public

Why Are We Doing This?

City committed to better manage business information Provide easy access to public Data Assets Improve accessibility, transparency and accountability for City Of Edmonton data Increase engagement of citizens, community organizations and private businesses

in creative community problem solving and development of innovative services, applications and solutions

Promote ‘One City’ concept - one location for all city data Integrate data from various discrete data-sets to provide more value and

intelligence

About Edmonton’s Open DataThe City of Edmonton - dedicated to using technology to make municipal information more open, transparent and accessible.

History

Open Data Principles ……

1. Complete Data not subject to privacy, legislative, security or privilege limitations

2. Primary Data collected at source Granularity at lowest level possible not in aggregate or modified form

3. Timely Data is current and relevant

4. Accessible to wide range of users for many different purposes

Open Data Principles ……

5. Machine processable Data structured for automated processing

6. Non-discriminatory Access Data available to anyone, with no requirements

7. Non-Proprietary Format Data available in format no entity has exclusive control

8. Data license-free Data not subject to any copyright, patent, trademark or

regulations

Open Data Infrastructure

Why Use FME?

Transform (restructure, reformat, and integrate) spatial and non-spatial data into proper format and structure for loading into Socrata’s Open Data Catalogue

Add spatial components to non-spatial data so it can be readily visualized on a map in Socrata’s Open Data Catalogue

How to publish Open Data Cycle

Choose Data for Release Prepare data extract in Open Format (csv, xls,

shp) Create Open Data page on Catalogue Create Meta Data for dataset Make Dataset public

FME Transformers

AttributeKeeper AttributeRenamer CoordinateExtractor NullAttributeReplacer Reprojector Sorter StringConcatenator StringSearcher Tester

• extract data from SLIM (Oracle)• format and filter in FME• output as csv file

https://data.edmonton.ca/Environmental-Services/Trees-Species-Map-View-/cggb-hzzm

TreesList of all trees owned and maintained by the City of Edmonton

https://data.edmonton.ca/City-Administration/City-of-Edmonton-s-Properties-for-Sale/yutc-c5ju

• extract view data from SLIM (Oracle) and POSSE

• format and filter in FME• output as csv file

City Properties for SaleList of all City-owned properties for sale

City of Edmonton/NAIT Solar Panel Reference System - Daily Power Production 

Daily production of electrical power from solar panels located at NAIT

https://data.edmonton.ca/Environmental-Services/City-of-Edmonton-NAIT-Solar-Panel-Reference-System/ysgw-us2d

• assemble “scraped” data for each panel from vendor using FME

• format and consolidate each panel• output as csv file

Portable SignsList of all licensed portable signs in the City

https://data.edmonton.ca/Sustainable-Development/Temporary-Signs-Map/xt3a-rjpj

• extract data from POSSE Data Warehouse

• format and filter in FME• output as csv file

•

Thank You!

Questions?

For more information: Bohdan Hrynyshyn – City of Edmonton, IT Branch

bohdan.hrynyshyn@edmonton.ca

opendata@edmonton.ca