sharing environmental data, our experiences at epa

Sharing Environmental Data

Our experiences at EPA

Dr. Peter Mooney

EPA Research FellowDept of Computer Science, NUIM

As of .. May 26th 2014

The EPA and Environmental Data and Information

Generator and Collector(monitoring, position as legal entity, etc)

Facilitator and enabler (through R&D programmes)

Manager and distributor(Water Framework Directive....)

Providing ‘open-data’ as PDF reports: knowledge distribution - but not actionable

“like funding James Cameron to make Avatar, and then releasing it in a black and white flip book. We are missing all the good stuff”

http://www.theguardian.com/global-development-professionals-network/2013/oct/21/development-open-data-action

The EPA have committed significant resources to making

GIS data available

Envision

Pre-packaged data downloads

Service-based access to geospatial data

WFS, WMS and REST Services

Envision Download Services

EPA Research

Over 700 funded Over 700 funded projects (> €10K)projects (> €10K)

since 2002since 2002

https://www.flickr.com/photos/33280166@N02/5354725682

EPA Research Programme has been operating an open access approach over the last 6 years

All EPA funded research projects must provide “significant outputs”

(datasets, info resources, etc) for public access via SAFER

Crucially and innovatively, this couples the final reports/papers with the actual data/information used to generate the findings/recommendations

SAFER -Secure Archive for Environmental Research Data

High rates of download

SAFER has driven excellent resource efficiencies in EPA Research

● AQ PM10 Data ArchiveOpen Access

● 2300 downloads (May 2013)● 2300 X 6 minutes per request = 13,800 minutes

● 13,800 / 60 = 230 hours

● 8 hour working day

● TOTAL 28.75 days SAVED

Example

Several key EPA dataset archives are available on SAFER

http://erc.epa.ie/safer/resourcelisting.jsp?oID=10206&username=EPA%20Drinking%20Water

SAFER is beginning to offer some web-service access

The open nature of the Catalogue has exceeded our expecations

EPA Air Quality

The Air Quality Index for Health

(AQIH) Launched May 2013

The AQIH was developed through collaboration between EPA, HSE, Met Eireann, and other public bodies and experts

The AQIH is true real-time open data from the EPA

Monitoring Network

Ongoing Sampling

EPA Servers andDatabases

Processing and Analysis

Delivery to the public and

stakeholders

Example

Sampling between09:00 and 10:00

Receive from monitoring stations (09:00 – 10:00)

Storage in Databases

Processing and analysis

10:00 – 11:00

Data from the periods including 09:00 – 10:00 is now available for access

AQIH Map Access

Screenshot taken at 14:33 on 26-May-2014

AQIH Twitter

Automated tweets @ 08:00 and 17:00 each day – hourly if AQIH becomes poor in one or more regions

AQIH Web Pages for Stations

AQIH as JSON Open Data

AQIH JSON (wget call)

We get lots of request for the raw hourly data from individual

monitoring stations

This type of data requires expert interpretation – so the AQIH is made available openly instead

SAFER contains the entire archive of validated AQ data from EPA

http://erc.epa.ie/safer/resourcelisting.jsp?oID=10136&username=EPA%20Air%20Quality

Summary of thoughts

Our advice is to L.E.A.R.N

ListenEngage

Activity/ActionsResearch

New/Innovatehttps://www.flickr.com/photos/arichards-gallery/8358810783

The unbounded evolution of Internet The unbounded evolution of Internet technologies has completely changed the technologies has completely changed the data playing field and rules of engagementdata playing field and rules of engagement

10 Years Ago

“Fear of being open”

Today

“Fear of not being open”

DRIVERS CHANGE

INSPIRE,SEIS,PSI Directive,Open Data, Initiatives, etc

Open Internet,Social Media,User-generated content,“The Internet Generation”, etc

Open Data is high on the EPA agenda for future work

● AQIH: Make historical archive available (JSON)● Up-to-date AQ data: AQ Directive changes are

being implemented – potential to make INSPIRE-compliant AQ UTD available

● Explore potential linkages to DUBLINKED: Are there datasets/services which can be shared?

● Upgrading tools such as SAFER: Investigate adding more web-services (pull of information)

Emphasis on delivery of mobile-focused applications

The EPA must carefully consider open-data on a case-by-case basis

Concerns about data misuse or misinterpretation

The resource costs – making open data available isn't free

Working with stakeholders – when data has been generated or owned by several parties

Privacy and security issues – particularly in point-based geospatial datasets

Dr. Peter Mooney email peter.mooney@nuim.ie and

p.mooney@epa.ie