uk climate projections user interface
DESCRIPTION
Centre for Environmental Data Archival RAL, UK. UK Climate Projections User Interface. Stephen Pascoe, Ag Stephens, Alan Iwi, Peter Norton, David Alderson, Philip James, Simon Abele. Introduction. Peter Norton A contractor from Tessella - PowerPoint PPT PresentationTRANSCRIPT
http://ukclimateprojections-ui.defra.gov.uk
UK Climate Projections User Interface
Centre for Environmental Data ArchivalRAL, UK
Stephen Pascoe, Ag Stephens, Alan Iwi, Peter Norton, David Alderson, Philip James, Simon Abele
http://ukclimateprojections-ui.defra.gov.uk
Introduction
Peter Norton
• A contractor from Tessella
• Worked on the Web Processing Service (WPS) and the data processing code
www.tessella.com
http://ukclimateprojections-ui.defra.gov.uk
Overview
• What are the UK Climate Projections
• Quick look at the UKCP User Interface
• A Closer look at the WPS
• The UKCP-UI Deployment Architecture
http://ukclimateprojections-ui.defra.gov.uk
UK Climate Projections
• Funded by UK Department of Environment DEFRA
• Designed to serve as the evidence base for climate change mitigation decisions by public and private UK bodies
• 7 years work by UK Met. Office
• 2 year project at CEDA to build the user interface UKCP-UI
http://ukclimateprojections-ui.defra.gov.uk
UKCP09 Data Product
• Probabilistic:
• Ensemble of model runs
• Summarised into CDFs and PDFs
• Regional and sub-regional resolution
• 25km grid over land
• Administrative regions and river basins
• Weather Generator
http://ukclimateprojections-ui.defra.gov.uk
UKCP09 Reports
http://ukclimateprojections-ui.defra.gov.uk
UKCP-UI a quick look
http://ukclimateprojections-ui.defra.gov.uk
Starting the request
http://ukclimateprojections-ui.defra.gov.uk
Selecting a datasource
http://ukclimateprojections-ui.defra.gov.uk
Emissions Scenario and Time
http://ukclimateprojections-ui.defra.gov.uk
Selecting a location
• Different methods of selection
25Km Rotated Grid
http://ukclimateprojections-ui.defra.gov.uk
Selecting a location
5Km Grid
Regions
http://ukclimateprojections-ui.defra.gov.uk
Choosing your output
• Select from different output types• Can name the request
http://ukclimateprojections-ui.defra.gov.uk
Graphical Output
Preview Image
Adjust Variables
Download Options
Plot Settings
http://ukclimateprojections-ui.defra.gov.uk
Graphical Output (Map)
http://ukclimateprojections-ui.defra.gov.uk
User jobs summary
http://ukclimateprojections-ui.defra.gov.uk
Data product delivery
• The UKCP output formats• Publication quality plots: PNG, PDF, PS, JPG• CF-NetCDF. via subsetting with cdat_lite
(cdms)• CSV via nappy NetCDF<->NASA Ames library• Shapefiles via shapelib
• Delivered in a zipfile with project-specific metadata
• Duplicate outputs avoided using caching• Resubmit jobs via the “MyJobs” page• Share some jobs via URLs
http://ukclimateprojections-ui.defra.gov.uk
Architecture
Geoserver (Java)Geoserver (Java)
UI (PHP)UI (PHP)
COWS-WPS (Python)
COWS-WPS (Python) COWS-WMS
(Python)
COWS-WMS(Python)
UKCP Data libUKCP Data lib
geoplotgeoplot
chartplotchartplot
Weather GeneratorWeather Generator
Threshold DetectorThreshold Detector
Sun Grid EngineSun Grid Engine
cdat_lite, matplotlib, rpy, nappycdat_lite, matplotlib, rpy, nappy
User Interface Layer
Application Layer
Library Layer
Processing Layer
http://ukclimateprojections-ui.defra.gov.uk
The Application Layer
http://ukclimateprojections-ui.defra.gov.uk
Web Processing Service
• OGC Standard for asynchronous processing
• A WPS service offers a set of Processes that operate on Inputs producing Outputs
• Supports process status polling
• Percent complete
• Outputs available
http://ukclimateprojections-ui.defra.gov.uk
COWS WPS
• COWS handles OGC interfaces• “Plug-In” process modules
• Synchronous processes run in the server• Asynchronous processes run via Sun Grid
Engine• Cost estimation
• SGE scheduling• Configurable queues for fast and slow jobs• Control number of simultaneous requests• Processing occurs on separate nodes
http://ukclimateprojections-ui.defra.gov.uk
WPS Protocol
http://ukclimateprojections-ui.defra.gov.uk
Evaluating the use of a WPS
Advantages:• Someone has thought about the interface
before• Useful access (via http) directly to a process• Clear division between UI and Application layer
code
Disadvantages:• Need to negotiate a clear interface (XML)
between UI and Application layers• Had to follow (or extend) a specification
http://ukclimateprojections-ui.defra.gov.uk
Deployment
http://ukclimateprojections-ui.defra.gov.uk
Original Server Structure
ddp-ps1 ddp-ps2 ddp-ps3
ddp-ui1
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app1
WMS (python)WPS (python)
ddp-store1
SGE execd
archive
ddp-u2i
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app2
WMS (python)WPS (python)
ddp-store2
SGE execd
archive
ddp-ui3
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app3
WMS (python)WPS (python)
ddp-store3
SGE execd
archive
ddp-ps5
ddp-admin1
haproxySGE master
userdb (postgres)
cache
ddp-ps4
ddp-adminbak1
mirror statecachebak
acache1
acache2
acache3
Master server and backup
Physical servers
Virtual Machines
Temporary servers for high-demand launch period
http://ukclimateprojections-ui.defra.gov.uk
Requirements shift
“We need 1,000 simultaneous users” from 50
• Extensive system testing:
• Login, Request Builder, WMS, WPS, download
• Scaled up Xen VM system x3
• Found several critical optimisations
http://ukclimateprojections-ui.defra.gov.uk
Modified Server Structure
ddp-ps1 ddp-ps2 ddp-ps3
ddp-ps5 ddp-ps4
ddp-ui1
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app1
WMS (python)WPS (python)
ddp-store1
SGE execd
archive
ddp-u2i
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app2
WMS (python)WPS (python)
ddp-store2
SGE execd
archive
ddp-ui3
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app3
WMS (python)WPS (python)
ddp-store3
SGE execd
archive
ddp-ui6
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app6
WMS (python)WPS (python)
ddp-ps6
ddp-ui9
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app9
WMS (python)WPS (python)
ddp-ps9
ddp-ui7
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app7
WMS (python)WPS (python)
ddp-ps7
ddp-ui8
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app8
WMS (python)WPS (python)
ddp-ps8
ddp-ui10
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app10
WMS (python)WPS (python)
ddp-ps10
ddp-ui11
UI (php)geoserver (tomcat)spatialdb (postgres)
ddp-app11
WMS (python)WPS (python)
ddp-ps11
ddp-admin1
haproxySGE master
userdb (postgres)
ddp-adminbak1
mirror statecache cachebak
acache1
acache2
acache3
Master server and backup
Physical servers
Virtual Machines
Temporary servers for high-demand launch period
http://ukclimateprojections-ui.defra.gov.uk
Thank you
Stephen Pascoe, Ag Stephens, Alan Iwi, Peter Norton, David Alderson, Philip James, Simon Abele