wtf-ceop implementation plan #1 status (wtf-ceop first prototype, by jaxa) september 12, 2005
Post on 11-Jan-2016
37 Views
Preview:
DESCRIPTION
TRANSCRIPT
CEOS/WGISS 20, Kyev, September 12-16, 2005 1
WTF-CEOPImplementation Plan #1 Status(WTF-CEOP first prototype, by JAXA)
September 12, 2005
Osamu Ochiai - JAXA
Ben Burford - JAXA/RESTEC
CEOS/WGISS 20, Kyev, September 12-16, 2005 2
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 3
WTF-CEOP IP #1
• Provide access to CEOP project data (data integration)– In-situ (archive at UCAR)– MOLTS (archive at MPI)– Satellite (archive at Univ. of Tokyo)– Model Output (archive at MPI)
• Basic functions– Access data from archive sites– View data (plots, view data values on screen)– Download (e.g. NetCDF format)
CEOS/WGISS 20, Kyev, September 12-16, 2005 4
WTF-CEOP IP #1
• “Use existing software”– Live Access Server (LAS)– DODS/OPeNDAP server
• Advantages/capabilities– Detailed access (“6 way subsetting”)– Flexible – can be modified in several ways– Analysis capabilities/services can be added– Well documented– Good support – users group and developers
CEOS/WGISS 20, Kyev, September 12-16, 2005 5
WTF-CEOP by JAXA
Live Access Server(LAS)
Ferret
Menus – Select DataFunctions – Plot, Display Download, etc.
UsersWeb Browser CSDIC Archive
Satellite
(Japan)
DODS
UCAR Archive
In-situ
(U.S.)
DODS
MPI Archive
MOLTS, Model Output
(Germany)
DODS
Coordination:
1. Selection, modification and test of OPeNDAP Freeform server
2. Coordination of DODS server installation at UCAR, MPI (not yet) and CSDIC (tuning)
IP #1/JAXA PrototypeSystem Overview
Development:
1. Develop “User Interface” menus (XML files, converted to populate MySql database)
2. Modify “Data Server” software (to access various CEOP data types on OPeNDAP Freeform server)
CEOS/WGISS 20, Kyev, September 12-16, 2005 6
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 7
Implementation Plan #1
Status
Last Fiscal Year Activities (ended March 31, 2005)
• Select data type• Select location (Reference Site, Station name)• Select variable name• Select by height or depth• Select by time range (start time / stop time)• “See the data” (plot and view values on the screen) • Download data (NetCDF, ascii or spreadsheet format) • Registration (user log on and access logs, to satisfy CEOP data policy)
CEOS/WGISS 20, Kyev, September 12-16, 2005 8
Implementation Plan #1
Current Work
This Fiscal Year Activities (final - ends March 31, 2006)• Upgrade LAS software (from version 6.3 to 6.5)• In-situ data (update menus, add download of QC flags)• MOLTS (access by HTTP string from new MPI interface)• Satellite (band selection, period selection, download multiple scenes)• Model Output (access by HTTP string, convert from GRIB to NetCDF)• Choose to be notified by email when data is ready• Automate update of menus as new data is added• Expand User’s Manual• Write Maintenance Manual• Update and increase DIFs in WTF-CEOP portal in GCMD
Additional R&D, beginning this Fiscal Year• Presented by Osamu
CEOS/WGISS 20, Kyev, September 12-16, 2005 9
Upgrade LAS Software:From Version 6.3 to Version 6.5
• Contains a prototype THREDDS Catalog • New Browser Management
– A new XML file (las/server/browsers.xml) contains a list of known browsers with an indication of whether they are compatible with the applet or not.
• New <ui_timeout> Property– The <ui_timeout> property is the number of seconds the user interface serv
er should wait for a product request to finish before 'backgrounding' the request.
– If the requested product finishes in less than <ui_timeout> seconds it is simply returned when ready. Otherwise, an HTML page is returned that allows the end user to 1) check on the status of this request; 2) ask to be notified by
email when the request is finished; or 3) cancel the request.
CEOS/WGISS 20, Kyev, September 12-16, 2005 10
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 11
In-situ Data Archive Development Status
• Archive at UCAR (Boulder, Colorado, US)• OPeNDAP Freeform Server was implemented• In-situ data – Surface, Tower, Soil, Flux (time s
eries)• All current in-situ data is online• Continuing – update menus as new in-situ data
is added at UCAR• New capability – include QC flags with data in d
ata download
CEOS/WGISS 20, Kyev, September 12-16, 2005 12
In-situ Data - 7 QC Flags
G Good
B Bad
U Unchecked
I Interpolated/Estimated
C Reported value exceeds output format field size OR was negative precipitation.
D Questionable/Dubious
M Parameter value missing OR Derived parameter can not be computed.
CEOS/WGISS 20, Kyev, September 12-16, 2005 13
MOLTS Data Archive(Development Status)
• MPI (Max Planck Institute), M&D (Hamburg, Germany) – MOLTS archive center for CEOP
• MPI decided to ingest MOLTS data into Oracle database– Develop Java routines or install DODS server– Request using HTTP string
• CEOP telecon on MOLTS data format– MPI will convert all (10 agencies) MOLTS data to NetCDF forma
t.
• 3 copies of MOLTS data (original, netCDF, Oracle).• MPI upgraded Oracle software during August.• MPI beginning development of MPI/WTF-CEOP interface
for MOLTS data in September (this month).
CEOS/WGISS 20, Kyev, September 12-16, 2005 14
Model Output Data Archive(Development Status)
• MPI (Max Planck Institute), M&D (Hamburg, Germany) – Model Output archive center for CEOP
• MPI will reprocess all model output data (10 agencies, about 50 Terrabytes) to individual variables at individual levels– Some sample files are available
• WTF-CEOP access will be via HTTP string using MPI “Jblob” interface.
• WTF-CEOP will receive model output data in GRIB format, and must convert from GRIB format to netCDF format.
• MPI beginning development of MPI/WTF-CEOP interface for model output data in September (this month).
CEOS/WGISS 20, Kyev, September 12-16, 2005 15
Satellite Data Archive (Development Status)
• CSDIC (University of Tokyo, Japan)– Satellite data archive for CEOP
• Working on installing Freeform Server at CSDIC• Upgraded libraries on CSDIC computer• Changed environment variables for HTTP string access via
Apache server to DODS server• Problem – now encountering DODS internal error• Satellite data format has changed (separate header and data
files)• Band selection is being added – select one band or all bands.• Time period selection is being added – all scenes in a time
period.• Package download function is being added – download is for all
scenes in a time period.
CEOS/WGISS 20, Kyev, September 12-16, 2005 16
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 17
Additional Functions
• Add background processing capability– “Wait for processing to finish, or be notified by email when file
is ready to be picked up from server?”• Write utilities to automate updating of WTF-CEOP
menus as more data is added– Possibly use metadata provided by UCAR to update WTF-
CEOP menus– Especially satellite data (over 1,000,000 scenes of satellite
data)– Read satellite data directories and create XML file with menu
contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 18
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 19
• Dr. Koike requested that WTF-CEOP should include a data discovery function for CEOP data, using CEOP metadata.– JAXA agreed to develop a prototype CEOP data d
iscovery system under GCMD as a first step.– Lola established a CEOP portal in GCMD.– JAXA/RESTEC prepared samples of DIFs, based
on CEOP 19115 metadata samples and DIFs were ingested into CEOP portal.
CEOP InternationalMetadata Meeting
CEOS/WGISS 20, Kyev, September 12-16, 2005 20
CEOP GCMD Portal
CEOS/WGISS 20, Kyev, September 12-16, 2005 21
CEOP GCMD Portal
In-Situ
Surface datasets (EOP-3)
Tibet Himalayas Mongolia
Satellite
Subset datasets (EOP-1)
Aqua/AMSR-E
(Tibet)
ADEOS-II/GLI
(Inner Mongolia)
GCMD CEOP Portal:5 New DIF Samples
GCMD CEOP Portal:Data Discovery
CEOS/WGISS 20, Kyev, September 12-16, 2005 22
CEOP GCMD Portal
• Work this year– Modify existing DIFs to provide a clear path to acc
ess WTF-CEOP from DIFs– Create DIFs for MOLTS data– Create DIFs for Model Output data
CEOS/WGISS 20, Kyev, September 12-16, 2005 23
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 24
Documentation
• Revise and expand User’s Manual
• Write System Maintenance Manual
• Recently updated WTF-CEOP Project Plan– Added information R&D plans for 3 years (to be
presented by Osamu)
CEOS/WGISS 20, Kyev, September 12-16, 2005 25
• Implementation Plan #1 Review
• Implementation Plan #1 Status
• Status of archives (In-situ, MO, Satellite)
• Adding additional functions
• CEOP GCMD Portal
• Documentation and manuals
• Conclusions and JAXA Future Plan
Table of Contents
CEOS/WGISS 20, Kyev, September 12-16, 2005 26
Implementation Plan #1 will finish at the end of this Fiscal year (March 31, 2006)
All 4 types of CEOP data will be fully functional
A new R&D effort is beginning this year and will continue for two more years (until March 31, 2008)
To be presented by Osamu
(c) Prof. Shibasaki@Univ. Tokyo
Conclusions andFuture plan
top related