didbase: intelligent, interactive archiving technology for ionogram data b. w. reinisch, g. khmyrov,...

DIDBase:  Intelligent, Interactive Archiving Technology for Ionogram

Data

B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov

Environmental, Earth, and Atmospheric Sciences Department, Center for Atmospheric Research, University of Massachusetts Lowell, USA

Our choice

• RDBMS Server - Firebird (Interbase)

• JDBC - Database Client

• Language – Java– Applications– Applets– Servlets

• Application Server - Tomcat

Ionogram processing software• DIDBase (ionograms, scaling data,

ionospheric characteristics)

• Ionogram software suite– SAO Explorer– DIDB Fill, Smart Backup– DIDB Request Processor (CalVal project)– Analyze Scaling Quality– Representative Ionospheric Characteristics– Visualization programs: Ion2Png, Drg2Png,

DrgMaker

Data Management Automation

• DIDBase data ingestion

• ADRES

• Smart Backup

• Database backup

Extended data quality analysis

• Bad data• Incomplete data• Future data

ADRESDIDBase

at UMLCAR

Digisondes

DIDBase Request Processor

Requests(REQ files)

Reports(SAO files)

Offline archivesAutomated

Data RequestExecution System

USER

Online archives

DIDB Fill

SAO Explorer

Query only data for requests

Save scaled data

ADRES

Ion2Png thumbnail mode

http://car.uml.edu/DIDBase

WEB-interface

Station list

Inventory for Station

Calendar list

One day

Ionogram example

LoginStep 1 Step 2

Step 3

Manual scaling

• Total manually scaled data – 144297 (more than 4 years of 15 min data)

DMSP F16 weather satellite

• DMSP (Defense Meteorological Satellite Program ) F16 weather satellite was launched 18 October 2003

• Orbital parameters are period 101.9 min , apogee 853 km, perigee 843 km, and inclination 98.9°.

• Two optical instruments onboard:– SSULI - limb scanning ultraviolet imager / spectrometer

(built by the Naval Research Laboratory)

– SSUSI - nadir scanning ultraviolet imager /spectrometer and photometer (built by the Applied Physics Laboratory at Johns Hopkins University)

• SSULI and SSUSI sensors make measurements of far and extreme ultraviolet airglow, which are converted into electron and neutral density profiles by ground processing software.

• Measurement calibrated and validated with ground truth– Digisondes

– Incoherent Scatter Radars (ISR)

– F16 SSJ5 Particle Detector Data for auroral measurements

CalVal campaign

Digisonde network

• 50 Digisondes in the network• 27 Digisondes are participating in CalVal

campaign• Products:

– Ionospheric characteristics (foF2, foE, hmF2, …)– Regular profiles– Auroral profiles

CalVal Requests

• Nominally 4 requests per Station per Day when overpass happen for limb and surface measurements on ascending and descending part of orbit

• Each Request has one hour time interval (± 30 min from overpass time)

• CalVal is planned for period from Dec 15 2002 to Mar 2005

• Some “dry run” data for 2001

Statistics

Request status Total, as of Oct 2004

Total requests 12230

Request data loaded to DIDBase 10826

Manually scaled and reported requests

10027

Manually scaled and reported ionograms

50598

Interactive Ionogram Processing

• Ionogram Data Visualization– ionogram display with zoom-in and details-on-demand– ionogram surveys (using “thumbnails”)– ionogram movies

• Interactive Ionogram Scaling• Profile Inversion• Time series of Scaled Data

– Ionospheric characteristics– Plasma density contour/profilogram– Directogram– Text tables for ASCII export

Data Formats Supported

• Ionograms– MMM

– BEM

– SBF

– RSF-flex

– PGH

– RSF

• Scalings– SAO 3.0 - 4.3

– ART

– ADP binary

Ionogram surveys

Profilogram

Char Table

Ionogram Editor

Directogram

Welcome to SAO-X

Contours

Characteristics Plot

Main screen

List of records

Ionosphericcharacteristics set.

Drag and drop to change order.

Scaler (DIDB only)

Ionogram frame

Acton buttons

Mode controls

Context menu(mouse right button)

Details on mouse position

Database connection

• Read-only connection to DIDB over the Internet

• Remembers last connection information

Database queries• List of available data

with the station name and the start/end times

• DIDB inventory available

• Use of ionospheric characteristics as search criteria

• Query only manual data

• By default, the queries bring the data of the highest available quality

Manually scaled

Select measurements with auroral layer

Inventory tree (DIDB only)

• Simple navigation

• Easy to check what is stored in DIDB

• Comprehensive information about data

• “One click” to set station/date in the Query dialog

DIDB write access

• Scaler password protected login

• Submit current record

• Auto submit option

SAO-X homepage

Installation

http://car.uml.edu/Installation/SAO-X_3/install.htm

InstallAnywhere:commercial strength installers for various platforms.

Windows, Solaris: installers with Java VM included.

Others: JVM has to be installed first.

Digisonde and Internet

• Web Publishing

(via homepage)– Latest displays

– Retro displays

– SAO archive

• Dissemination

(by FTP deliveries)– Near real-time

– Works from behind a firewall

Digisonde Homepage

1997

1995

1998

1996

Digisonde Homepage (2)

1999

2000

2002

2001

Digisonde Homepage (3)

2003 - 2004

2004 Digisonde Homepage

Digisonde Data Tree

DIGISONDERaw Ionograms

DFT: Raw DriftSAO: Scaled data

Time Series

Contours

Profilograms

MUF

MMM, BEM, PGH, SBF, RSF

Ionogram displays

DRG: Directograms

Old Formats ION, IHT, IOS, TEC, ADP, ART

HousekeepingBIT, ERR

SKY: Skymaps

DVL: Velocities

TLT: Tilt

online pictures

16 channel D-256

References

• UMLCAR home page http://ulcar.uml.edu

• Digisonde home page http://ulcar.uml.edu/digisonde.html

• SAOExplorer home page http://ulcar.uml.edu/SAO-X/SAO-X.html

• DIDBase home page http://ulcar.uml.edu/DIDBase/

Publications

• Reinisch, B. W., I. A. Galkin, G. Khmyrov, A. Kozlov, and D. F. Kitrosser, Automated collection and dissemination of ionospheric data from the digisonde network, Adv. Radio Sci. (2004), 2: 241-247.