DIDBase: Intelligent, Interactive Archiving Technology for Ionogram
Data
B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov
Environmental, Earth, and Atmospheric Sciences Department, Center for Atmospheric Research, University of Massachusetts Lowell, USA
Our choice
• RDBMS Server - Firebird (Interbase)
• JDBC - Database Client
• Language – Java– Applications– Applets– Servlets
• Application Server - Tomcat
Ionogram processing software• DIDBase (ionograms, scaling data,
ionospheric characteristics)
• Ionogram software suite– SAO Explorer– DIDB Fill, Smart Backup– DIDB Request Processor (CalVal project)– Analyze Scaling Quality– Representative Ionospheric Characteristics– Visualization programs: Ion2Png, Drg2Png,
DrgMaker
Data Management Automation
• DIDBase data ingestion
• ADRES
• Smart Backup
• Database backup
Extended data quality analysis
• Bad data• Incomplete data• Future data
ADRESDIDBase
at UMLCAR
Digisondes
DIDBase Request Processor
Requests(REQ files)
Reports(SAO files)
Offline archivesAutomated
Data RequestExecution System
USER
Online archives
DIDB Fill
SAO Explorer
Query only data for requests
Save scaled data
ADRES
Ion2Png thumbnail mode
Station list
Inventory for Station
Calendar list
One day
Ionogram example
LoginStep 1 Step 2
Step 3
Manual scaling
• Total manually scaled data – 144297 (more than 4 years of 15 min data)
DMSP F16 weather satellite
• DMSP (Defense Meteorological Satellite Program ) F16 weather satellite was launched 18 October 2003
• Orbital parameters are period 101.9 min , apogee 853 km, perigee 843 km, and inclination 98.9°.
• Two optical instruments onboard:– SSULI - limb scanning ultraviolet imager / spectrometer
(built by the Naval Research Laboratory)
– SSUSI - nadir scanning ultraviolet imager /spectrometer and photometer (built by the Applied Physics Laboratory at Johns Hopkins University)
• SSULI and SSUSI sensors make measurements of far and extreme ultraviolet airglow, which are converted into electron and neutral density profiles by ground processing software.
• Measurement calibrated and validated with ground truth– Digisondes
– Incoherent Scatter Radars (ISR)
– F16 SSJ5 Particle Detector Data for auroral measurements
CalVal campaign
Digisonde network
• 50 Digisondes in the network• 27 Digisondes are participating in CalVal
campaign• Products:
– Ionospheric characteristics (foF2, foE, hmF2, …)– Regular profiles– Auroral profiles
CalVal Requests
• Nominally 4 requests per Station per Day when overpass happen for limb and surface measurements on ascending and descending part of orbit
• Each Request has one hour time interval (± 30 min from overpass time)
• CalVal is planned for period from Dec 15 2002 to Mar 2005
• Some “dry run” data for 2001
Statistics
Request status Total, as of Oct 2004
Total requests 12230
Request data loaded to DIDBase 10826
Manually scaled and reported requests
10027
Manually scaled and reported ionograms
50598
Interactive Ionogram Processing
• Ionogram Data Visualization– ionogram display with zoom-in and details-on-demand– ionogram surveys (using “thumbnails”)– ionogram movies
• Interactive Ionogram Scaling• Profile Inversion• Time series of Scaled Data
– Ionospheric characteristics– Plasma density contour/profilogram– Directogram– Text tables for ASCII export
Data Formats Supported
• Ionograms– MMM
– BEM
– SBF
– RSF-flex
– PGH
– RSF
• Scalings– SAO 3.0 - 4.3
– ART
– ADP binary
Ionogram surveys
Profilogram
Char Table
Ionogram Editor
Directogram
Welcome to SAO-X
Contours
Characteristics Plot
Main screen
List of records
Ionosphericcharacteristics set.
Drag and drop to change order.
Scaler (DIDB only)
Ionogram frame
Acton buttons
Mode controls
Context menu(mouse right button)
Details on mouse position
Database connection
• Read-only connection to DIDB over the Internet
• Remembers last connection information
Database queries• List of available data
with the station name and the start/end times
• DIDB inventory available
• Use of ionospheric characteristics as search criteria
• Query only manual data
• By default, the queries bring the data of the highest available quality
Manually scaled
Select measurements with auroral layer
Inventory tree (DIDB only)
• Simple navigation
• Easy to check what is stored in DIDB
• Comprehensive information about data
• “One click” to set station/date in the Query dialog
DIDB write access
• Scaler password protected login
• Submit current record
• Auto submit option
SAO-X homepage
Installation
http://car.uml.edu/Installation/SAO-X_3/install.htm
InstallAnywhere:commercial strength installers for various platforms.
Windows, Solaris: installers with Java VM included.
Others: JVM has to be installed first.
Digisonde and Internet
• Web Publishing
(via homepage)– Latest displays
– Retro displays
– SAO archive
• Dissemination
(by FTP deliveries)– Near real-time
– Works from behind a firewall
Digisonde Homepage
1997
1995
1998
1996
Digisonde Homepage (2)
1999
2000
2002
2001
Digisonde Homepage (3)
2003 - 2004
2004 Digisonde Homepage
Digisonde Data Tree
DIGISONDERaw Ionograms
DFT: Raw DriftSAO: Scaled data
Time Series
Contours
Profilograms
MUF
MMM, BEM, PGH, SBF, RSF
Ionogram displays
DRG: Directograms
Old Formats ION, IHT, IOS, TEC, ADP, ART
HousekeepingBIT, ERR
SKY: Skymaps
DVL: Velocities
TLT: Tilt
online pictures
16 channel D-256
References
• UMLCAR home page http://ulcar.uml.edu
• Digisonde home page http://ulcar.uml.edu/digisonde.html
• SAOExplorer home page http://ulcar.uml.edu/SAO-X/SAO-X.html
• DIDBase home page http://ulcar.uml.edu/DIDBase/
Publications
• Reinisch, B. W., I. A. Galkin, G. Khmyrov, A. Kozlov, and D. F. Kitrosser, Automated collection and dissemination of ionospheric data from the digisonde network, Adv. Radio Sci. (2004), 2: 241-247.