New Technologies for New Technologies for Storage and Display of Storage and Display of Meteorological DataMeteorological Data

Dale PaynterOperations Management Group

May, 2002NUMUG Conference

10 May 2002 OMG 2

Benefits of Technology?Benefits of Technology?

Move to modern technology path that will be developed and supported

Make initial installation, maintenance & upgrades easier Change is the name of the game

External requirements & standards Infrastructure changes Organizational re-alignment

Meet rising expectations Simple systems interfaces Capitalize on staff knowledge & skills

Let Meteorologists manage, analyze and use the data

10 May 2002 OMG 3

New TechnologiesNew Technologies

What are they?

Data Driven System Architecture

Relational Databases

Web Applications

10 May 2002 OMG 4

Data Driven ArchitectureData Driven Architecture

What is it?Write code to be independent of

specificsMake system configurable to define:

Input stream Parameters for internal calculations Reports Interfaces to other systems

10 May 2002 OMG 5

Data Driven ArchitectureData Driven Architecture

What are the benefits?Respond to new requirements and

standardsReduce dependency on vendors & ITGet out of program change queueReduce change orders & site visitsGive users control through configuring

system

10 May 2002 OMG 6

What does this let you do What does this let you do easily?easily?

Add new sensors with no code changes

Create new list reportsChange corporate informationRevise alarm / flag limitsAdd or decommission sampling

locationsInterface to different systems

10 May 2002 OMG 7

So how do we do this?So how do we do this?

Parameters, parameters, parameters...Make data base ‘self-descriptive’

Store ‘what’ is being stored as well as ‘values’ Eliminate need to reorganize / reload data to

accommodate changes

Make variables for all ‘constants’

Tower namesTime zonesDaylight saving timeMeasurement codes

Report titles & headingsData to display / orderReport field formatsConversion formulae

10 May 2002 OMG 8

Adding a new sensorAdding a new sensor

Traditional “Programming” Approach User configures or reprograms

datalogger Programmer adds new field to file

formats. Then dumps existing data, reloads and verifies

Programmer changes data loading to process new input layout & update new field

Programmer changes updating to display / edit new field

Programmer changes reporting to display new field

System administrator installs final programs on all affected computers

Data Driven Architecture Approach User configures datalogger User defines sensor in

database User tests and verifies results System administrator copies

configurations to production system

10 May 2002 OMG 9

Relational DatabasesRelational Databases

What are they?What are they?Data is stored in Tables made up of rows

of fields (columns)Data is referenced through name of the

fieldSubsets of data may be selectedTables may be joined to retrieve related

records

10 May 2002 OMG 10

Relational DatabasesRelational Databases

Usage comparison exampleSequential file - 30’ wind speed is in columns

26 to 31 (format F6.2)Typical relational database - 30’ wind speed is

stored in a field named WS30Relational database using data driven

architecture - 30’ wind speed is stored in a field named Value where the field MeasCode contains ‘WS’ and Height contains ‘30’

10 May 2002 OMG 11

Relational DatabasesRelational Databases

What are the the general benefits?Built-in securitySupports simultaneous accesses Built-in data recovery techniquesBuilt-in data retrieval algorithms &

optimizationStandard database interaction (SQL) In-house expertise can be leveraged

10 May 2002 OMG 12

Relational DatabasesRelational Databases

And... for meteorological data?Concurrent loading, updating and reportingPermits non-sequential loading for legacy

data and data recoveryHandles data retention and archiving

requirementsAd hoc queriesTime-stamping data changesKeep original values as well as edited values

10 May 2002 OMG 13

Web ApplicationsWeb Applications

In general... User requests report and submits parameters through

browser Web application generates report on server and sends

to browser User views / prints report through the desktop browser

Benefits? Removes need to run / maintain specific report

programs on desktop Just ask your LAN Administrator…

10 May 2002 OMG 14

Web Server ConfigurationWeb Server Configuration

Meteorologist’sWorkstationrunningUpdater

Meteorological Monitoring SystemServer

MMSDatabase

InterceptTM

&Loader

Web Server

Reporter

InterfaceFiles Other

Servers

Meteorologist’s& Users’

Workstationsrunning

Web Browsers

Sensors ZenosZenos

Zeno®s

10 May 2002 OMG 15

Web TechnologyWeb Technology

Internet or Intranet Server Data manipulation and report

generation performed on serverOnly results sent to browserAutomatic report refresh optionAdditional security levels possible

10 May 2002 OMG 16

How does it perform?How does it perform?

Even running on a notebook:Current reports are instantaneousHistorical reports take a few secondsFrequency Distribution produces a 1

year report in 12 secondsData Capture Summary produces a 1

year report in 55 seconds

10 May 2002 OMG 17

Benefits of TechnologyBenefits of Technology

Move to modern technology path that will be developed and supported

Get out of the program change queue

Flexibility to respond to change

Let Meteorologists manage, analyze and use the data