Data Assembly Center (DAC)Example Data Quality Procedures

VRCO Discussions, 06 December 2006

The Mission of the National Data Buoy Center (NDBC) is to Deliver Real-Time Meteorological and Oceanographic Data to Weather Forecasting Offices

Real-Time is generally considered < 2 hours after measurement acquisition (< 6 hr, NDBC receives credit)

The NDBC is currently quality controlling nearly 500 remote marine stations (buoys and fixed-stations) and number is growing

Data Management and Communications Branch Status

11 September 2006Weekly Platform Count

0

50

100

150

200

250

300

350

400

450

500

Date

Nu

mb

er

of

Pla

tfo

rms

NDBC DART IOOS NOS O&G TAO Total

Generalized Data Flow

Automated Real-Time QC isperformed on incoming data atthe National Weather Service Telecommunications Gateway. Limited algorithms are applied (range checks and time continuity).

DAC Watch-stander Schedules

Data Assembly CenterDay Watch Routine (Weekday)

I:\\DAC\Procedures\Daywatch Routine.xls

Date: _______________________________ Analyst: __________________________________

Time   Task Report Initials Remarks

6:00   Watch Turnover      

6:15   Review Synoptic situation and Yosimite displays      

6:30   WEB, Yosimite and Iridium Checks      

7:00   Evaluate Location Report      

7:30   Review Brief, add late changes Final Brief    

8:00  Post Briefing Slides to web and send 5-Day Forecast Report

5-Day Forecast Report

   

8:30   WEB, Yosimite and Iridium Checks      

9:00   Evaluate DAPS reports      

9:30   Perform DART review on web      

10:00   Analyze IOOS/NOS metdiffs (12z-12z)      

10:30   WEB, Yosimite and Iridium Checks      

11:00   Analyze IOOS/NOS metdiffs (12z-12z)    

11:30          

12:00   Analyze IOOS/NOS plots IOOS/NOS Daily

Update   

12:30   WEB, Yosimite and Iridium Checks      

13:00   Review Wave ValidationWave Validation

Report   

14:00   Analyze DART Summary Report      

14:30   WEB, Yosimite and Iridium Checks      

15:00   Evaluation Long Island Time Tab      

15:30   Review Critical BuoysCritical Buoy

Report   

16:00   Evaluate Communication Summary      

16:30   WEB, Yosimite and Iridium Checks      

17:00   Review Buoys no reporting No Data Report    

17:30   Prepare for Turnover      

18:00   Turnover      

Data Assembly CenterNight Watch Routine (Weekday)

I:\\DAC\Procedures\Nightwatch Routine.xls

Date: _______________________________ Analyst: __________________________________

Time   Task Report Initials Remarks

18:00   Watch Turnover      

18:15   Review Synoptic situation and Yosimite displays      

18:30   WEB, Yosimite and Iridium Checks      

18:45   AIS check      

19:00   Evaluate Location Report      

19:30   Preliminary Briefing Slides Preparations      

20:00   Analyze Power Data      

20:30   WEB, Yosimite and Iridium Checks      

21:00          

21:30   Perform DART review on web      

22:00   Analyze NDBC metdiffs (00z-00z)      

22:30          

23:00   WEB, Yosimite and Iridium Checks      

23:30   Analyze NDBC plots NDBC QC Update    

0:00          

0:30   WEB, Yosimite and Iridium Checks      

1:00   Evaluate Communications Summary      

1:30  Evaluate Station Location Report and prepare Drifting Buoy Report

Drifting Buoy Report    

2:00   Analyze DART Summary Report      

2:30   WEB, Yosimite and Iridium Checks      

3:00          

3:30   Prepare Briefing Slides      

4:00          

4:30   WEB, Yosimite and Iridium Checks      

5:00   Finalize Briefing Slides      

5:30   Prepare for Turnover      

6:00   Turnover      

Manual (DAC Watch-stander)Real-Time QCMarta Systems (Yosemite WeatherWorkstation)

•Wind barbs•Yellow circles indicate comms loss

(data > 2 hours old)•Overlaid satellite stitched graphics

and looping

Watch-standers must actively look at Screen – Not as many screens asGeographical Regions

Marta Display – Overall View

Marta Display – Gulf of Mexico

Near-Real-time QC(After Automated Real-Time QC)

•Intensive Database Set-up•ASCII IDs Associated with

measured parameters•Analysts run programs to search for

anomalous data•Program output on line printers

•DAC Analysts set-up all stations indatabase

Oracle Database – NEMIS GUI Front-end

Oracle Database – ASCII ID Assignment

Example METDIFF Printout Page

Lower-case flags, warning-onlyDoes not stop data

Generates stacks of paper

Example METDIFF Printout Page‘D-Flagged’ Data

Upper-case FlagsData flow is terminateduntil released by analyst

Example METDIFF Plots Region

Sta

tio

ns

Dat

a D

isp

lays

Oracle Database – Raw Data Report

Oracle Database – Raw Data Report

Oracle Database – Interactive Plotting

Communications Report

•Looking for data dropouts dueto communication problems

•Analyst runs program and scansoutput

Oracle Database – Communications Report

“M” – Missing Data (Comms)

Buoy Location Report

•Looking for buoys off-station“drifting buoys”

•Analyst runs program and scansoutput

Oracle Database – Buoy Location Report

% of Watch circle>100% is drifting