1 nodc quality control : automatic checks - reveal systematic errors in incoming data and metadata -...

11
1 C Quality Control : Automatic Checks reveal systematic errors in incoming data and metadata eliminate most non-representative data from consideration minates ~6% of perature data from sideration 00,000 profiles) cks include: ge check ke check sity inversion ed check ndard deviation Red – Unflagged Black – Flagged

Upload: theodore-burns

Post on 18-Jan-2016

216 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

1

NODC Quality Control : Automatic Checks

- reveal systematic errors in incoming data and metadata

- eliminate most non-representative data from consideration

Eliminates ~6% of

temperature data from

Consideration

(>800,000 profiles)

Checks include:

Range check

Spike check

Density inversion

Speed check

Standard deviation

Red – Unflagged Black – Flagged

Page 2: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

31 distinct regions with ocean variable ranges at standard depths, includes separate ranges for coastal regions

177,000 temperature profiles affected (1.5%)

Min/Max Range Checks

Page 3: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

Inversion and Large Gradient Checks

For Temperature: Inversion ≤ -0.3°C/meter Gradient ≥ 0.7°C/meter

MBT with large inversion

°Celsius

Dep

th (m

eter

s)

Inversions presentIn 276,000 temperatureProfiles (2% of profiles)

Large positive gradientsPresent in 534,000Temperature profiles (4% of profiles)

Page 4: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

• Density inversions – when local stability with respects to density less than zero

• Check applied at observed levels, but not used to exclude data from interpolation to standard levels

• Check applied at standard levels. Interpolation procedure can introduce inversions. Two or more density inversions at same station and whole temperature and salinity profiles are marked and not used to calculate climatologies.

• 214,00 temperature and salinity casts not used due to two or more density inversions on standard levels (1.6% of all temperature profiles, 3.7% of all salinity profiles)

Density inversion check

Page 5: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

Standard Deviation Checks

• check is on standard levels • 5° means and standard deviations used• values flagged if they exceed 3 to 5 times standard deviations from mean - 3 times for upper 50 meters for coastal data - 4 times for near coastal and near bottom data - 5 times for all other data

• profile not used for climatologies if it has 2 or more standard deviation flags

• separate checks for annual, seasonal, monthly time periods

• 367,000 temperature profiles flagged with more than 2 annual mean/standard deviation failures (3% of total profiles) - 233,000 salinity profiles flagged (4% of total)

Page 6: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

May – August 1976 Cruise of Russian Ship Priboy (Russian Far East Scientific Research Institute)

Page 7: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

Blue Dots = Phosphate data from Priboy cruise, 2839 stations.Red Dots = other Phosphate in representative 5° square in North Pacific, 5455 stationsRed Solid Line = Mean phospate values at standard depths for same 5° squareRed Dotted Line = ±3 to 5 standard deviations for 5° squareGreen Line= Mean Priboy cruise phosphate with correction applied

Estimated fix of data:Correct value = (3.189 X initial value) -0.094

Problem is due to difference inMolecular weight conventionally Used by Russians, Americans

Page 8: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

Flag values: 0-9

Flag value: 2

Flag values: 3-9

Depth: 0-50 m Depth: 51-100 m Depth: 101-400 m

Years: 1900-2013. Month: Jan - Mar

Courtesy Igor Smolyar

Page 9: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

Region: 73º-75º N, 10º-45º E 1900-2013, Jan-AprTemperature data 0-20 m

All measurements

Flagged values: Standard Deviation outliers

Fuzzy logic method

Temperature (°C)Courtesy Igor Smolyar

Page 10: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

# CRUZTRACK REPORT FORM:# Statn Date Time Latitde Longitde Dist / TLS = Speed * UniqStat Access# Ship MP#O 1 2/ 7/1955 10.00 13.700 144.733 0.0 0.0 0.0 1048747 7501234 2129 1. 2 2/ 7/1955 14.00 14.200 145.083 1.5 0.2 9.1 1048757 7501234 2129 1. 3 2/ 7/1955 18.00 14.600 145.300 1.1 0.2 6.8 1048772 7501234 2129 1. 4 2/ 7/1955 22.00 14.833 145.583 0.9 0.2 5.4 1048789 7501234 2129 1. 5 2/ 7/1955 99.99 31.700 114.734 0.0 0.0 0.0 9507544 507 2129 1. 6 2/ 8/1955 7.00 15.617 145.550 81.0 1.3 62.7 * 1048805 7501234 2129 1

Cruise Speed/Cruise TrackCheck

Page 11: 1 NODC Quality Control : Automatic Checks - reveal systematic errors in incoming data and metadata - eliminate most non-representative data from consideration

“Bullseyes” in December Salinity Climatology at Surface

Before cleanup After cleanup

Problems along 180 longitude caused by December, 1979Russian cruise. An archiving error involving a computer at theRussian National Oceanographic Data Center caused decreaseIn salinity values. We are able to recover original values for these data

Thank you – Questions?