Homogenization of monthly Benchmark temperature series of network no. 3 – using ProClimDB software

COST Benchmark meeting in Zürich 13-14 September 2010 – Lars

Andresen

Norwegian Meteorological Institute met.no

Software package

– AnClim• Homogeneity analysis (using txt-files)

– ProClimDB• Automating the homogenization procedure

(using mainly dbf-files)

• Petr Štěpánek

Norwegian Meteorological Institute met.no

Normal homogenization procedure

Original Data

Quality control

Reconstruction of series

Homogeneity testing

Adjusting Data

Reference series (40 years, 10 years overlap) from correl. / weights

SNHT (Alexandersson test) Assessment of hom. results

Standardization to base station (AVG/STD)

Stations within 10 km Demands on data coverage

Merging of different series

Iteration process

Reference series (10 years around inhomogeneity) from distances

Standardization to base station (AVG/STD)

Smoothing monthly adjustments / Demands on corr. after adjustm.

Rank of monthly values Comparing with neighbours

Replacing suspicious valuesDist. / Stand. to alt. / Outliers

Norwegian Meteorological Institute met.no

Detecting breaks of network 3 (15 series)

• Outliers removed from manipulated series– 10 outliers from 8 stations

• Testing settings of ProClimDB – 40 year periods, 10 years overlap versus 20

years– Excluding breaks closer than 4 years to edge of

series or to nearest break– Finding the more distinct breaks before the less

distinct ones

Norwegian Meteorological Institute met.no

Removing outliers

Station 01400

Value of 5/1978 changed from 14.8°C (outlier) to 10.8°C (true)

1976, 14.3/14.3

1977, 11.5/11.5

1978, 10.8/14.8

1979, 13.2/13.2

1980, 8.8/8.8

Norwegian Meteorological Institute met.no

Consequences by changing overlap years – A case study, using SNHT method

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Years from edge of series

%

FaultNo sign. breaksNearly approvedApproved

0

10

20

30

40

50

60

70

80

90

100

%

FaultNo sign. breaksNearly approvedApproved

0.3° 0.5° 0.7°

• Single shift of +/- 0.5°

• 2, 4, 9, 19 years from edge of a homogeneous temperature series of 40 years

• Single shift of +/- 0.3, 0.5, 0.7°

• Each pair 9 and 19 years from edge of the series

Norwegian Meteorological Institute met.no

Criteria for detection

• Approved– Correct year (two years involved, both correct)– Adjustment within ± 0.1 degrees, e.g. 0.5 ± 0.1– T0 ≥ 8.1 (40 years – significance level 95%)

• Nearly approved– Correct year, T0 ≥ 8.1, Adj = 0.5 ± 0.3 degrees

– Correct year ± 1, T0 ≥ 8.1, Adj = 0.5 ± 0.2

– Correct year, T0 ≥ 7.0 (s.l.90%), Adj = 0.5 ± 0.1

• Fault– Significant break not approved or nearly approved

Norwegian Meteorological Institute met.no

Network 3 – comparing 46 breaks

B: Breaks detected , M: Missing detection , F: Fault detection

After 0 1 2 iterations

05

101520253035

B M F

05

101520253035

B M F

05

101520253035

B M F

Overlap

10 years

20 years

05

101520253035

B M F

05

101520253035

B M F

05

101520253035

B M F

Y_Poss ≥30

Y_Poss ≥25

Y_Poss ≥20

Norwegian Meteorological Institute met.no

Left: ”Official result” (46 breaks)

Y_Poss ≥15, no iteration

05

101520253035

B M FY_Poss ≥30, 25 and 20, 2

iterations

Case study

05

10152025303540

B M F

Norwegian Meteorological Institute met.no

Discussion – 1Homogeneity analysis

Reference series for finding breaks• Using correlations• Using distances• Weighting of neighbour values (0.5 or 1.0?)• Period (40 years) / Overlap (10 or 20 years?)

Processing of results• Method (SNHT alone or in combination with others?)• Finding most probable breaks (Y_POSSIBLE). How?• Weighting of month, season, year (1, 2, 5)• Metadata (improving?)• Nearness to begin/end/other breaks (2 or 4 years?)

Norwegian Meteorological Institute met.no

Discussion – 2Adjustments of the series

Reference series for making adjustments• Using distance alone (limitation on distance)• Using distance and correlation (limitations on distance

and correlation)

Smoothing monthly adjustments• Gauss filter (0~no smoothing, 2~period of 5 values is

recommended, other?)

Checking correlation after adjustments• Keep smoothed adjustment if correlation improvement

between candidate and neighbours (Corr+value) ≥ 0.005 or ≥ 0.000 ?

Norwegian Meteorological Institute met.no

Discussion – 3

Iterations

• Using adjusted file for new analysis• How finding most probable breaks

– More stringent criteria when automating procedure (depends on metadata and Y_POSSIBLE)?

Norwegian Meteorological Institute met.no

Conclusion

• It is reason for concern about the high number of fault detections

• Use of metadata is necessary in homogenization! Using metadata allows lower values of Y_Possible

• It’s important to find the optimal conditions of a procedure before comparing methods

• Homogenization has no correct answer !