executable test suite (ets) · 2 name b01 version 1.0 date 10-03-2014 author agnieszka tarko &...

1

LPIS Quality Assurance Framework

Executable Test Suite (ETS)

PACKAGE CONSISTENCY TESTS, version 5.3

March 2015

2

Name

b01

Version

1.0

Date

10-03-2014

Author

Agnieszka Tarko & Paolo Isoardi

Description

Check if the number of imported parcels in LpisSamplePreselectionStatus.xml equals the

optimal pre-selection size

Input

SP package: lpis qa.sample_preselections (field: optimal_preselection_size)

ETS package: LpisSamplePreselectionStatus.xml

Procedure

Imported_parcels = The sum of all parcels from LpisSamplePreselectionStatus.xml

IF

Imported_parcels = optimal_preselection_size

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

The number of imported parcels in LpisSamplePreselectionStatus.xml file

(imported_parcels) does not equal the optimal pre-selection size (optimal preselection

size).

Comment

Possible exception: parcels holding specific code in “comment F1” field (2012 SI only)

3

Name

b02

Version

1.0

Date

10-03-2014

Author


Description

Check if there are differences in rpid between the original sample pre-selection provided

by JRC and the LpisSamplePreselectionStatus.xml file delivered in the ETS report

package

Input

SP package: samplepreselection xml file provided by JRC to the MS (lpis

qa.sample_preselections_details)


Procedure

IF

each samplepreselection_rpid have only one equal samplepreselectionstatus_rpid and all

samplepreselectionstatus_rpid have a “corresponding” rpid in samplepreselection THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Some parcels in LpisSamplePreselectionStatus.xml file delivered in the ETS report

package do not match the original sample pre-selection provided by JRC.

Details:

- (count) unexpected additional parcels: (list of rpid)

- (count) parcels not found: (list of rpid)

Comment


4

Name

b03

Version

1.0

Date

10-03-2014

Author


Description

Check correctness of reporting inspected and skipped parcels in the

LpisSamplePreselectionSatus.xml

Input

SP package: lpis qa.sample_preselections (fields: optimal_preselection_size,

sample_size)


Procedure

IF

a) Number of records where RP_FSI_ skipped = “false” and belongsToQC_Sample =

“true” equals sample_size

AND

b) each parcel with RP_FSI_ skipped ="true” have attribute belongsToQC_Sample as

“true”

AND

c) (Number of records where RP_FSI_ skipped = “false” and belongsToQC_Sample

= “false”) equals ((optimal_preselection_size) minus (sample_size) minus

(number of parcels holding skipped occurence="true”))

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Incorrect reporting of inspected and skipped parcels in LpisSamplePreselectionSatus.xml

file.

Details:

- The number of parcels feasible for inspection (sps_sample_size) does NOT equal

the sample size value (sample_size);

- Not all parcels feasible for inspection belong to QC sample;

- The optimal preselection size (optimal_preselection_size value) does NOT equal

the sum of parcels not belonging to QC sample (count) plus inspected parcels

(sample_size count) plus skipped parcels (count).

Comment


5

Name

b04

Version

1.0

Date

10-03-2014

Author


Description

Check if the reference area declared by the MS in the sample pre-selection

(LpisPointZeroState.gml) equals area in the ETS package (LpisPolygonZeroState.gml)

Input

SP package: LpisPointZeroState.gml

ETS package: LpisPolygonZeroState.gml

Procedure

IF

Reference area of parcels in LpisPolygonZeroState.gml equals reference area of parcels

with the same rpid in the LpisPointZeroState.gml

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Reference area of parcels in LpisPolygonZeroState.gml file does not equal reference area

of parcels with the same rpid in LpisPointZeroState.gml file.

Details:

(count) parcels with area mismatch: (list of rpid)

Comment

Values rounded to 4 decimals.

This test executes tests b07 and b11 internally.

6

Name

b05

Version

1.0

Date

10-03-2014

Author


Description

Check if the rpid is unique (if there are no multipolygons)

Input


Procedure

IF

There are no duplicate rpid

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Some rpid in LpisPolygonZeroState.gml file are NOT unique. Risk of multi-polygon/ multi-parcel critical defect in the sample. Details:

(count) duplicate rpid: (list of rpid)

Comment

7

Name

b06

Version

1.0

Date

10-03-2014

Author


Description

Check if all inspected parcels belonging to the QC sample are present in the

PolygonZeroState.gml

Input



Procedure

IF

All parcels rpid where “belong to QC sample=true” in LpisSamplePreselectionStatus.xml

have only one corresponding rpid in LpisPolygonZeroState.gml

(exception: parcels flagged in LpisSamplePreselectionStatus as

referenceParcelIdFoundNotPersistentInLpisA3 occurence="true" or

parcelGeometryIsNotAvailableT5 occurence="true")

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Not all inspected parcels belonging to the QC sample have a corresponding polygon

(exactly one) in the PolygonZeroState.gml file.

Details:

1. Duplicate rpid: (list of rpid)

2. Missing rpid: (list of rpid)

Comment

8

Name

b07

Version

1.0

Date

10-03-2014

Author


Description

Check if reference area in the ETS package (LpisPolygonZeroState.gml file) is in

hectares

Input


Procedure

IF

Reference area is roughly 1/10000 of area calculated by FME scripts (in m2)

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Reference area in LpisPolygonZeroState.gml file is not in hectares.

Comment

Measures rounded to 4 digit precision.

This test is called internally by test b04.

9

Name

b08

Version

1.0

Date

10-03-2014

Author


Description

Check if RP_FSI (rp_fsi_skipped_occ) attribute in LpisSamplePreselectionStatus.xml is

correct

Input


Procedure

IF

rp_fsi_refparidnotpera3_occ=FALSE and

rp_fsi_pargeonotavat5_occ=FALSE and

rp_fsi_parnotcovimgt2_occ=FALSE and

rp_fsi_parcovclohazt4_occ=FALSE and

rp_fsi_luiintimportc4_occ=FALSE and

rp_fsi_formajcirobsluif1_occ=FALSE

THEN

IF rp_fsi_skipped_occ=FALSE THEN

Test passed

ELSE

Test failed

ELSE

IF rp_fsi_skipped_occ=TRUE THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

(count) parcels in LpisSamplePreselectionStatus.xml file have incorrect RP_FSI attribute.

Details: (list of rpid)

Comment

10

Name

b09

Version

1.0

Date

10-03-2014

Author


Description

Check if all parcels flagged as feasible for inspection in

LpisSamplePreselectionStatus.xml file have only one corresponding record in

EtsObservations.xml file

Input


ETS package: EtsObservations.xml

Procedure

IF

All parcel rpid flagged as “rp_fsi_skipped_occ=FALSE” in

LpisSamplePreselectionStatus.xml have only one corresponding record with the same

rpid in EtsObservations.xml (1:1)

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Not all parcels flagged as feasible for inspection in LpisSamplePreselectionStatus.xml file

have corresponding record (exactly one) in EtsObservations.xml file.

Details:



Comment

11

Name

b10

Version

1.0

Date

10-03-2014

Author


Description

Check if all parcels flagged as feasible for inspection in

LpisSamplePreselectionStatus.xml file have only one corresponding polygon in

LpisPolygonZeroState.gml file

Input



Procedure

IF

All parcel rpid flagged as “rp_fsi_skipped_occ=FALSE” in

LpisSamplePreselectionStatus.xml file have only one corresponding polygon rpid in

LpisPolygonZeroState.gml file (1:1)

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Not all parcels flagged as feasible for inspection in LpisSamplePreselectionStatus.xml file

have corresponding polygon (exactly one) in LpisPolygonZeroState.gml file.

Details:



Comment

12

Name

b11

Version

1.0

Date

16-06-2014

Author

Paolo Isoardi

Description

Check if reference area has 4 decimal precision

Input

SP package: LpisPointZeroState.gml


Procedure

IF

The count of (parcels with reference area digit < 4 in LpisPointZeroState) is = 0 AND the

count of (parcels with reference area digit < 4 in LpisPolygonZeroState) is = 0

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Reference area in LpisPolygonZeroState.gml file has not 4 decimal precision.

Comment

This test is called internally by test b04.

13

Name

b12

Version

1.0

Date

14-08-2014

Author


Description

Check if all parcels flagged as “feasible for measurement” have reference polygon in

EtsInspectionMeasurements.gml, and “not feasible for measurement” have no polygon

entry

Input


ETS package: EtsInspectionMeasurements.gml

Procedure

IF

All parcels rpid flagged as “RP_FSM = true” in EtsObservations.xml have at least one

corresponding polygon with the same rpid in EtsInspectionMeasurements.gml

AND

all rpid in EtsInspectionMeasurements.gml have been “paired”

AND

all parcels rpid flagged as “RP_FSM = false” in EtsObservations.xml have NO

corresponding polygon with the same rpid in EtsInspectionMeasurements.gml

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Message in log table ets.lqtestlog

Comment

14

Name

b13

Version

1.0

Date

18-08-2014

Author


Description

Check if RP_CNF_observedToRecordedAreaPercentage in Observations.xml file is

correctly calculated

Input

ETS package: ETSObservations.xml


Procedure

IF

observedToRecordedAreaPercentage (EtsObservations.xml) = (RP_MEA

(EtsObservations.xml) / (referenceArea * 10000 (LpisPolygonZeroState.gml)) ) * 100

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

RP_CNF_observedToRecordedAreaPercentage in Observations.xml file is not correctly

calculated

Details:

(count) parcels with wrong RP_CNF_observedToRecordedAreaPercentage: (list of rpid)

Comment



15

Name

b14

Version

1.0

Date

18-08-2014

Author


Description

Check if RP_CNF_observedRecordedAreaDifference in Observations.xml file is correctly

calculated

Input

ETS package: ETSObservations.xml


Procedure

IF

observedRecordedAreaDifference (EtsObservations.xml) = RP_MEA

(EtsObservations.xml) – referenceArea * 10000 (LpisPolygonZeroState.gml)

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

RP_CNF_observedRecordedAreaDifference in Observations.xml file is not correctly

calculated

Details:

(count) parcels with wrong RP_CNF_observedRecordedAreaDifference: (list of rpid)

Comment



16

Name

b15

Version

1.0

Date

20-11-2013

Author

Natasa Luketic & Paolo Isoardi

Description

Check if all “userDefinedLegendCode” used in EtsObservation.xml match the ones

defined in EligibilityProfile.xml

Input

ATS package: EligibilityProfile.xml


Procedure

IF

at least one “userDefinedLegendCode” in EligibilityProfile.xml is equal to

“userDefinedLegendCode” assigned in EtsObservation.xml for each rpid

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Some userDefinedLegendCode used in EtsObservation.xml are not among the ones

defined in EligibilityProfile.xml file

Details:

(count) undefined userDefinedLegendCode: (list of userDefinedLegendCode)

Comment

17

Name

b16

Version

1.0

Date

20-11-2013

Author


Description

Check if all “userDefinedLegendCode” used in InspectionMeasurement.gml match the

ones defined in EligibilityProfile.xml

Input

ATS package: EligibilityProfile.xml

ETS package: InspectionMeasurement.gml

Procedure

IF

at least one “userDefinedLegendCode” in EligibilityProfile.xml is equal to

“userDefinedLegendCode” assigned in InspectionMeasurement.gml for each rpid

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Some userDefinedLegendCode used in InspectionMeasurement.gml are not among the

ones defined in EligibilityProfile.xml file

Details:

(count) undefined userDefinedLegendCode: (list of userDefinedLegendCode)

Comment

18

Name

b17

Version

1.0

Date

06-06-2014

Author


Description

Check if the zoneid names defined in OrthoimagerySet.xml file are unique

Input

ETS package: OrthoimagerySet.xml

Procedure

IF

zoneid is unique

THEN

Test passed

ELSE

Test failed

Output

message in

case of

failure

Some zoneid in OrthoimagerySet.xml file are duplicate.

Details:

(count) duplicate zoneid: (list of zoneid)

Comment

19

Name

b18

Version

1.0

Date

06-06-2014

Author


Description

Check if all reference parcels reported as skipped in LpisSamplePreselectionStatus.xml

file are consistent

Input


Procedure

IF

sum (

count (referenceparcelidfoundnotpersistentinlpisa3 occurence=TRUE),

count (parcelgeometryisnotavailablet5 occurence=TRUE),

count (parcelispartiallyorfullynotcoveredbyimaget2 occurence=TRUE),

count (parcelispartiallyorfullycoveredbycloudsorhazet4 occurence=TRUE),

count (luiinterpretationimpossiblewithgivenorthoimagec4 occurence=TRUE),

count (forcemajeurecircumstancesobservedonluif1 occurence=TRUE),

count (erroneousscopings1 occurence=TRUE)

)

IS EQUAL TO

count (skipped occurence=TRUE)

THEN

test passed

ELSE

test failed

Output

message in

case of

failure

Some reference parcels reported as skipped in LpisSamplePreselectionStatus.xml file are

not consistent.

Comment

20

Name

b19

Version

1.0

Date

06-06-2014

Author


Description

Check if all reference parcels in EtsObservations.xml file are conformant to 3% area

purity test:

For all reference parcels that have area recorded greater than 5.000 m2 and area

difference either bigger than 3% or bigger than 10.000 m2, a cause of non-conformity

should be given.

Input


Procedure

For each RPid:

IF RP_MEA > 5000

AND

( “observedToRecordedAreaPercentage” > 103

OR

“observedToRecordedAreaPercentage” < 97

OR

“observedRecordedAreaDifference” > 10000 )

AND

all categories of “RP_CEA” are false

THEN

Test failed

ELSE

Test passed

Output

message in

case of

failure

Some reference parcels in EtsObservations.xml file are non-conformant to 3% area

purity test.

Details:

(count) non-conformant reference parcels: (list of rpid)

Comment

21

Name

b20

Version

1.0

Date

06-06-2014

Author


Description


purity test:

For all reference parcels that have area recorded between 2.000 – 5.000 m2 and area

difference bigger than 5%, a cause of non-conformity should be given.

Input


Procedure

For each RPid:

IF RP_MEA > 2000

AND RP_MEA < 5000

AND


OR

“observedToRecordedAreaPercentage” < 95 )

AND


THEN

Test failed

ELSE

Test passed

Output

message in

case of

failure


purity test.

Details:


Comment

22

Name

b21

Version

1.0

Date

06-06-2014

Author


Description


purity test:

For all reference parcels that have area recorded less than 2.000 m2 and area difference

bigger than 7%, a cause of non-conformity should be given.

Input

ETS package: ETS_Observations.xml

Procedure

For each RPid:

IF RP_MEA < 2000

AND


OR

“observedToRecordedAreaPercentage” < 93 )

AND


THEN

Test failed

ELSE

Test passed

Output

message in

case of

failure


purity test.

Details:


Comment

23

Name

b22

Version

1.0

Date

06-06-2014

Author


Description

Check if all reference parcels in EtsObservations.xml file have consistent contamination

reporting.

Input


Procedure

For each RPid:

IF

At least one category of “RP_ANF” abundance > 0

AND

all categories of “RP_CNT” are false

THEN

Test failed

ELSE

Test passed

Output

message in

case of

failure

Some reference parcels in EtsObservations.xml file have inconsistent contamination

reporting.

Details:

(count) reference parcels with inconsistent contamination reporting: (list of rpid)

Comment

24

Name

b23

Version

1.0

Date

06-06-2014

Author


Description

Check if all inspected parcels that are not feasible for measurement have assigned cause

of non-conformities.

Input


Procedure

For each RPid:

IF

“RP_FSM” is false

AND


THEN

Test failed

ELSE

Test passed

Output

message in

case of

failure

Some inspected parcels that are not feasible for measurement do not have cause of

non-conformities assigned.

Details:

(count) parcels without assigned cause of non-conformities: (list of rpid)

Comment

25

Name

b24

Version

1.0

Date

06-06-2014

Author


Description

Check if all inspected parcels that are contaminated have a category of non-conformities

assigned.

Input

ETS package: ETS_Observations.xml

Procedure

For each RPid:

IF

“RP_FSM” is true

AND

at least one category of “RP_CNT” is true

AND


THEN

Test failed

ELSE

Test passed

Output

message in

case of

failure

Some inspected parcels that are contaminated do not have a category of non-

conformities assigned.

Details:

(count) parcels without assigned cause of non-conformities: (list of rpid)

Comment

executable test suite (ets) · 2 name b01 version 1.0 date 10-03-2014 author agnieszka tarko &...

Documents