a methodology for translating positional error into measures of attribute error, and combining the...
Post on 18-Dec-2015
226 views
TRANSCRIPT
![Page 1: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/1.jpg)
A methodology for translating positional A methodology for translating positional error into measures of attribute error, and error into measures of attribute error, and combining the two error sourcescombining the two error sources
Yohay Carmel1, Curtis Flather2 and Denis Dean3
1 The Thechnion, Haifa, Israel2 USDA, Forest Service3 Colorado State University
![Page 2: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/2.jpg)
Part 1: bridging the gap between positional error and classification error
![Page 3: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/3.jpg)
classification error -- difference in pixel class between the map and a reference
![Page 4: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/4.jpg)
Positional error(misregistration, location error)
Is the gap between the true location of an item and its location on the
map / image
![Page 5: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/5.jpg)
Positional error may translate to thematic error
positional error largely affects overall thematic error (often more than classification error)(Townshend et al 1992, Dai and Khorram 1998)
![Page 6: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/6.jpg)
Positional error:
RMSE = 2.51 m
classification error:
Accuracy Matrix:
Forest Shrubs GrassForest 221 32 6Shrubs 12 185 18Grass 3 27 149Classified
Reference
Goal 1: find a common denominator for both error typesGoal 2: combine the two error types to get an overall estimate of error (in the context of temporal change)
THE PROBABILITY THAT AN OBSERVED TRANSITION IS CORRECT
![Page 7: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/7.jpg)
positional error affects thematic error
![Page 8: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/8.jpg)
Expressing positional error in terms of thematic error
Shift = 1, 0Shift = 15, 7
Shift = 2, 3
![Page 9: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/9.jpg)
Forest Shrubs GrassForestShrubsGrassClassified
ReferenceExpressing positional error in terms of thematic error
![Page 10: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/10.jpg)
Error model: step 1 (and step 2) of 5
RMSEPositionalError
ClassificationError
Forest Shrubs GrassForest 221 32 6Shrubs 12 185 18Grass 3 27 149
Forest Shrubs GrassForest 24411 3708 1236Shrubs 2627 34763 4172Grass 1700 4944 20240
![Page 11: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/11.jpg)
1 2 3123
Reference
1 2 3123
positional accuracy matrix
ALOC
classification accuracy matrix
ACLASS
Combined
ABOTH
ALOC1,1*ACLASS
2,1 / n+1 1 2 3123
Total n+1 n+2 n+3
ALOC2,1*ACLASS
2,2 / n+2
ALOC3,1*ACLASS
2,3 / n+3
ABOTH2,1
=
+
+
Classified
Classified
Classified
error model: step 3 (of 5)Combining the two accuracy matrices
![Page 12: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/12.jpg)
Forest Shrubs Grass Forest 29652 7458 3668 Shrubs 6450 27885 4885 Grass 3459 5214 15332
Classified
Reference
Error model: step 3 Combined Error Matrix
![Page 13: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/13.jpg)
Error model: step 4
Calculating the combined PCC, and the Combined user accuracy, p(C)
Forest Shrubs Grass User accuracy
Forest 29652 7458 3668 0.73 Shrubs 6450 27885 4885 0.71 Grass 3459 5214 15332 0.64
Classified
Reference
PCC = 0.70
THE PROBABILITY THAT AN OBSERVED STATE IS CORRECT
![Page 14: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/14.jpg)
Error model: step 5
Calculating multi-temporal indices
One such index is The probability that an observed transition is correct
The context of this model is temporal change. The goal is to provide indices for the reliability of an observed change
![Page 15: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/15.jpg)
Example: vegetation changes in Hastings Nature Reserve, California
![Page 16: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/16.jpg)
1939
1956
1971
1995
![Page 17: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/17.jpg)
Example: vegetation changes in Hastings, California
RMSE 1939 = 3.53 m 1995 = 2.51 m
User accuracy for:Grass in 1939 = 0.92Trees in 1995 = 0.91
positional accuracy Classification accuracy
1939 1995
C1 C4
![Page 18: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/18.jpg)
The probability that an observed transition is
correct
p(C1C2…Cn) = p(C1) * p(C2) * … * p(Cn)
C1 C2 C3 C4
1939 1956 1971 1995
![Page 19: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/19.jpg)
Error model: step 5
p(C1C2…Cn) = p(C1) * p(C2) * … * p(Cn)
1939 1956 1971 1995
C1 C2 C3 C4
The probability that an observed transition is correct:
This probability may be calculated as the product of the respective user-accuracy value for the respective
year and class
![Page 20: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/20.jpg)
Example: vegetation changes in Hastings, California
1939 1995
G T
User accuracy for positional Classification Combined
Grass in 1939 0.82 0.92 0.74
Trees in 1995 0.77 0.91 0.72
p(GRASS1939TREE1995) = 0.530.53
p(G1939G1956G1971T1995) = 0.220.221939 1956 1971 1995
G G G T
![Page 21: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/21.jpg)
Transition type
Nature of transition
Proportion in 1939
Probability of being correct, given:
positional error
Classif. error
Combined error
GGGG
Grassland does not change
0.43 0.28 0.69 0.21
CGGG Chaparral burnt in 1955 fire
0.06 0.29 0.60 0.22
All 69 transitions involving forest 0.66 0.26 0.59 0.20
Averaged across the entire study area
1 0.29 0.67 0.22
Indices of accuracy of multi-temporal datasets
![Page 22: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/22.jpg)
(1) errors in each time step are independent of errors in other time steps
(2) positional and classification errors are independent of one another
A simulation study was conducted in order to evaluate how robust is the model in general
and in particular -- to violations of two assumptions:
![Page 23: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/23.jpg)
Maps for simulationsHigh autocorrelation, equal class proportions
Low autocorrelation, unequal class proportions
Int. J. Rem. Sens. 2004
![Page 24: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/24.jpg)
Original map
Spatial error
Classification error
Both error types
SimulationSimulation
![Page 25: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/25.jpg)
Model simulations were conducted under a range of values for:
• Number of map categories (2-4)
• Class Proportions in the original map
• Auto-correlation in the original map
• Auto-correlation in classification error
• Classification error rate
• Positional error rate
• Correlation in error structure between time steps
• Correlation between the two error types
![Page 26: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/26.jpg)
characteristics PCC D
1 2 3 Both
One class dominates 92% 4% 4.00% 0.89 0.08%
One class nearly absent 32% 68% 0.03% 0.77 0.20%
Positional error Large 36% 33% 31.0% 0.56 0.39%
Classification error large 30% 41% 30.0% 0.62 0.25%
Class proportions
Some results of simulation runsN
AA
D
k
i
k
j
bothobs
1 1
||
![Page 27: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/27.jpg)
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.0 0.1 0.2 0.3 0.4 0.5 0.6
Correlation between location and classification errors
Dif
fere
nce
in t
rans
itio
n pr
obab
iliti
es b
etw
een
mod
el
pred
icti
ons
and
sim
ulat
ions
maximum difference average difference
When models assumptions are not met, model fit decreases
Maximum correlation found in real datasets*
*IEEE GRSL 2004
Maximum correlation found in real datasets*
*IEEE GRSL 2004
![Page 28: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/28.jpg)
Transition type
Nature of transition
Proportion in 1939
Probability of being correct, given:
positional error
Classif. error
Combined error
GGGG
Grassland does not change
0.43 0.28 0.69 0.21
CGGG Chaparral burnt in 1955 fire
0.06 0.29 0.60 0.22
All 69 transitions involving forest 0.66 0.26 0.59 0.20
Averaged across the entire study area
1 0.29 0.67 0.22
Indices of accuracy of multi-temporal datasets
![Page 29: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/29.jpg)
PART 2:Controlling data-uncertainty
via aggregation
Pixel size = 0.3 mGrid cell size = 15 m = 2500 pixels
![Page 30: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/30.jpg)
Map aggregation = image degradationoverlay a grid of cells on the image (cell >>pixel)
and define the larger cell as the basic unit
Pixel size = 0.3 mGrid cell size = 15 m = 2500 pixels
![Page 31: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/31.jpg)
31%45%
24%
‘‘soft’ aggregationsoft’ aggregation‘‘hard’ aggregationhard’ aggregation
Map aggregation = image degradationoverlay a grid of cells on the image (cell >>pixel)
and define the larger cell as the basic unit
![Page 32: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/32.jpg)
Impact of positional error is largely reduced when cells are aggregated
ab
c
![Page 33: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/33.jpg)
At the pixel level:
only 55% of the pixels remained unaffected by a minor shift
At the grid cell level:
post-shiftpre-shift0.170.210.340.330.490.46
Impact of positional error is largely reduced when cells are aggregated
![Page 34: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/34.jpg)
This trade-off calls for a model that quantifies the process
to aid decisions on optimal level of aggregation
Aggregation:
Gain in accuracy BUT loss of information
![Page 35: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/35.jpg)
A geometric approach to the impact of positional error
![Page 36: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/36.jpg)
Effective positional error at the grid cell level
2A
eeeAeA yxyx is the proportion of
pixels that transgress into neighboring cells
(RMSE units)
![Page 37: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/37.jpg)
Positional error at the GRID CELL level
p(loc) is the probability that positional error translates into attribute error
pA(loc) is the same probability – in the context of a larger grid cell
)()( locplocp A
![Page 38: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/38.jpg)
The impact of aggregation on thematic accuracy
0.23
A p(loc) cell size error
0.6 m 0.23
6 m 0.14
60 m 0.01
![Page 39: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/39.jpg)
Conclusions
• positional error has a large impact on thematic accuracy, particularly in the context of change
• But can be easily mitigated: increase MMU to >10X[positional error] and do not worry about it.
• Within overall thematic error at the pixel level –classification error component is typically smaller than the positional error component, but is more difficult to get rid of by aggregation.
![Page 40: A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and](https://reader030.vdocuments.us/reader030/viewer/2022033100/56649d265503460f949fca4e/html5/thumbnails/40.jpg)
TODA
THANK YOU