lecture #5: maps with gaps-- small geographic area estimation, kriging, and kernel smoothing
DESCRIPTION
Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing. Spatial statistics in practice Center for Tropical Ecology and Biodiversity, Tunghai University & Fushan Botanical Garden. Topics for today’s lecture. The E-M algorithm The spatial E-M algorithm - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/1.jpg)
Lecture #5:Lecture #5: MAPS WITH GAPS--MAPS WITH GAPS-- Small geographic Small geographic area estimation, area estimation,
kriging, and kernel kriging, and kernel smoothingsmoothing
Spatial statistics in Spatial statistics in practicepractice
Center for Tropical Ecology and Center for Tropical Ecology and Biodiversity, Tunghai University & Fushan Biodiversity, Tunghai University & Fushan
Botanical GardenBotanical Garden
![Page 2: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/2.jpg)
Topics for today’s lecture
• The E-M algorithm
• The spatial E-M algorithm
• Kriging in ArcGIS
• geographically weighted regression (GWR)
• approaches to map smoothing
![Page 3: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/3.jpg)
THEOREM 1When missing values occur only in a response variable, Y, then the iterative solution to the EM algorithm produces the regression coefficients calculated with only the complete data.
PF: Let b denote the vector of regression coefficients that is converged upon. Then if ,bXY mm
ˆ
ooTo
1o
To
mTmo
To
1-m
Tmo
To
m
o
T
m
o
1
m
o
T
m
o
)(
)()(
bYXXX
bXXYXXXXX
bX
Y
X
X
X
X
X
Xb
![Page 4: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/4.jpg)
THEOREM 2 When missing values occur only in a response variable, Y, then by replacing the missing values with zeroes and intro-ducing a binary 0/-1 indicator variable covariate -Im for each missing value m, such that Im is 0 for all but missing value observation m and 1 for missing value observation m, the estimated regression coefficient bm is equivalent to the point estimate for a new obser-vation, and hence furnishes EM algorithm imputations.PF:Let bm denote the vector of regression coefficients for the missing values, and partition the data matrices such that
,
and ,)(
)()(
)()(
omm
oTo
1o
Too
m
oTo
Tm
1o
Tommm
1o
Tom
Tm
1o
To
1o
To
m
o
T
mmm
omo
1
mmm
omo
T
mmm
omo
m
o
bXb
YXXXb
0
YX
XXXXIXXX
XXXXX
0
Y
IX
0X
IX
0X
IX
0X
b
b
![Page 5: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/5.jpg)
The EM algorithm solution
M
1mmm εIyXβY
where:the missing values are replaced by 0 in Y, andIm is an indicator variable for missing value m that contains n-m 0s and a single 1
m
om
mm,
mo,
mm
oo
m
o
β
α
0
εy
I
0
X1
X1
0
Y
![Page 6: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/6.jpg)
THEOREM 3 For imputations computed based upon Theorem 2, each standard error of the estimated regression coefficients bm is equivalent to the conventional standard deviation used to construct a prediction interval for a new observation, and as such furnishes the corresponding EM algorithm imputation standard error.PF:
2εT
m1
oTommm
1o
Tom
Tm
1o
To
1o
To2
ε
1
mmm
omo
T
mmm
omo σ)()(
)()(σ
XXXXIXXX
XXXXX
IX
0X
IX
0X
2εdiag
Tm
1o
Tommm σ])([
mXXXXIsb
![Page 7: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/7.jpg)
What is the set of equations for the following case?
10 7
7 y4 = ?
080
1087
1087
20810
4
y
M
1mmm εIy1Y α
![Page 8: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/8.jpg)
Some preliminary assessments
![Page 9: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/9.jpg)
simulations
Calculations from ANCOVA regression and the EM algorithm Data Source quantity Reported value OLS/NLS estimate Schafer (1997) p. 43 )( 48.1000 48.10000
)( 59.4260 59.42600
p. 195 2,3y average (n=5) 226.2 228.0 (se = 32.86)
4,3y average (n=5) 146.8 146.2 (se = 38.37)
5,3y average (n=5) 190.8 192.5 (se = 34.11)
10,3y average (n=5) 250.2 271.7 (se = 36.20)
13,3y average (n=5) 234.2 241.3 (se = 35.18)
16,3y average (n=5) 269.2 269.9 (se = 34.53)
18,3y average (n=5) 192.4 201.9 (se = 32.91)
23,3y average (n=5) 215.6 207.4 (se = 33.09)
25,3y average (n=5) 250.0 255.7 (se = 33.39)
![Page 10: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/10.jpg)
simulatedimputations
0.99 R ;0.987Y0.044Y 2imputedreported
![Page 11: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/11.jpg)
EM algorithm solution for aggregated georeferenced
data: vandalized turnips plots
![Page 12: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/12.jpg)
MTB > regress c4 8 c7-c14Regression Analysis: C4 versus C7, C8, C9, C10, C11, C12, C13, C14The regression equation isC4 = 28.9 - 6.32 C7 - 18.2 C8 - 1.10 C9 - 11.4 C10 - 10.1 C11 + 28.9 C12 + 18.8 C13 + 27.8 C14
Predictor Coef SE Coef T PConstant 28.900 2.404 12.02 0.000C7 [I1-I6] -6.317 3.254 -1.94 0.063C8 [I2-I6] -18.200 3.254 -5.59 0.000C9 [I3-I6] -1.100 3.399 -0.32 0.749C10 [I4-I6] -11.400 3.254 -3.50 0.002C11 [I5-I6] -10.100 3.399 -2.97 0.006
C12 [plot(6,5)] 28.900 5.887 4.91 0.000C13 [plot(5,6)] 18.800 5.887 3.19 0.004C14 [plot(6,6)] 27.800 5.887 4.72 0.000
![Page 13: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/13.jpg)
Analysis of Variance for C4 Source DF SS MS F PC5 5 1289.0 257.8 8.92 0.000Error 27 779.9 28.9Total 32 2068.9
Individual 95% CIs For Mean Based on Pooled StDevLevel N Mean StDev ---+---------+---------+---------+---1 5 28.900 4.407 (-----*-----) 2 6 22.583 6.391 (----*-----) 3 6 10.700 2.585 (----*-----) 4 5 27.800 5.082 (-----*-----)5 6 17.500 6.648 (-----*-----) 6 5 18.800 5.922 (------*-----) ---+---------+---------+---------+---Pooled StDev = 5.375 8.0 16.0 24.0 32.0
![Page 14: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/14.jpg)
Residual spatial autocorrelation
What does this mean?
![Page 15: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/15.jpg)
![Page 16: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/16.jpg)
SAR-based missing data estimation
where ym is a missing value (replaced by 0 in Y),
Im is an indicator variable for ym, and
is the mth column of geographic weights
matrix W
εWI
XβWIWYY
M
1m
*mm )ρ(y
)ρ(ρ
om
*omW
![Page 17: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/17.jpg)
The Jacobian term
1
mmmo
omoo2 detJVV
VV
])ρωLN(1 )ρλLN(1[n-n
2 mn
1kk
n
1ii
m
NOTE: denominator becomes (n-nm)
![Page 18: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/18.jpg)
What is the set of equations for the following case?
32
2231
12
eyρ)ρ(1μ0ρ10
ey)ρ(1μ2
yyρ0
eyρ)ρ(1μ0ρ7
7 Y2 = ? 10
εWI1WIWYY
M
1m
*mm )ρ(yμ)ρ(ρ
om
![Page 19: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/19.jpg)
0
ε
X
X
1
1
YIY
Y
WW
WW
0
Y
k
1
m
o
m
o
mm
o
m
o
mmmo
omoo
m
o
β
β
ρ)α-(1
0
ρ
spatial autoregressive (AR)
)ˆ(ˆˆˆoo
1oomomm βXYΣΣβXY
kriging
estimate withsemivariogram model
fit semivariogram model with
![Page 20: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/20.jpg)
The pure spatial autocorrelation CAR model
)β()ρ(ρβˆ0oomo
-1mm0mm 1YCCI1Y
Dispersed missing values:
NOTE: exactly the same algebraic structure as the kriging equation
)β(ρβˆ0oomo0mm 1YC1Y
ImputationImputation = the observed mean plus a weighted average of the surrounding residuals
![Page 21: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/21.jpg)
Employing rook’s adjacency and a CAR model, what is the equation for
the following imputation?
10 3 7
6 y5 = ? 4
9 5 5
)]b(5 )b(4
)b(6)b[(3ρby
00
0005
![Page 22: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/22.jpg)
The spatial filter EM algorithm solution
M
1mmm εβEIyXβY
kEkX
where:
the missing values are replaced by 0 in Y, and
Im is an indicator variable for missing value m that contains n-m 0s and a single 1
![Page 23: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/23.jpg)
Field plot
Conven-tional EM estimate
Spatial SAR-EM estimate
= 0.443
Spatial filter: 3 selected
eigenvectors
(6,5) 28.9 29.99 24.31
(5,6) 18.8 17.66 13.62
(6,6) 27.8 28.26 23.93
SARρ
Imputation Imputation of turnipof turnip
productionproductionin 3 in 3
vandalizedvandalizedfield plotsfield plots
![Page 24: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/24.jpg)
Cressie’s PA coal ash
model estimate
Cressie 10.27%
Spherical 10.62%
Gaussian 10.18%
exponential 10.12%
SAR 10.17%
spatial filter 10.71%
min mean max
7.00 9.78 17.61
![Page 25: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/25.jpg)
![Page 26: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/26.jpg)
Missing 1992 georeferenced density of milk
production in Puerto Rico: constrained (total = 1918)
Predicted from 1991 DMILK
Predicted from spatial filter
Predicted from both
235 70 385
1,339 1,848 1,065
344 0 468
predictionsMoran scatterplot
![Page 27: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/27.jpg)
USDA-NASSestimation ofPennsylvania
crop production
covariate
totalconstraints
map gaps
![Page 28: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/28.jpg)
![Page 29: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/29.jpg)
USDA-NASS estimation of Michigan crop production
If this is2% milk, how mucham I paying for the
other 98%?
![Page 30: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/30.jpg)
Michigan imputations
differentresponse variable
specifications
![Page 31: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/31.jpg)
USDA-NASS estimation of Tennessee crop production
![Page 32: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/32.jpg)
Tennesseeimputations
![Page 33: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/33.jpg)
An EM specification when some data for both Y and the Xs are missing
Ym,y
m,y
m,y,x
m,y,o
Ym,x
m,x,y
m,x
m,x,o
Y
Y
m,y
m,x
o
m,y
m,x
o
m,y
m,x
o
Y
I
0
0
X
0
I
0
X
0
X
1
1
1
0
Y
Y
XXm,y
m,y
m,y,x
m,y,o
m,x
m,x,y
m,x
m,x,o
X
X
m,y
m,x
o
m,y
m,x
o
m,y
m,x
o
Y
I
0
0
X
0
I
0
0
Y
Y
1
1
1
X
0
X
![Page 34: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/34.jpg)
Concatenation results:
m,x
o,x
m,y
o,y
m,xm,x
o,xm,y
m,y
o,yx
m,y
o,yxx2
m,ym,y
o,ym,x
m,x
o,xy
m,x
o,xyy1
m,x
o,x
m,y
o,y
0
0X
I
0Y
I
0
0
Yeq
YI
0X
I
0
0
Xeq
0
X
0
Y
![Page 35: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/35.jpg)
( )
( )
( )
yie ld y
acresarea a
productionarea p
y
a
p
0
0
0
y ij j yj
n n
ij
prp
p
ara
a
yj n
n
y y yx
area xy
y ijx
area xyj
n
y y y xy y ijx
area xyj
n
w yie ld w
C
C
w w
y
m
jr
krk
n r
jr
krk
n r
y
m
xy
j
j
xy xy j
j
xy
[ ( ) ( ) ] ( ) [( )
( ) ] { ( ) [ ( )
1
1
1
1
1 1
1
1
1
1 ]
[ ( ) ( ) ]}
[ ( ) ( ) ]
y
y ij j yj
n n
ij
prp
p
ara
a
yj n
n
y ij j yj
n n
ij
prp
p
ara
a
yj n
n
y
y
y
m
jr
krk
n r
jr
krk
n r
y
m
y
m
jr
krk
n r
jr
krk
n r
y
m
w yie ld w
C
CI
w yie ld w
C
C
1
1
1
10
1
1
1
1
1
1
1
1
( ) [( )
( ) ] [( ) ]
1
0
0
0
0
1
1
1
1
1
y yx
area xy
y ijx
area xyj
npr
p
p
ara
a
y m
xy
j
j
xy
ir
krk
n r
ir
krk
n r
y
rw
C
CI
The spatialmodel
powertransformation
spatial autocorrelation
totalsconstraints
covariate
![Page 36: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/36.jpg)
0
0
1
1
1
1
1
1 1
1
a ij
acres
area aj
n n
ij
a r
a
a
ja
j n
n
a a ax
area xa
a ij
x
area xaj
n
a a a xa a ij
x
area xaj
n
a
w w
C
area
w w
j
j
a
m
jr
krk
n r
a
m
xa
j
j
xa xaj
j
xa a
[ ( ) ( ) ] ( ) [( )
( ) ] { ( ) [ ( ) ]
a ij
acres
area aj
n n
ij
a r
a
a
ja
j n
n
a ij
acres
area aj
n n
ij
a r
a
a
ja
j n
n
a a ax
area xa
a ij
x
area
w w
C
areaI
w w
C
area
w
j
j
a
m
jr
krk
n r
a
m
j
j
a
m
jr
krk
n r
a
m
xa
j
[ ( ) ( ) ]}
[ ( ) ( ) ] ( ) [( )
(
1
1
10
1
1
1
1
11
j
xa
ir
krk
n r
a
rxaj
nar
a
a
a m
C
areaI
) ] [( ) ]1
11
0
0
![Page 37: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/37.jpg)
0
0
0
0
1
1
1
1
1
1 1
1
p ij
p roduction
area pj
n n
ij
p r
p
p
jp
j n
n
p p px
area xp
p ij
x
area xpj
n
p p p xp p ij
x
area xpj
n
p
w w
C
area
w w
j
j
p
m
jr
krk
n r
p
m
xp
j
j
xp xpj
j
xp
[ ( ) ( ) ] ( ) [( )
( ) ] { ( ) [ ( ) ]
p
j
j
p
m
jr
krk
n r
p
m
j
j
p
m
jr
krk
n r
p
m
xp
p ij
p roduction
area pj
n n
ij
p r
p
p
jp
j n
n
p ij
p roduction
area pj
n n
ij
p r
p
p
jp
j n
n
p p px
area xp
p ij
w w
C
areaI
w w
C
area
w
[ ( ) ( ) ]}
[ ( ) ( ) ] ( ) [( )
(
1
1
10
1
1
1
1
11
x
area xpj
npr
p
p
p mj
j
xp
ir
krk
n r
p
r
C
areaI
) ] [( ) ]
1
11
y ie ld
y ie ld
acres
acres
produc tion
produc tion
residua l
residua l
residua l
,
![Page 38: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/38.jpg)
Field plot Spatial filter: 3 selected eigenvectors
(6,5) 24.31
(5,6) 13.62
(6,6) 23.93
Imputation Imputation of turnipof turnip
productionproductionin 3 in 3
vandalizedvandalizedfield plotsfield plots
![Page 39: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/39.jpg)
Cross-validation of spatial filter for observed turnip data
![Page 40: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/40.jpg)
The accompanying table contains a test set of sixteen random samples (#17-32) used to evaluate three maps. The “Actual” column lists the measured values for the test locations identified by “Col, Row” coordinates. The difference between these values and those predicted by the three interpolation techniques form the residuals shown in parentheses. The “Average” column compares the whole field arithmetic mean of 23 (guess 23 everywhere) for each test location.
Kriging: best linear unbiased spatial interpolator (i.e.,
predictor)
![Page 41: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/41.jpg)
ArcGIS: Geostatistical Wizard
anisotropycheck
density ofGerman workers
![Page 42: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/42.jpg)
Cross-validation check of krigged values
This is one use ofthe missing spatial data
imputation methods.
![Page 43: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/43.jpg)
Unclipped krigged surface
krigged (mean response) surface prediction error surface
exponential semivariogram modelvalues increase with darkness of brown
extrapolation
![Page 44: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/44.jpg)
Clipped krigged surface
prediction error surface
krigged (mean response) surface
values increase with darkness of brown
![Page 45: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/45.jpg)
Detrended population density across China
anisotropycheck
![Page 46: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/46.jpg)
Cross-validation check of krigged values
This is one use ofthe missing spatial data
imputation methods.
![Page 47: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/47.jpg)
Unclipped krigged surface
krigged (mean response) surface prediction error surface
exponential semivariogram modelvalues increase with darkness of brown
extrapolation
![Page 48: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/48.jpg)
Clipped krigged surface
krigged (mean response) surface
prediction error surface
values increase with darkness of brown
![Page 49: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/49.jpg)
THEOREM 4
The maximum likelihood estimate for missing georeferenced values described by a spatial autoregressive model specification is equivalent to the best linear unbiased predictor kriging equation of geostatistics.
![Page 50: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/50.jpg)
Geographically weighted regression: GWR
Spatial filtering enables easier implementation of GWR, as well as proper assessment of its dfs
•Step #1: compute the eigenvectors of a geographic connectivity matrix, say C
•Step #2: compute all of the interactions terms XjEk for the P covariates times the K candidate eigenvectors (e.g., with MC > 0.25)
•Step #3: select from the total set, including the individual eigenvectors, with stepwise regression
![Page 51: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/51.jpg)
• Step #4: the geographically varying intercept term is given by:
• Step #5: the geographically varying covariate coefficient is given by factoring Xj out of its appropriate selected interaction terms:
K
1kEki,i ki,
bEaa
j
K
1kEXki,jjji, XbEbXb
ki,j
![Page 52: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/52.jpg)
A Puerto Rico DEM example
Mean elevation (Y) is a function of: standard deviation of elevation (X), eigenvectors E1-E18, and 18 interaction terms (XE)
Results
intercept: 1, E2, E5-E7, E9, E11-E13, E15, E18
slope: 1, E4, E6, E9, E10
R2 increases from 0.576 (with X only) to 0.911 (with geographically varying coefficients)
P(S-W) = 0.52 for the final model
![Page 53: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/53.jpg)
GWR-spatialfilter intercept(MC = 0.692)
GWR-spatialfilter slope(MC = 0.721)
![Page 54: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/54.jpg)
Spatial moving averages
Local smoothing of attribute values
where: wij is a spatial weights matrix yi is the attribute value for each areal unit n is the number of areal units
n...,2,1,in
1jij
n
1jiij
i
w
yw
μ
n...,2,1,in
1jij
n
1jiij
i
w
yw
μ
![Page 55: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/55.jpg)
![Page 56: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/56.jpg)
A summary: what have we learned during the 5 lectures?
Lecture #1• The nature of data and its information content.• What is spatial autocorrelation?• Visualizing spatial autocorrelation: Moran scatterplots,
semivariogram plots, and maps.• Defining and articulating spatial structure: topology and
distance perspectives; contagion and hierarchy concepts.
• Necessary concepts from multivariate statistics.• An example of the elusive negative spatial
autocorrelation.• Some comments about spatial sampling.• Implications about space-time data structure.
![Page 57: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/57.jpg)
Lecture #2• Multivariate grouping, and location-allocation
modeling.• Going from the global to the local: variability and
heterogeneity.• Impacts of spatial autocorrelation on histograms.• The LISA and Getis-Ord statistics.• Cluster analysis: multivariate analysis, cluster
detection, and spider diagrams.– An overview of geographic and space-time clusters.
• Regression diagnostics and geographic clusters
![Page 58: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/58.jpg)
Lecture #3• Autoregressive specifications and normal curve
theory (PROC NLIN).• Auto-binomial and auto-Poisson models: the
need for MCMC.• Relationships between spatial autoregressive
and geostatistical models• Spatial filtering specifications and linear and
generalized linear models (PROC GENMOD).• Autoregressive specifications and linear mixed
models (PROC MIXED).• Implications for space-time datasets (PROC
NLMIXED)
![Page 59: Lecture #5: MAPS WITH GAPS-- Small geographic area estimation, kriging, and kernel smoothing](https://reader035.vdocuments.us/reader035/viewer/2022062517/568138e7550346895da0995d/html5/thumbnails/59.jpg)
Lecture #4• Frequentist versus Bayesian perspectives.• Implementing random effects models in
GeoBUGS.• Spatially structured and unstructured random
effects: the CAR, the ICAR, and the spatial filter specifications
Lecture #5• The E-M algorithm• The spatial E-M algorithm• Kriging in ArcGIS• Approaches to map smoothing