overview: 2005 protocol comparison test
DESCRIPTION
Overview: 2005 Protocol Comparison Test. Brett Roper National Aquatic Ecologist, USDA Forest Service [email protected] (435) 755-3566. Analysis. To determine stream means and difference among streams proc mixed data=***; classes stream ; model BW = stream; - PowerPoint PPT PresentationTRANSCRIPT
Overview: 2005 Protocol Comparison Test
Brett RoperNational Aquatic Ecologist, USDA Forest Service
[email protected](435) 755-3566
Analysis
For Means, STD, CVproc glm data= ***;class stream;model BW = stream;run;
To decomposevarianceproc mixed data=***;classes stream;model BW =;
random stream;run;
To determine stream means and difference among streamsproc mixed data=***;classes stream ;model BW = stream;lsmeans stream /pdiff adjust=tukey; ods output diffs=ppp lsmeans=mmm; ods listing exclude diffs lsmeans;run; %include 'c:\BBRfile\stats\sasmacros\pdmix800.sas'; %pdmix800(ppp,mmm,alpha=0.1,sort=yes);run;
No analysis yet to determine significant differences among groups
Proc GLM statement Source DF Squares Mean Square F Value Pr > F
Model 11 254.5044526 23.1367684 148.70 <.0001 Error 24 3.7342700 0.1555946Corrected Total 35 258.2387225 R-Square Coeff Var Root MSE grad Mean 0.985539 10.57097 0.394455 3.731490
Proc Mixed statementCovariance Parameter Estimates Cov Parm Estimate
Stream 7.6604 Residual 0.1556
Output of the PDMIX statement for gradient from one group
Standard Letter Obs Stream Estimate Error Group 1 Myrtle 9.8156 0.2277 A 2 Whisky 7.1933 0.2277 B 3 Indian 6.0044 0.2277 C 4 Crawfish 5.2956 0.2277 C 5 WF Lick 3.5644 0.2277 D 6 Tinker 2.9444 0.2277 DE 7 Potamus 2.7622 0.2277 DE 8 Trail 1.9511 0.2277 EF 9 Big 1.3800 0.2277 F 10 Crane 1.3756 0.2277 F 11 Camus 1.3601 0.2277 F 12 Bridge 1.1311 0.2277 F
S tre a mC re w
0
5
1 0
1 5
2 0
2 5
3 0
S tre a m A S tre a m B S tre a m C
Str
eam
Att
rib
ute
C re w 1
C re w 2
C re w 3
0
5
1 0
1 5
2 0
2 5
3 0
S tre a m A S tre a m B S tre a m C
Str
eam
Att
rib
ute
G ro u p 1
G ro u p 2
Group 1 = Group 2 - 5
What should the results of an aquatic habitat protocol comparison look like?
What is a good attribute
• For categorization– Very little Rosgen large classes
• For status and trend– Minimum
• S:N of around 2, % stream ≈ 70% (I would prefer S:N of 4 and stream ≈ 80%
• Coefficient of variation ≈ 20%
Attributes
• Gradient• Bankfull Width• Wetted Width• Width-to-Depth• Sinuosity• Entrenchment• % Pool• Residual Pool Depth
• %Fines• Median Particle Size• D84
• Large Wood• Large wood volume
Gradient
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
Mean 3.33 3.73 3.84 3.83 3.60 3.12
RMSE 0.24 0.39 0.53 1.13 0.49 0.91
CV 7.13 10.57 13.91 29.62 13.68 28.86
Stream 7.02 7.66 8.82 5.97 6.95 4.27
Error 0.06 0.16 0.28 1.29 0.24 0.81
Total 7.08 7.82 9.10 7.25 7.19 5.09
S:N
124.42 49.23 30.99 4.64 28.65 5.27
%Observer 0.01 0.02 0.03 0.18 0.03 0.16
%Stream 0.99 0.98 0.97 0.82 0.97 0.84
Groups 9.00 6.00 6.00 3.00 6.00 5.00
0
2
4
6
8
10
12
G R P 1 G R P 2 G R P 3 G R P 4 G R P 5 G R P 6
Gra
die
nt
(%)
Myrtle
Crane
Gradient; Results Can be shared?
0
2
4
6
8
10
12
0 2 4 6 8 10 12
G rad ien t (G R P 1 )
Gra
die
nt
G R P 2 (99 .7 )
G R P 3 (98 .1 )
G R P 4 (93 .1 )
G R P 5 (99 .4 )
G R P 6 (94 .0 )
(R2)
Bankfull Width
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
Mean 4.57 4.01 6.79 5.92 5.27 5.89
RMSE 0.33 0.57 0.96 2.48 1.89 1.37
CV 7.26 14.23 14.12 41.93 35.86 23.21
Stream 6.37 6.57 22.46 18.95 8.86 11.87
Error 0.11 0.32 0.92 6.17 3.57 1.87
Total 6.48 6.89 23.38 25.12 12.43 13.74
S:N 58.06 20.23 24.45 3.07 2.48 6.36
%Observer 0.02 0.05 0.04 0.25 0.29 0.14
%Stream 0.98 0.95 0.96 0.75 0.71 0.86
Groups 6 4 7 2 3 4
02468
101214161820
Cam
as
Po
tom
as
Tra
il
Cra
ne
Cra
wfi
sh
Bri
dg
e
Big
Wh
isky
Ind
ian
My
rtle
Tin
ker
Str
ea
m W
Idth
(m
) truth
Group 1
Group 2
Group 3
Group 4
Group 5
Group 6
truth GRP 1 GRP 2 GRP 3 GRP 4 GRP 5 GRP 6truth 1.00 0.97 0.96 0.89 0.96 0.97 0.97Group 1 1.00 0.99 0.90 0.97 0.96 0.97Group 2 1.00 0.93 0.98 0.97 0.98Group 3 1.00 0.89 0.95 0.95Group 4 1.00 0.93 0.96Group 5 1.00 0.99Group 6 1.00
Correlations among all stream width groups and the truth (remember these are mean values compared to the truth)
Wetted Width
Group 2 Group 4 Group 5 Group 7 Group 8
Mean 2.94 3.36 3.18 3.01 3.07
RMSE 0.20 0.63 0.17 0.65 0.26
CV 6.68 18.83 5.41 21.67 8.48
Stream 5.88 5.36 5.00 7.54 6.13
Error 0.04 0.40 0.03 0.43 0.07
Total 5.92 5.76 5.03 7.96 6.20
S:N 152.49 13.43 168.79 17.69 90.56
%Observer 0.01 0.07 0.01 0.05 0.01
%Stream 0.99 0.93 0.99 0.95 0.99
Groups 8 5 7 6 5
Width to Depth
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
Mean 18.63 27.28 15.45 16.73 14.26 23.43
RMSE 3.97 7.89 2.94 5.22 4.30 6.22
CV 21.31 28.91 19.05 31.20 30.12 26.89
Stream 23.88 102.31 18.32 12.51 30.66 65.63
Error 15.90 62.18 8.66 27.24 18.45 38.73
Total 39.78 164.49 26.98 39.74 49.11 104.36
S:N 1.50 1.65 2.12 0.46 1.66 1.69
%Observer 0.40 0.38 0.32 0.69 0.38 .37
%Stream 0.60 0.62 0.68 0.31 0.62 .63
Groups 3 3 4 2 3 3
Width to Depth
5
10
15
20
25
30
G R P 1 G R P 2 G R P 3 G R P 4 G R P 5 G R P 6
Wid
th t
o D
ep
thB
igC
rawfish
Sinuosity
Group 1 Group 2 Group 3 Group 5
Mean 1.25 1.22 1.22 1.19
RMSE 0.11 0.11 0.04 0.06
CV 8.43 8.79 3.13 5.12
Stream 0.01 0.03 0.02 0.02
Error 0.01 0.01 0.00 0.00
Total 0.02 0.04 0.02 0.02
S:N 0.82 2.44 13.00 5.54
%Observer 0.55 0.29 0.07 0.15
%Stream 0.45 0.71 0.93 0.85
Groups 2.00 4.00 5.00 4.00
Entrenchment
Group 1 Group 2 Group 3 Group 4 Group 6
Mean 1.99 1.71 2.18 2.17 2.50
RMSE 0.59 0.48 0.95 1.06 1.22
CV 29.82 28.22 43.68 48.76 48.89
Stream 0.07 0.04 1.07 0.00 0.56
Error 0.33 0.23 0.91 1.03 1.48
Total 0.40 0.27 1.98 1.03 2.04
S:N 0.21 0.16 1.17 0.00 0.38
%Observer 0.82 0.86 0.46 1.00 0.73
%Stream 0.18 0.14 0.54 0.00 0.27
Groups 1.00 1.00 2.00 1.00 1.00
Entrenchment
1
2
3
4
5
6
7
8
9
G R P 1 G R P 2 G R P 3 G R P 4 G R P 6
En
tre
nc
hm
en
tB
igC
rawfish
Percent Pool
Group 1 Group 2 Group 3 Group 4 Group 5 Group 7 Group 8
Mean 29.44 21.52 9.99 17.70 10.28 22.57 24.35
RMSE 12.91 11.01 4.14 6.31 8.30 10.42 5.24
CV 43.85 51.18 41.45 35.66 80.73 46.20 21.54
Stream
237.49 227.36 12.34 201.18 109.55 122.13 394.26
Error
165.87 121.32 17.13 39.85 68.86 108.59 27.47
Total
403.36 348.68 29.47 241.03 178.41 230.72 421.73
S:N 1.43 1.87 0.72 5.05 1.59 1.12 14.35
%Observer 0.41 0.35 0.58 0.17 0.39 0.47 0.07
%Stream 0.59 0.65 0.42 0.83 0.61 0.53 0.93
Groups 2 4 2 3 2 2 6
0
10
20
30
40
50
60
70
80
90
G R P 1 G R P 2 G R P 3 G R P 4 G R P 5 G R P 7 G R P 8
% P
oo
lM
yrtleC
raneLets look at Percent Pool
Can data be shared? Some yes Some No.
0
20
40
60
80
0 20 40 60 80
% P o o l (G R P 1 )
% P
oo
l
G R P 2 (84 .2 )
G R P 3 (0 .0 )
G R P 4 (75 .1 )
G R P 5 (64 .8 )
G R P 7 (74 .9 )
G R P 8 (69 .6 )
(R2)
Residual Pool Depth
ResPoDep RPD RPD RESIDPD RPD AvgRPDm RPD
Group 1 Group 2 Group 3 Group 4 Group 5 Group 7 Group 8
Mean 0.21 0.25 0.19 0.37 0.07 0.33 0.33
RMSE 0.03 0.03 0.03 0.06 0.01 0.08 0.05
CV 12.72 11.40 15.11 16.80 12.39 24.60 13.94
Stream 0.01 0.01 0.01 0.01 0.00 0.01 0.01
Error 0.00 0.00 0.00 0.00 0.00 0.01 0.00
Total 0.01 0.01 0.01 0.02 0.00 0.02 0.01
S:N 7.43 10.13 8.65 3.47 17.04 2.06 4.93
%Observer 0.12 0.09 0.10 0.22 0.06 0.33 0.17
%Stream 0.88 0.91 0.90 0.78 0.94 0.67 0.83
Groups 5 4 5 4 6 4 6
Median Particle Size (54.4 mm)
Group 1 Group 2 Group 3 Group 5 Group 7
Mean 49.28 27.93 39.25 36.44 36.46
RMSE 13.80 14.20 18.19 24.29 22.98
CV 28.01 50.84 46.33 66.65 63.04
Stream 1144.90 728.99 807.29 584.22 601.60
Error 190.50 201.63 330.69 589.85 526.83
Total 1335.40 930.62 1137.98 1174.07 1128.43
S:N 6.01 3.62 2.44 0.99 1.14
%Observer 0.14 0.22 0.29 0.50 0.47
%Stream 0.86 0.78 0.71 0.50 0.53
Groups 4 4 4 3 3
D84 (155.1 mm)
Group 1 Group 2 Group 3 Group 5 Group 7
Mean 143.81 192.85 158.28 196.05 118.12
RMSE 44.81 86.35 35.27 64.14 54.28
CV 31.16 44.77 22.28 32.72 45.95
Stream 11733.00 38644.00 12075.00 28912.00 5519.79
Error 2008.00 7455.55 1243.92 4114.43 2942.16
Total 13741.00 46099.55 13318.92 33026.43 8461.95
S:N 5.84 5.18 9.71 7.03 1.88
%Observer 0.15 0.16 0.09 0.12 0.35
%Stream 0.85 0.84 0.91 0.88 0.65
Groups 3.00 3.00 5.00 4.00 3.00
% Fines
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7
Mean 18.83 29.90 10.77 20.15 19.52 22.34 29.15
RMSE 4.88 8.22 9.55 7.59 6.45 13.45 16.93
CV 25.92 27.49 88.67 37.64 33.03 60.17 58.97
Stream
170.39 106.94 44.36 124.53 149.18 63.03 39.11
Error 23.83 67.54 91.28 57.53 41.56 183.18 284.35
Total
194.22 174.48 135.64 182.06 190.74 246.21 323.46
S:N 7.15 1.58 0.49 2.16 3.59 0.34 0.14
%Observer 0.12 0.39 0.67 0.32 0.22 0.74 0.88
%Stream 0.88 0.61 0.33 0.68 0.78 0.26 0.12
Groups 3 4 1 2 4 2 1
Bank Stability
stab2pct 100-unstabBNK 100-pcterosion PctStab
Group 1 Group 2 Group 4 Group 7
Mean 78.25 79.56 95.66 96.80
RMSE 11.12 9.95 1.73 7.10
CV 14.21 12.51 1.81 7.33
Stream 115.65 462.86 26.85 5.17
Error 123.61 99.04 2.99 50.41
Total 239.26 561.90 29.85 55.57
S:N 0.94 4.67 8.97 0.10
%Observer 0.52 0.18 0.10 0.91
%Stream 0.48 0.82 0.90 0.09
Groups 2 4 4 1
Large Wood
>=3m_Cnt100m LWD3x30 LWD lwdpiece1 LWD_L LWD_L
Group 1 Group 2 Group 3 Group 4 Group 7 Group 8
Mean 18.60 26.48 14.72 21.18 4.63 2.50
RMSE 4.27 12.86 3.35 6.52 4.92 1.35
CV 22.97 48.57 22.78 30.79 106.15 54.16
Stream 248.18 334.94 72.17 271.28 35.14 5.78
Error 18.25 165.43 11.25 42.61 24.19 1.82
Total 266.43 500.37 83.42 313.89 59.33 7.60
S:N 13.60 2.02 6.41 6.37 1.45 3.18
%Observer 0.07 0.33 0.13 0.14 0.41 0.24
%Stream 0.93 0.67 0.87 0.86 0.59 0.76
Groups 6 4 6 4 2 5
Large Wood (volume)LWD_Cat1-
2_allLen_Vol100mLWD_Cat1_allLen_
Vol100m VLWD lwdvol1
Group 1 Group 2 Group 3 Group 4
Mean 19.96 9.60 12.59 12.21
RMSE 6.89 3.90 8.24 7.06
CV 34.50 40.58 65.44 57.80
Stream 158.24 38.87 70.15 31.20
Error 47.67 15.18 67.90 51.07
Total 205.91 54.05 138.05 82.28
S:N 3.32 2.56 1.03 0.61
%Observer 0.23 0.28 0.49 0.62
%Stream 0.77 0.72 0.51 0.38
Groups 4 4 3 1
How I summarized
• A: S:N >9, stream variability 90%, CV < 20%
• B: S:N >4, stream variability 80%, CV < 20%
• C: S:N > 2, Stream variability 70% or CV around 20%
• D: S:N close to 2, stream variability more than 50%, or CV around 20%.
• F: Anything lower.
GRP 1
GRP2
GRP3
GRP 4
GRP 5
GRP 6
GRP 7
GRP 8
Gradient A(1) A A B A B
BF Width A(1) A A C C B
Wetted Width A A A(1) A A
WD D F C(1) F F F
Sinuosity D C A(1) B
Entrenchment F F F(1) F F
% Pool F F F B F D A(1)
Res Pool Depth A A A B A(1) C B
D50 B(1) C C F F
D84 B B A(1) B C
Fines B(1) F F C B F F
Bank Stability D B A(1) F
LWD # A(1) C B B D C
LWD Volume B(1) C F F
Preliminary Observations• Some attributes everyone does passable at:
gradient, bankfull width, wetted width, residual pool depth.– For these attributes it is likely that cross walks can be
determined not only with e ach other but with the truth. • Only one attribute that nobody does well at;
entrenchment.– Although nobody does well with this one it may not
matter since it will only be used for classification; but if that is true why not as a group agree on an AML.
• For remaining attributes – width to depth, sinousity, % pool, D50D84, fines, bank stability, and large wood volume – some groups have better (more consistent within the group) protocols than others.
Some thoughts on why certain attributes were done better when
there was variation in the protocols
• Ratio things (Sinuosity, width-to-depth, and entrenchment) were done better with laser level.
• Extensive training resulted in more consistent sediment (fines and particle distributions, and large wood (counts and volume).
• Pools were best done with a fixed length.
How do we decide what to measure
Bankfull width vs wetted width
• Bankfull width has hydrologic meaning, can be measured if the stream is dry. Not affect by season.
• Wetted with more consistently measured, is a measure of summer aquatic habitat (nice to know a stream is dry)
What’s next?
• LiDAR
• Determine what steps should be taken to standardize protocols.– Continue efforts to develop crosswalks.
– Should data quality control recommendations be made.
– Seek consensus on the best protocol(s) to use.
• Determine which attributes provide useful data.
• Proposal was submitted for BPA funding for follow up work.
• Publication of the John Day basin protocol test results.
Questions?