1

In Situ Characterization of Soil Clay Content with Visible Near-Infrared 1 Diffuse Reflectance Spectroscopy 2

3 Travis H. Waiser, Cristine L.S. Morgan*, David J. Brown, and C. Tom Hallmark 4

5

T.H. Waiser, C.L.S. Morgan, and C.T. Hallmark Dep. of Soil and Crop Sciences, Texas 6 A&M University, 2474 TAMU, College Station, TX 77843-2474. D. J. Brown, Dep. of 7 Land Resources and Environmental Sciences, Montana State University, PO Box 8 173120, Bozeman, MT 59717-3120. Received 06 June 2006. *Corresponding author 9 (cmorgan@ag.tamu.edu). 10

KEYWORDS 11 Diffuse reflectance spectroscopy; in situ; VNIR; PLS regression; clay content; water 12 potential 13 14

ACKNOWLEDGEMENTS 15

The authors would like to thank the Texas Water Resource Institute and Texas 16

Agricultural Experiment Station for their financial support. The extension agents from 17

Erath and Comanche County, Texas were instrumental in putting us in contact with 18

landowners of the study sites. We appreciate the work the Texas Agricultural 19

Experiment Station Soil Characterization Laboratory did in running some of the analyses 20

especially Dr. Richard Drees and Ms. Donna Prochaska. 21

2

In Situ Characterization of Soil Clay Content with Visible Near-Infrared 22

Diffuse Reflectance Spectroscopy 23

ABSTRACT 24

Visible and near-infrared (VNIR, 400-2500 nm) diffuse reflectance spectroscopy 25

(DRS) is a rapid, proximal-sensing method that has proven useful in quantifying 26

constituents of dried and ground soil samples. However, very little is known about how 27

DRS performs in a field setting on soils scanned in-situ. The overall goal of this research 28

was to evaluate the feasibility of VNIR-DRS for in situ quantification of clay content of 29

soil from a variety of parent materials. Seventy-two soil cores were obtained from six 30

fields in Erath and Comanche Counties, Texas. Each soil core was scanned with a visible 31

near-infrared spectrometer, with a spectral range of 350-2500 nm, at four different 32

combinations of moisture content and pre-treatment: field-moist in situ, air-dried in situ, 33

field-moist smeared in situ, and air-dried ground. The VNIR spectra were used to predict 34

total and fine clay content of the soil using partial least squares (PLS) regression. The 35

PLS model was validated with 30% of the original soil cores that were randomly selected 36

and not used in the calibration model. The validation clay predictions had a root mean 37

squared deviation (RMSD) of 61 g kg-1 and 41 g kg-1 dry soil for the field-moist and air-38

dried in situ cores, respectively. The RMSD of the air-dry ground samples was between 39

the two in situ RMSDs and comparable to values in the literature. Smearing the samples 40

increased the field-moist in situ RMSD to 74 g kg-1. Whole-field holdout validation 41

results showed that soils from all parent materials need to be represented in the 42

calibration samples for maximum predictability. In summary, DRS is an acceptable 43

3

technique for rapidly measuring soil clay content in situ at variable water contents and 44

parent materials. 45

46

Abbreviations: DRS, diffuse reflectance spectroscopy; N, the total number of samples in 47

the validation data; PLS, partial least squares; RMSD, root mean squared deviation; RPD, 48

ratio of standard deviation to RMSD; SD, standard deviation; VNIR, visible and near-49

infrared; Ypred, predicted values of the validation set using the PLS model; and Ymeas, 50

laboratory measurements of the validation set. 51

52

INTRODUCTION 53

The resolution of a 1:24,000 scale soil survey map is too coarse to capture soil 54

variability within soil mapping units and transitions between soil mapping units. 55

However, soil maps that capture soil variability at a 10-50-m scale are necessary for 56

resource management such as, precision agriculture, non-point source pollution 57

modeling, and resource use planning (Pachepsky et al., 2001; Ellert et al., 2002). 58

Researchers have used external landscape features such as terrain modeling, soil surface 59

reflectance, vegetative cover, and rapid surface sensing techniques like electromagnetic 60

devices to capture the spatial variability of soil properties across landscapes followed by 61

collecting soil cores and laboratory analyses of those cores for model calibration and 62

validation (Moore et al., 1993; Sudduth et al., 1997; Zhu et al., 1997, McBratney et al., 63

2003, Zhu et al., 2004). Each of these methods has the advantage of creating a high-64

resolution map of horizontal soil variability, but is limited in the ability to get high-65

resolution, vertical soil information. Soil profile (vertical) information is limited by the 66

4

capacity to collect and analyze soil cores. Model calibration and validation are currently 67

the weak points in digital soil mapping because collecting and analyzing soil cores to 68

capture vertical variability is time consuming and cost prohibitive. For example, current 69

laboratory soil analyses can take several weeks to months for each soil pedon and cost 70

upwards of $2,000. The lack of soil sensors that can rapidly quantify soil profile 71

information demonstrates the need to find new methods for soil mapping. 72

A proximal soil sensor that might be useful for rapidly quantifying soil profile 73

information in the field is visible and near-infrared diffuse reflectance spectroscopy 74

(VNIR-DRS). Recent research has shown the effectiveness of VNIR-DRS in providing a 75

non-destructive rapid prediction of soil physical, chemical, and biological properties of 76

air-dried ground soil samples in the laboratory (Shepherd and Walsh, 2002). In the VNIR 77

spectrum, absorptions by water bonds associated with clay content and other bonding 78

associated with clay type provide the opportunity to use VNIR for quantifying clay 79

information in soil. Previous research on air-dried, ground soil samples has shown VNIR 80

predictions of soil clay content with r2-values ranging from 0.56 to 0.91 and RMSD’s 81

ranging from 23 to 11 g kg-1 (Ben-Dor and Banin, 1995; Janik et al., 1998, Shepherd and 82

Walsh, 2002; Islam et al., 2003; Sorensen and Dalsgaard, 2005; Brown et al., 2006). Air-83

drying and grinding the soil sample prior to scanning is believed to improve prediction 84

accuracy of DRS-based models. Air-drying the sample reduces the intensity of bands that 85

are related to water so signals associated with other soil properties are not masked or 86

hidden. Additionally, particle size affects accuracy of the spectral scan. Smaller particles 87

increase reflectance scatter, which reduces the absorption peak height (Workman and 88

5

Shenk, 2004). Soil homogenization, accomplished while grinding, may also be beneficial 89

when scanning with VNIR-DRS. 90

The determination of clay content by VNIR measurements is possible in part due 91

to the distinctive spectral signatures of common clay minerals. The spectral signatures 92

include overtones and combination bands that occur from chemical bonds within soil 93

minerals (Hunt and Salisbury, 1970; Clark, 1999). For example, kaolinite, smectite, and 94

muscovite occur in the clay fraction of soils and have distinct spectral absorption 95

features. Kaolinite [Al2Si2O5(OH)4] is an aluminum silicate with two very strong 96

hydroxyl bands near 1400 nm, OH stretch, and 2200 nm, OH stretch Al-OH bend 97

combination (Hunt and Salisbury, 1970; Clark et al., 1990). Smectite (montmorillonite) 98

[M+0.3Al2.7Si3.3O10(0H)2; M=Ca2+, Mg2+, K+, etc.] has two very strong water bands around 99

1400 and 1900 nm from molecular water and another at 2200 nm (Hunt and Salisbury, 100

1970; Goetz et al., 2001). Muscovite [KAl3Si3O10 (OH)2] displays hydroxyl bands at 101

1400 nm and between 2200 to 2600 nm (Hunt and Salisbury, 1970). Brown et al. (2006) 102

built a VNIR calibration for clay minerals and was able to predict kaolinite and 103

montmorillonite within one ordinal unit from X-ray diffraction data 96% and 88% of the 104

time, respectively. Goetz et al. (2001) investigated using NIR spectroscopy to measure 105

smectite content in selected Colorado soils (r=0.83). Because the literature shows VNIR 106

absorbance associated with clay mineralogy, one may expect that the wavebands 107

significant to predicting soil clay content to be similar to mineralogy wavebands. 108

There have been few reported studies of in situ VNIR-DRS soil characterization. 109

Sudduth and Hummel (1993) tested a portable near-infrared (NIR) spectrometer to 110

measure soil properties in situ on the fly. Their research concluded that VNIR-DRS 111

6

could be used to accurately predict cation exchange capacity, soil organic matter, and 112

moisture content in the laboratory, but the technique was not sufficiently accurate at 113

quantifying soil organic matter in a furrow. The poor prediction with this in situ 114

technique was attributed to movement of the soil past the spectrometer. Hence low 115

prediction accuracy for in situ scanning may be avoided by holding the spectrometer 116

scanning device more stable, and by holding the soil stationary while scanning. 117

In the field, DRS measurements may not work as well as laboratory DRS 118

measurements because of potential problems that have not been addressed by research on 119

air-dried ground soils. The potential problems that may reduce the prediction accuracy of 120

in situ DRS analysis include, varying amounts of soil moisture, varying particle size and 121

aggregation, smearing of soil surfaces, small scale heterogeneity (mottles, accumulations, 122

and redox features), and geographic regionality of calibration models. Sudduth and 123

Hummel (1993) indicated that local buried residue and roughness of the soil surface 124

caused a reduction in their prediction accuracies. Smearing of the soil causes reflectance 125

properties to change, possibly altering prediction accuracies. Geographic regionality, 126

such as changes in soil parent material, may affect prediction accuracies when using 127

VNIR-DRS (Ben-Dor and Banin, 1995; Dalal and Henry, 1986; Sudduth and Hummel, 128

1996). 129

The overall goal of our study is to evaluate the feasibility of VNIR-DRS for in 130

situ quantification of clay content for soil profiles from a variety of parent materials. 131

Specifically, this research addresses the following objectives: 1) evaluate the precision of 132

350 to 2500 nm (VNIR region) soil reflectance measurements in quantifying soil clay 133

content of in situ soils at field-moist and air-dry water content; 2) quantify any change in 134

7

measurement error or prediction accuracy for in situ, field-moist soil with a smeared 135

surface; and 3) quantify any change in prediction accuracies with consideration to field-136

specific calibrations. 137

This research will help evaluate the feasibility of using a VNIR spectrometer in 138

the field as a proximal sensor to aid soil mapping activities. For VNIR-DRS to be 139

considered useful in the field, it must be an improvement on current field techniques. 140

The VNIR DRS methods should be reliable, meaning that the errors should be 141

comparable to current field techniques with similar data acquisition speeds, and VNIR-142

DRS should be rapid, meaning that it should quantify clay much faster than any current 143

techniques. Although time and money are required to calibrate the VNIR instrument, 144

once calibrated, VNIR-DRS can scan whole soil pedons more rapidly than hand texturing 145

in the field or laboratory particle-size methods (i.e. pipette and hydrometer), allowing for 146

more soil core information to be quantified in one day. Therefore VNIR-DRS is 147

considered adequate if VNIR-DRS can quantify clay content accurate enough for 148

precision management, watershed modeling needs, or other applications where soil 149

information is need. In general, most precision management strategies and watershed 150

models require a stronger emphasis on the change in soil texture in space (vertical and 151

horizontal) than on lab-quality soil data (Morgan et al., 2003). Quickly collecting higher 152

spatial resolution data is where VNIR-DRS has an advantage over current field 153

techniques. 154

155

MATERIALS AND METHODS 156

Soil Coring 157

8

Seventy-two soil cores were collected from six fields in Erath County (fields 1, 2, 158

3, and 6) and Comanche County (fields 4 and 5), Texas in May 2004 using a Giddings 159

hydraulic soil sampler (Windsor, CO) attached to a truck. Each soil core was collected to 160

a maximum depth of 105 cm or to the depth of a coring-restrictive horizon. The soil 161

cores were contained in a 6.0-cm diameter plastic sleeve that was capped on both ends. 162

The soil cores that were collected were chosen to represent the large variability in soil 163

properties over the two counties. For example, 21 soil series were mapped by the Natural 164

Resources Conservation Service in these fields (Table 1), and the parent material of these 165

soils included alluvium, sandstone, shale, and limestone. The large variability was 166

selected to ensure that the VNIR prediction models would not be mineralogy specific. 167

Additionally, though many cores represented the same soil series, variability within each 168

soil series because of 1) differences of morphology within a series, 2) inclusions of other 169

soils within each mapping unit, and 3) varying landscape positions all lead to 170

representation of high morphological variability within each mapped series. 171

VNIR-DRS Scanning 172

The soil cores were scanned using an ASD “FieldSpec® Pro FR” VNIR 173

spectroradiometer (Analytical Spectral Devices, Boulder, CO) with a spectral range of 174

350-2500 nm, 2-nm sampling resolution and spectral resolution of 3 nm at 700 nm and 175

10 nm at 1400 and 2100 nm. The spectroradiometer was equipped with a contact probe. 176

The contact probe has a viewing area defined by a 2-cm diameter circle and its own light 177

source. A Spectralon® panel with 99% reflectance was used to optimize the 178

spectrometer each day; the same panel was used as a white reference before scanning 179

each core. The 72 soil cores were prepared for the first, moist in situ, scan by slicing the 180

9

plastic sleeve lengthwise with a utility knife, and then halving the soil core lengthwise 181

(surface to subsoil) with a piano wire. One-half of the core was smeared using a stainless 182

steel spatula to simulate possible smearing by a soil probe and the other half was left 183

unsmeared. The smearing test was performed to quantify any reduction in accuracy that 184

might occur if a soil probe, pushed into the ground, smeared the sides of the hole. In this 185

laboratory exercise, smearing moist, high clay content soil required multiple passes with 186

the spatula. Therefore, this test was a worst-case scenario, as the soil was smeared to its 187

maximum capacity. A wire grid was used to identify two 3-cm wide columns and 3-cm 188

wide rows within each core half (Fig. 1). Each row within each column was scanned 189

twice with the FieldSpec® Pro FR with a 90º rotation of the contact probe between scans. 190

Both halves of each core, smeared and unsmeared, were scanned at field-moist water 191

content. 192

Water potential was measured at each soil horizon from each core, with a 193

maximum of six samples per core, using a SC-10 thermocouple psychrometer (Decagon, 194

Pullman, WA) (Rawlins and Campbell, 1986). The water potential samples were 195

collected immediately after scanning each core to minimize drying. Water potential 196

measurements were made to confirm variability of water content and to quantify the 197

range of soil moisture among the soils scanned. The cores were then placed in a drier at 198

44°C for two days for air-dried in situ scans. Before taking air-dried scans, the cores 199

were removed from the oven and left on the countertop in the laboratory to equilibrate to 200

room temperature. The soil core half that was scanned in unsmeared condition was then 201

rescanned, in situ, at air-dry moisture content. 202

10

Each row of the unsmeared soil core was ground and passed through a 2-mm 203

sieve, and re-scanned as air-dried, ground soil. The air-dried, ground soils were scanned 204

from below with an ASD mug lamp connected to the FieldSpec® Pro FR. A Spectralon® 205

99% reflectance panel was used to optimize and white reference the spectrometer. 206

Approximately 28 g of ground soil was placed into a Duraplan® borosilicate optical-207

glass Petri dish. Each sample was scanned twice with a 90º rotation between scans. 208

Pretreatment of Data 209

The reflectance spectra were spliced where the three detectors overlapped using 210

an empirical algorithm. Reflectance for the 0º and 90º scans were averaged (mean). 211

Scans on the same row, column 1 and 2, were also averaged, creating a mean of four 212

scans for each measured value of clay content. Smoothed reflectance and 1st and 2nd 213

derivatives were extracted on 10-nm intervals (360-2490 nm) from a weighted cubic 214

smoothing spline fit to the mean reflectance data using the R “smooth spline” function (R 215

Development Core Team, 2004), following the methods of Brown et al. (2006). 216

Laboratory Analysis 217

Although every 3 cm of the soil core was scanned, only selected rows were used 218

for laboratory particle size analysis. Only the VNIR scans from those rows were 219

subsequently used in the VNIR models. Whole rows of soil where combined for 220

laboratory analysis; hence, each row of soil had a total of four scans to represent its 221

spectral properties (Fig. 1). Rows were chosen to represent the primary horizons within a 222

soil core. The entire row from each half of the core was used for particle size analysis. 223

Particle size distribution was determined in the laboratory using the pipette method with 224

an error of ±1% clay (Steele and Bradfield, 1934; Kilmer and Alexander, 1949; Gee and 225

11

Or, 2002). Fine clays were measured using centrifugation, USDA NRCS method 3A1b 226

(USDA, 1996). Eighteen samples were selected for determination of clay mineralogy. 227

Clay mineralogy by X-ray diffraction was completed following techniques described in 228

Hallmark et al. (1986); no pretreatment was done. The mineralogy technique used was to 229

determine the absence or presence of clay minerals existing in quantities of greater than 5 230

% by volume of clay fraction. 231

232

Model Calibration and Validation 233

Four types of clay prediction models were built to help determine how in situ 234

scans and variable water content affect prediction accuracies for clay content. Air-dried 235

ground, air-dried in situ, field-moist in situ, and smeared field-moist in situ scans were 236

the four soil preparation types used in the models. The clay content models were 237

produced using 70% of the soil cores randomly chosen as the calibration samples. Scans 238

from an individual soil core were not split between validation and calibration datasets. 239

Entire cores were selected for validation or calibration to maintain independence between 240

the calibration and validation data (Brown et al., 2005). For each of the four pretreatment 241

scans (air-dried ground, air-dried in situ, field-moist in situ, and smeared field-moist in 242

situ), three prediction models for clay content were built using soil reflectance, 1st 243

derivative of reflectance and 2nd derivative of reflectance. The calibration models were 244

built using segmented cross validation PLS method in Unscrambler 9.0 (CAMO Tech, 245

Woodbridge, NJ). The segments for cross validation are randomly chosen and represent 246

four percent of the calibration dataset. The remaining 30% of the cores were used to 247

validate the models. 248

12

Model calibration and validation sets were also completed on whole-field 249

holdouts of moist in-situ scans, as a means to compare to the findings of Brown et al. 250

(2005). Whole-field holdouts were achieved by calibrating a model using PLS with five 251

of the six fields. The sixth field was held out as the validation samples. Six models were 252

created so all six fields were represented as a validation set. 253

The significant wavelengths in each model were plotted to help determine what 254

portions of the spectra were important for clay content predictions. Significant 255

wavelengths were chosen by Unscrambler 9.0 by comparing Student t-values of the PLS 256

regression coefficients to a p-value of 0.05. Negative clay content predictions were 257

changed to zero clay content before comparison of measured to predicted clay. 258

Measured versus predicted values of the validation samples were compared using 259

simple regression. The coefficient of determination (r2), root mean squared deviation 260

(RMSD), ratio of standard deviation (SD) to RMSD (RPD) and bias were calculated to 261

compare the accuracy of different PLS models. Statistical formulas to calculate RMSD 262

and bias follow Gauch et al. (2003), Brown et al. (2005) and RPD follows Chang et al. 263

(2005): 264

( )∑ −=n

2measpred N/YYRMSD , [1] 265

RPD = SD/RMSD, and [2] 266

( )∑ −=n

measpred ;N/YYBias [3] 267

where Ypred are predicted values of the validation set using the PLS model, and Ymeas are 268

laboratory measurements of the validation set, and N is the total number of samples in the 269

validation data. 270

13

RESULTS AND DISCUSSION 271

Sample Descriptions 272

From the 72 soil cores, 270 soil samples were analyzed for clay content and water 273

potential. Of these samples, 188 were used in the calibration model. The other 82 274

samples were used to validate the PLS model. Though selected randomly, the calibration 275

and validation data sets were similar (Table 2). Clay content range of the 270 soil 276

samples was 12 g kg-1 to 578 g kg-1 dry soil. The mean and median for the calibration 277

and validation data were similar, indicating that the samples were evenly split above and 278

below the mean, and that the prediction model produced should not be skewed. The 279

minimum water potential measured, -5.8 MPa, in the calibration samples was from a 280

tilled surface (0-3 cm) of a loamy-textured soil. The range of water potential confirms 281

that in situ moist scans involved a large range of water contents that may be found in 282

most field situations. The maximum water potential, 0 MPa, occurred in deep horizons in 283

several of the irrigated fields. The samples represented a range of clay mineralogy. The 284

minerals found in the clay fractions were: smectite, kaolinite, mica, quartz, calcite, and 285

feldspar (Table 3). The soils in field 1 were notably high in clay content and were dark 286

colored Vertisols; field 3 soils were low in clay content. 287

Field-moist vs. Air-dried Validation 288

The 1st derivative PLS model performed better than the reflectance and 2nd 289

derivative PLS models in the field-moist in situ scans. The r2 value for the 1st derivative 290

model was 0.83 compared to 0.71 and 0.58 for the reflectance and 2nd derivative models, 291

respectively. Because the 1st derivative PLS model performed the best for field-moist in 292

situ, and the 1st derivative worked the best in other VNIR work (Reeves et al., 1999; 293

14

Reeves and McCarty, 2001; Brown et al., 2006), the remaining PLS models reported in 294

this paper all use the 1st derivative of the VNIR spectra between 350 and 2500 nm. The 295

prediction accuracy of three of the four models used to predict clay content were very 296

similar, and the fourth, field-moist in situ smeared, had the largest prediction error and 297

scatter about the validation regression. Table 4 summarizes the prediction accuracies 298

between the four VNIR models. The r2 value of the dried-ground samples were similar to 299

the literature, while the RMSD value of this work was smaller than those cited (Ben-Dor 300

and Banin, 1995; Janik et al., 1998, Shepherd and Walsh, 2002; Islam et al., 2003; 301

Sorensen and Dalsgaard, 2005; Brown et al., 2006). The air-dried ground model was 302

expected to produce the most accurate prediction model because the sample was reduced 303

to a uniform size, moisture was uniform, and grinding homogenized the soil, but the air-304

dried in situ model did slightly better (Fig. 2). There are two possible explanations for 305

why the air-dried ground model did not perform as well as the air-dried in situ model. 306

One explanation could be random error. The second possible explanation is that in situ 307

soil has a higher bulk density than dried ground soil; and therefore, the in situ soil may 308

have a stronger reflectance signal. The air-dried ground and in situ moist models had a 309

similar number of wavelengths with significant (p-value ≤ 0.05) regression coefficients, 310

83 and 78 significant wavelengths, respectively (Fig. 3). 311

The air-dried ground prediction had the most bias (-16.3), compared to the other 312

predictions. The large bias associated with the air-dried ground prediction was attributed 313

to 60% of the samples being under predicted in clay content and large discrepancies 314

between measured and predicted clay content of soils greater than 450 g kg-1 clay. 315

Because the air-dried in situ model slightly outperformed the air-dried ground model 316

15

using the 1st derivative of VNIR reflectance spectra, we conclude natural soil 317

heterogeneity, as a result of clay films, translocations, redox features etc…, and 318

variability associated with aggregation had little effect on model predictions. Gaffey 319

(1986) found that aggregate size had little effect on prediction accuracy, but Gaffey was 320

looking at carbonate sizes. His results showed a difference in reflectance, but the number 321

of bands, band positions and width, and relative band intensities were unchanged. 322

Another significant finding was that the clay content model using the field-moist 323

in situ model had slightly less prediction accuracy than the air-dried in situ model. 324

However, the field-moist in situ model performed just as well when compared to the air-325

dried ground model (Fig. 2). These results indicate that the amount of water in the soil 326

sample did not change the prediction accuracy compared to air-dried laboratory 327

situations. These results show that soil water may reduce accuracy somewhat (increased 328

RMSD by 20 g kg-1), but this reduction in accuracy was no more than air-drying, 329

grinding, and scanning the soil through a borosilicate Petri dish (Fig. 2). Chang et al. 330

(2005) showed that r2-values decreased slightly from 0.79 to 0.76 from an air-dried soil to 331

a moist soil, respectively, which agrees with other findings. The field-moist in situ 332

models used 102 significant wavelengths in determining clay content (Fig. 3). Figure 3 333

shows the 32 significant wavelengths that were common in all three models. 334

The same set of VNIR models was created to predict fine clay. Fine clay models 335

have lower RMSD values than total clay models but the RPD values are similar to the 336

total clay models (Table 4). The similar RPD values indicate that the smaller RMSD 337

values for fine clay models were due to smaller standard deviations (smaller range); so 338

the total clay and fine clay models performed similarly. VNIR-DRS has proven 339

16

effective in identifying smectitic clays (Brown et al., 2006) and studies suggest that 340

Vertisol fine clays are comprised primarily of smectites (Anderson et al., 1972; Yerima 341

et al., 1989). Therefore, our ability to predict fine clay in this study might be due to the 342

mineral signature of this size fraction. 343

Field-moist Smeared Validation 344

Smearing of the field-moist in situ cores reduced the accuracy of the prediction 345

model (Fig. 2). The decrease in the prediction accuracy of the smeared cores was 346

probably due to the change in reflectance properties of the soil. Smearing causes the soil 347

surface to become shiny, which changes the reflectance from diffuse to diffuse specular 348

reflectance. Specular reflectance can mask diffuse absorptions and otherwise confuse the 349

analysis of reflectance spectra (Coates, 1998). When comparing the significant 350

wavelengths between the field-moist in situ and field-moist in situ smeared cores, a 351

number of wavelengths found in the visible portion of the spectra are absent from the 352

smeared core (Fig. 4). The smeared core model had 70 significant wavelengths and the 353

unsmeared core model had 102, with 50 of these wavelengths being common in both 354

models. The absence of these wavelengths may be the source of loss in prediction 355

accuracy. In a field situation, the loss of prediction accuracy due to smearing might be 356

less because it took multiple passes with the knife to smear many of the soil cores. 357

Whole-field Holdout Validation 358

As expected (Brown et al., 2005), whole-field cross-validation yielded degraded 359

predictions relative to random cross-validation. For all six fields, whole field validation 360

statistics (Table 5, RMSD and RPD) were worse than for the randomly selected cross-361

validation results (Table 4). The RPD statistics are particularly telling, with fields 1 and 362

17

3 having RPD values < 1 (no predictive value), fields 4-6 having RPD values < 1.5 (little 363

predictive value), and only field 2 yielding acceptable results (RPD = 1.92). 364

Both field 1 and 3 contained soil at the extremes for clay content. Field 1 365

consisted of high clay soils (median 46 % clay) that were dark in color through the entire 366

length of the soil core—high clay contents that were not represented in the calibration 367

soils. Not surprisingly, the prediction bias for this field was -128.1 g kg-1 (Table 5) 368

indicating a significant underprediction of these high clay contents. Conversely, field 3 369

contained relatively low clay soils (median 3 % clay) not found in the other five fields. 370

Regional clay calibrations will require a far larger and more diverse soil-spectral library 371

than that presented in this study. 372

DRS as a Proximal Sensor for Clay Content 373

Figure 5 illustrates the performance of VNIR-DRS used as a proximal soil sensor 374

for in-situ soils at field soil moisture. Three very different soil profiles are shown. The 375

first is a Vertisol from field 1. This soil is very high in clay and somewhat uniform in 376

clay content with depth. The VNIR-DRS prediction shows the same uniformity, with 10 377

% clay under prediction (Fig. 5a.). Nonetheless, the prediction does put the profile in the 378

correct clay texture category. The second soil, Fig. 5b., is an Alfisol, with a sandy 379

surface and an E horizon. In the case of the Alfisol, VNIR-DRS predictions show the 380

continuous change of clay content, the A, E, and Bt horizons are clearly seen. The third 381

soil is a Mollisol with secondary carbonate concentrations in the B horizon (Fig. 5c). The 382

VNIR-DRS prediction of this core was encouraging, because the secondary carbonates 383

did not appear add random scatter to the prediction. The concentrations may have biased 384

the clay content prediction toward the bottom of the profile. In summary, though the 385

18

VNIR-DRS predictions do no perfectly agree with the particle size analysis in these three 386

cores, the predictions do provide a rapid, continuous measurement of at least relative soil 387

clay content fluctuations with depth. 388

CONCLUSIONS 389

In this study of 72 soil cores from Central Texas, we found a strong relationship 390

between VNIR reflectance and clay content for air-dried in situ (RMSD = 41 g kg-1), 391

field-moist in situ (RMSD = 61 g kg-1), and air-dried ground (RMSD = 62 g kg-1) scans. 392

Visible near-infrared DRS was capable of predicting soil clay content in-situ at varying 393

water contents. There was a decrease in predictive accuracy for smeared field-moist in 394

situ (RMSD = 74 g kg-1) scans. However, we obtained worse results for ground and 395

sieved samples than for the non-smeared in situ scans—suggesting that soil aggregates 396

and natural heterogeneity did not degrade in situ predictions. Variable water contents did 397

reduce predictive accuracy as evidenced by a comparison of air-dried in situ and field-398

moist in situ results, but the field-moist in situ were still slightly better than those 399

obtained from scans of air-dried, ground-and-sieved samples. Validation RPD statistics 400

were similar for fine clay and total clay predictions. 401

These results indicate that VNIR-DRS may be useful as a proximal soil sensor in 402

field conditions. In particular, we envision a VNIR-DRS sensor placed in a soil probe for 403

in situ soil profile characterization. Such a probe would give soil scientists the ability to 404

acquire soil profile data (e.g. clay content) at greater spatial and vertical sampling 405

densities than is practical with conventional soil excavation, extraction and 406

characterization techniques. To make this vision a reality, continued research is needed 407

19

on in situ VNIR-DRS applications with greater soil diversity and a wider range of soil 408

properties. 409

410

REFERENCES 411

Anderson, J.U., K.E. Fadul, and G.A. O’Connor. 1972. Factors affecting the coefficient 412

of linear extensibility in vertisols. Soil Sci. Soc. Am. Proc. 37:296-299. 413

Ben-Dor, E. and A. Banin. 1995. Near-infrared analysis as a rapid method to 414

simultaneously evaluate several soil properties. Soil Sci. Soc. Am. J. 59:364-372. 415

Brown D.J., R.S. Bricklemeyer, and P.R. Miller. 2005. Validation requirements for 416

diffuse reflectance soil characterization models with a case study of VNIR soil C 417

prediction in Montana. Geoderma. 129:251-267. 418

Brown, D.J., K.D. Shepherd, M.G. Walsh, M.D. Mays, and T.G. Reinsch. 2006. Global 419

soil characterization with VNIR diffuse reflectance spectroscopy. Geoderma. 420

132:273-290. 421

Chang, C.W., D.A. Laird, and C.R. Hurburgh, Jr. 2005. Influence of soil moisture on 422

near-infrared reflectance spectroscopic measurement of soil properties. Soil Sci. 423

70:244-255. 424

Clark, R.N. 1999. Spectroscopy of rocks and minerals, and principles of spectroscopy. 425

p. 3-52. In A.N. Rencz (ed.) Remote sensing for the earth sciences: Manual of 426

remote sensing. John Wiley & Sons, New York. 427

Clark, R.N., T.V.V. King, M. Klejwa, and G.A. Swaze. 1990. High spectral resolution 428

reflectance spectroscopy of minerals. J. Geophys. Res. 95:12653-12680. 429

20

Coates, J. 1998. A review of sampling methods for infrared spectroscopy. p. 49-91. In 430

J. Workman and A.W. Springsteen (ed.) Applied spectroscopy, a compact 431

reference for practioners. Academic Press, San Diego, CA. 432

Dalal, R.C. and R.J. Henry. 1986. Simultaneous determinations of moisture, organic 433

carbon, and total nitrogen by near infrared reflectance spectrophotometry. Soil Sci. 434

Soc. Am. J. 50:120-123. 435

Ellert, B.H., H.H. Janzen, and T. Entz. 2002. Assessment of a method to measure 436

temporal change in soil carbon storage. Soil Sci. Soc. Am. J. 66:1687-1695. 437

Gaffey, S.J. 1986. Spectral reflectance of calcite minerals in the visible and near 438

infrared (0.35-2.55 microns): Calcite, aragonite, and dolomite. Am. Mineral. 439

71:151-162. 440

Gauch, H.G., Jr., J.T.G. Hwang, and G.W. Fick. 2003. Model evaluation by comparison 441

of model-based predictions and measured values. Agron. J. 95:1442-1446. 442

Gee, G.W. and D. Or. 2002. Particle-size analysis. p. 255-293 In J.H. Dane and G.C. 443

Topp (ed.) Methods of soil analysis. Part 4. ASA, CSSA, and SSSA, Madison WI. 444

Goetz, A.F., S. Chabrillat, and Z. Lu. 2001. Field reflectance spectroscopy for detection 445

of swelling clays at construction sites. Field Anal. Chem. Technol. 5:143-155. 446

Hallmark, C.T., L.T. West, L.P. Wilding, and L.R. Drees. 1986. Characterization data 447

for selected Texas soils. Texas Agricultural Experiment Station. College Station, 448

TX. 449

Hunt, G.R. and J.W. Salisbury. 1970. Visible and near-infrared spectra of minerals and 450

rocks: I. Silicate minerals. Modern Geol. 1:283-300. 451

21

Islam, K., B. Singh, and A. McBratney. 2003. Simultaneous estimation of several soil 452

properties by ultra-violet, visible, and near-infrared reflectance spectroscopy. Aust. 453

J. Soil Res. 41:1101-1114. 454

Janik, L.J., R.H. Merry, and J.O. Skjemstad. 1998. Can mid infrared diffuse reflectance 455

analysis replace soil extractions? Aust. J. Exp. Agric. 38:681-696. 456

Kilmer, V.H., and L.Z. Alexander. 1949. Methods for making mechanical analyses of 457

soil. Soil Sci. 68:15-24. 458

McBratney, A.B., M.L.M. Santos, and B. Minasny. 2003. On digital soil mapping. 459

Geoderma. 117:3-52. 460

Moore, I.D., P.E. Gessler, G.A. Nielsen, and G.A. Peterson. 1993. Soil attribute 461

prediction using terrain analysis. Soil Sci. Soc. Am. J. 57:443-452. 462

Morgan, C.L.S., J.M. Norman, C.C. Molling, K. McSweeney, and B. Lowery. 2003. 463

Evaluating soil data from several sources using a landscape model. p. 243-260. In 464

Y. Pachepsky, D.E. Radcliffe, and H.M. Selim (ed.) Scaling methods in soil 465

physics. CRC Press, Boca Raton, FL. 466

Pachepsky, Ya.A., D.J. Timlin, and W.J. Rawls. 2001. Soil water retention as related to 467

topographic variables. Soil Sci. Soc. Am. J. 65:1787-1795. 468

R Development Core Team, 2004. R: A language and environment for statistical 469

computing. R Foundation for Statistical Computing, Vienna, Austria. (Avaliable on-470

line with updates at http://www.Rproject.org) (Verified 04 Oct. 2006) 471

Rawlins, S.L. and G.S. Campbell. 1986. Water potential: Thermocouple psychrometry. 472

p. 597-662. In A. Klute (ed.) Methods of soil analysis. Part I. ASA, CSSA, and 473

SSSA Madison, WI. 474

22

Reeves III, J.B. and G.W. McCarty. 2001. Quantitative analysis of agricultural soils using 475

near infrared reflectance spectroscopy and a fibre-optic probe. J. Near Infrared 476

Spectrosc. 9:25-34. 477

Reeves III, J.B., G.W. McCarty, and J.J. Meisinger. 1999. Near infrared reflectance 478

spectroscopy for the analysis of agricultural soils. J. Near Infrared Spectrosc. 7:179-479

193. 480

Shepherd, K.D. and M.G. Walsh. 2002. Development of reflectance spectral libraries for 481

characterization of soil properties. Soil Sci. Soc. Am. J. 66:988-998. 482

Sorensen, L.K. and S. Dalsgaard. 2005. Determination of clay and other soil properties 483

by near infrared spectroscopy. Soil Sci. Soc. Am. J. 69:159-167. 484

Steele, J.G. and R. Bradfield. 1934. The significance of size distribution in the clay 485

fraction. Report of the 14th annual meeting. Am. Soil Sci. Assn., Bull. 15, 88-93. 486

Sudduth, K.A. and J.W. Hummel. 1993. Soil organic matter, CEC and moisture sensing 487

with a portable NIR spectrophotometer. Trans. ASAE. 36:1571-1582. 488

Sudduth, K.A. and J.W. Hummel. 1996. Geographic operating range evaluation of a NIR 489

soil sensor. Trans. ASAE. 39:1599-1604. 490

Sudduth, K.A., J.W. Hummel, and S.J. Birrell. 1997. Sensors for site specific 491

management for agriculture. p. 183-210 In F. J. Pierce and E. J. Sadler (ed.) The 492

state of site specific management for agriculture. ASA, CSSA, and SSSA, Madison 493

WI. 494

U.S. Department of Agriculture. 1973. Soil survey of Erath County, Texas. Soil 495

Conservation Service. U.S. Gov. Printing Office, Washington, DC. 496

U.S. Department of Agriculture. 1977. Soil survey of Comanche County, Texas. Soil 497

23

Conservation Service. U.S. Gov. Printing Office, Washington, DC. 498

U.S. Department of Agriculture. 1996. Soil survey lab method manual. Soil Survey 499

Investigations report no.42. Version no. 3. USDA-NRCS, Washington D.C. 500

U.S. Department of Agriculture. 2005. Official soil series descriptions [Online]. 501

Available at http://soils.usda.gov/technical/classification/osd/index.html (Verified 502

04 Oct. 2006) 503

Workman, J., Jr. and J. Shenk. 2004. Understanding and using the near-infrared spectrum 504

as an analytical method. p. 3-10. In C.A. Roberts, J. Workman, Jr., and J.B. Reeves 505

III (ed.) Near-infrared spectroscopy in agriculture. ASA, CSSA, and SSSA, 506

Madison, WI. 507

Yerima, B.P.K., L.P. Wilding, C.T. Hallmark, and F.G. Calhoun. 1989. Statistical 508

relationships among selected properties on Northern Cameroon vertisols and 509

associated alfisols. Soil Sci. Soc. Am. J. 53:1758-1763. 510

Zhu, A.X., L. Band, R. Vertessy, and B. Dutton. 1997. Derivation of soil properties using 511

a soil land inference model (SoLIM). Soil Sci. Soc. Am. J. 61:523-533. 512

Zhu, J., C.L.S. Morgan, J.M. Norman, W. Yue, B. Lowery. 2004. Combined mapping of 513

soil properties using a multi-scale spatial model. Geoderma. 118:321-334. 514

515

24

Fig. 1. Schematic of a vertically sliced soil core. Columns and rows indicate locations 516

scanned using the contact probe for in situ scans. Dried ground soil samples represent the 517

soil from one row, both columns. 518

519

Fig. 2. Predicted vs. measured clay content of the validation data set for (a) air-dried 520

ground (b) air-dried in situ (c) field-moist in situ (d) field-moist in situ smeared. Clay 521

content predictions were made from models built with partial least square regression 522

using the 1st derivative of visible near-infrared reflectance (VNIR) spectra (350-2500 523

nm). RMSD is root mean squared deviation. 524

525

Fig. 3. The wavelengths that contributed to significant regression coefficients (p-value ≤ 526

0.05) in the prediction of clay content are shown for field-moist and air-dry in situ and 527

air-dry ground models. The relative magnitude of each regression coefficient indicates 528

the strength of the correlation. The top plot shows the mean of the regression coefficients 529

common in all three models. All plots are on the same x-axis. Values of the y-axis are 530

not shown, but all y-axes are on the same scale. 531

532

Fig. 4. The wavelengths that contributed to significant regression coefficients (p-value ≤ 533

0.05) in the prediction of clay content are shown for field-moist, smeared and unsmeared, 534

in situ models. The relative magnitude of each regression coefficient indicates the 535

strength of the correlation. The top plot shows the mean of the regression coefficients 536

common in both models. All plots are on the same x-axis. Values of the y-axis are not 537

shown, but all y-axes are on the same scale. 538

25

539

Fig. 5. Laboratory measured (+) and visible near-infrared (VNIR) predicted (◊) clay 540

content for three soil cores, a) a dark-colored Vertisol, b) an Alfisol with distinct A, E, Bt 541

horizons, and c) a Mollisol with secondary calcium carbonate concentrations in the B 542

horizons. Laboratory measurements were taken to represent a row within a horizon, and 543

VNIR predictions were made every 3-cm. 544

545

26

Table 1. Series and family classification of the soils mapped by the Natural Resources 546

Conservation Service within the fields which were used for VNIR-DRS predictions 547

(USDA, 1973; USDA, 1977; USDA, 2005). 548

Soil series Family classification Field number Abilene Fine, mixed, superactive, thermic Pachic Argiustolls 4 Altoga Fine-silty, carbonatic, thermic Udic Haplustepts 6 Blanket Fine, mixed, superactive, thermic Pachic Argiustolls 6 Bolar Fine-loamy, carbonatic, thermic Udic Calciustolls 1,5,6 Bosque Fine-loamy, mixed, superactive, thermic Cumulic

Haplustolls 5

Brackett Loamy, carbonatic, thermic, shallow Typic Haplustepts 5 Bunyan Fine-loamy, mixed, active, nonacid, thermic Typic

Ustifluvents 2

Chaney Fine, mixed, active, thermic Oxyaquic Paleustalfs 4 Cisco Fine-loamy, siliceous, superactive, thermic Typic

Haplustalfs 4

Denton Fine-silty, carbonatic, thermic Udic Calciustolls 1,6 Frio Fine, smectitic, thermic Cumulic Haplustolls 1,4,5 Houston Black Fine, smectitic, thermic Udic Haplusterts 1 Karnes Coarse-loamy, carbonatic, thermic Typic Calciustepts 5 Lewisville Fine-silty, mixed, active, thermic Udic Calciustolls 5,6 Maloterre Loamy, carbonatic, thermic Lithic Ustorthents 1 Nimrod Loamy, siliceous, active, thermic Aquic Arenic Paleustalfs 3,4 Pedernales Fine, mixed, superactive, thermic Typic Paleustalfs 4 Purves Clayey, smectitic, thermic Lithic Calciustolls 1,5,6 Selden Fine-loamy, siliceous, active, thermic Aquic Paleustalfs 3 Venus Fine-loamy, mixed, superactive, thermic Udic Calciustolls 5 Windthorst Fine, mixed, active, thermic Udic Paleustalfs 2,3,6 549

27

Table 2. Clay content and water potential summary statistics for soil samples used in 550

calibration and validation datasets. 551

N Min. Max. Mean SD† Median Calibration samples

Total clay (g kg-1) 188 12 525 255 139 262 Fine clay (g kg-1) 188 3 380 138 85 140 Water potential (MPa) 188 -5.8 0.0 -0.49 0.60 -0.35

Validation samples Total clay (g kg-1) 82 28 578 271 144 247 Fine clay (g kg-1) 82 18 362 152 79 147 Water potential (MPa) 82 -2.2 0.0 -0.48 0.47 -0.38 † SD, standard deviation. 552

553

Table 3. Clay minerals present in soil cores from each field. Mineralogy is presented as 554

a summary by field and not all cores are represented. Cores selected for mineralogy were 555

chosen to represent clay minerals within a field. 556

Clay Minerals Field no. Vermiculite Smectite Kaolinite Mica Calcite Quartz Feldspar

1 XXX† X XX X 2 XX X X X 3 t X XX X X t 4 XX X X X 5 X X X X X 6 XX X t X X

† XXX, greater than 50% of mineral present; XX, 20-50% of mineral present; X, 5-20% 557

of mineral present; t, trace amounts of mineral present. 558

28

Table 4. Prediction accuracies used for total clay and fine clay content models using the 559

1st derivative of visible near-infrared reflectance spectra (350-2500 nm) and partial least 560

squares regression. Prediction accuracies were calculated by using entire soil cores (30 % 561

of the total data set) randomly held out of the calibration model. Also included are the 562

number of principal components (PCs) used in each partial least squares regression 563

model. 564

r2 RMSD† RPD† Bias PCs g kg-1 g kg-1

Total clay Air-dried ground 0.84 62 2.32 -16.3 9 Air-dried in situ 0.92 41 3.51 -2.0 10 Field-moist in situ 0.83 61 2.36 3.0 9 Field-moist smeared 0.75 74 1.95 7.4 6 Fine clay Air-dried ground 0.81 34 2.32 0.65 4 Air-dried in situ 0.85 31 2.55 4.2 4 Field-moist in situ 0.84 32 2.47 4.4 8 Field-moist smeared 0.75 41 1.93 11 4 † RMSD, root mean squared deviation; RPD, ratio of standard deviation of clay content to 565

RMSD. 566

Table 5. Prediction accuracies of clay content using whole-field holdouts of field-moist 567

in-situ scans. Models were created using partial least squares regression with five fields 568

and then validated using soil cores from the sixth field. Six separate models were 569

calibrated and validated. 570

Validation field RMSD† RPD‡ Bias ---------------------g kg-1--------------------- 1 143 0.72 -128.1 2 64 1.92 4.6 3 97 0.98 9.2 4 89 1.18 44.0 5 72 1.21 31.7 6 68 1.09 30.0

29

† RMSD, root mean squared deviation; RPD, ratio of standard deviation of clay content to 571

RMSD. 572

573

574

575

576

Inserted:

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

Ro

F

30

0 100 200 300 400 500 600Lab clay (g kg-1)

0

100

200

300

400

500

600

VN

IR c

lay

(g k

g-1)

0

100

200

300

400

500

600V

NIR

cla

y (g

kg-1

)

0 100 200 300 400 500 600Lab clay (g kg-1)

Fig. 2

(a)y= 0.72x + 60r2= 0.84

(b)y= 0.88x + 31r2= 0.92

(c)y= 0.76x + 69r2= 0.83

(d)y= 0.65x + 103r2= 0.75

RMSD= 62 g kg-1 RMSD= 41 g kg-1

RMSD= 61 g kg-1 RMSD= 74 g kg-1

1:1 lin

e

1:1 lin

e

1:1 lin

e

1:1 lin

e

577

31

500 750 1000 1250 1500 1750 2000 2250 2500Wavelength (nm)

Sig

nific

ant r

egre

ssio

n co

effic

ient

(b)

Fig. 3

Similar wavelengths

Air-dried ground

Field-moist in situ

Air-dried in situ

578 579 580

32

500 750 1000 1250 1500 1750 2000 2250 2500Wavelength (nm)

Sig

nific

ant r

egre

ssio

n co

effic

ient

(b) Similar wavelengths

Smeared in situ

Field-moist in situ

Fig. 4581 582

583 584

0 150 300 450 600

80

60

40

20

0

dept

h (c

m)

0 150 300 450 0 150 300 450

clay content (g kg-1)

Fig. 5

" Measured% VNIR prediction

a) b) c)

585 586