Hands-on Soil Infrared Spectroscopy Training Course

Getting the best out of light11 – 15 November 2013

Why Soil Spectroscopy?

Keith D Shepherd

Surveillance Science• Measure frequency of problems and associated risk factors in populations

using statistical sampling designs & standardized measurement protocolsUNEP. 2012. Land Health Surveillance: An Evidence-Based Approach to Land Ecosystem Management. Illustrated with a Case Study in the West Africa Sahel. United Nations Environment Programme, Nairobi.http://www.unep.org/dewa/Portals/67/pdf/LHS_Report_lowres.pdf

Shepherd KD and Walsh MG (2007) Infrared spectroscopy—enabling an evidence-based diagnostic surveillance approach to agricultural and environmental management in developing countries. Journal of Near Infrared Spectroscopy 15: 1-19.

Simplicity of light

Wavelength unit converter.xls

Spectral shape relates to basic soil properties

• Mineral composition• Iron oxides• Organic matter• Water (hydration,

hygroscopic, free)• Carbonates• Soluble salts• Particle size distribution

Functional properties

•Soil mineralogy•nutrient quantity (stock) and intensity (strength of

retention by soil)•pH and buffering, variable charge•anion and cation exchange capacity•carbon saturation; protection•aggregate stability, dispersion/flocculation•resistance to erosion

•Soil organic matter•soil structure•aggregate stability, resistance to erosion; water

holding capacity•carbon storage and turnover•cation exchange capacity•nitrogen, organic P, sulphur supply

Soil function largely determined by soil mineralogy and soil organic matter

Origin of infrared spectral absorption features

Water vibrations movie

Carbon dioxide-vibrations movie

SpectraSchool - Royal Society of Chemistryhttp://www.rsc.org/

Soil IR fundamentals

1 = Fingerprint region e.g Si-O-Si stretching/bending2 = Double-bond region (e.g. C=O, C=C, C=N)3 = Triple bond (e.g. C≡C, C≡N)4 = X–H stretching (e.g. O–H stretching)NIR = Overtones; key features clay lattice and water OH; SOM affects overall shape

Field spectroscopy

Shepherd KD and Walsh MG. (2002) Development of reflectance spectral libraries for characterization of soil properties. Soil Science Society of America Journal 66:988-998.

Infrared spectroscopy Dispersive VNIR FT-NIR FT-MIR Robotic FT-MIR Portable

Handheld MIR ?Mobile phone cameras ?

Brown D, Shepherd KD, Walsh MG (2006). Global soil characterization using a VNIR diffuse reflectance library and boosted regression trees. Geoderma 132:273–290.

Shepherd KD and Walsh MG (2007) Infrared spectroscopy—enabling an evidence-based diagnostic surveillance approach to agricultural and environmental management in developing countries. Journal of Near Infrared Spectroscopy 15: 1-19.

Terhoeven-Urselmans T, Vagen T-G, Spaargaren O, Shepherd KD. 2010. Prediction of soil fertility properties from a globally distributed soil mid-infrared spectral library. Soil Sci. Soc. Am. J. 74:1792–1799

Sample preparation/presentation

Instrument protocols

Fourier Transform Spectrometer

Dispersive spectrometer

Reference analyses

Data & soil library management

Barcoding

Soil archiving system

1.2 km shelving to hold over 40 t of soil

CalibrationSoil organic carbon Spectral pretreatments

• Derivatives, smoothing

Data mining algorithms:• PLS +• Support Vector

Machines• Neural networks• Multivariate Adaptive

Regression Splines• Boosted Regression

Trees• Random Forests• Bayesian Additive

Regression Trees

Training Out-of-bag validation

Soil pH

R package soil.specSoil spectral file conversion, data exploration and regression functions

Spectral libraries

Inter-instrument calibration transfer

Robotoic high throughput MIR

• Submit batch of spectra online

• Uncertainties estimated for each sample

• Samples with large error submitted for reference analysis

• Calibration models improve as more samples submitted

• All subscribers benefit

Soil-Plant Spectral Diagnostics Lab

•500 visitors/yr again

•338 instruction

•13 PhD, 4 MSc training

Spectral Lab Network

•IAMM, Mozambique

•AfSIS, Sotuba, Mali

•AfSIS, Salien, Tanzania

•AfSIS, Chitedze, Malawi

•CNLS, Nairobi, Kenya

•ICRAF, Nairobi, Kenya

•CNRA, Abidjan, Cote D’Ivoire

•KARI, Nairobi, Kenya

•ICRAF, Yaounde, Cameroon

•Obafemi Awolowo University, Ibadan, Nigeria

•IAR, Zaria, Nigeria

•ATA, Addis Ababa, Ethiopia (+ 5 on order)

•IITA, Ibadan, Nigeria

•IITA, Yaounde, Cameroon

•ICRAF, Nairobi, Kenya

Planned

•Eggerton University, Kenya

•MoA, Liberia

•IER, Arusha, Tanzania

•FMARD, Nigeria

•NIFOR, Nigeria

•CNLS, Nairobi

•BLGG, Kenya (mobile labs)

Spectral fingerprintingTotal X-ray fluorescence spectroscopy

X-ray diffraction spectroscopy

Mineral Semi-quant (%)

Quartz

Albite

Microclin

e

Kaolinite

Hematite

Muscovit

e

Diopside

69.2

5.0

4.3

9.9

2.8

4.3

4.6

Infrared spectroscopy

Land Health Surveillance

Consistent field protocol

Soil spectroscopyCoupling with

remote sensingPrevalence, Risk factors, Digital mapping

Sentinel sites Randomized sampling schemes

✓60 primary sentinel sites➡ 9,600 sampling plots➡ 19,200 “standard” soil samples➡ ~ 38,000 soil spectra➡ 3,000 infiltration tests➡ ~ 1,000 Landsat scenes➡ ~ 16 TB of remote sensing data

to date

AfSIS

Spectral prediction performance

Main AfSIS workflow, products & services overview

Markus Walsh, August 2013

Ethiopia: current spatial coverage of new ground observations and measurements

Africa Soil Information Service

www.africasoils.net

Markus Walsh, August 2013

Probability topsoil pH < 5.5 ... very acid soils

prob(pH < 5.5)Africa Soil

Information Servicewww.africasoils.net

Markus Walsh

[N] ppm [P] ppm [K] ppm

[S] ppm [Ca] ppm [Mg] ppm

“Best” current topsoil macro-nutrient (N,P,K,S,Ca & Mg) concentration predictions

Africa Soil Information Service

www.africasoils.net

Markus Walsh

Living Standards Measurement StudyIntegrated Surveys on Agriculture

LSMS-IMS

Improve measurements of agricultural productivity through methodological validation and research

Responding to policy needs to provide data to understand the determinants of social sector outcomes.

Soil fertility monitoring componentTwo pilot countries

MTT-Finland FoodAfrica

Soil Micronutrients

Healthy soils

Healthy crops

Healthy livestock

Healthy people

Evidence-based micronutrient management

Land HealthSurveillance Out-scaling

Tibetan Plateau/ Mekong

Vital signs

Cocoa - CDIParklands Malawi

National surveillance systems

Regional Information Systems

Project baselines

Ethiosis

Rangelands E/W AfricaSLM Cameroon MICCA EAfrica

Global-Continental Monitoring Systems

Evergreen Ag / Horn of Africa

CRP pan-tropical sites

AfSIS

Future directions

• Centralized calibration service on-line

• Direct calibration of MIR to plant/soil response data

• Rural MIR labs providing low cost soil testing for smallholder farmers

• Complementarity of IR, TXRF, XRD, Handheld XRF

• Decision cases