www.irstea.fr

IrsteaNational Research Institute of Science and

Technology for Environment et Agriculture

2

www.irstea.fr

▪A public research institute

▪9 centres

▪1,500 employees

Irstea

3

www.irstea.fr

Irstea : from Research to Action

Scientific and technical

support of public policy

decision-making and

innovation

A multidisciplinary approach

combining human and social

sciences as well as biophysics

Improving knowlegde in

Water, Land Use and

Environmental

Technologies

www.irstea.fr

Research organisation in

Irstea : 3 departments

- water

- environmental technologies

- land use

www.irstea.fr

25 avril 2016

Irstea

Montpellier Centre

6

Localization : north of Montpellier

7Localization : in Agropolis campus

CIRAD

Agropolis

International

IRD

Irstea

AgroParisTech

MTD

IAM

7

8

JRU G-EAU : Water resource management, actors and

uses

9

JRU TETIS

Land, environment, remote sensing and spatial information

9

10

JRU ITAP

Information – Technologies – environmental Analysis –

agricultural Processes

10

Challenges for image conversionin management data:

the case of UAV

Sylvain Labbé, Gilles RabatelIRSTEA – Montpellier, France

« New technologies for precision agriculture » Agropolo Campinas 24-25 may 2017

Part 1 UAV image acquisition for agricultural

monitoring: what technologies ?

Summary

• 1. RGB cameras

• 2. Multispectral cameras

• 3. Hyperspectral cameras

• 4. Thermal cameras

• 5. Radiometric issues

1. RGB cameras

The simplest, but already a lot of things to do with….

RGB sensor technology

• The most used: Bayer matrix– Very cheap solution, huge application market

– Actual spatial resolution < number of pixels

• Foveon sensor– better spatial accuracy

– more complex colorimetry

A necessary step: image mosaicing

Structure from Motion (SfM)

Direct merging of images impossible (variations of UAV pitch and roll) 3D model mandatory to get orthorectified images

Basic principles

• Image overlapping (> 60%): each point must be seen several times

• Complex optimizing processing to recover camera positions

• Image georeferencing based on UAV embedded GPS (5-10 m accuracy) or ground control points (5-10 cm accuracy)

SfM in practice• Open source or commercial software suites

(MicMac, VisualSfM, Photoscan, Pix4D…)

• Up to several hours to get an accurate mosaicing

• Digital Surface Model (DSM) as a by-product

RGB camera for agriculture

• Sometimes used for soil-vegetation discrimination(but not that robust)

• Well-adapted to visual inspection of crop parcels(storm and animal damages, parcel contours and area…)

Excessive green index: Eg = 2G - R - B

• Low cost, high accuracy 3D modelling

RGB camera for agriculture

An alternative for 3D modelling: the LIDAR

• Able to detect both ground and vegetation above

• No need for mosaicing process

• Expansive !!! (~80 k€)

2. Multispectral cameras

Towards more specific spectral information…

Why get NIR spectral bands ?

• Compared to other natural materials, vegetation has the highest ratio betweenNIR and visible reflectance, especially between NIR and R Useful for robust vegetation discrimination or biomass evaluation

Visible NIR

Reflectance (%)

0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 µm

Water Bare soil

Vegetation

0

10

20

30

40

50

60

70

80

90

100

Ratio vegetation index : RVI = NIR/R [0, ∞ [

Normalized Difference Vegetation Index:NDVI = (NIR-R)/(NIR+R) = (RVI-1)/(RVI +1) [-1, 1]

How to get NIR spectral bands ?

Basic issue:

A CDD sensor with a Bayer matrixincluding a NIR filter is technically feasible, but the

agricultural market is not large enough(should be x1000)

Mono-lens solutions

• Industrial multi-CCD camera:

– requires embedded electronics for camera control and image storage

Removing the

NIR blocking filter

Setting a blue or red

band-pass filter

• Modified camera + linear combinationof channels (e.g. MAPIR cameras)

– Lose color information

– NDVI is not that accurate

Multi-lens solutions

Removing the

NIR blocking filter

Setting a NIR

band-pass filter

R+G+B NIRR+G+B NIR

• Camera coupling:

• Commercial multi-lens devices– Supplementary bands for better crop

assessment and phenotyping

– Integrated GPS receiver

– SD card storage

Sequoia(www.parrot.com)

Airphen(www.hiphen-plant.com)

Band registration issue

Due to inter-lens distance,band registration accuracy limited

to a few centimeters

Not adapted to high resolution imagery

Band registration by Fourier-Mellin transform(Rabatel & Labbé, 2016)

• Developed for centimetric imagerywith coupled still cameras

• Registration based on local Fourier spectra analyses

Successfully applied alsoon Airphen multi-lens camera

~0.3 pixel accuracy

3. Hyperspectral cameras

Still more spectral information…

Hyperspectral imagery

• Each pixel contains hundredsof bands (instead of a few ones)

• Allows very accurate discriminations and assessments:

- Variety discrimination

- Chemical content assessment

- Disease detection

…

• Can be used to compute specific vegetationindexes (MCARI, SAVI, etc…) or to deploymultivariate analysis tools (PLS-R, SVM, etc.)

Acquisition techniques

• Push-broom acquisition• Image is acquired line after line

• Requires a scanning motion and spatial reconstruction

• Sequential acquisition• Images in various bands are acquired successively,

using a tunable bandpass filter (LCTF or interferometer)

• Requires a perfect immobility

Some in-field examples

Weed/crop discrimination(Irstea, 2012)

Leaf nitrogen content mapping(Irstea, 2014)

UAV embedding

• Push-broom• Requires a high accuracy IMU for line acquisition

control and image reconstruction high quality but expansive solution (>100 keuros)

• Sequential acquisition• Requires band registration to compensate UAV instability Fourier-Mellin solution has been tested successfullymedium cost solution (~ 40 k€)

5. Thermal cameras

Water stress assessment…

Main characteristics

• Provides images in 8-12 µm spectral range, where material are light emitters according to their temperature.

• Low spatial resolution (usually 320x256 or 640x512)

• Temperature drift effects (especially with UAV uncooled cameras)

• ~ 3000-5000 Euros

Plant water stress assessment

• Leaf temperature versus ambiant temperature gives an indication on water stress (stressed plants reduce their evapo-transpiration to savewater)

• Relationship between signal and temperature depends on camera calibration and drift requires temperature references in the field

• Usually coupled with NDVI measurements(to compensate for pixel mixing)

5. Radiometric issues

From luminance to reflectance

• Only the reflectance R() is valuable for crop assessment,but the camera provides L()

E() L() = E().R()

R()

L()R()

• A radiometric correction must be applied

• The more spectral information is expected (e.g. hyperspectral imagery),the more accurate the correction must be

Radiometric correction techniques

• Solution 1: to measure incident light E() some cameras include an irradiance sensor(Sequoia multispectral camera, Rikola hyperspectral camera)

E() L() = E().R()

R()

L()R()

• Solution 2: to measure luminance Lref() = E() . Rref() on a reference groundtarget with known reflectance Rref()(and recover it in the final image)

Canopy effects

Due to 3D canopy structure, leavesreceive a part of light from

surrounding leaves, modifying the apparent reflectance

• Some hyperspectral indices have been designed to take these effects into accountat low spatial resolution (e.g. Wu et al, 2008 for chlorophyll content estimation)

• No formal solution right now, but we do work hard on it !(PhD thesis Irstea, 2015-2017)

To summarizeSensor type Usage Operation

difficultyPrice

Color (RGB) Visual observation3D modelling

+ +

Multi-spectral Biomass estimation (NDVI)

Robust vegetation/soildiscrimination

++ ++

Hyperspectral Variety discriminationLeaf chemical components

++++ ++++

Thermal Temperature measurement ++++ ++

Lidar 3D modelling ++ ++++

Part 2UAV Applications to precision agriculture

• Various kinds of UAV

• Regulatory and economical issues

• Applications in precision agriculture

• Few applications in environmental field

Drone or UAV (unmanned aerial vehicle) or RPAS (remotely piloted aircraft system)

Journée Optitec 2014

33

The range of different drones according to US classifications

Micro UAV Evolution

Choice of architecture(speed, yield, robustness).

- Propulsion (electric, thermal).

- Energy source- Battery- Hydrogen cell- Laser- Hybrid- Solar

Fixed wings Multirotor

UAV Upcoming evolution

-New design-Convertible plane/helicopter-Aerial and submarine-Flapping wings

-

ONERA

Regulatory and economical issues

• Coexistence with aerial traffic– In France over unpopulated areas:

• 150m (500ft) high and 200m horizontally from pilot

• 50m high and 1000m from pilot (with FPV)

• 25 kg max

• Pilot theorical and practical licenses

– Business solutions• service provider or owner of UAV and/or sensors

and/or data processing chain

Remote sensing typical use of UAV

How UAV can help with precision agriculture

Remote sensing

• Information on soil and field (heterogeneity)

• Crop monitoring: yields, damage assessment, weeds

• Optimized application (yields): fertilizer, pesticides

• Optimized application (balanced management of resources) : water, variety phenotyping

• Health assessment: early warning, targeted treatment

Other UAV payloads

• Crop spraying, crop planting, etc.

Information on soil and field

• Palm tree plantation (Indonesia)

Missing trees and heterogeneityInterpolation

NDVINDVI(Visible) Visible + parcelle

Fertilization

Parrot drone and sensors – output : fertilization recommendation map

Weed detection (RHEA project)

43

Weed detection

45

Weed detection (RHEA project)

Source: Institute for Sustainable Agriculture

(CSIC – Espagne)

Hydric stress

• Without stressCrops transpire and temperature of leaves

is lower than air temperature

• With stressCrops temperature is higher than

Ambiant air temaperature

Exemple of hydric stress on thermal image

(Irstea Montpellier)

Thermal camera

UAV with visible, NIR and TIR

48

49

UAV with visible, NIR and TIR

Wheatplot

UAV and precision irrigation

April

WDI

1

0

UAV and precision irrigation

Irrigation

En cours

WDI

1

0

May

UAV and precision irrigation

Visible image

Water stress study

High throughput Phenotyping

Exemple of a orchard with 120 apple varieties

High throughput Phenotyping

Water stress indexes

55 / nb total

VISIBLE PROCHE INRAROUGE

NDVI

High throughput PhenotypingSugarcane (Reunion island)

26°C

39°C

28°C

27°C

35°C

Temperature

Few other examples with UAV

• Spraying

• Sample collection in water bodies

• Environmental impact assesment

• Lidar and forestry

57

SPRAYING UAV

Yamaha Rmax

Aerial spraying of pesticides is currently forbidden in Europe

NIR false color image – Guyane

500 m

Wetland monitoring(French Guyana)

Classification result

Cartographie des zones humides en 2010

Wetland monitoring(French Guyana)

ForestryLidar examples

Forestry : wood volume estimation

[Bouvier, 2015]

To summarizeApplication Usage Development

stageIssues

Crop monitoring Visual observationYield forecastDamage assesment

Operational

Precisionagriculture

Fertilization

Weed detection

Precision irrigation

Operational

Prototype stage

Feasability stage

Adapted groundmaterials

Phenotyping Variety yield

Variety resistance to stress

Operational

Research stage Sensors sensitivity

Health assesment Early alert and treatment Research stage Detection and riskmodelling

Other payloads Spraying Operational Regulation

Thank you