the practical application of state of the art aerial...

34
The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924 1 Project Title: Weed Detection and Classification Using Low Altitude Aerial Images University of Sydney Reference: 176924 Sponsor Reference: NTLLS-0135 Item: Department of Agriculture Report Authors: Nasir Ahsan, Zhe Xu, Richard Murphy, Salah Sukkarieh Prepared for: Northern Tablelands Local Land Services Contacts: Zhe Xu [email protected] Salah Sukkarieh [email protected] Date: 05 April 2016

Upload: others

Post on 24-Jul-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

1

Project Title: Weed Detection and Classification Using Low Altitude Aerial Images University of Sydney Reference: 176924 Sponsor Reference: NTLLS-0135 Item: Department of Agriculture Report Authors: Nasir Ahsan, Zhe Xu, Richard Murphy, Salah Sukkarieh Prepared for: Northern Tablelands Local Land Services

Contacts: Zhe Xu [email protected] Salah Sukkarieh [email protected] Date: 05 April 2016

Page 2: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

2

1 Project Title The practical application of state-of-the-art un-manned aerial vehicles and imaging technology to on farm management of invasive weeds.

2 Lead Organisation and Partner Organisations Lead Organisation: Northern Tablelands Local Land Service

Partner Organisation: Australian Centre for Field Robotics, The University of Sydney

3 Primary Contact and Contact Details Lead Organisation: Jonathan Lawson ([email protected])

Partner Organisation:

Dr. Zhe Xu ([email protected]), Prof. Salah Sukkarieh ([email protected]).

4 Acknowledgements The authors would like to acknowledge Northern Tablelands Local Land Services staff, National Parks and Wildlife Services staff, North West Local Land Services, the Department of Agriculture and property owners for their support of this project.

5 Executive Summary This report presents the research and development work towards detecting invasive weed species from unmanned aerial vehicle (UAV) imagery and satellite data and how this technology can be applied to a practical weed management program to provide positive outcomes. The objective of this study was to demonstrate the weed detection algorithms developed at the Australian Centre for Field Robotics (ACFR) operating at the property and regional scale. The combination of high (spatial and temporal) resolution UAV- or satellite-based aerial imagery and automated weed detection software operating at this scale complements more traditional weed surveillance and weed management tools such as ground surveys. The ability to collect data at high spatial resolutions means that characteristics that differentiate weed species from native species are more apparent, thus improving detection rates and accuracies. Achieving a high temporal resolution means that surveys can be conducted more regularly, which may be important in fast-moving outbreaks. The increased scale removes the need for some of the “guess-work” required when developing weed management practices based on sparse site surveys or transects.

To demonstrate the practicability of our approach, we collected data at four locations across New South Wales at Marulan, Deepwater, Moree and Kosciuszko National Park, targeting six different weed species: serrated tussock (Marulan and Deepwater), African boxthorn, mimosa, galvanised burr and harrisia cactus (Moree) and orange hawkweed (Kosciuszko National Park). A multi-class, multi-scale neural network algorithm was trained to detect weeds in aerial imagery covering these properties. Further, we generated

Page 3: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

3

property level weed distribution maps that can be used to inform weed management by estimating the coordinates of each weed detection. The survey areas ranged from several hectares to areas in excess of 2km2. The detection algorithms had accuracies of up to 73% (serrated tussock), 80% (African boxthorn), 73% (mimosa) and 89% (galvanised burr). The UAV-based detections were used to bootstrap the detection of the larger woody weeds (African boxthorn and mimosa) in satellite data. Analysis of satellite imagery coinciding with the UAV field trials demonstrated that larger woody weed species such as mimosa and African boxthorn can be detected in satellite imagery at large scales (>25km2).

6 Background This report presents the research and development work towards detecting invasive weed species from unmanned aerial vehicle (UAV) and satellite data using automated software. The first objective was to demonstrate the practicability of weed detection algorithms developed at the Australian Centre for Field Robotics (ACFR) operating at the property scale. . This study targeted six weed species (serrated tussock, African boxthorn, mimosa, galvanised burr, harrisia cactus and orange hawkweed) at four locations across New South Wales (Marulan, Deepwater, Moree and Kosciuszko National Park). The second objective was to study how these results can be applied to satellite data to detect weeds at a regional scale.

The combination of high resolution UAV- or satellite-based aerial imagery and automated weed detection software complements more traditional weed surveillance and weed management tools such as ground surveys. UAVs in particular provide the capability to collect aerial imagery at high resolutions, both spatially and temporally. A high spatial resolution means that characteristics that differentiate weed species from native species are more apparent. Achieving a high temporal resolution means that surveys can be conducted more regularly, which may be important in fast-moving outbreaks. The ability to automatically detect weeds in this aerial imagery using computer software reduces the manual effort required and thus increases the scale at which weed surveys can be conducted. This increased scale removes the need for some of the “guess-work” required when developing weed management practices based on sparse site surveys or transects. The ability to detect weeds in satellite data further increases the spatial scale at which these tools can aid weed management and surveillance, which in turn provides benefits to the agriculture industry and for ecological management by bodies such as the National Parks and Wildlife Service. These benefits include greater situational awareness of the invasive weed species to support management decisions, as well as more timely information on weed outbreaks.

There are a number of challenges in the practical use of UAV-based imagery as weed surveillance and management tools. While advances in the use of UAVs in agricultural and ecological management applications have allowed end-users to acquire large amounts of data, the analysis of this data can be a time-consuming and error-prone exercise. To address this issue, we apply an automated classification algorithm that can be trained to detect different weed species. Once trained by expert users, this classification algorithm can be applied to large sets of aerial imagery and operate autonomously. Further, today’s UAVs are limited in their endurance and thus the area that can be practically surveyed is also limited. To address this limitation, we demonstrate that UAV-based weed detections can be used to bootstrap the use of satellite imagery to detect the same weed species on a far larger scale.

Page 4: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

4

In this study, aerial imagery was collected using a combination of a rotary-wing UAVs, used to survey smaller areas and areas that are difficult to access, and a fixed wing UAV used to cover larger areas. A multi-class, multi-scale neural network algorithm was applied to detect weeds in these datasets. This algorithm uses machine learning and computer vision techniques to detect objects of interest in images. A human expert trained this algorithm on each weed species by labelling examples of the weed in the aerial imagery. Based on these examples, the algorithm learned a model and searched the remainder of the dataset for the weed(s) of interest. Due to the specialised nature of orange hawkweed and harrisia cactus, separate colour- and phase-based detection algorithms were used. Weed detections were geo-registered to generate property level weed distribution maps.

These techniques were demonstrated for the six target weed species. The survey areas ranged from several hectares to areas in excess of 2km2. The detection algorithm had accuracies of up to 73% (serrated tussock), 80% (African boxthorn), 73% (mimosa) and 89% (galvanised burr). Satellite data covering an area of 25km2

was also analysed. Detections of larger woody weed species such as mimosa and African boxthorn in the satellite data showed a good correspondence with detections from UAV-based imagery, demonstrating the efficacy of using satellite imagery for weed management at larger scales. Finally, our orange hawkweed detection algorithms were used to analyse aerial imagery covering approximately 4ha of previously un-surveyed land. A number of detections were made and used to guide the hawkweed eradication program conducted by the National Parks and Wildlife Service.

7 Methodology Our approach to detecting weed species from UAV-based imagery is composed of three steps:

1. UAVs are flown over the area of interest to collect aerial imagery. Ground truth data is collected to train the classification algorithm to detect the weed(s) of interest.

2. A classification algorithm is trained to detect the weed(s) of interest using a number of examples provided by a human expert. This classification algorithm is run over the aerial imagery and searches for other occurrences of the same weed(s). Two types of classification algorithms are demonstrated in this report: (1) a multi-scale, multi-class neural network used for serrated tussock, African boxthorn, mimosa and galvanised burr and (2) specialised classification algorithms used to detect orange hawkweed and harrisia cactus.

3. The location of weed detections is estimated based on the navigation data recorded by the UAVs as the imagery was taken in a process called geo-registration. Geo-registration allows us to generate weed distribution maps that can then be used to inform weed surveillance and management activities.

Finally, we will describe the method by which UAV-based weed detections can be used to bootstrap the detection of weeds in satellite data.

7.1 Data Collection with UAVs Two UAV platforms were used to collect aerial imagery: a multi-rotor UAV, the Ascending Technologies Falcon 8 and a fixed-wing UAV, the Elimco E300. Both platforms were equipped with a Sony NEX 7 camera, but with lenses of different focal lengths – 19mm on the Falcon 8 and 16mm on the E300. The Falcon 8 shown in Figure 1 is able to hover above targets. In this study it was flown over 50m x 50m grids at different altitudes (8m, 30m and 100m). Multiple flights were performed to cover larger areas.

Page 5: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

5

Figure 1: The Falcon 8 used in data collection. The Falcon 8 was used to collect images from different altitudes. The camera is mounted in the front of the vehicle and is oriented downwards during data collection.

The E300 shown in Figure 2 is a fixed wing platform launched using a catapult; it is able to fly much longer than the multi-rotor (more than one hour) and is able to cover larger areas (1 x 2 km). The E300 was used to collect images for property-scale study.

Figure 2: The E300 used in data collection. The E300 can remain airborne for more than one hour and was used to cover larger areas.

In addition to aerial imagery, ground truth data was also collected to support this study. The ground truth data comprised the locations of known examples of the weed species and other plant species with similar appearance. These examples were further identified by placing checker-board targets nearby to make them easily identifiable in the aerial imagery.

7.2 Weed Detection Using Multi-scale Neural Networks In this study, we applied a multi-scale neural network classifier to detect serrated tussock, African boxthorn, mimosa and galvanised burr. A classifier is a computer algorithm that separates the target weed species from other background objects in an image. The multi-scale neural network classifier is a classifier that can be “taught” to recognise various objects such as weeds. In this approach, a subset of images was manually labelled by an expert, who identified all instances of the target weed species in the images. These labels were used to build a model of the appearance of the target weed species using features such as colour and texture that were learned by the neural network classifier.

Once a model is trained, the classifier applies the model across the entire data set, which could comprise 1500 images. This process allows the expert’s labels to be extrapolated across the entire set of aerial images,

Page 6: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

6

making the analysis of large amounts of aerial imagery tractable. The output of this classification is a grey-scale prediction image describing the probability of weed presence, with lighter colours indicating higher probabilities.

7.3 Orange Hawkweed Detection The advanced state of the orange hawkweed control in the Kosciuszko National Park means there were very few orange hawkweed plants in the survey area. This scarcity limited the number of training examples that could be labelled, which limits the applicability of the multi-scale neural network classifier described in the previous section.

In this study, we applied a two-step, colour- and phase-based classification algorithm that exploited the distinctive colour of orange hawkweed flowers to distinguish hawkweed from other vegetation. In the first step, pixels that were a particular hue of orange associated with hawkweed flowers were identified in the L-a-b colour space. This set of pixels, however, correspond to both hawkweed and other orange-coloured objects in the environment and resulted in a large number of false positive detections – false positive detections are weed detections generated by the classifier that do not actually correspond to a weed.

The second step reduces the false positive rate by examining the surrounding pixels, in particular the distribution of colour across these pixels. Orange hawkweed has a particular colour “signature”; this signature was compared against that of potential detections. The similarity of these signatures gave a probability that a detection was indeed an orange hawkweed flower. This second stage of the classification algorithm is illustrated in Figure 3.

Figure 3: Extracting colour signatures from 3x3 pixel neighbourhoods containing orange hawkweed and other vegetation in an image. The colour signature is essentially the probability density over each of the channels. The greater the resemblance along each of the channels the higher the probability that the candidate is an orange hawkweed flower.

Page 7: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

7

7.4 Harrisia Cactus Detection A colour based classifier was also applied to detect harrisia cactus. This classifier was tuned to detect the unique pink/red colour of the harrisia cactus fruit. This colour is distinct in the Hue-Saturation-Value (HSV) colour space, as shown in Figure 4. A set of tuning parameters (thresholds in the H-S-V space) that define the colour of harrisia cactus fruit were manually identified based on training images collected in the field. These parameters were then applied to the entire dataset to identify the harrisia cactus.

Figure 4: Channels in various colour spaces (Red-Green-Blue, Hue-Saturation-Value, YCbCr and L-a-b). Harrisia cactus fruit are distinct from the background and other plants in the H-S-V colour space, as seen in the bright dots on the top image.

7.5 Geo-Registration Geo-registration is the process of determining the physical locations of weeds detected by the classifier, and allows us to generate weed distribution maps. The classification algorithms described in the previous sections operates on an image-by-image basis. To produce a single weed detection map, we detected regions in each image where there is a high likelihood of weed presence. First, pixels in the grey-scale prediction image with a probability of weed presence below a given threshold (probability) were removed. The remaining portions of the prediction image were consolidated into groups of pixels (blobs). The physical location of each blob was then calculated using navigation data logged by the UAVs. Finally, the blobs were clustered in order to remove duplicate detections caused by overlaps across multiple images.

7.6 Weed Detection in Satellite Imagery In this section, we describe the methodology used to detect the larger woody weed species – African boxthorn and mimosa – in satellite data. The characteristics of satellite data are very different from UAV-based aerial imagery used in this study; while the satellite data has comparatively low spatial resolution (meters rather than centimetres), it has a high spectral resolution in the sense that satellites typically image both inside and outside the visible spectrum, for example, in infra-red. This disparity means that a different

Page 8: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

8

approach is needed for classifying satellite data – instead of classifying based on properties such as colour and texture of image patches, we examine the satellite data pixel-by-pixel. However, the higher spectral resolution of satellite data means that each pixel contains more information (intensities at multiple bands across the electromagnetic spectrum) than in UAV-based aerial imagery (only the red, green and blue bands). In this section, we first describe the nature of the satellite data used for this study and the ground truth data collected to support this analysis. We then describe the classification process.

7.6.1 Satellite Data Two sources of satellite data were used for this study: (1) the Landsat Operational Land Imager and (2) World View 2. Characteristics of these data sources are summarised in Table 1. Landsat is a commonly used source of satellite imagery for land use mapping applications and has a relatively coarse resolution of 30m/pixel. World View 2 is another source of satellite data that has a finer resolution (2m/pixel).

Table 1: Characteristics of satellite data used for this study.

Landsat World View 2

Resolution (m) 30 2

Acquisition Date 8-Oct-2015 11-Nov-2015

Bands • Costal aerosol • Blue • Green • Red • Near infra-red • Short wave infra-red

band 1 • Short wave infra-red

band 2

• Blue • Green • Red • Near infra-red band 1 • Near infra-red band 2 • Red-edge • Costal • Yellow

7.6.2 Ground Truth Data Collection In addition to the ground truth data collected to support the analysis of UAV data, we also recorded the reflectance spectra of the target weed species, other species of similar appearance as well as the background soil. The reflectance spectra refers to the signature of the weed species not only in parts of the electromagnetic spectrum visible to the human eye, but also in bands beyond the visible spectrum such as infra-red. Reflectance spectra were acquired from different ground targets on the site, specifically, boxthorn, mimosa, saltbush, dry grass and soil, and are shown in Figure 5. These spectra were used to confirm that the satellite imagery was correctly calibrated by the data providers and to aid in the extraction of training pixels from imagery. Reflectance spectra were acquired using an Analytical Spectral Devices (ASD) field spectrometer between 350-2500 nm. This ASD sensor is able to capture the signature of a weed species to a much higher spectral resolution than what is available in satellite imagery, allowing us to simulate what the reflectance spectra as seen in both Landsat and World View 2 satellite data. These simulated spectra are shown for the World View 2 data in Figure 5b,d.

Page 9: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

9

Figure 5: Reflectance spectra of the target weed species as collected by the ASD spectrometer (a,c) and simulations of the spectra as seen in World View 2 satellite data (b,d).

Spectra of boxthorn, and particularly mimosa, show a distinct peak in reflectance at green wavelengths and very low reflectance at blue and red wavelengths. This explains why these types of vegetation, appeared visually to be greener than the other vegetation in the field. Spectra of saltbush were relatively flat across the entire visible range with the exception of a slight increase in reflectance at green and red wavelengths. Boxthorn and mimosa also exhibited a steep rise in reflectance between red and near infra-red wavelengths caused by the scattering of light in the mesophyll layers of the leaves. Saltbush exhibited a smaller rise in reflectance between these wavelengths and in dead grass the rise was almost non-existent. The reflectance of soils increased monotonically with wavelength with the exception of a small dip at red wavelengths caused by absorption by chlorophyll. This could be caused either by algae in the soil or the presence of small amounts of live vegetation. These spectra suggests that boxthorn and mimosa could be spectrally separated from soils and saltbush at wavelengths sensed by both the Landsat and the World View 2 sensors.

7.6.3 Classification of Satellite Data The classification of satellite data into different surface cover types was a two stage process:

• In the first stage, we identified examples of spectrally distinct components in the satellite imagery. These distinct components correspond to various types of surface cover (soil, various weed species and various other plant species), or particular mixtures of surface cover types.

Page 10: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

10

• In the second stage, a set of classifiers were trained to detect these components, and used to separate the entire set of satellite data into distinct classes. The class that best corresponded to the UAV-based weed detections were then identified as the target weed species.

The details of these two steps are detailed in the subsections below.

7.6.3.1 Identification of Spectrally Distinct Image Components The following analysis was applied separately to the Landsat and World View 2 imagery with the objective of identifying pixels in the image that were spectrally representative of the different cover types in the image. To do this the method described by Murphy & Wadge (1994) 1 was used. First, a principal component analysis was applied to the imagery. This analysis examines the variation of spectra in the satellite data and identifies spectral combinations that are most distinct. The n-dimensional visualiser (ENVI; Exelis) was used to examine scatter-plots of the principle components in 3-dimensional space. Spectrally distinct pixels were located at the vertices of the polygon that bounded the data space, as shown in Figure 6; these pixels were selected for training various classifiers that will be described in the next section. The classes identified in the Landsat imagery were different to those identified from the World View 2 imagery as the spatial resolutions of the sensors was greatly different. The coarser resolution of the Landsat imagery (30m) compared to the World View 2 imagery (2m) meant that the pixels comprising the Landsat data had a much greater chance of containing mixtures of surface cover types (for example, mimosa, soil, eucalypt) than did the World View 2 imagery. This meant that cover types that were ostensibly spectrally distinct were in reality different cover types. For example, one spectrally distinct class in the Landsat data could correspond to a particular mixture of soil and boxthorn.

Figure 6: Visualisation of spectrally distinct components of satellite imagery.

1 Murphy RJ, Wadge G (1994) The effects of vegetation on the ability to map soils using imaging

spectrometer data. Int J Remote Sens, 15, 63-86.

Page 11: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

11

7.6.3.2 Classification Techniques Six classification techniques were trained using the examples of spectrally distinct pixels identified in the process described above:

• Maximum likelihood classifier2 (MLC); • Mahalanobis distance-based classifier1; • Minimum distance to means-based classifier1 (MDM); • Spectral angle mapper using the average of the spectrally distinct pixels3 (SAM); • Neural networks4; and • Support vector machine5 (SVM).

In all cases classification was done on the multispectral data without spatial sharpening using the panchromatic band.

Each of these classifiers was applied to a 2km2 subset of the satellite data that overlapped spatially with the coverage of the UAV-based aerial imagery. The classes of spectrally distinct surface cover types generated by each of these classifiers were compared with UAV-based detections. The class/classifier combination that showed the greatest (visual) correspondence was applied over the entire (25km2) satellite survey area.

8 Results

8.1 Summary of Survey Sites The data shown in this report was collected from 20 different sites at six locations, summarised in Table 2.

Table 2: Summary of survey sites.

Site Weed species present

Marulan Serrated tussock

Deepwater Serrated tussock

Moree gum flats African boxthorn, mimosa and galvanised burr

2 Richards JA (1999) Remote Sensing Digital Image Analysis, Springer-Verlag, Berlin. 3 Kruse FA, Lefkoff AB, Boardman JB, Heidebrecht KB, Shapiro AT, Barloon PJ, Goetz AFH (1993) The

Spectral Image Processing System (SIPS) - Interactive Visualization and Analysis of Imaging Spectrometer Data. Remote Sens Environ, 44, 145-163.

4 Rumelhart DE, Mc Clelland J (1987) Learning Internal Representation by Error Propagation. In: Parallel

Distributed Processing (eds Rumelhart D, Hinton GE, Williams RJ). MIT Press. 5 Wu T-F, Lin C-J, Weng RC (2004) Probability estimates for multi-class classification by pairwise coupling Journal of Machine Learning Research, 5, 975-1005.

Page 12: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

12

Moree property African boxthorn and mimosa

Moree cactus site Harrisia cactus

Kosciuszko National Park Orange hawkweed

The Marulan site (-34.599792, 150.053596) is a property owned by the University of Sydney. A section of this property is covered by serrated tussock at various densities. The trial was performed in June 2015 with the E300 UAV. Data was collected at an altitude of 100m.

The property at Deepwater is privately owned and has serrated tussock presence at various densities, with some sites (Sites 4 and 5) experiencing a higher density compared to others. This trial was performed in September 2015. The Falcon 8 was flown at each of the sites in 50m x 50m grids at both 30m and 100m altitudes. The details of the sites are shown in Table 3.

Table 3: Site locations and flights at Deepwater. All flights were conducted with the Falcon 8 UAV.

Site Location (latitude, longitude)

1 -29.4513369, 151.939578

2 -29.4540156, 151.9518805

3 -29.4502703, 151.9525649

4 -29.4488089, 151.9479203

5 -29.4477125, 151.9447241

We surveyed two sites near Moree for African boxthorn, mimosa and galvanised burr: the gum flat area and a private property. Boxthorn was the dominant weed at the gum flat with a minority infestation of mimosa and galvanised burr. The private property was dominated by mimosa with a significant minority of boxthorn. This trial was conducted in October 2015 using the Falcon 8 at the gum flat and the E300 on the property. The altitude used in this study was 100m throughout. Details of the sites are shown in Table 4.

Table 4: Site locations and platforms flow at Moree.

Site UAV Platform Location (latitude, longitude)

1 Falcon 8 -29.4581221, 150.085764

2 Falcon 8 -29.4709268, 150.0803999

3 E300 -28.7204433333, 150.239486944

Page 13: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

13

Further, we conducted vertical profile studies at two sites near Moree to determine an appropriate altitude for detecting harrisia cactus. These vertical profile studied were conducted using the Falcon 8 UAV in May 2015. The site locations for this study is summarised in Table 5.

Table 5: Location of harrisia cactus sites near Moree.

Site Location (latitude, longitude)

1 -28.73533, 150.223

2 -29.48064, 150.000

We surveyed 11 sites across Kosciuszko National Park. These include both known orange hawkweed and unknown orange hawkweed sites with 16 different flights over a period of 3 days, summarised in Table 6. A known orange hawkweed site was previously visited by rangers whereas an unknown orange hawkweed site has been inspected for the first time. Vertical transects were carried out over known sites to understand what spatial resolution (and thus altitude) was required to reliably detect orange hawkweed. Survey grids were flown at unknown sites.

Table 6: Site locations and Flights across Kosciuszko National Park. All flights were conducted with the Falcon 8 UAV.

Site Known orange hawkweed presence Location (latitude, longitude)

RMT11 Y -36.0555029,148.3716937

RMT2 N -36.0522791,148.3719402

RMT1 N -36.0601143,148.3721452

CP01 N -36.0537055,148.3493487

CP02 N -36.0514075,148.3464067

FMR07 Y -36.0206071,148.3932846

FMR03 Y -36.0224347,148.3932519

FMR144 N -36.0188118,148.3948513

OAS2 N -36.050643,148.3205733

OAS5 N -36.0483324,148.3204654

OAS12 Y -36.0498076,148.3261595

Page 14: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

14

8.2 Results From Marulan In this section we present the results from a serrated tussock survey at Marulan. Figure 7 shows a subset of manually labelled images used to train the multi-class neural network classifier, while Figure 8 shows an example classifier prediction. These labelled images were used to assess the accuracy of the classification using cross validation, and demonstrated that the classification accuracy was 77.3%. Cross validation is an experimental design that involves splitting the manually labelled patches into a training set (90% of the patches for the 10-fold cross validation used in this study) and a test set (the remaining patches). The model is trained on the training set and its performance is recorded on the test set. Repeating this procedure numerous times gives a distribution of the actual generalisation error, that is, how robust the model is to unseen training data. On this dataset, the standard deviation of the accuracy across the folds was 5.3%, indicating that the model is robust.

Figure 7: Manually labelled patches containing Serrated Tussock at a 100m altitude.

Page 15: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

15

Figure 8: Example classifier prediction on one aerial image. The white regions in the prediction images indicate areas that are likely to be covered by serrated tussock. This result demonstrates that the classifier is able to differentiate between trees, grass and serrated tussock.

The property level weed distribution map is shown in Figure 9. The results indicate dense patches of serrated tussock at the north-east end of the property (top right corner in Figure 9), and along both sides of the river. The central part of the property only contained sparse populations of serrated tussock, with some presence within the high tree density region.

Figure 9: Spatial profile of serrated tussock at Marulan.

8.3 Results From Deepwater In this section we present the results from a serrated tussock survey at Deepwater. A set of image patches labelled as serrated tussock is shown in Figure 10, while examples of other objects in the environment are shown in Figure 11. Both sets of labels were used for training the classification algorithm. The classification

Page 16: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

16

accuracies are shown in Table 7. Site 1 had only three serrated tussock plants and hence insufficient for training a classifier. Thus a cross validation test could not be performed for those flights. An example prediction is shown in Figure 12.

Figure 10: Serrated tussock labelled patches at 100m.

Figure 11: Non-serrated tussock labelled patches at 100m, including rocks, trees, ground, and native vegetation.

Page 17: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

17

Figure 12: Example classifier prediction from Deepwater. This image was taken at an altitude of 100m.

Table 7: Summary of 10-fold cross validation statistics for Deep Water. Images are acquired at each site at two different altitudes (100m and 30m).

Site Altitude (m) Accuracy (%) Accuracy Standard Deviation (%)

2 30 67.1 13.7

2 100 79.5 8.9

3 30 68.2 27.8

3 100 59.0 15.8

4 30 69.6 5.3

4 100 72.9 5.7

5 30 71.7 9.7

5 100 71.0 6.9

Firstly, it should be noted that the results at both altitudes for Site 3 are comparatively lower than those at other sites, with the 30m altitude resulting in worse accuracy than at 100m. This can be explained by the fact that most of the serrated tussock present at this site was found to be < 1 year old, which is difficult to detect even at the 30m altitude due to the plants' small size. However, due to the high variance in the predictive accuracies the difference between the two altitudes for this site can be considered insignificant.

For the remaining sites, the predictive accuracy varies between 69-73% at 100m and 70-80% at 30m. However, given the variances in the predictive accuracies at the two altitudes the differences are not very significant. This suggests that we are not compromising predictive accuracy by increasing the altitude. Increasing the altitude is beneficial from an operational and a practical perspective as much more area can be covered at 100m. It should be noted that at these sites, mature serrated tussock (>2 years) was present, with mature serrated tussock dominating Sites 4 and 5.

Page 18: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

18

The serrated tussock distribution maps for Sites 2 - 5 are shown in Figure 13 - Figure 16 respectively. The results show significant distributions of serrated tussock at Sites 2, 4 and 5 and a more sparse distribution at Site 3. This observation corresponds with the greater proportion of young serrated tussock that could not be detected by the classifier at Site 3.

Figure 13: Spatial distribution of weeds at Deepwater Site 2.

Figure 14: Spatial distribution of weeds at Deepwater Site 3.

Page 19: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

19

Figure 15: Spatial distribution of weeds at Deepwater Site 4.

Figure 16: Spatial distribution of weeds at Deepwater Site 5.

8.4 Results from Moree (African Boxthorn, Mimosa and Galvanised Burr) In this section, we present the results obtained at Moree on both at the gum flat and the private property. Unlike previous sections, the results presented in this section deals with detecting multiple targets; African boxthorn, mimosa, and galvanised burr.

Aerial imagery was acquired at 100m for all three sites. Figure 17 and Figure 18 provide some examples of the patches that were manually labelled to train the neural network classifier. Multiple classes were labelled including trees, rocks, branches, ground, to enhance the ability of the classifier to discern between the target

Page 20: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

20

weeds and the similar vegetation and/or objects. The gum flat was predominantly populated by African boxthorn, whereas the property had a significantly larger proportion of mimosa. Site 1 at the gum flat also had a minority population of galvanised burr. Example predictions are shown in Figure 19.

Figure 17: Training data from the gum flat for African boxthorn (top), mimosa (middle), and galvanized burr (bottom).

Figure 18: Training data from the private property for African boxthorn (top), mimosa (middle), and galvanized burr (bottom).

Page 21: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

21

Figure 19: Sample predictions of African boxthorn (top 2 rows) and galvanized burr (bottom row).

The predictive accuracies for all sites are presented in Table 8. From the results we can see that boxthorn was detected with accuracies within 78-82% with the exception of Flight 240 at the gum flat Site 1. This site experiences not only significantly lower prediction accuracy but also has a high variance. This performance may be due to poorer labelling.

The accuracy statistics for mimosa on the gum flat are not as high as for boxthorn, whereas for the private property (Flight 009), we achieved 73.6% accuracy with a smaller variance. This accuracy can be credited to the fact that there was an insufficient amount of mimosa to label at the gum flat, and therefore, only a few patches were provided to the classifier for training. On the other hand, there is a significant distribution of mimosa on the private property and an adequate distribution of boxthorn, which provided a good set of training data for both weed species.

Galvanized burr was a minority at both locations. Nonetheless, its distinctness (visually) required only a few training labels to achieve a 70-98% accuracy. However, the difference between the 3 estimates of accuracy are significantly spaced apart which also suggests perhaps the classifier overfit the data, that is, it performs well on training data but might not be as accurate on unseen test data. Thus, to conclude any further on the classification of galvanized burr required more training data.

The most accurate results we obtained were for the combined class (mimosa and boxthorn) ranged between 80-93%. This result suggests that the similar appearance of boxthorn and mimosa resulted in a degree of confusion between these two classes. This is further evident at Site 3 where the difference between the individual class accuracies and the combined accuracy is smaller, especially when noting that Site 3 had sufficient training data for the two. Nonetheless, the high accuracy of the combined class suggests that the classifier can still confuse the two weed species.

Page 22: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

22

Table 8: Summary of 10-fold cross validation statistics for Moree for African boxthorn, mimosa, and galvanised burr. An empty cell means that the weed species was not present for that flight. The standard deviation of the accuracy across the folds is given in (parentheses) after the accuracy.

Site UAV / Flight Boxthorn (%)

Mimosa (%)

Galvanised Burr (%)

Mimosa and Boxthorn (%)

1 Falcon 8/Flight 240 64.2(14.9) 53.1(19.5) 98.5(7.00) 79.6(17.1)

1 Falcon 8/Flight 241 80.1(10.6) 57.7(13.0) - 84.6(11.5)

1 Falcon 8/Flight 242 80.3(10.6) - 80.6(10.5) -

2 Falcon 8/Flight 246 78.3(8.40) 49.6(21.2) - 93.0(10.4)

2 Falcon 8/Flight 247 82.4(7.00) 64.4(22.2) - 85.3(8.40)

3 Elimco E300/Flight 009

79.5(7.70) 73.6(6.10) 73.4(11.7) 89.6(6.20)

The property level weed distribution maps are shown in Figure 20 for Site 1, Figure 21 for Site 2 and Figure 22 for Site 3. It can be seen that African boxthorn is the majority species at the gum flat, with mimosa and galvanised burr being minority species. However, at Site 3, the majority species was mimosa. Further, it is noted that both African boxthorn and mimosa could be detected at Site 2, despite partial canopy coverage by taller native species such as eucalyptus. This result demonstrates that the classifier is able to differentiate crowns of eucalyptus from the weed species. Further, these results demonstrates that weed detections are possible in challenging environments where there is partial occlusion of the ground and target weed species.

Page 23: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

23

Figure 20: Spatial distribution for African boxthorn, mimosa, and galvanized burr over the gum flat at Site 1.

Page 24: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

24

Figure 21: Spatial distribution for African boxthorn and mimosa over the gum flat at Site 2. Note there was no galvanized burr at this site.

Page 25: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

25

African boxthorn Mimosa

Figure 22: Spatial distribution for African boxthorn and mimosa at Site 3.

8.5 Results from Moree (Harrisia Cactus) In this section, we present results for the detection of harrisia cactus. An example detection demonstrating the separation of harrisia cactus fruit is shown in Figure 23. Our results indicated that harrisia cactus could be detected at altitudes over 30m, which was the highest altitude flown as part of this study. Image patches showing the fruit at various altitudes is shown in Figure 24.

Page 26: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

26

Figure 23: Example detection of harrisia cactus fruit.

Figure 24: Image patches of harrisia cactus fruit at various altitudes. The fruit can be identified at over 30m, which is the highest altitude considered in this study.

8.6 Results from Kosciuszko National Park In this section, we present the results obtained at Kosciuszko National Park for orange hawkweed. Examples of orange hawkweed that have been verified by ground observations are show in the top row of Figure 25. However, there are a number of objects in the environment with a similar colour to orange hawkweed. Examples of these are other flowers, grasses and trees as illustrated on the bottom row of Figure 25.

Figure 26 shows classifier detections on known examples of orange hawkweed while Figure 27 shows detections of other objects in the environment with similar colour. These figures demonstrate that the second

Page 27: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

27

phase-based stage removes a large proportion of the false positive detections (detections classified as orange hawkweed but are in fact something else), while still detecting all of the known examples of orange hawkweed.

Figure 25: The top row shows examples of orange hawkweed found in the aerial imagery that have been verified by ground observations. The bottom row shows vegetation with similar colour to orange hawkweed. The first and the third images in the bottom row are of a non-hawkweed flower with a similar colour. The second and fourth images are of leaves and grass.

Figure 26: Results of the orange hawkweed classifier on known examples of orange hawkweed. The green boxes represent individual weed detections. The red boxes represent detections based on colour alone. These detections were filtered out by the phase-detection stage. It can be seen that in all instances, the phase-detection did not unintentionally remove any detections.

Page 28: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

28

Figure 27: Results of the orange hawkweed classifier on examples of other objects in the environment. The green boxes represent classifier detections. The red boxes represent detections based on colour alone. It can be seen that the phase-based stage removes many of these false positive detections.

We applied the two-stage classifier on 2900 aerial images captured over approximately 4ha of previously un-surveyed areas where the presence of orange hawkweed were unknown. The classifier detected 33 potential occurrences of orange hawkweed. Analysis by weed experts indicated 7 could be orange hawkweed plants. The high false positive rate was a consequence of the need to minimise the false negative rate, that is, the classifier was tuned so as to not miss any potential hawkweed plants. Having a low false negative rate is important in this application as the detections were used to the hawkweed eradication program conducted by the National Parks and Wildlife Service.

8.7 Weed Detection in Satellite Imagery

8.7.1 Landsat Imagery Four classes were found in the Landsat data: (1) green vegetation and mimosa, (2) soil and vegetation, (3) dry and green vegetation and (4) dry vegetation. Green vegetation appeared in two separate classes (1 and 2) as did dry vegetation (3 and 4). This separation into different classes could be caused by different types of green or dry vegetation having a different spectral signature. The presence of more than one material in the area of ground measured by a single pixel causes the spectral signatures of the materials to be mixed. In

Page 29: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

29

effect, the mixture of the different classes becomes a class of its own. In this case, the classes describe a spectrally discrete terrain type comprising more than one class. The coarse spatial resolution of the Landsat data (30m) increases the chance that more than one material type would be present on a single “class”.

MLC Mahalanobis distance MDM

SAM Neural network SVM

Figure 28 Classified maps derived from Landsat data. The classes are: green vegetation, African boxthorn and mimosa (light green), soil and vegetation (blue), dry and green vegetation (red) and dry vegetation (dark green).

Results from the classification were very different depending on the method used. With the exception of spectral angle mapper, all methods performed poorly using the classes identified from the UAV imagery. Comparison with high resolution colour images acquired with the UAV (Figure 22) showed that Mahalanobis distance, MDM and SVM performed particularly poorly in mapping cover types. Although these methods did correctly classify areas with green vegetation and mimosa (light green in classified map; Figure 28) they either under- or over-estimated areas occupied by other classes. This mis-classification can be seen in Figure 28, which shows that the north-eastern corner is dominated by other green vegetation types

Page 30: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

30

(red) rather than mimosa and boxthorn (light green). SAM correctly classified areas of green vegetation, mimosa and boxthorn (light green in classified map; Figure 28). However, the low resolution meant that many examples of mimosa were mistakenly classified as dry vegetation. These false negative detections are particularly evident in the south-western corner; the UAV based weed detections (Figure 22) shows significant mimosa presence, while the classified Landsat results indicate the area is wholly dry vegetation.

8.7.2 World View 2 Imagery True and false colour World View 2 imagery is shown in comparison to Landsat data in Figure 29. The greatly increased spatial resolution of the World View 2 sensor (2m) is evident, with individual crowns of bushes and trees being resolved in the true and false colour imagery (mimosa and boxthorn are shown as bright green pixels in the imagery; Figure 29). The increased resolution of the sensor meant that it was less likely for different materials in the scene to be mixed together to form a discrete terrain type, as was observed for the Landsat imagery. Instead, classes comprised individual material types, specifically: (1) soil, (2) mimosa, (3) eucalyptus and (4) background vegetation – that is, vegetation that was dried to varying amounts.

Landsat World View 2

Figure 29: Comparison of colour composite images from Landsat and World View 2. The resolution of the World View 2 data is much higher than Landsat (2m vs. 30m) and allows individual trees to be identified visually.

Results from the classification showed that all methods performed well. However, a visual comparison with weed detections from the UAV-based, high-resolution colour imagery (Figure 22) indicated that SAM gave the best results, as seen in Figure 30. Mimosa and boxthorn were mapped as a single class (blue in classified

Page 31: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

31

map; Figure 30) by SAM at the scale of individual bushes. Blue areas with red indicate mixed assemblages of mimosa, boxthorn and eucalyptus. In comparison to all other methods, MLC performed poorly and overestimates areas of mimosa/boxthorn. The classified results from the World View 2 imagery present a significant improvement of over those from the Landsat data. The greater spatial resolution of the World View 2 imagery enable areas affected by mimosa and boxthorn to be mapped at a much finer spatial scale and with a greater specificity than could be achieved with Landsat data.

MLC Mahalanobis distance MDM

SAM Neural network SVM

Figure 30 Classified maps derived from World View 2 data by different classification methods. The classes are: soil (yellow), mimosa and boxthorn (blue), eucalypt (red) and other vegetation (green).

Finally, the SAM classifier was applied across the entire satellite dataset, which covers an area of 25km2. The results are shown in Figure 31.

Page 32: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

32

Figure 31: Results of the SAM classification of World View 2 data across the entire satellite data set. The classes are: soil (yellow), mimosa and boxthorn (blue), eucalypt (red) and other vegetation (green). The UAV survey area is outlined in red.

9 Discussion The results of this study demonstrated that the combination of UAV-based aerial imagery and automated classification algorithms can be applied to a range of weed species found in NSW to generate weed distribution maps that can inform weed surveillance and management. In particular, this study showed that the use of UAVs allowed relatively large (2km2) areas to be surveyed at high resolutions. This high resolution enabled detection accuracies ranging from 73 – 89%, depending on the weed species; the average

Page 33: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

33

detection accuracy for each weed species is given in Table 9. The low standard deviation of the detection accuracy across the folds shown in Table 9 demonstrates the robustness of the classification algorithms to unseen data.

Table 9: Summary of the expected 10-fold cross validation statistics over multiple sites at each of the locations.

Weed Species Detection accuracy (%) Standard deviation (%)

Serrated tussock 72.7 8.1

African boxthorn 80.4 8.8

Mimosa 73.6 6.1

African boxthorn and mimosa 84.2 9.7

Galvanised Burr 88.8 9.95

Our analysis of satellite data demonstrated that a combination of ground truth data collection and weed detections from UAV-based aerial imagery could be used to bootstrap weed detection in high resolution (2m/pixel) satellite data. These results mean that weed detections from ground- and UAV-based surveys can be extrapolated to much larger areas than what would otherwise be practical.

10 Future Needs for Innovation Uptake Recent advances in UAV technology and satellite imaging means that high-resolution aerial imagery is becoming increasingly available to end users at lower costs. In such an environment, the key gap that needs to be closed to increase the uptake of the technology demonstrated in this study is to make the automated weed detection algorithms more readily available to end-users, through mechanisms such as cloud-based computing. Providing a cloud-based weed detection service will increase the uptake of this innovation by allowing end users across NSW who have aerial imagery (either from their own UAVs or through a UAV operator) to detect weeds relevant to them.

While the use of UAV- and ground- based weed detections to bootstrap weed detection in satellite data demonstrated in this study has shown good results, it is a manual process that requires a degree of domain knowledge. Further automation of this process through the use of computer algorithms to match UAV- and ground-based weed detections to classes identified in satellite data can increase the uptake of this innovation by encapsulating the domain knowledge required to do this matching in intelligent software, thus making the technology more readily accessible. A further extension would be to analyse discrepancies between the classification of satellite data and ground- and UAV-based detections to identify areas where the satellite classification could be refined, again using intelligent software. This information can be exploited to target future weed surveys and plan weed surveillance programs using quantifiable metrics such as confidence of detection.

Page 34: The practical application of state of the art aerial ...northerntablelands.lls.nsw.gov.au/__data/assets/... · The combination of high resolution UAV- or satellite-based aerial imagery

The practical application of state of the art aerial vechicles and imaging technology to on Farm management of invasive weeds. University of Sydney Reference 176924

34

11 Communications Outcomes of this study have been communicated at the following forums:

• Towards Weed Detection in the Cloud, Hung, C., Xu, Z., Ashan, N. and Sukkarieh, S., Queensland Weed Symposium, 14th Sep. 2015

• Drone Applications for Broadacre Agriculture Field Day, 8th Oct. 2015. • NSW Weeds Conference, 12th Oct. 2015. • Algorithms for Weed, Pest and Tree Detection in UAS Data, UAS for Remote Sensing Conference,

15th Feb. 2016.