digital terrains
Post on 04-Apr-2018
227 Views
Preview:
TRANSCRIPT
-
7/30/2019 digital terrains
1/42
CRC for Spatial Information
info@crcsi.com.auwww.crcsi.com.au
Report on
Performance of DEM GenerationTechnologies in Coastal
Environments
Clive Fraser and Mehdi RavanbakshCooperative Research Centre for Spatial Information
-
7/30/2019 digital terrains
2/42
2
Table of ContentsPage
Executive Summary 3
1. Introduction 5
2. Project Overview 5
3. Overview of DEM Generation Technologies 5
3.1 Technology Options 5
3.2 Accuracy Considerations 6
3.3 LiDAR 7
3.4 Photogrammetry 8
3.5 IfSAR 8
4. Project Work Plan 9
5. Test Area Locations 11
6. Specifications for DEM Data Sets 15
7. Benchmark Elevation Data 16
7.1 Permanent Survey Marks 16
7.2 GPS Survey of Height Profiles 16
7.3 Comparison of GPS and Ground Survey Elevations 16
7.4 GPS Heighting versus LiDAR DEMs 20
8. Analysis of Different DEMs Against LiDAR Reference DEM 23
8.1 Discrepancies in Elevation 23
8.2 SRTM DEM 26
8.3 SPOT5 DEM 28
8.4 Topo DEM (from 1:25,000 map data) 28
8.5 Airborne IfSAR DEM 28
8.6 ADS40 DEM 31
9. Impact of Land Cover on DEM Accuracy 31
9.1 Urban areas 31
9.2 Open Rural Areas 35
9.3 Forest/Bushland Areas 36
9.4 Mixed Coastal Land Cover 38
10. Influence of Terrain Slope 39
11. Conclusions 40
-
7/30/2019 digital terrains
3/42
3
Executive Summary
Reliable digital elevation models (DEMs) are vital to better understand and prepare for the
impacts of sea level rise and storm surges caused by climate change. A number of satellite
and airborne remote sensing technologies can be used to generate digital elevation models,
however each technology possesses its own advantages and limitations. The primary aim of
this project has been to evaluate the performance of different technologies for the generation
of digital elevation models, specifically in coastal environments. The accuracy characteristics
of six such technologies have been assessed within four test areas on the mid north coast of
New South Wales. These test sites were chosen as being representative of low-lying coastal
zones of differing land cover, topography and geomorphology.
The DEM technologies investigated were:
Airborne LiDAR (airborne laser scanning)
Airborne IfSAR (interferometric synthetic aperture radar)
SPOT5 HRS satellite imagery
1-second SRTM-based national DEM
Aerial photography:o with the DEM sourced from existing 1:25,000 digital topographic mappingo with the DEM derived from recent ADS40 digital imagery
An objective of the project was to look beyond differences in vertical resolution, cost and
productivity, and to consider the overall performance of different DEMs in the context of
fulfilling anticipated requirements for fit-for-purpose elevation data in Australias vulnerable
coastal zones. Outcomes of the project can be used to inform the development of future
guidelines covering optimal DEM generation technologies for programs such as UDEM and
the National Digital Elevation Framework (NEDF).
Recent forecasts of sea level rise are in the range of 0.5m to upwards of 1m over the
remainder of this century. Digital elevation modelling in support of prediction andmonitoring of the inundation impacts of sea level rise and storm surges will therefore require
vertical resolution at the sub-metre and even decimetre level. A principal finding of this
project has been to reinforce the prevailing view that LiDAR is the optimal DEM generation
technology for this application. While DEMs produced photogrammetrically from aerial
imagery can match the vertical resolution of LiDAR, namely around 10cm, they are
invariably more expensive and exhibit significant shortcomings in bare-earth elevation
modelling if automated classification and filtering are solely relied upon.
Beyond highlighting the recognized superiorities of LiDAR, this project has identified DEM
characteristics that are perhaps not as widely appreciated, but are nevertheless important in
the context of producing accurate bare-earth DEMs of coastal terrain. One of these concerns
the accuracy gap between LiDAR DEMs and those derived from airborne IfSAR and aerial
photography. Comparing LiDAR accuracy (10cm) to IfSAR and ADS40 accuracy (50-
100cm), one would expect LiDAR DEMs to be at least 5 times better. However the
difference are accentuated by shortcomings in the automated classification and filtering of
both vegetation and, to a lesser extent, man-made structures, within the process of producing
a bare-earth DEM from the latter two technologies. Multiple-return LiDAR on the other hand
displays significant advantages by way of last-pulse ground definition, which cannot be
-
7/30/2019 digital terrains
4/42
4
matched in densely vegetated areas by radar and photogrammetry techniques, except through
skill-intensive and expensive manual editing processes.
The results obtained for DEM performance in open areas, largely free of trees and buildings,
highlighted the fact that distinctions in DEM accuracy are as much due to different terrain
and land cover, and consequently to filtering, as to differences in the basic metric resolution
of DEM technologies. In the case of open pasture, sub-metre accuracy was obtained for theSRTM DEM while the 1:25,000 mapping, the IfSAR DEM and the aerial imagery DEM all
displayed sub-half metre accuracy. Although the accuracy of all these lower-resolution
DEMs exceeded specifications, they are nevertheless still not likely to fulfill requirements for
fit-for-purpose high-resolution elevation models for decision support and risk analysis
associated with sea level rise.
-
7/30/2019 digital terrains
5/42
5
1. Introduction
This report summarises the objectives, work plan, conduct and outcomes of the project
Performance of DEM Generation Technologies in Coastal Environments, which formed a
research project under the Urban Digital Elevation Modelling in High Priority Regions
Program(UDEM). The focus of the analyses required to evaluate the performance of different
Digital Elevation Model (DEM) technologies has been upon detailed assessments of the
heighting accuracy produced by six DEM generation technologies within four test areas
representing typical low-lying Australian coastal environments, with land cover types
including urban, rural and forest. Outcomes of the project will provide an increased
understanding of the characteristics of different elevation data technologies and how well
they perform in Australian coastal environments. Results will therefore inform the
development of guidelines covering optimal DEM generation technologies for vulnerable
coastal zones. Results will also be of benefit to producers of DEMs, particularly to those
producing elevation models under the National Digital Elevation Framework (NEDF) and
UDEM.
2. Project Overview
The objective of this project has been to investigate the performance of different DEM
generation technologies within a range of coastal environments. The quality of DEMs is a
function not only of the data acquisition and subsequent data processing, but also of the
characteristics of the terrain being mapped, especially in regard to topography and vegetation,
and to the presence of cities and urban land cover. DEMs produced from different imaging
and ranging sensors need to be analysed in order to better understand their characteristics and
accuracy, and also their cost-benefit ratios in relation to producing fit-for-purpose elevation
models for coastal assessments.
3. Overview of DEM Generation Technologies
3.1 Technology Options
It was initially envisaged that the research would investigate the accuracy capabilities of four
categories of DEMs/DEM generation technologies, namely:
Airborne LiDAR (Light Detection and Ranging technology).
The new national Australian mid-resolution SRTM 1 DEM (derived from the ShuttleRadar Topography Mission IfSAR data).
The mid-resolution SPOT DEM (derived from approximately 5m resolutionstereoscopic SPOT5 HRS satellite imagery).
High-resolution airborne IfSAR.
In addition to comparing DEMs, and in cases DSMs (digital surface models), generated fromthese four data sources, the work plan was extended to also encompass analysis of DEMs
derived from the following sources:
Photogrammetrically derived DEMs from digital aerial imagery, and specificallyautomatically produced DEMs from the Leica ADS40 3-line scanning system.
DEM data from current 1:25,000 topographic mapping, the data having beengenerated over several decades, originally from manual stereo-compilation of analog
-
7/30/2019 digital terrains
6/42
6
aerial imagery and subsequently from digitisation of the resulting contour maps. This
elevation model is referred to throughout this report as the Topo DEM.
Photogrammetrically derived DEMs from 3-line ALOS PRISM satellite imagery.
IfSAR derived DEMS from the TerraSAR-X/TanDEM-X satellite radar system.
DEM data from six of the above technologies was successfully sourced for the project.
LiDAR, ADS40, airborne IfSAR and Topo DEM data was made available by LPMA (NSW
Dept. of Lands), and Geoscience Australia supplied the SPOT5 and SRTM DEMs. The
project team was unable to source both ALOS PRISM and TanDEM-X DEM data. In the
case of tandem-X, the system is still within its initial commissioning phase, with commercial
operations not anticipated to commence for several months. The ALOS satellite unfortunately
ceased to operate in late April, 2011, which lessened the imperative to examine this DEM
data source.
In the absence of PRISM and TanDEM-X data, it is still possible to infer to some degree the
overall performance of these two technologies from the results obtained for the SPOT5 HRS
and airborne IfSAR DEMs, respectively. However, in the IfSAR case, TanDEM-X is
anticipated to produce vertical accuracies at the 2m level as opposed to the 0.5-1m expected
accuracy for airborne IfSAR DEMs. Also ALOS PRISM DEMs should display higher overall
accuracy than those from SPOT5 HRS, since PRISM has double the spatial resolution and 3-
line scanner geometry.
Prior to describing the methodology and workflow of the project, it is useful to recall the
accuracy associated with each DEM technology and salient characteristics of the DEM
sources considered. These are briefly summarized in the following sections.
3.2 Accuracy Considerations
Associated with each DEM technology is an accuracy specification. This is generally
expressed as a bound, since DEM accuracy is a function of both sensor and topographic/land
cover characteristics, and in the case of LiDAR and photogrammetry it can vary according toproject design requirements. More will be said in the following paragraphs about the
accuracy specifications for each of the DEM technologies considered in this investigation,
but it is initially useful to appreciate the range of accuracy anticipated, this range being
shown in Figure 1. The figure shows representative 1-sigma accuracy bounds (68%
confidence level) for each of the six DEM technologies. The bounds shown are indicative
only, and their purpose is to highlight the order of magnitude and more difference between
the representative 15cm vertical accuracy of LiDAR and the 8-9m accuracy of the SPOT5
HRS and SRTM DEMs. The considerable variation in vertical resolution needs to be kept in
mind when comparing the merits of different DEM generation options.
Another important aspect related to DEM quality is the presence or absence of height bias,
which can be local, for example as a consequence of incomplete filtering of above-groundfeatures in the DSM-to-DEM conversion process, or large-area, as a consequence of
systematic errors in sensor positioning and orientation. Through reference to Figure 1 the
reader can visualize that whereas a DEM may have high precision, of let us say a standard
error (1-sigma value) of +/- 15cm, it may be inaccurate by the extent of the bias, which is
indicated by the dashed line in the figure.
-
7/30/2019 digital terrains
7/42
7
Figure 1. Representative 1-sigma vertical accuracy bounds for DEM technologies.
3.3 LiDAR
Airborne laser scanning or LiDAR is today the clear technology of choice for the generation
of high-resolution DEMs with post spacings of of 1-3m. The advantages of LiDAR centre
upon its relatively high-accuracy of generally 10-15cm in height and around 1/2000th
of the
flying height in the horizontal, and upon the very high mass point density of nowadays
around 4 points/m2. This high point density greatly assists in filtering out non-groundartefacts in the conversion from the directly acquired DSM to the final bare-earth DEM.
Moreover, LiDAR has high productivity of around 300 km2
of coverage per hour, and it can
be operated locally, day or night. In practice, data acquisition is generally confined to
daylight hours since most LiDAR units nowadays come with dedicated digital cameras
(usually medium format), the resulting imagery being used both to assist in the artefact
removal process and for orthoimage production.
One of the most significant attributes of LiDAR is multiple-return sensing, where the first
return of a pulse indicates the highest point encountered and the last the lowest point. There
may also be mid pulse returns. As a consequence, LIDAR has the ability to see through all
but thick vegetation. Whereas it might not be certain from where in the canopy the first pulse
was reflected, it can be safely assumed that a good number of the last returns will be frombare earth. This greatly simplifies the DSM-to-DEM conversion process in vegetated areas.
Whereas aerial photogrammetry techniques can yield DSMs of vertical accuracy equivalent
to LiDAR, it is generally not economical to opt for photogrammetry over LiDAR for DEM
generation at vertical resolutions of 10-20cm. Thus, in the context of UDEM, LiDAR stands
alone in most practical respects as the most accurate and comprehensive means to produce
highest resolution DEMs of coastal environments. For this reason, LiDAR has been chosen in
-
7/30/2019 digital terrains
8/42
8
this project as the standard against which the other DEM generation technologies are
compared. In order to quantify the accuracy of LiDAR against ground-truth, elevation data
from a kinematic GPS survey of several thousand points has been used, along with data from
permanent survey marks.
3.4 Photogrammetry
As a tool for topographic mapping, photogrammetry has a long history. Traditionally
elevation data was extracted from stereo aerial photography in the form of contours, as
exemplified in this project by the Topo DEM, which was obtained from 1:25,000 topographic
map data. DSM generation was automated with the advent of analytical stereoplotters and
then further process automation accompanied the introduction of digital aerial imagery. The
generation of a DSM from digital aerial or satellite imagery is today a fully automatic batch
process, with the resulting elevation model often being employed to support orthoimage
generation.
Broad area DEM generation via photogrammetry is presently not the preferred approach,
except in special circumstances such as very high accuracy DSMs for 3D city modelling. The
latter is exemplified to some extent by current programs to create high definition,photorealistic models of major cities. For example, one approach employs the Vexcel
Ultracam digital camera flown in a block configuration of 80% forward overlap and 60% side
overlap at an imaging scale that yields a 15cm ground sample distance (GSD). DSM and
subsequently DEMs to around 30cm vertical and 2-3m horizontal resolution can be generated
with a high degree of automation through such a process. For the present project, DEM data
generated from 50 cm GSD imagery recorded by LPMAs Leica ADS40 line scanning
camera to a nominal vertical resolution of 0.5-1m has been adopted as representative of the
capabilities of fully automated DSM production from digital aerial imagery, followed up with
initial stage automated DSM-to-DEM conversion. However, it is noteworthy that the final
stage, manually intensive classification and filtering was not carried out, and thus the ADS40
data should be thought of as constituting a bare-earth DEM over open terrain, and to some
extent in urban areas, but only as a smoothed DSM for land cover comprising densevegetation.
Satellite imaging systems have gained popularity for DSM generation at vertical resolutions
within the range of 1m to 10m. For example, the GeoEye-1 and World View-1 and -2
satellites have a 50cm GSD, which will support DSM extraction to around 1-2m vertical
accuracy. Also, the dedicated DEM generation program of SPOT Image, which uses the
SPOT 5 HRS system, yields DEMs with a nominal 5-10m height accuracy (1-sigma) and 20-
30m horizontal resolution. All satellite imaging systems used for 3D terrain modelling use
line scanner technology, with the 2.5m resolution ALOS PRISM satellite having a 3-line
scanner geometry similar to that in the ADS40 aerial camera. DEMs produced from ALOS
PRISM can be expected to display height accuracies of around 3-5m.
3.5 IfSAR
Synthetic Aperture Radar (SAR) has been employed for a few decades as an imaging
technology in remote sensing. Through an augmentation of a conventional airborne or
spaceborne SAR system with a second receiving antenna, spatially separated from the first, it
has been possible to utilise the principles of interferometry to extend SAR from a 2D imaging
system to a 3D topographic modelling technology. The resulting Interferometric Synthetic
Aperture Radar(IfSAR) system determines the relative heights of imaged ground points as a
-
7/30/2019 digital terrains
9/42
9
function of the phase difference of the coherently combined signals received at the two
antennas. The first commercial IfSAR system for DEM generation, the Intermap STAR 3i
system, appeared in the mid 1990s and global focus was brought onto the capabilities of
IfSAR to produce DEMs with the successful completion of the Shuttle Radar Topography
Mission (SRTM) in 2000.
There have been a number of refinements made to the SRTM DEM of Australia over the pastfew years, to the point where an updated DTED 2 DEM with a post spacing of 1 second
(30m) and a nominal vertical accuracy in the range of 6-12m has recently been released. This
new SRTM DEM data was accessed for the current project from Geoscience Australia.
Beyond the heavily built-up areas of major cities and very rough mountainous areas, the
Australian terrain can be characterized as being ideal for DEM generation via airborne radar.
Intermap Technologies have recently completed a large project within the Murray Darling
Basin with their STAR system and produced DSMs with a stated 0.5 - 1m vertical accuracy,
and a post spacing of 5m. Moreover, use of stereo radar imagery as a complement to the
process allows for semi-automated DSM-to-DEM conversion. Airborne IfSAR can record
data at the very rapid rate of around 100-200 km2
per minute, which is some 10-20 times the
area acquisition rate of LiDAR (the IfSAR swath width is generally 8-20km). Moreover, datacollection in not impeded by clouds. Over the past two or three years, there has been a
considerable upsurge in 1m-accurate DEM generation via IfSAR, with national DEMs being
commercially available through Intermaps Nextmap product line.
A second source of radar DEMs is single-pass spaceborne IfSAR. Under the TanDEM-X
program of Germanys DLR and the Infoterra company, the current TerraSAR-X satellite has
been joined in space by a second X-band SAR unit. With the orbits of the two satellites being
tightly controlled, single-pass IfSAR operation is possible, as is vegetation removal using
new techniques for polarmetric radar interferometry. The intended elevation model product
from TanDEM-X is a global DTED3 DEM of 12m post spacing and 2m vertical accuracy. As
at late May 2011, full commercial operation of TanDEM-X had not commenced, though
initial results from the system are reported as being very encouraging. From the standpoint of
a DEM generation system that can economically provide 2m accuracy elevation models of
Australia to horizontal resolutions of 10m, TanDEM-X has considerable potential.
4. Project Work Plan
Shown in Figure 2 is the workflow designed for the DEM analysis. Given that a main focus
of this analysis is upon DEM accuracy, it is useful to keep in mind that the DEMs being
compared have accuracy ranges that differ by more than an order of magnitude. Recall that
nominal vertical resolutions of the DEMS are: approximately 5-15m for SRTM and SPOT5
HRS, 3-5m for the 1:25000 Topo DEM (referred to here as the Topo DEM), 0.51m for
airborne IfSAR and the ADS40 DEM, and 15cm for LiDAR. Thus, the principal aim of the
analysis to be conducted is to better characterize the performance of these different DEM
technologies within a typical Australian coastal environment, rather than to reinforce well-
recognised differences in resolution and accuracy.
Also shown in Figure 2 is the work flow adopted for the production of the reference LiDAR
DEM from the measured mass points. Automated classification and filtering was based upon
analysis of the multiple-pulse returns, after which interpolation was adopted to generate the
final grid of 2m horizontal spacing.
-
7/30/2019 digital terrains
10/42
10
As can be seen from Figure 2, the initial step in the accuracy assessment and analysis of
differently sourced DEMs of varying resolution against the reference dataset, which is taken
to be LiDAR data, involves bringing all DEM datasets into a uniform reference coordinate
system. Especially important is uniformity within the height datum. All current DEM data
acquisition technologies utilize GPS for absolute positioning and consequently the DEM
datum is initially referenced to the WGS84 ellipsoid. A height conversion from ellipsoidal toorthometric is then carried out using both geoid height information from AusGEOID09 and,
where applicable and if known, the local relationship between AusGEOID09 and the
Australian Height Datum (AHD) to facilitate a transformation of the DEM to AHD. In the
conversion of height data recorded in the kinematic GPS survey conducted as part of the
project, AusGEOID09 was employed to facilitate a one-step WGS84-to-AHD reference
datum conversion.
It is noteworthy that there can be discrepancies in actual local MSL and AHD amounting to
70cm or more as a consequence of sea-surface topography. However, localized distortions in
AHD will have no significant impact in the accuracy analysis for two reasons. Firstly, height
differences are being determined, which nullifies the effect of absolute biases in the datum, at
least when all DEMs are nominally referenced to AHD. Secondly, the anticipated localizedMSL versus AHD biases can be anticipated to be very small in relation to the overall error
budget for all DEM data other than the LiDAR reference data.
Figure 2. Project workflow.
In order to compare height values from different DEMs at specific positions, interpolation is
needed because of the multiple horizontal resolutions (post spacings) involved. Within the
current project the principle adopted is that the interpolation should occur in the higher
-
7/30/2019 digital terrains
11/42
11
resolution DEM. Thus, height comparisons require interpolation within the 2m horizontal
resolution LiDAR DEM, this interpolation being bilinear as opposed to bicubic, in order to
minimize smoothing effects. A result of this approach is that the number of sample points
will vary proportionally to the horizontal resolution of the DEM being compared to the
LiDAR reference data.
In accordance with the different height resolutions of the DEMs being considered, differentlevels of initial artefact removal and filtering have been applied in the DSM-to-DEM
conversion, with automated processes alone being largely relied upon. It is important to keep
in mind that the characteristics of both the underlying terrain and the particular sensor
technology will dictate the degree of complexity of the DSM-to-DEM process. Issues
include, for example, the fact that photogrammetry techniques beneficially support manual
artefact removal in a visual 3D environment, whereas removal of above ground features in
LiDAR DSMs is greatly aided by both the high density and vertical resolution of the mass
points and the provision of multiple returns (ranges) which allow penetration of the
vegetation layer. Also, IfSAR DSMs can be accompanied by intensity images that support
stereo visual interpretation to aid in the DSM-to-DEM conversion.
5. Test Area Locations
Two criteria governed selection of geographic location for the DEM analysis: 1) suitability in
the context of overall assessment of coastal zone vulnerability to climate change; and 2)
availability of elevation model coverage from as many data acquisition sources as possible.
Fulfilment of the latter criterion turned out to be the factor that most influenced the selection
of test area locations, since the choice was essentially limited to the mid north coast of NSW,
where there had been recent production of medium- and high-resolution DEMs from airborne
IfSAR, LiDAR and photogrammetry (from ADS40 aerial imagery). Moreover, there was
coverage from SRTM, 1:25000 topographic mapping and SPOT5 HRS.
Following the selection of the general test area based on data availability, it was necessary to
select specific test sites, which in combination fulfilled the following requirements:
1) Coastal zone with mixed vegetation, ranging from grassland to scrubland and forest.
2) Topographic variation, ranging from floodplains, to undulating low-level coastal sanddunes to low- and medium level hills.
3) Variation in landcover, from urban to rural to bushland and forests.
4) Containing extensive areas below 10m elevation and open to the coastline.
A principal aim of the project was to assess the influence of both man-made structures in an
urban environment, and different land and vegetation cover, on the accuracy and integrity of
bare-earth DEMs. Although there have been a number of published reports on the
performance of different DEM generation techniques in different topography, especially as afunction of ground slope, this factor has only been briefly analysed here. The reasons for this
are, firstly, that by its very nature the vulnerable coastal zone is low-lying, with only mild
topographic variation; and, secondly, the metadata necessary to comprehensively consider
slope and aspect for the IfSAR, ADS40 and LiDAR were not available. The analysis was thus
limited to gridded DEM data only.
Shown in Figure 3 are the four selected test areas: Area 1 (128 km2) extends from South West
Rocks to the Stuarts Point/Grassy Head area and comprises varied coastal topography and
-
7/30/2019 digital terrains
12/42
12
vegetation cover. Area 2 (76 km2), which is centred on the town of Kempsey, constitutes the
sample low-lying urban area. Area 3 (24 km2) covers Crescent Head and this was selected
based on the varying terrain of the headland. Area 4 (72 km2
) was added to the initial three in
order to provide further coverage of dense coastal forest areas, as well as an additional urban
area, namely the settlement of Scotts Head. The DEMs within each of the test areas are
shown in Figure 4, and Figure 5 highlights the areas below 10m elevation within each of the
four test sites.
Figure 3: Test areas, with locations shown for 9 permanent survey marks used as GPS checkpoints.
Area 1
Area 3
Area 2
Area 4
10 km
-
7/30/2019 digital terrains
13/42
13
(a) Area 1 (b) Area 2
(c) Area 3 (d) Area 4
Figure 4: LiDAR DEMs for each test area.
-
7/30/2019 digital terrains
14/42
14
(a) Area 1 (b) Area 2
(c) Area 3 (d) Area 4
Figure 5: Areas below 10m elevation (black areas are >10m or outside area).
-
7/30/2019 digital terrains
15/42
15
6. Specifications of DEM Datasets
Shown in Table 1 is a summary of the specification for the different DEM datasets employed
in the project. With the exception of the 1-second SRTM and SPOT5 data, which were made
available by Geoscience Australia (GA) , all DEM data was kindly provided to the project by
the Land and Property Management Authority (LPMA) of the NSW Department of Lands.
The project is indebted to LPMA and GA for this support, which was crucial to realization of
the project objectives.
Table 1. Specifications of DEM Datasets and GPS survey data.
Dataset Technology Data format
Horizontal
accuracy
(RMSExy)
Vertical
accuracy
(RMSEz)
SPOT5 DEMSpace
photogrammetry
ESRI binary
30m grid
(.ADF)
10m 5-10m
ADS40 DSM
Aerial
photogrammetry
(50 cm GSD)
ERDAS
8 m grid
(.IMG)
0.5m 0.5-1m
Airborne IfSAR
DEM
Intermap STAR 3
& 4 IfSAR
5 m grid
(.BIL)1.5m 0.5 - 1m
SRTM DEM Space-borne IfSAR
ESRI binary
30m grid
(.ADF)
7m 6-12m
LiDAR mass pointsAirborne Laser
Scanning
2m grid
(.LAS)0.3m 0.15m
Topo DEM
Aerial
photogrammetry,
1:25000 mapping
ERDAS 25m
Grid (.IMG)6m 3m
Ground check points Kinematic GPS ASCII 0.03m 0.03m
7. Benchmark Elevation Data
7.1 Permanent Survey Marks
The reference elevation model against which DEMs from different data sources are compared
is taken as the LiDAR DEM. In order to assess the quality of the LiDAR standard against
ground survey data that is directly referenced to AHD to a nominal accuracy of better than
10cm, surveyed benchmark data within the test area was accessed. The elevations of nine
benchmarks, the locations of which are indicated in Figure 3, were used in a comparison of
GPS-derived AHD heights versus those of the permanent survey marks. It had originally been
intended to employ additional benchmarks as ground checkpoints, however time constraintsand difficulties imposed in locating the permanent survey marks beyond township areas
meant that the number of checkpoints was restricted to nine. This number would be sufficient
to indicate the presence of any localized biases in the AHD reference system that were not
modeled via the AusGEOID09 Geoid model.
-
7/30/2019 digital terrains
16/42
16
7.2 GPS Survey of Height Profiles
Prior to the adoption of airborne LiDAR as the highest accuracy master elevation data set
against which other DEM generation technologies are compared, it was necessary to validate
the absolute accuracy of the LiDAR DEM. This is by no means a simple matter in practise,
since the only available basis for comparison is elevation data acquired from ground surveys,
either via GPS or standard surveying techniques of spirit or trigonometric levelling. As willbe explained in a following section, there are practical limitations to utilization of thinly
distributed benchmark and permanent survey mark data as an accurate base against which to
assess LiDAR DEMs. Not only are there uncertainties of several cm in the height relationship
between the ellipsoidal WGS84 and AHD reference systems, but there is also the inherent
accuracy limitation, again several cm, of the ground surveyed elevations.
The only feasible approach for assessing the absolute accuracy of LiDAR DEM data covering
the UDEM test areas is through the provision of GPS surveyed bare-earth elevations. The
most practical way of acquiring such data is through the use of real-time kinematic GPS
(RTKGPS) surveying where a GPS receiver is mounted in a vehicle and 3D positions to an
accuracy of a few cm are determined through the use of either a nearby radio-linked base
station or a CORS network. For the present project, RTKGPS surveys were conducted in five
areas: Scotts Head, Stuarts Point, South West Rocks, Kempsey and Crescent Head.
The surveyed height profiles were mainly restricted to areas in or near townships, for two
reasons. Firstly, the mode of operation was to utilize a base station that broadcast corrections
to the vehicle-borne roving receiver via a radio link, and the effective maximum distance for
radio reception was about 4km depending upon topography. Secondly, beyond townships,
roads tended to be covered by overhanging trees, which blocked reception of the GPS
signals. This accounts for most of the broken height profiles shown in Figures 6-11.
Notwithstanding these shortcomings, some 27,000 elevation readings at generally 3-5m
intervals were made to 2-4cm accuracy over the roads indicated in the figures. One benefit of
being restricted to open roadways was that heights to the same points would have been
readily recorded within the LiDAR survey. An illustration of the problems posed by
vegetation in the RTKGPS surveys is provided in Figure 12, which shows favourable and
unfavourable areas for data collection within the Stuarts Point area. The vegetation cover is
indicative of most of the native forest areas within the region, with only the low-lying test
area around Kempsey (Area 2) being largely free of forest cover.
7.3 Comparison of GPS and Ground Survey Elevations
In order to ascertain the absolute accuracy of the LiDAR DEM data, comparisons were to be
made with the elevation data recorded within the vehicle-borne Real-Time Kinematic GPS
(RTKGPS) Survey. Both technologies yield elevations, in the first instance, within an
ellipsoidal height reference system, namely WGS84. In this sense, discrepancies between the
RTKGPS heights and those determined from LiDAR yield an indication of the accuracy ofthe LiDAR system free of the effects of uncertainty in the relationship between the ellipsoidal
and the orthometric height datums. For the conversion of both LiDAR and GPS surveyed
heights to elevations referenced to AHD, it is necessary to apply a geoid correction, in this
case via the AUSGeoid09 correction model. GPS surveyed AHD heights can then be directly
compared to elevations of benchmarks (BMs) and permanent survey marks (PMs), which
have traditionally been established via spirit leveling.
-
7/30/2019 digital terrains
17/42
17
Figure 6. Elevation profiles recorded by real-time kinematic GPS in the four test areas.
Figure 7. Elevation profiles recorded by kinematic GPS in Scotts Head (Area 4).
Scotts Head
SW Rocks
Stuarts Point
Kempsey
Crescent Head
-
7/30/2019 digital terrains
18/42
18
Figure 8. Elevation profiles recorded by kinematic GPS in Stuarts Point (Area 4).
Figure 9. Elevation profiles recorded by kinematic GPS in South West Rocks (Area 1).
Figure 10. Elevation profiles recorded by kinematic GPS in Kempsey (Area 2).
-
7/30/2019 digital terrains
19/42
19
Figure 11. Elevation profiles recorded by kinematic GPS in Crescent Head (Area 3).
Figure 12. Constraints on kinematic GPS surveying: unfavourable vegetation conditions (left) and generally
favourable conditions (right).
A principal cause of discrepancies between GPS surveyed AHD heights and those for
BMs/PMs can be anticipated to be localized biases in Geoid modeling. In the case of the test
areas considered, the geoid correction value N varies by 1.1m over the 50km from Crescent
Head to Scotts Head, from 30.7m to 31.8m, and by 0.4m over the 20km from Crescent Head
to Kempsey. This fact, coupled with the anticipated accuracy (95% confidence) level of only
5-8cm for ground surveyed BM/PM elevations suggests that RMS discrepancies in the order
of 10cm might well be expected between GPS and ground surveyed elevations.
Table 2 lists the results of the GPS to BM/PM height data comparison for nine survey marks
in the Kempsey, Scotts Head and Crescent Head areas. The overall RMS value of height
discrepancies is 9.4cm and it is noteworthy that there is a systematic trend in the discrepancy
values at two of the locations. These are indicative of either one or two factors: firstly,
localized biases in AUSGeoid09 or, secondly, systematic errors in the BM/PM data. Either
way, the results suggest that in order to independently ascertain the accuracy of the LiDAR
data, it is more appropriate to use RTK GPS heights rather than benchmark data.
-
7/30/2019 digital terrains
20/42
20
For the purposes of this study it suffices to note that the level of agreement between BM and
PM data and RTKGPS is of a similar magnitude to the 1-sigma elevation accuracy
anticipated from airborne LiDAR data, namely 10-15cm. Subsequent comparisons of the
LiDAR DEM heights to ground surveyed data will utilize only RTKGPS data.
Table 2. Comparison between GPS surveyed and published elevations for nine benchmarks/permanent survey
marks (units are metres).
7.4 GPS Heighting versus LiDAR DEMsThe LiDAR DEM has been adopted as the reference DEM in view of its significantly higher
accuracy, and generally also resolution, as compared to the other DEM generation
technologies. In order to validate, as far as was practical, the absolute accuracy of the LiDAR
derived elevations, a comparison with the profiles of RTKGPS data described above was
conducted. Within this process, some 27,000 individual RTKGPS height measurements were
compared to elevations interpolated from the gridded LiDAR DEM via bilinear interpolation.
The resulting discrepancies in elevation are summarized in Tables 3 and 4, where the
heighting bias of LiDAR (-ve value indicates higher LiDAR elevation), the RMS
discrepancy, the bias-free standard deviation of the discrepancies H and the size of the
sample within each of the four test areas is listed. Table 3 shows results when all RTKGPS
points are included, whereas Table 4 lists the corresponding results when height discrepancy
values of greater than three times the standard deviation (ie 99% confidence level) of H
values are omitted.
In assessing the heighting discrepancies between the RTKGPS and corresponding points
from the LiDAR DEM, it should be kept in mind that given the 2-3cm accuracy of the laser
ranging component, and the fact that both data sets were transformed from ellipsoidal to
orthometric heights via the AusGeoid09 geoid model, elevation differences will primarily be
a function of:
GPS
PointPM
Ellipsoidal
Height
from GPS
Geoid
Separation
Orthometric
height
H from
Levelling
Height of
PM from
GPS
True
height of
PM
Error in
Height
(m)
Kempsey
GPS06 PM25886 54.24 31.06 23.18 0.70 23.88 23.89 -0.01
GPS10 PM25983 39.92 31.12 8.80 -0.77 8.03 8.02 0.01
GPS16 PM26032 41.36 31.08 10.28 -0.35 9.93 9.86 0.07
Scotts Head
GPS10 PM56083 36.39 31.78 4.61 1.46 6.07 6.11 -0.04
GPS12 PM93242 61.63 31.79 29.84 -0.19 29.65 29.77 -0.12
GPS13 PM72384 45.40 31.77 13.63 0.10 13.73 13.77 -0.04
Crescent Head
GPS11 PM12869 41.66 30.75 10.91 0.04 10.95 11.07 -0.12
GPS14 PM12867 34.39 30.76 3.63 -0.09 3.54 3.64 -0.10
GPS17 PM12884 120.78 30.74 90.04 1.61 91.65 91.83 -0.18
-
7/30/2019 digital terrains
21/42
21
discrepancies in the GPS surveying of platform positions, airborne and terrestrial; and
errors in the filtering of the LiDAR data, ie in the removal of above bare-ground features.
In many respects the comparison of elevations along roadways would be expected to yield
optimal results, since the filtering issue is minimized. However, in the case of the test areas
considered, there were instances were roadside vegetation appeared to influence localized
filtering results. This issue will be addressed following a general summary of the results ofthe RTKGPS versus LiDAR comparison.
Table 3. Comparison between RTK GPS surveyed elevations and those from the LiDAR DEM; all GPS
points included (Units are metres).
Table 4. Comparison between RTK GPS surveyed elevations and those from the LiDAR DEM; GPS points
where H is greater than 3 times the standard deviation are omitted (Units are metres).
In the context of validating the LiDAR DEM via RTKGPS data, the results listed in Tables 3
and 4 are quite encouraging for two of the test areas, Areas 1 and 2, where neither the bias
value nor the standard deviation of height discrepancies is significant given the 1-sigma
accuracy of the LiDAR of around 15cm. However, the level of compatibility is less than
expected within the remaining two areas, in Area 3 because of a higher than expected positive
height bias for the LiDAR, and in Area 4 because of a high RMS discrepancy value. It is also
noteworthy in Table 4 that, for the Stuarts Point and Kempsey test fields, only 1% of
discrepancy values fell outside 3-sigma error bounds, which is consistent with a normal
distribution. The corresponding figures for rejected points (H >3 in Areas 3 and 4 are
much higher at 11% and 8%, respectively.
It is difficult to definitively establish the reasons for the larger mean LiDAR heighting bias in
Crescent Head, though preliminary analysis suggests that it may in fact be due to a
combination of both errors in the LiDAR DEM and lower than expected accuracy within the
RTKGPS data. Shown in Figure 13 are plots of the positions of RTKGPS points, with the
height discrepancy at each point being indicated by a coloured dot. White indicates within 1-
standard deviation ofH (ie within 1-sigma), blue between 1- and 2-sigma, green between 2-
and 3-sigma, and red greater than 3-sigma, the sigma values being those listed in Table 4.
Test Area
Mean elevation
discrepancy
(heighting bias)
RMS elevation
discrepancy
Std. deviation
of HNo. of points
1, Stuarts Pt 0.06 0.12 0.10 4880
2, Kempsey 0.02 0.12 0.11 10188
3, Crescent Hd -0.14 0.16 0.07 5740
4, Scotts Hd -0.09 0.24 0.22 6311
Test Area
Mean elevation
discrepancy
(heighting bias)
RMS elevation
discrepancy
Std. deviation
of HNo. of points
% of points
removed
(H >3
1, Stuarts Pt 0.06 0.10 0.07 4842 1%
2, Kempsey 0.02 0.04 0.04 10130 1%
3, Crescent Hd -0.13 0.14 0.05 5097 11%
4, Scotts Hd -0.04 0.12 0.12 5831 8%
-
7/30/2019 digital terrains
22/42
22
Note in the upper two of the three images how GPS errors are suggested by distinctly
different H values being obtained in overlapping runs of the vehicle borne GPS survey. This
is particularly apparent in the right-hand image covering a road roundabout. On the other
hand, the lower image of Figure 13 shows systematic error in a double run along the edge of
what is essentially a cliff face, and here one could infer that the heighting error is more likely
to have arisen within the LiDAR processing.
Figure 13. Sample discrepancies in LiDAR DEM versus RTKGPS elevation data, Crescent Head. White
indicates within 1-sigma; blue, 1-2 sigma; green, 2-3 sigma; and red, >3 sigma.
A further example of where the height discrepancies are more likely attributable to
shortcomings in LiDAR classification and filtering is shown in Figure 14. Note how the
discrepancies increase for a double-run RTKGPS survey exactly at the transition between an
open urban area and a heavily forested area. The elevation cross section through the LiDAR
DEM, at the position indicated by the yellow line, is also shown. A final example, which
needs no explanation, is indicated by Figure 15. This shows the error arising when the
vehicle borne GPS crosses a railway bridge, some 4-5m above the underlying DEM.
It can be difficult to accurately attribute errors in the determination of absolute elevation to
the LiDAR DEM versus the RTKGPS data. However, there is the consolation in this
investigation that height discrepancies are of a sufficiently small magnitude where they are
consistent overall with the 1-sigma vertical accuracy specification of around 15cm for the
LiDAR DEM. Given that the next highest resolution DEM to be considered has a nominal
vertical accuracy of 50cm, the LiDAR DEM can be safely taken as the benchmark against
-
7/30/2019 digital terrains
23/42
23
which to assess the remaining DEM generation technologies. Notwithstanding the acceptance
of this benchmark status, the RTKGPS versus LiDAR DEM analysis has highlighted
practical issues that still hinder the acquisition of DEMs with vertical accuracies of better
than, say, 10cm. This analysis has indicated that remaining shortcomings in the DSM-to-
DEM conversion for LiDAR data are most apparent in the classification and filtering of
vegetation as opposed to man-made, above-ground structures such as buildings.
Figure 14. Sample discrepancies in LiDAR DEM versus RTKGPS elevation data, South West Rocks. White
indicates within 1-sigma; and blue 1-2 sigma. Also shown is the cross section height profile corresponding to
the yellow line.
Figure 15. Discrepancies in LiDAR DEM versus RTKGPS elevation data at bridge crossing, Kempsey. White
indicates within 1-sigma; and red greater than 3-sigma.
8. Analysis of Different DEMs against LiDAR Reference DEM
8.1 Discrepancies in Elevation
Shown in Tables 5 and 6 are results from initial comparisons of DEMs against the LiDARstandard, for each different data acquisition technology investigated. The areas of
comparison have been restricted to those indicated in Figure 5, ie to areas with an elevation
of 10m or less, which are deemed most vulnerable to the impact of rising sea level and storm
surges. The results represent an initial summary of overall accuracy in these regions, as
quantified by both the Root Mean Square height discrepancy/Error value (RMSE) and the
estimated standard error (h), both being relative to the LiDAR DEM. The distinction
between these two measures is that the RMSE includes the error arising from systematic
height biases, whereas the h is free of the overall mean bias. Thus, h will always be equal
-
7/30/2019 digital terrains
24/42
24
to or smaller than the RMSE, with the two estimates being equal when there is no mean
height bias.
Table 5. Accuracy evaluation result against LiDAR derived reference DEM. Only height differences below
listed thresholds were included and those above removed, as per the %-removed column.
DatasetHeight
bias (m)
RMSE
(m)h(m)
Sample Size % removed
SRTM DEM (Area 1,Threshold=15m)
0.5 3.4 3.4 85740 0.06
SRTM DEM (Area 2,
Threshold=15m)-0.6 2.2 2.1 51625 0.03
SRTM DEM (Area 3,
Threshold=15m)1.9 4.1 3.6 13075 0
SRTM DEM (Area 4,
Threshold=15m)2.4 4.3 3.6 20927 0.8
SPOT5 DEM (Area 1,
Threshold=15m)4.3 5.1 2.8 79077 7.8
SPOT5 DEM (Area 2,
Threshold=15m)4.3 4.7 1.9 51443 0.4
SPOT5 DEM (Area 3,Threshold=15m)
4.8 5.5 2.8 12359 5.5
SPOT5 DEM (Area 4,
Threshold=15m) 5.3 6.0 3.0 18962 10.1Topo DEM (Area 1,
Threshold=10m)0.8 2.3 2.2 123454 0.01
Topo DEM (Area 2,Threshold=10m)
2.0 3.3 2.7 74354 0.02
Topo DEM (Area 3,Threshold=10m)
2.5 3.2 1.9 18906 0.1
Topo DEM (Area 4,Threshold=10m)
1.4 2.6 2.2 30377 0.2
IfSAR DEM (Area 1,
Threshold=5m)0.0 1.4 1.4 3038739 2.0
IfSAR DEM (Area 2,Threshold=5m)
0.1 0.8 0.8 1855859 0.3
IfSAR DEM (Area 3,
Threshold=5m)0.4 1.1 1.0 473092 0.7
IfSAR DEM (Area 4,Threshold=5m)
0.3 1.5 1.5 701487 8.7
ADS40 DSM (Area 1,Threshold=5m)
0.8 1.9 1.8 852493 29.5
ADS40 DSM (Area 2,
Threshold=5m)0.3 0.9 0.9 695044 4.4
ADS40 DSM (Area 3,
Threshold=5m)0.9 1.7 1.4 116038 37.6
ADS40 DSM (Area 4,
Threshold=5m)0.6 1.6 1.5 179993 39.9
The distinction between Tables 5 and 6 lies in the adopted threshold for classification of
particular height discrepancy values as outliers, or gross errors. These are removed from the
computation of the RMSE and standard deviation values. The outlier thresholds (cut-off
values) in Table 6 impose a tighter tolerance on data acceptance than those of Table 5, and
the different threshold values afford an indication of the extent of noise within each DEM
data set. The cut-off height discrepancy values in Table 5 were set at 15m for SRTM and
SPOT5 data, 10m for the Topo DEM and 5m for both the IfSAR and ADS40 DEMs. These
values correspond roughly to multiples of three to five times the respective standard
deviations. The area that was most noise-free, as expected, was Area 2 and the ADS40 DEM
constituted the noisiest data. Some 40% of ADS40 data points in Area 4 were classed as
-
7/30/2019 digital terrains
25/42
25
outliers, which is no doubt attributable to incomplete classification and filtering within
forested areas.
Table 6. Accuracy evaluation result against LiDAR derived reference DEM. Only height differences below
listed thresholds were included and those above removed, as per the %-removed column.
Dataset Heightbias (m)
RMSE(m)
h(m)
Sample Size % removed
SRTM DEM (Area 1,
Threshold=10m)0.4 3.2 3.2 84665 1.3
SRTM DEM (Area 2,
Threshold=10m)-0.6 2.2 2.1 51508 0.3
SRTM DEM (Area 3,Threshold=10m)
1.7 3.7 3.3 12735 2.6
SRTM DEM (Area 4,Threshold=10m)
1.9 3.5 3.0 19908 5.6
SPOT5 DEM (Area 1,Threshold=10m)
4.0 4.6 2.3 75763 11.7
SPOT5 DEM (Area 2,
Threshold=10m)4.2 4.5 1.7 50700 1.8
SPOT5 DEM (Area 3,
Threshold=10m) 4.3 4.8 2.1 11655 10.9SPOT5 DEM (Area 4,
Threshold=10m)4.6 5.1 2.1 17325 17.8
Topo DEM (Area 1,
Threshold=5m)0.7 2.2 2.0 119997 2.8
Topo DEM (Area 2,
Threshold=5m)1.3 2.5 2.2 62730 15.7
Topo DEM (Area 3,
Threshold=5m)2.3 2.8 1.6 17461 7.7
Topo DEM (Area 4,
Threshold=5m)1.1 2.2 1.9 28473 6.5
IfSAR DEM (Area 1,
Threshold=3m)0.0 1.1 1.1 2889079 6.8
IfSAR DEM (Area 2,
Threshold=3m)0.1 0.7 0.7 1828944 1.7
IfSAR DEM (Area 3,Threshold=3m)
0.4 0.9 0.8 458564 3.7
IfSAR DEM (Area 4,
Threshold=3m)0.1 1.1 1.1 651056 15.2
ADS40 DSM (Area 1,
Threshold=3m)0.4 1.3 1.2 723999 40.1
ADS40 DSM (Area 2,
Threshold=3m)0.3 0.8 0.8 683764 6.0
ADS40 DSM (Area 3,
Threshold=3m)0.6 1.1 0.9 103057 44.5
ADS40 DSM (Area 4,
Threshold=3m)0.3 1.0 1.0 160370 46.4
A feature to note is that due to the restriction of the analysis to elevations of less than 10m,
there is limited initial consideration of DEM performance within urban environments, sincemost of the town of Kempsey, as well as significant parts Southwest Rocks, Crescent Head
and Scotts Head, all lie at elevations above 10m. DEM performance in urban areas will be
addressed in a later section of this report.
The results in Tables 5 and 6, coupled with the plots in Figures 16-20 showing height
discrepancies above given thresholds, reveal a number of characteristics, some unique to
particular DEM data acquisition technologies and others common to all. In the latter
category, findings could be briefly summarizes as follows:
-
7/30/2019 digital terrains
26/42
26
The accuracy associated with each DEM technology, as assessed via the RMSE and
h values was basically consistent with or better than suggested by specifications. In
the case of the SRTM data the RMSE values of around 2 - 4m were significantly
lower than anticipated, whereas the standard error of the SPOT5 DEM displayed
lower than expected standard error values of 2 - 3m, but a disturbing, persistent height
bias of close to 5m. The accuracy of the Topo DEM was close to specifications,namely around 3m, whereas the IfSAR and ADS40 DEMs displayed an accuracy
level in the range of 0.7m to 1.5m, which is equal to or slightly below expectations.
As anticipated, both heighting biases and height RMSE values are generally larger forAreas 1, 3 and 4 than for Area 2. The lack of forest cover in the extensive open
floodplain area around Kempsey accounts to a large degree for this characteristic,
since the positive bias effect of the DEM being in reality more of a canopy DSM in
forest areas is absent. This enhances the prospect for a better fit to the bare-earth
LiDAR DEM. It can be seen that the bias and RMSE values follow this trend for the
SRTM, IfSAR and ADS40 DEMs, but not for the SPOT5 and Topo DEMs. In the
case of the SPOT5 DEM there is a relatively uniform bias of 4-5m across all three
areas, with corresponding uniform RMSE values of 4.5-5.5m.
Also as anticipated, the distribution of RMSE values and standard errors for each caseare correlated to the presence or absence of forest. There should be an expectation that
automated DSM-to-DEM conversion will yield better results for IfSAR versus
photogrammetrically derived DEMs generated through image matching because of
the ability of radar to penetrate vegetation, at least to a moderate extent. It is
noteworthy that the mean biases for the SRTM and airborne IfSAR DEMs are 0.9m
and 0.3m, respectively. In the case of the Topo DEM, where extensive manual
filtering has been carried out, the systematic errors in DEM heights, although
influenced by the presence of forest, tend to be concentrated in a small number of
areas, as opposed to being distributed widely throughout forested regions.
Based on results obtained in the foregoing analysis, as summarized in Tables 5 and 6, the
following general summaries of DEM accuracy can be offered:
8.2 SRTM DEM
When assessed against the basic accuracy specifications for the 1-second SRTM DEM, the
achieved RMSE, standard error (1-sigma) and mean height bias values are very impressive.
Instead of finding an RMSE in the range of 6-12m, the values instead range from 2.2m for
Area 2 to 4.3m for the heavily forested Area 4. The corresponding 1-sigma values are 2.1m
and 3.6m. The number of points with height discrepancies exceeding 10m (roughly 3-sigma)
reaches 5.6% in the worst case (Area 4) and 0.3% in the best (Area 2). At a 15m or approx. 5-
sigma threshold the number of rejected points falls below 1%. A further encouraging feature
of the SRTM DEM, which can be seen for Areas 1, 3 and 4 in Figure 16, is that the
distribution of height discrepancies exceeding the 10m threshold is characterized by
concentrations in a few, mainly forested locations, with the majority of the area being free
from rejected points. It is also noteworthy that there is a concentration of outlier points both
within vegetated valley areas, which increase with increasing elevation, and along two
watercourses. Heighting blunders exceeding 15m are confined to a small number of local
vegetation clusters in Area 4. The main conclusion regarding the GA-supplied 1-second
-
7/30/2019 digital terrains
27/42
27
SRTM DEM is that within the coastal areas considered it is more accurate than specifications
would suggest, and it is free of significant height biases when assessed against RMSE values.
Area 1 Area 2
Area 3 Area4
Figure 16: Points within the SRTM DEM with height discrepancies greater than threshold values when
compared to the LiDAR reference DEM. Red areas representing a 10m threshold are overlaid by blue areas
representing a 15m cutoff (areas not to scale).
-
7/30/2019 digital terrains
28/42
28
8.3 SPOT5 DEM
In the absence of height biases, DEMs generated from SPOT5 HRS imagery could be
expected to show a standard error in elevation within the range of 5-10m. It is encouraging to
see that with a point rejection threshold of 15m, the resulting standard errors for the SPOT5DEM are 3m or just under in Areas 1, 3 and 4, and just below 2m in Area 2. This is well
within specifications. Of concern, however, is the very significant height bias of over 4 - 5m
in all four test areas, which results in RMSE values ranging from 4.7 to 6.0m. One can only
speculate as to the cause of the systematic heighting error. For example, it is could arise in
large part in this case from errors in the exterior orientation of the stereo satellite imagery,
perhaps as a consequence of insufficient or inaccurate ground control within the block
adjustment process. Alternatively, it might be attributable to shortcomings in the filtering of
vegetation within the DSM-to-DEM conversion. The latter assumption is supported to some
degree by the percentages of the rejected points where the height error exceeded a 15m
threshold, there being over 8% in Area 1, 6% in Area 3, 10% in Area 4, and a predictably
lower 0.4% in Area 2, which is largely devoid of forest cover. The rejections grow to greater
than 10% in Areas 1 and 3, and to 18% in Area 4, when the threshold is reduced to 10m, withthe distribution of the rejected points being shown in Figure 17. The rejected points are
concentrated mainly in areas of dense coastal forest. Initial indications are that whereas the
precision of relative heights is within specifications for SPOT5 data, the DEM exhibits
degraded accuracy due to the presence of significant height biases, even in the absence of
vegetation.
8.4 Topo DEM (from 1:25,000 map data)
The vertical accuracy specification typically associated with 1:25,000 topographic mapping is
3m, corresponding to a third of the contour interval of 10m. Initial expectations for the Topo
DEM would then be an RMSE at the 3m level, with localized occurrences of height biases as
opposed to the area wide bias seen in the SPOT5 DEM. The mean height biases obtained forthe Topo DEM, with a 10m removal threshold for height discrepancies, were 0.8m in Area 1,
2m in Area 2, 2.5m in Area 3 and 1.4m in Area 4. While the biases in Areas 2 and 3 are
higher than one would anticipate for a 3m-accurate DEM, they are not viewed as significant
given the corresponding 1-sigma values, which had a range of 1.9 - 2.7m. The number of
points with height discrepancies greater than the 10m cutoff (nominal 3-sigma value) was
0.2% or less for all four areas. This is consistent with the expectation that the Topo DEM
should have fewer filtering errors and thus fewer %-removals because of the map compilation
process being based on manual stereoplotting from aerial photography. The higher %-
removal values shown for a 5m cutoff in Table 6 can be discounted somewhat because the
threshold is set too tight at only 2-sigma, but it is nevertheless interesting that the points
removed are concentrated in localized, mainly forested areas, as shown in Figure 18.
8.5 Airborne IfSAR DEM
With the relatively coarse rejection threshold value of 5m or approximately 5-sigma assigned
to the airborne IfSAR DEM, resulting RMSE values were 1.4m in Area 1, 0.8m in Area 2,
1.1m in Area 3 and 1.5m in Area 4. The corresponding 1-sigma values were basically the
same as a consequence of the modest bias values of 0.5m or less. Unlike the three lower
resolution DEMs discussed above, the attained accuracy of the IfSAR DEM was not well
within specifications.
-
7/30/2019 digital terrains
29/42
29
Area 1 Area 2
Area 3 Area4
Figure 17: Points within the SPOT5 DEM with height discrepancies greater than threshold values when
compared to the LiDAR reference DEM. Red areas representing a 10m threshold are overlaid by blue areas
representing a 15m cutoff (areas not to scale).
-
7/30/2019 digital terrains
30/42
30
Area 1 Area 2
Area 3 Area4
-
7/30/2019 digital terrains
31/42
31
Figure 18: Points within the Topo DEM (1:25,000 map data) with height discrepancies greater than threshold
values when compared to the LiDAR reference DEM. Red areas representing a 5m threshold are overlaid by
blue areas representing a 10m cutoff (areas not to scale).
Instead, the accuracy was generally consistent with expectations and even a little worse than
anticipated. The accuracy indicators of RMSE and standard error changed marginally when
the rejection threshold was lowered from 5m to 3m, and significantly more points were
removed. The %-removal values climbed to 7% and 15% in Areas 1 and 4, respectively, and
to 2% and 4%, respectively, for Areas 2 and 3. As can be seen in Figure 19, the regions with
most rejected points correspond to hilly terrain with steeper slopes, and to a lesser extent to
forested areas. Generally speaking, the results obtained with the IfSAR DEM were in
accordance with accuracy expectations, with the technology performing best in low lying
areas.
8.6 ADS40 DEM
The DEM derived from ADS40 digital 3-line scanner aerial imagery was in fact a
smoothed DSM that had undergone some initial automated classification and filtering. The
first indication of the partial filtering of the ADS40 DSM is indicated in Figure 20, where it
can be seen that the majority of the elevations within forested areas were rejected as outliers,
their associated discrepancy values against the LiDAR data being greater than 5m or roughly5-sigma. Some 40% of the height discrepancy values in Area 4 were rejected. The
assumption that the RMSE values were inflated by an incomplete DSM-to-DEM conversion
is reinforced by the results of the mostly forest free Area 2, where the RMSE value for the
5m threshold falls from the near 2m level of Areas 1, 3 and 4 to 1m, and the %-removal value
drops from 30% or more to 4%. The height bias for Area 2 is also reduced to 0.3m from
closer to 1m for the remaining areas. Given the incomplete filtering, it is difficult to
characterize the accuracy of the ADS40 DEM (actually DSM), but it is encouraging to see
results in Area 2 which are consistent with accuracy specifications, ie an RMSE value of less
than 1m.
9. Impact of Land Cover on DEM Accuracy
Based on the results obtained in the analysis of performance of the five DEMs against the
LIDAR reference DEM, it is apparent that a significant factor limiting vertical accuracy in
the generation of supposedly bare-earth DEMs is the automated classification and filtering in
forest and urban areas, with vegetation cover appearing as a more significant issue than the
presence of buildings and other man-made structures. In order to gain further insight into the
impact of different land cover on the DEM technologies considered, analyses were carried
out for samples of four specific land cover types: urban, forest/bushland, open farm land, and
mixed coastal cover of vegetated dunes and housing. Once again, elevation bias, RMSE and
standard error of height discrepancies were quantified using the LIDAR data as the reference
DEM.
9.1 Urban AreasFigure 21 shows three sample urban areas: (a) a part of the coastal settlement of Scotts
Head (taken from Test Area 4), (b) the commercial centre of South West Rocks (Area 1), and
(c) a low-lying residential area of West Kempsey (Area 2). The results of the analysis for
these three test sites are shown in Table 7, which has the same structure as the earlier Tables
5 and 6.
-
7/30/2019 digital terrains
32/42
32
Area 1 Area 2
Area 3 Area4
Figure 19: Points within the airborne IfSAR DEM with height discrepancies greater than threshold values when
compared to the LiDAR reference DEM. Red areas representing a 3m threshold are overlaid by blue areas
representing a 5m cutoff (areas not to scale).
-
7/30/2019 digital terrains
33/42
33
Area 1 Area 2
Area 3 Area4
Figure 20: Points within the ADS40 DEM with height discrepancies greater than threshold values when
compared to the LiDAR reference DEM. Red areas representing a 3m threshold are overlaid by blue areas
representing a 5m cutoff (areas not to scale).
-
7/30/2019 digital terrains
34/42
34
(a)
(b)
Table 7. Accuracy evaluation result against LiDAR derived reference DEM for three Urban Test Areas. Only height
differences below listed thresholds were included and those above removed, as per the %-removed column. Sample labels
correspond with those in Figure 21.
DatasetHeight
bias (m)
RMSE
(m)h(m)
Sample Size % removed
SRTM DEM (Area a,Threshold=15m)
1.1 2.5 2.3 273 0
SRTM DEM (Area b,
Threshold=15m)1.4 2.9 2.5 150 0
SRTM DEM (Area c,
Threshold=15m)1.1 1.9 1.5 403 0
SPOT5 DEM (Area a,Threshold=15m)
6.1 6.6 2.6 273 0
SPOT5 DEM (Area b,
Threshold=15m)7.2 7.7 3 150 0
SPOT5 DEM (Area c,
Threshold=15m)6.7 6.9 1.6 403 0
TopoDEM (Area a,Threshold=10m)
-0.8 3.8 3.7 243 11
Topo DEM (Area b,Threshold=10m)
-1.7 3 2.5 150 0
Topo DEM (Area c,
Threshold=10m)0.1 1.9 1.9 403 0
IfSAR DEM (Area a,
Threshold=5m) 0.3 1.6 1.5 10043 0.9IfSAR DEM (Area b,
Threshold=5m)-0.8 1.4 1.2 5512 0.1
IfSAR DEM (Area c,
Threshold=5m)-1 1.4 1 14800 0
ADS40 DSM (Area a,Threshold=5m)
0 1.3 1.3 3828 1.1
ADS40 DSM (Area b,
Threshold=5m)-0.4 1.2 1.1 2135 0.5
ADS40 DSM (Area c,
Threshold=5m)0.5 1 0.9 5725 0.4
(c)
Figure 21: Urban test areas, (a) Scotts Head, (b)
South West Rocks and (c) West Kempsey.
-
7/30/2019 digital terrains
35/42
35
The first feature of note in Table 7 is that for the SRTM and SPOT5 DEMs, the bias value
has increased over that listed in Tables 5 and 6. In the case of SRTM, it is safe to assume that
this is attributable to an incomplete removal of buildings in the DSM-to-DEM conversion.
The South West Rocks town centre, Figure 21b, is characterized by buildings taller than a
single story and it is thus not unexpected to see a more significant bias being present. The
bias value for SPOT5, at between 6m and 7m, is not at all consistent with a shortcoming in
building classification and filtering. Instead, it is a gross positive height error likelyattributable to a failure to utilize local ground control in the exterior orientation determination
for the HRS imagery. Upon compensation for the bias, both SRTM and SPOT5 yield
standard errors of height discrepancies in the range of 1.5m to 3m.
The results achieved for the three urban areas for the Topo, IfSAR and ADS40 DEMs show
an overall reduction in height bias, which is indicative of a more successful filtering of
buildings in the automated DSM-to-DEM conversion. In terms of accuracy, the RMSE values
obtained are largely consistent with those obtained in the full-area evaluations.
9.2 Open Rural Areas
Figure 22 shows the three selected open rural area sites: (a) open grassland with thinlydistributed houses and trees in West Kempsey, (b) open fields near Yarrahappini and (c)
ploughed fields south of Stuarts Point. The first two areas are gently undulating, while the
third is flat. The results of the analysis for these test sites are shown in Table 8, where it can
be immediately seen that the DEM accuracy improves significantly when the need for
extensive filtering is removed from the DSM-to-DEM transformation.
(a)
(b)
Shortcomings in the DSM filtering required in the area shown in Figure 21a, which
comprises a relatively small number of houses and trees, is enough to significantly inflate the
RMSE value compared to that for the bare-ground areas of Figs. 21b and 21c, for all five
(c)
Figure 22: Open rural test areas, (a) West
Kempsey, (b) Yarrahappini and (c) Stuarts Point
-
7/30/2019 digital terrains
36/42
36
DEMs. In the open areas, the accuracy of SRTM, as expressed through the RMSE, is better
than 1m, and the corresponding values for the IfSAR and ADS40 DEMs are between 0.4m
and 0.8m. The absolute accuracy for all DEMs is within specifications for all three test sites.
Table 8. Accuracy evaluation result against LiDAR derived reference DEM for three Open Rural Test Areas.
Only height differences below listed thresholds were included and those above removed, as per the %-removed
column. Sample labels correspond with those in Figure 22.
DatasetHeight
bias (m)RMSE
(m)h(m)
Sample Size % removed
SRTM DEM (Area a,Threshold=10m)
-1.6 2.3 1.7 703 0
SRTM DEM (Area b,Threshold=10m)
-0.4 0.6 0.5 374 0
SRTM DEM (Area c,Threshold=10m)
0.7 0.9 0.6 286 0
SPOT5 DEM (Area a,Threshold=10m)
4.2 4.4 1.5 702 0.1
SPOT5 DEM (Area b,
Threshold=10m)3.3 3.5 1.2 374 0
SPOT5 DEM (Area c,
Threshold=10m)
3.3 3.4 1 286 0
Topo DEM (Area a,Threshold=5m)
0.2 1.6 1.6 691 1.7
Topo DEM (Area b,
Threshold=5m)0.8 0.9 0.6 374 0
Topo DEM (Area c,
Threshold=5m)-2.3 2.4 0.4 286 0
IfSAR DEM (Area a,
Threshold=3m)-0.2 0.6 0.6 25087 1.3
IfSAR DEM (Area b,
Threshold=3m)-0.1 0.4 0.4 13802 0
IfSAR DEM (Area c,Threshold=3m)
-0.6 0.8 0.6 9726 0.3
ADS40 DSM (Area a,Threshold=3m)
0.9 1 0.6 9860 0.1
ADS40 DSM (Area b,Threshold=3m) 0.2 0.3 0.2 5376 0
ADS40 DSM (Area c,
Threshold=3m)-0.3 0.7 0.6 3854 0
Given that the cultivated area shown in Figure 21c was likely bushland at the time the Topo
DEM was produced, the probable reason for the bias figure of -2.3m is land clearing and
subsequent earthworks to create the cultivated fields. Also exhibiting a large positive bias is,
once again, the SPOT5 DEM. Given the largely insignificant height biases and RMSE values
that are within specifications, it is not surprising to see so few points classified as outliers,
with virtually all of these being found in the DEMs covering the scene with houses and trees.
9.3 Forest/Bushland AreasFigure 23 shows the three selected forest/bushland sites: (a) Dense tall (>10m) eucalypt forest
at Yarrahappini, (b) Tall forest near Grassy Head and (c) scrubland covering a coastal dune at
Stuarts Point, including an area of mangroves. The results of the analysis for these test sites
are shown in Table 9. The table indicates a number of interesting features worthy of note.
Firstly, in the heavily forested area, Figure 23a, the accuracy of the SPOT5 DEM is no better
than 10m in absolute terms. Indeed, it can be seen that some 91% of the sample points are
-
7/30/2019 digital terrains
37/42
37
rejected as outliers, meaning they are in error by more than 10m, the cause no doubt being a
combination of the already referred to exterior orientation bias and an inadequate removal of
vegetation from the DSM.
(a) (b) (c)
Figure 23: Forest/Bushland test areas (a) Yarrahappini, (b) Grassy Head Road, and (c) Stuarts Point Beach.
Table 9. Accuracy evaluation result against LiDAR derived reference DEM for three forest/bushland areas.
Only height differences below listed thresholds were included and those above removed, as per the %-removed
column. Sample labels correspond with those in Figure 23.
DatasetHeight
bias (m)
RMSE
(m)h(m)
Sample Size % removed
SRTM DEM ( Area a,
Threshold=15m)2.2 2.9 1.8 240 0
SRTM DEM (Area b,
Threshold=15m)2.2 2.2 0.5 60 0
SRTM DEM (Area c,
Threshold=15m)-2.3 2.7 1.4 395 0
SPOT5 DEM (Area a,
Threshold=15m)9.7 10.2 3.3 21 91
SPOT5 DEM (Area b,
Threshold=15m)4 4.1 1.2 60 0
SPOT5 DEM (Area c,
Threshold=15m)4.3 4.8 2 395 0
Topo DEM (Area a,Threshold=10m)
1.5 1.6 0.5 240 0
Topo DEM (Area b,
Threshold=10m)-3.2 3.2 0.4 60 0
Topo DEM (Area c,Threshold=10m)
-0.4 1.3 1.3 395 0
IfSAR DEM (Area a,Threshold=5m)
0.4 0.5 0.2 8910 0
IfSAR DEM (Area b,Threshold=5m)
1.5 1.7 0.7 1888 0
IfSAR DEM (Area c,Threshold=5m)
-0.8 1.4 1.2 15017 0.8
ADS40 DSM (Area a,Threshold=5m)
3.9 4 0.9 5 99.9
ADS40 DSM (Area b,
Threshold=5m)1.8 2.4 1.6 529 28
ADS40 DSM (Area c,
Threshold=5m)0.1 1.7 1.7 5907 0.7
Another DEM showing a high bias value in thick forest was that from ADS40 imagery,
though this was to be anticipated given the low level of filtering undertaken with this data.
Contrasting to the poor accuracy of the SPOT5 and ADS40 DEMs is the result for the
airborne IfSAR DEM in the same area, where agreement with the LIDAR DEM is 0.5m
RMS.
-
7/30/2019 digital terrains
38/42
38
Also noteworthy in Table 9 are the negative height biases of the SRTM DEM in the coastal
scrubland and mangrove environment of Figure 23c and the Topo DEM in the tall bushland
of Figure 23b. Neither systematic error is immediately explainable. Overall, the results for the
forested areas are consistent with expectations, namely that the RMSE is higher than
specifications for the DEM technologies would suggest, with the achievable accuracy being
inversely proportional to vegetation density.
9.4 Mixed Coastal Land Cover
The final land cover type sampled could be characterized as mixed coastal dunes, scrubland,
bush and built-up urban area. The chosen test area shown in Figure 24 is representative of
much of the low-lying coastal environment along Australias eastern seaboard that is
vulnerable to sea level rise and storm surges.
Figure 24: Coastal area of mixed land cover, Arakoon.
The results shown in Table 10 show largely the same characteristics as those presented in
Table 5 for the full test areas. The RMSE of the SRTM elevations is a commendable 3m,
with a modest bias, whereas the SPOT5 elevations show an RMSE of 6m, largely due to the
now quite familiar large bias of also close to 6m. The Topo DEM also has a larger than
expected bias given that the test area is right on the coast, with its RMSE value being
marginally higher than expected. Both the ADS40 and IfSAR DEMs have small biases,
though RMSE values which are outside specifications by approximately 0.7m. Given these
results it is observed that the particular combination of land cover types does not reveal any
distinctive performance characteristics which might not be apparent in the data covering the
broader test areas.
Table 10. Accuracy evaluation result against LiDAR derived reference DEM for coastal area of mixed land
cover, as shown in Fig. 24. Only height differences below listed thresholds were included and those above
removed, as per the %-removed column.
DatasetHeight
bias (m)
RMSE
(m)h(m)
Sample Size % removed
Smoothed SRTM DEM
(Sample 1, Threshold=15m) -1.2 2.9 2.6 244 0
SPOT5 DEM (Sample 1,
Threshold=15m)5.6 6 2.1 233 4.51
Topo DEM (Sample 1,
Threshold=10m)2.7 3.7 2.5 244 0
IfSAR DEM (Sample 1,
Threshold=5m)0.2 1.8 1.7 8557 9.39
Smoothed ADS40 DSM
(Sample 1, Threshold=5m)0.1 1.7 1.7 3281 10.87
-
7/30/2019 digital terrains
39/42
39
10. Influence of Terrain Slope
It is well established that the performance of DEM generation technologies is generally
degraded as a function of increasing terrain slope. Steep terrain adversely impacts especially
upon image matching in stereo photogrammetric techniques and upon radar interferometry.
Whereas, the impact of slope might not be of prime importance within low-lying coastal
topography potentially affected by sea-level rise, the current project offered a favourableopportunity to examine how different DEM technologies behaved in areas of differing
topography. Shown in Figure 25a-d are plots of the variation of RMSE values for DEM
elevations for slopes from 50
to 500
within the four test areas. The values shown represent
discrepancies between the LiDAR reference elevations and the SRTM, SPOT5, Topo,
airborne IfSAR and ADS40 DEMs.
(a) Area 1 (b) Area 2
(c) Area 3 (d) Area 4
Figure 25. Plots of RMSE values against LiDAR elevations for different DEM technologies for different terrain
slope. Error cut-off thresholds of 15m apply for the SRTM, SPOT5 and Topo DEMs, whereas a cut-off of 5m
applies to the IfSAR and ADS40 data.
The accuracy degradation with slope is clearly apparent, being most pronounced in Area 1,
which contains both the steepest and most forested terrain. Whereas the impact of forest is
not anticipated to significantly influence the character of the plots, it is interesting to note that
the mean errors, essentially heighting biases, are impacted in the cases of SRTM and SPOT5.
The mean error for SPOT5 derived elevations increases from around 4.5m in near-flat areas
to 7.5m in areas displaying slopes of between 250
and 500. The biases are considerably less
for SRTM, but a value of 5m is obtained for Area 4. Another feature of the plots is that in
addition to the airborne IfSAR and ADS40 DEMs displaying significantly higher accuracy
-
7/30/2019 digital terrains
40/42
40
than the SPOT5, SRTM and Topo DEMs, their performance is less influenced by terrain
slope, though a mild fall off in accuracy with increasing slope is apparent.
Overall, in the context of terrain modelling within low-lying coastal zones, the results of the
analysis of the influence of slope on DEM technologies has served to further emphasise that
of the five DEMs considered, only the airborne IfSAR and the ADS40 DEMs, as well as the
LiDAR reference data of course, are of sufficient accuracy and reliability. Yet, as can be seenfrom Figure 25 and from Table 11, RMSE values for IfSAR and the ADS40 DEMS are
nevertheless at a higher than desired level of 1-2m.
Table 11. RMSE values for DEMs assessed against LiDAR DEM at different ground slopes. Units are metres.
Slope category % 0-5 5-15 15-25 25-50
DEMHeight
biasRMSE
Heightbias
RMSEHeight
biasRMSE
Heightbias
RMSE
SRTM (Area 1) 0.5 3.1 1.2 4.2 1 5.6 1.7 7
SRTM (Area 2) -0.8 2.3 0 3.2 0.4 3.9 1.4 4.7
SRTM (Area 3) 1.4 3.5 2.8 4.8 0.3 5.9 0 7.1
SRTM (Area 4) 1.6 3.4 2.3 5.1 2.7 6.4 4.9 8SPOT5 (Area 1) 4.2 5.1 4.8 5.5 4.9 6 4.5 7.2
SPOT5 (Area 2) 4.2 4.6 5.1 5.4 5.5 6.1 6.5 7.4
SPOT5 (Area 3) 4.7 5.4 5.2 5.9 5.1 6.4 5.9 8
SPOT5 (Area 4) 4.9 5.7 4.9 5.8 4.9 6.4 5.5 7.7
Topo (Area 1) 0.9 2.3 0 2.7 -1.5 3.4 -1.7 4.7
Topo (Area 2) 1.2 3.6 -0.8 3.4 -1.3 3.5 -2.2 4.1
Topo (Area 3) 2.9 3.4 1.4 3 -0.3 4.2 -0.3 5
Topo (Area 4) 1.3 2.9 -0.9 3.7 -1.7 4.8 -0.8 5.1
IfSAR (Area 1) 0 1.2 0.1 1.5 -0.5 1.8 -0.9 2.1
IfSAR (Area 2) -0.2 0.9 -0.4 1.4 -0.2 1.7 -0.1 1.8
IfSAR (Area 3) 0.4 1 0.3 1.4 0 2.2 0.5 2.6
IfSAR (Area 4) 0.2 1.2 0.3 1.7 0.4 1.9 0.3 2.2
ADS40 (Area 1) 0.4 1.8 1.2 2 0.6 1.7 0.6 2.1
ADS40 (Area 2) 0.3 0.9 0.5 1.2 0.6 1.5 0.4 1.7
ADS40 (Area 3) 0.8 1.5 1 1.8 0.8 1.9 1 2.2
ADS40 (Area 4) 0.5 1.3 0.8 1.8 0.6 1.9 0.8 2.3
Conclusions
In most respects, the findings from the evaluation of the performance of DEM generation
technologies are consistent with expectations regarding both accuracy and recognisedattributes and limitations of the different DEM data sources considered. However, this project
has also revealed characteristics of the different DEMs that are perhaps not as widely
recognized, but are nevertheless important in the context of producing accurate bare-earth
DEMs of coastal terrain vulnerable to the impact of climate change.
The accuracy gap between LiDAR DEMs and those from airborne IfSAR and ADS40 aerial
photography at 50cm GSD might only be a factor of three to four according to specifications,
eg 15-25cm elevation accuracy for LiDAR versus 0.5-1m for IfSAR and the ADS40.
-
7/30/2019 digital terrains
41/42
41
However, this difference is accentuated by shortcomings in the automated classification and
filtering of both vegetation and, to a lesser extent, man-made structures within the DSM-to-
DEM conversion of the radar and photogrammetrically produced DEMs. Multiple-return
LiDAR displays significant advantages by way of last-pulse ground definition, which cannot
be matched in densely vegetated areas by radar and photogrammetry techniques, except
through skill-intensive and expensive manual editing processes.
The residual systematic elevation errors attributable to incomplete filtering of DEMs have the
potential to compromise the integrity of bare-earth elevation models in low-lying coastal
areas that are either heavily vegetated or urbanized. Such land cover accounts for the majority
of the populated coastal regions of Australia. As a consequence of the classification/filtering
issues, and to a lesser extent, the difference in vertical resolution between different DEM data
sources, it can be concluded that LiDAR is very much the preferred option for DEM
generation in coastal regions vulnerable to sea level rise and storm surges.
The results listed in Table 8 for DEM performance in open areas, largely free of trees and
buildings, highlight the fact that distinctions in DEM accuracy are as much due to different
terrain and land cover, and consequently to filtering, as to differences in basic metric
resolution of the different technologies. In the case of the open pasture (Area b), sub-metre
RMSE values were obtained for the SRTM and Topo DEMs, and the IfSAR and ADS40
DEMs showed sub-half metre RMSE values.
In regard to the five DEM generation technologies evaluated against LiDAR within the
project, each produced localized, relative vertical accuracy within specifications, and indeed
in the case of the 1-second SRTM accuracy significantly exceeded specifications. When
corrected for bias, the SPOT5 DEM also produced a relative accuracy well within
specifications, but here the bias problem was very significant, to the point where this DEM
has little utility for higher resolution terrain modeling. Elevation biases are generally
attributable to incomplete filtering of vegetation and buildings, and are therefore generally
positive in sign. In the case of the SPOT5 DEM, a 4-7m systematic elevation error was
present, irrespective of land cover. This bias likely arises due to accuracy shortcomings in the
exterior orientation determination for the SPOT5 line scanner imagery, which was performed
without the use of local ground control. The systematic error effects then flowed through to
the image matching and object point triangulation phases.
Finally, the following short summaries of the performance of each of the DEM generation
technologies in the four selected test sites on the mid north coast of NSW are offered:
The integrity if the LiDAR master DEM was validated through checks against the27,000-point kinematic GPS survey, which revealed an overall RMS height
discrepancy value of close to 0.1m. This is entirely consistent with the anticipated
RMS elevation accuracy of the LiDAR DEM of 0.15m. It must be kept in mind,
however, that all checkpoints were positioned along open roads where issues with
filtering in the DSM-to-DEM conversion do not arise. Whereas the removal ofvegetation and building from the DSM through automated classification and filtering
can be expected to be more complete with multiple-return LiDAR than with radar or
Topo DEMs, the presence of residual height errors over dense vegetation cover and
low-level man-made features can be anticipated to some extent, though there is an
absence of available tools to assess the extent of such systematic errors.
-
7/30/2019 digital terrains
42/42
The 1-second SRTM DEM, with its RMSE value range of 2.2 to 4.1m, appears to be amore accurate bare-earth elevation model than its accuracy specifications would
suggest.
The SPOT5 DEM also produces an accuracy, as quantified by an RMSE value ofaround 5m, which is inside specifications, but it displays a disturbingly high
systematic height bias averaging around 5m.
The Topo DEM derived from 1:25,000 topographic map data is internally quiteconsistent and displays an accuracy in accordance with its 3m specification. This
DEM displays localized areas of systematic height bias, in some cases due to changes
in land cover.
The airborne IfSAR DEM displays an accuracy at the high end of its anticipated 0.5 to1m range, and has optimal accuracy in low-lying areas with sparse vegetation
coverage.
Given that the ADS40 elevation model was really a smoothed, partially filtered DSM,a comprehensive DEM accuracy analysis was precluded. However, it is noteworthy
that within Area 2, which has only minor coverage of either scrubland or forest, thebetter than 1m height accuracy attained in the smoothed DSM is consistent with
accuracy specifications.
Acknowledgments
This work was funded by the Australian Governme
top related