csiro land and water science report 37/07 · science report 37/07. deriving moisture availability...
TRANSCRIPT
Tim R. McVicar, Tom G. Van Niel, Lingtao Li, Edward A. King and Randall J. Donohue
Deriving moisture availability from time series remote sensing for ecohydrological applications: Development of a prototype near real-time operational system.
CSIRO Land and Water Science Report 37/07
For further information:
CSIRO Land and WaterTim McVicarPhone: (02) 6246 5741Email: [email protected]: www.clw.csiro.au
Deriving m
oisture availability from tim
e series remote sensing C
SIRO Land and W
ater Science Report 37/07
Deriving moisture availability from time series remote sensing for ecohydrological applications:
Development of a prototype near real-time operational system.
Tim R. McVicar, Tom G. Van Niel, Lingtao Li,
Edward A. King and Randall J. Donohue
CSIRO Land and Water Science Report 37/07
June 2007
Copyright and Disclaimer © 2007 CSIRO To the extent permitted by law, all rights are reserved and no part of this publication covered by copyright may be reproduced or copied in any form or by any means except with the written permission of CSIRO. Important Disclaimer: CSIRO advises that the information contained in this publication comprises general statements based on scientific research. The reader is advised and needs to be aware that such information may be incomplete or unable to be used in any specific situation. No reliance or actions must therefore be made on that information without seeking prior expert professional, scientific and technical advice. To the extent permitted by law, CSIRO (including its employees and consultants) excludes all liability to any person for any consequences, including but not limited to all losses, damages, costs, expenses and any other compensation, arising directly or indirectly from using this publication (in part or in whole) and any information or material contained in it. Authors: Tim R. McVicar 1, Tom G. Van Niel 2, Lingtao Li 3, Edward A. King 4 and Randall J. Donohue5 1 CSIRO Land and Water and eWater CRC, GPO Box 1666, Canberra, 2601, ACT, Australia Tel.: +61-2-6246-5741, e-mail: [email protected]
2 CSIRO Land and Water and eWater CRC, Private Bag No. 5, Wembley, 6913,WA, Australia Tel.: +61-8-9333-6705, e-mail: [email protected] 3 CSIRO Land and Water and eWater CRC, GPO Box 1666, Canberra, 2601, ACT, Australia Tel.: +61-2-6246-5809, e-mail: [email protected] 4 CSIRO Marine and Atmospheric Research, GPO Box 3023, Canberra, 2601, ACT, Australia Tel.: +61-2-6246-5894, e-mail: [email protected] 5 CSIRO Land and Water and eWater CRC, GPO Box 1666, Canberra, 2601, ACT, Australia Tel.: +61-2-6246-5803, e-mail: [email protected] Cover: Time series of monthly Normalised Difference Temperature Index (NDTI) composites from April to September 1995 are shown starting from the top-left and ending at the bottom-right. When NDTI is low (shown in red) there is little moisture available for actual evapotranspiration (i.e., the environment is relatively dry) and when NDTI is high (shown in blue) moisture is freely available so actual evapotranspiration is occurring near its potential (i.e., the environment is relatively wet). Black is the ocean. For bibliographic purposes, this report may be cited as: McVicar T.R., Van Niel T.G., Li, L.T., King, E.A. and Donohue, R.J. (2007) Deriving moisture availability from time series remote sensing for ecohydrological applications: Development of a prototype near real-time operational system. CSIRO Land and Water Science Report 37/07, Canberra, Australia, 144 pp. A PDF version is available at: http://www.clw.csiro.au/publications/science/2007/sr37-07.pdf ISSN 1834-6618 ISBN 978 0 643 09505 2 (Print format) ISBN 978 0 643 09506 9 (Web format)
CSIRO Land and Water Page i
Acknowledgements This report is part of the National Heritage Trust (NHT) funded Australian Water Availability
Project (AWAP) and eWater CRC project D1. AWAP is a collaborative project between the
Bureau of Rural Sciences (BRS), Bureau of Meteorology (BoM) and CSIRO (both Land and
Water (CLW), and Marine and Atmospheric Research (CMAR)). Thanks to Drs John Sims
and David Barratt for their continued support for developing observational outputs from
optical (thermal and reflective) remote sensing suitable for use in ecohydrological
applications. Drs David Jones and William Wang (both from National Climate Centre, BoM),
answered many questions and unwaveringly supported the ideal that gridded daily
meteorological surfaces must be matched with daily error surfaces, thereby allowing error
propagation to be spatio-temporally explicit.
Dr Rob Vertessy (ex-Chief of CLW and now Chief Hydrologist BoM) advocated and strongly
supported the development of the WRON (Water Resources Observation Network)
computing facility in CLW, and without this resource the massive task of processing the
nearly 9,000 afternoon AVHRR overpasses (occupying 750 Gb of disk space) from April
1992 to current would have been insurmountable. The entire information system, including
specific time-of-day meteorological surfaces, requires approximately 2 Tb of disk space. Mr
Garry Swan, Brendan Spetch and Mick Hartcher (CLW IT specialists) provided wonderful
high-end IT support.
CSIRO Land and Water Page ii
Executive Summary The Normalised Difference Temperature Index (NDTI) is a measure of moisture availability
(ma), the ratio of actual evapotranspiration (ETa) to potential evapotranspiration (ETp). NDTI
is derived by linking meteorological point data and optical (both thermal and reflective)
remotely sensed data in a way that optimally utilises process understanding of ma and the
inherent strengths of these different data types. NDTI is generated using a ‘calculate-then-
interpolate’ approach to best use the strengths of the two primary datasets - meteorological
data are temporally dense (and spatially sparse), whereas remotely sensed data are spatially
dense (and temporally sparse). The spatial interpolation is performed using covariates that
are heavily governed by the spatially dense remotely sensed data, which allow for better
modelling of ma in-between meteorological stations (e.g., over water bodies, which would
otherwise not be well represented). Importantly, the NDTI is readily calculated across large regions; it does not require interpolated gridded precipitation or irrigation data as inputs, and therefore partly provides an independent means to monitor water application compliance. Given that ma ≅ NDTI, and that ma = ETa / ETp, then multiplying
the resultant NDTI surfaces by daily ETp surfaces provides a means to measure daily ETa
with units of mm day-1.
Fortnightly NDTI surfaces from April 1992 to 2006 for the Murray-Darling Basin, and adjacent
coastal catchments, with a spatial resolution of 1 km have been generated using almost
9,000 AVHRR (Advanced Very High Resolution Radiometer) images. To rapidly process
such a large number of files (750 Gb disk space), and to routinely process current and future
remotely sensed data from any number of possible sensors, a prototype operational system
has been developed. The system has a modular design, requires little operator input, and
automatic error checking methodologies have been implemented. The core of the NDTI is
the inversion of a 2-source resistance energy balance model (REBM) which is run at the
specific date-time of each remotely sensed overpass.
Ecohydrological assessment of interactions between the NDTI and the commonly used
Normalised Difference Vegetation Index (NDVI) for three hydrologically relevant land-uses (a
reservoir, an irrigation area, and an agriculture-forest interface) show, as expected, that the
NDTI responds more quickly to changes in water availability, than the NDVI does. This has
implications for adequately modelling soil evaporation, and evaporation from water bodies,
that NDVI-based models of ETa can not adequately account for.
CSIRO Land and Water Page iii
The prototype operational system is now ready to be applied to all Australia for the last 21
years, and to routinely process current observations. Following this, further research would
be required to assess how the improved spatial metrics from remote sensing can be used to
assess risks to ecohydrological stability and agricultural production by detailed analysis of
moisture availability (NDTI) and moisture utilization (NDVI), spatio-temporal variability, and
trends for all Australia over the last 15 years. These products, and resulting analysis, would
be suitable for distribution via on-line web-services as they could be routinely produced and
would be widely relevant to a number of water resource and environmental managers and
policy-makers. While the remotely sensed data currently used are AVHRR, any sensor with
both thermal and reflective capacity – such as MODIS (MODerate Resolution Imaging
Spectroradiometer), ETM+ (Enhanced Thematic Mapper Plus), AATSR (Advanced Along-
Track Scanning Radiometer) and ASTER (Advanced Spaceborne Thermal Emission and
Reflection Radiometer) – could be used.
CSIRO Land and Water Page iv
Table of Contents Copyright and Disclaimer ii Acknowledgements i Executive Summary ii 1 Introduction 1 2 Physical Models of Moisture Availability 3 3 Examples of NDTI / NDVI Characterisation of Hydrological Land-Uses 7
3.1 Lake Victoria 8 3.2 Coleambally Irrigation Area 12 3.3 Forest – dryland agriculture interface at Tumut 16
4 Developing a Prototype Operational NDTI Generation System 20 4.1 Data collection 22 4.2 Quality assess SILO Point Patch Database annually 23 4.3 Quality assess BoM wind data and convert from wind run to wind speed 25 4.4 Convert fPAR data to percent vegetation cover 26 4.5 Generate annual .DAT files and .SS files 27 4.6 Run PRERAD to generate .PSF files 28 4.7 Generate and interpolate daily atmospheric transmittance 29 4.8 Calculate Rs_in at the date-time of image acquisition 30 4.9 Use .ALL files in RADWAT to create .COV files 31 4.10 Interpolate Ta and ea at the date-time of image acquisition 33 4.11 Generate Rn and Ts-Ta at the date-time of image acquisition 34 4.12 Run RADWAT to create files containing overpass information 36 4.13 Generate overpass NDTI surfaces 39 4.14 Generate fortnightly and monthly NDTI composites 39
5 Monthly Gap-filled NDTI Composite Images from 1992 to 2006 42 6 Conclusions 59 7 References 61 8 Appendix A: Monthly NDTI / NDVI Characterisation 66
8.1 Lake Victoria 67 8.2 Coleambally Irrigation Area 69 8.3 Forest - dryland agriculture interface at Tumut 71
9 Appendix B: Fortnightly Gap-filled NDTI Composite Images from 1992 to 2006 73 10 Appendix C: IDL code 90
10.1 IDL code to quality assess SILO Point Patch Database annually 90 10.2 IDL code to quality assess BoM wind data and convert from wind run to wind speed 92 10.3 IDL code to convert fPAR data to percent vegetation cover 96 10.4 IDL code to generate annual .DAT files and .SS files 98
CSIRO Land and Water Page v
10.5 IDL code to run PRERAD to generate .PSF files 101 10.6 IDL code to generate and interpolate daily atmospheric transmittance 102 10.7 IDL code to calculate Rs_in at the date-time of image acquisition 106 10.8 IDL code to use .ALL files in RADWAT to create .COV files 106 10.9 IDL code to interpolate Ta and ea at the date-time of image acquisition 113 10.10 IDL code to generate Rn and Ts-Ta at the date-time of image acquisition 116 10.11 IDL code to run RADWAT to create files containing overpass information 119 10.12 IDL code to generate overpass NDTI surfaces 134 10.13 IDL code to generate initial fortnightly and monthly NDTI composites 138 10.14 IDL code to gap-fill fortnightly and monthly NDTI composites 142
CSIRO Land and Water Page vi
CSIRO Land and Water Page 1
1 Introduction In recent years reduced availability of water has resulted in several large cities in Australia
having water restrictions, crop production has been below-average (detectable in the
Nation’s current account deficit), and widespread severe wind erosion has resulted in large
dust storms being experienced. Hence there is heightened demand for accurate, timely and
consistent appraisals of the Nation’s water resources. To meet this demand, a partnership
project, called the Australian Water Availability Project (AWAP) has been established with
funding provided by the National Heritage Trust. AWAP partners are the Bureau of Rural
Sciences (BRS), the Bureau of Meteorology (BoM) and CSIRO (both Land and Water (CLW)
and Marine and Atmospheric Research (CMAR)). The objective of the project is: “To
develop an operational prototype of a new and integrated approach to monitoring and
predicting soil moisture and other components of the water balance, for spatial scales
ranging from 1km to all of Australia, and for time scales ranging from fortnightly to decades.”
Currently there is no nationally consistent source of information on water balance fluxes and
stores that utilises the inherent strengths of meteorological network and remote sensing data.
Meteorological data are temporally dense (and spatially sparse), whereas remotely sensed
data are spatially dense (and temporally sparse). In AWAP, both the direct interpretation of
moisture availability and prediction of select key components of the water balance (e.g., soil
moisture) will utilise both meteorological and regional time series remotely sensed
databases. In this report we document the development of prototype system that directly
interprets a key component of the regional water balance, moisture availability, from optical
(meaning both thermal and reflective) remotely sensed data. This is achieved by generating
a 15 year time-series of Normalised Difference Temperature Index (NDTI) images from the
AVHRR (Advanced Very High Resolution Radiometer) sensor. Any sensor with both thermal
and reflective capacity – such as MODIS (MODerate Resolution Imaging Spectroradiometer),
ETM+ (Enhanced Thematic Mapper Plus), AATSR (Advanced Along-Track Scanning
Radiometer) and ASTER (Advanced Spaceborne Thermal Emission and Reflection
Radiometer) could be used; here AVHRR is used to provide a long-term context of changes
to a key component of Austrlia’s water balance. Importantly, the NDTI can also be readily
calculated across large regions. It does not require interpolated gridded precipitation or
irrigation data as inputs, and therefore provides a means to detect ‘spurious and
unaccounted’ water application thus enabling compliance to be monitored.
This research integrates two data types with very different spatial and temporal scales.
Remotely sensed data are spatially dense, with a specific spatial resolution; they are
CSIRO Land and Water Page 2
recorded over large areas in a matter of seconds at a specific time for specific wavelengths.
This means for the extent of the image, remotely sensed data are a “census” at a particular
spatial scale recorded at a specific time. Depending on the amount of cloud cover and the
satellite repeat characteristics, optical remotely sensed data may only be available for an
area of interest once every week or even just once every month. On the other hand
meteorological data are recorded sparsely with the points often separated by tens to
hundreds of kilometres. The variables measured at these points represent a certain area.
However, the exact area being represented by a given point measurement is unknown as the
spatial autocorrelation is unknown. Thus remotely sensed data is spatially dense but
temporally sparse while meteorological data are spatially sparse but temporally dense.
Within the AWAP framework the idea is to best utilise these characteristics.
This report is organised as follows: Section 2 provides an overview of the methods that are
currently available (and how these methods may be refined) to produce spatial estimates of
moisture availability. Section 3 provides three examples of time series of NDTI and NDVI
(Normalised Difference Vegetation Index) imagery for three hydrologically relevant land-
uses: (a) Lake Victoria; (b) Coleambally Irrigation Area (CIA); and (c) the forest-dryland
agriculture interface in the vicinity of Tumut, New South Wales (NSW). Section 4 provides
an overview of the design of the current information system used to generate the NDTI in
near real time. Current monthly NDTI composites from April 1992 to December 2006 are
shown in Section 5. Conclusions, some specific issues related to making the current
information system operational and a review of future scientific directions to better utilise time
series remotely sensed based ecohydrological indices are presented in Section 6. The
information system developed here is suitable to generate all-Australian 1 km resolution daily
time-step moisture availability products from 1986 onwards.
CSIRO Land and Water Page 3
2 Physical Models of Moisture Availability Regional remotely sensed data can be used in an information system to monitor moisture
availability - ma, which is defined as the ratio of actual evapotranspiration (ETa) to potential
evapotranspiration (ETp). That is, ma = ETa / ETp. The remotely sensed variables required
to calculate ma are surface temperature (Ts), fraction vegetation cover (VegCov), and albedo
(α ). When these variables are combined with meteorological data (specifically air
temperature (Ta), net radiation (Rn), wind speed (u) and some measure of vapor pressure
(ea)) there exists the opportunity to regionally estimate ma.
Much international and Australian research has illustrated the worth of using optical remote
sensing (both thermal and reflective) to derive regional information about the key
components of the water balance. For detailed reviews see (Kustas and Norman 1996;
McVicar and Jupp 1998). In much previous research (Kustas and Humes 1997; Kustas and
Jackson 1999) spatial estimates of key components of the water balance have been
determined in small catchments for short experimental field campaigns. While this has lead
to vast improvements in process understanding, the data requirements to perform this style
of analysis are not routinely available for operational management of large regions for long
periods. To overcome this problem the NDTI was developed (McVicar and Jupp 2002) which
utilises the complimentary inherent data densities of meteorological (temporally dense and
spatially sparse) and remotely sensed data (temporally sparse and spatially dense).
The NDTI is defined as:
NDTI T TT T
s=−−
∞
∞
( )( )0
(1)
The two bounding temperatures, T0 and T∞, are derived by inverting a dual-source
Resistance Energy Balance Model (REBM), this is achieved by setting the vegetation
resistance (rsv) and ground resistance (rsg) to pre-defined values (Jupp et al. 1998). T0 is a
REBM inverted surface temperature when rsv = rsg = 0 s m-1, that is when it is assumed ETa =
ETp. T∞ is a REBM inverted surface temperature when the vegetation and ground (or soil)
resistances are set to infinity, mathematically that is rsv = rsg = ∞ s m-1, that is when it is
assumed ETa = 0 W m-2. Ts is the daytime remotely sensed measurement of surface
temperature; it can be derived from any space, or airborne thermal sensor. If the REBM and
the meteorological data (Ta, Rn, u and ea) estimated at the time of remotely sensed data
acquisition are well defined, a time series of Ts should fall within the envelope defined by the
limits T0 and T∞. Figure 1 shows this relationship for 3 years (July 1999 to June 2002) of
CSIRO Land and Water Page 4
AVHRR Ts data recorded at Cobar. At the time of change over from an ‘older’ NOAA satellite
and its ‘newer’ operational replacement there can be up to 3 hours differences in the local
observation time due to orbital decay in the NOAA series of satellites carrying the AVHRR
sensor (Price 1991). Figure 2 shows that the local time of image acquisition at the end of the
operational life of NOAA14 was approximately 3 hours later than data acquired early in the
operational life of NOAA16, i.e., the local observation time was approximately 1700 hours at
the end of the life of NOAA14, and early in the life of NOAA16 it was about 1400 hours.
Figure 1. Time series of AVHRR observed temperature, Ts, and the REBM-inverted temperatures, T0 and T∞ (denoted Tinf on the figure), at Cobar, central NSW from 1 July 1999 to 30 June 2002. The change over from NOAA14 to NOAA16 is shown by the vertical grey line. The large change in temperatures due to local changes in time of data acquisition for the two satellites is clearly seen when comparing the summer of 2000 with the summer of 2001 (and when comparing the winters of the same two years). Any anomalous data (i.e., Ts < T0 or Ts > T∞,) has been removed, and a box-car average with a width of 15 (i.e., 7 values on either side) was run from 1 Jan 1999 to 31 Dec 2002 to ensure edges of the time series were not adversely impacted by the box-car averaging.
Figure 2. A time series plot showing the local time of image acquisition of afternoon AVHRR images for the Murray-Darling Basin, south-east Australia from January 1992 to December 2006. The vertical grey lines show change over dates between NOAA satellites, which are labelled on the plot. Only AVHRR data with more than 5% valid land pixels in south-east Australia are shown.
CSIRO Land and Water Page 5
To make the most use of the spatial and temporal qualities of remote sensing and
meteorological datasets, the NDTI was calculated using a REBM at the BoM stations that
record maximum and minimum air temperature (Tx and Tn, respectively) and precipitation (P)
every day which are temporally interpolated to the image overpass times (McVicar and Jupp
1999). The NDTI values calculated at all of these BoM points were then spatially
interpolated using ANUSPLIN (Hutchinson 2004) to develop NDTI surfaces. A quint-variate
partial thin plate spline (PTPS), incorporating a bi-variate thin plate spline (TPS) function of
longitude and latitude with constant linear dependencies on Ts-Ta, VegCov and Rn, was used
to generate the NDTI surface (McVicar and Jupp 2002).
Scientifically the six main processes in the current system to generate spatial NDTI are:
1.) the specific time-of-day meteorological data was approximated by temporally interpolating
from daily extreme (maximum and minimum air temperatures) and daily integral
(precipitation) data (McVicar and Jupp 1999);
2.) remote sensing processing algorithms were utilised to convert the raw data into the
variables needed, specifically Ts, VegCov and α (McVicar and Jupp 2002);
3.) estimates of Ta and ea were generated using all meteorological stations, these were used
in the generation of the covariates Ts-Ta, and Rn (with VegCov being directly observed from
the remote sensing – as described in step 2 above);
4.) the NDTI was calculated by inverting the REBM (Jupp et al. 1998) where the
meteorological data and cloud-free AVHRR intersect. That is, the process based REBM was
ONLY run at the meteorological stations for the specific days and time-of-day that cloud free
remote sensing was recorded;
5.) for each afternoon overpass outputs from the inversion of the REBM were then spatially
interpolated between the meteorological stations using a quint-variate partial thin plate spline
(PTPS), incorporating a bi-variate thin plate spline (TPS) function of longitude and latitude
with constant linear dependencies on Ts-Ta, VegCov and Rn, using ANUSPLIN (Hutchinson
2004); and
6.) the resultant single overpass NDTI images were then temporally composited to provide
monthly or fortnightly assessments of ma at 1 km resolution.
McVicar and Jupp (2002) illustrated how the NDTI is generated from the point BoM
observation stations and linked with remotely sensed spatial modelling to provide single
afternoon NDTI images. To illustrate the potential of the NDTI for the derivation of
ecohydrological metrics an initial dataset was developed from 1986 to 1994 (McVicar and
Van Niel 2005). This initial dataset comprised 97 near monthly AVHRR images, the size of
CSIRO Land and Water Page 6
the AVHRR data on disk was 0.75Gb, and with all intermediate files the amount of disk
storage required was 11.5 Gb (i.e., a 15-fold increase of hard disk was required). The
afternoon overpasses in the current database from April 1992 to current included 8,889
AVHRR files (many days have two images that acquire data for part of south-east Australia)
which require approximately 750 Gb to store these data. If there was the same 15-fold
increase in data volume, this would mean that the processing would generate 10,950 Gb (or
10.7 Tb) of data – obviously this was unfeasible and the number of intermediate files had to
be reduced, by, wherever possible, concatenating processing steps. This meant that the
processing chain to produce the initial (or research) NDTI database required a major
overhaul; our primary aim was to develop a prototype operational system by judicious and
effective use of scripting, file naming and sub-folder structures to enable the 8,889 images to
be processed operationally. In the next section, case studies demonstrating the uses of
NDTI time series for different hydrologically-relevant land-uses are discussed.
CSIRO Land and Water Page 7
3 Examples of NDTI / NDVI Characterisation of Hydrological Land-Uses
To illustrate the utility of the NDTI images, and comparing it to the widely used NDVI, we
show these data for three areas with spatial and temporal variability of moisture availability
and vegetation vigour. This analysis is performed for fortnightly composites for 2001 and
highlights typical ecohydrological information present in NDTI, NDVI imagery and their
interactions. The three areas illustrated are: (1) Lake Victoria; (2) Coleambally Irrigation
Area; (3) forest-dryland agriculture interface in the vicinity of Tumut (see Figure 3). We first
show the NDTI and NDVI contexts of the feature of interest, then for 80 km transects located
across each area (Figure 3b-d) )the NDTI and NDVI values are extracted and shown, and
finally for individual pixels along the transects time series plots are shown.
(b)
(c)
(a)
(d) Figure 3. The south-eastern Australia study site and the location of the three transects are shown in (a). The transects for Lake Victoria (b), the Coleambally Irrigation Area (CIA) (c), and the forest-dryland agriculture interface near Tumut, NSW (d) are shown over the NDTI composite for the first fortnight in January 2001.
For the fortnightly images the layout of the 24 fortnights is shown in Figure 4.
CSIRO Land and Water Page 8
Jan_1 Jan_2 Feb_1 Feb_2
Mar_1 Mar_2 Apr_1 Apr_2
May_1 May_2 Jun_1 Jun_2
Jul_1 Jul_2 Aug_1 Aug_2
Sep_1 Sep_2 Oct_1 Oct_2
Nov_1 Nov_2 Dec_1 Dec_2
Figure 4. Schematic showing the layout of the progression of fortnightly composite images, where each month is given by the first 3 letters of that month, and 1 and 2 mean the first fortnight (days 1 through 15) and second fortnight (days 16 through to the end of the month) respectively.
3.1 Lake Victoria Lake Victoria can store 677 Gl and it provides a buffering capacity to ensure that reliable
daily flow of 3 Gl to 7 Gl per day (depending on season) can be provided to South Australia
(Murray-Darling Basin Commission 2006). It is located in the south-west of NSW, its location
being further west than all major tributaries flowing into the Murray River (including the
Darling River), and most of the NSW and Victorian irrigation water off-takes along the Murray
River. By locating this large storage further downstream of connections which can both add
and remove water from the system is the essence of the buffering capacity (Murray-Darling
Basin Commission 2006).
Figure 5a shows the fortnightly NDTI for 2001 in the vicinity of Lake Victoria. As expected
the water body has a persistently high NDTI throughout, and is easily visible in the summer
months where there is high contrast with the surrounding Mallee shrub. Over winter (from
CSIRO Land and Water Page 9
May to August, inclusive) due to widespread precipitation in the area, the actual water body
becomes more difficult to identify. Figure 5b is the NDVI imagery for the equivalent area and
sampling characteristics. As expected the water body has a continued low NDVI. Models
using NDVI-driven estimates of Leaf Area Index (LAI) or fraction of Photosynthecially
Available Radiation (fPAR) to characterise actual evaporation would be severely limited in
this environment, as they would likely estimate that ETa was also low. However, methods
using the differential between the surface and air temperatures (e.g., NDTI) are not adversely
impacted in these situations. Accurate characterisation of ETa from large water bodies such
as Lake Victoria is important for advanced water accounting systems that are currently being
built for Australia.
0 . 0 0 . 5 1 . 0 - 0 .0 5 0 .4 7 1 .0 0
(a) NDTI ≅ moisture availability (b) NDVI ≅ vegetation vigour Figure 5. Fortnightly NDTI images (a) and fortnightly NDVI images (b) around the Lake Victoria transect for 2001. NDTI measures moisture availability (high = wet) and NDVI reflects vegetation vigour (high = lush).
CSIRO Land and Water Page 10
Both NDTI and NDVI data were extracted for a 80 km long transect running through the
centre of the context images (and the Lake Victoria waterbody provided in Figure 3). Figure
6 represents the time series for 2001; the x-axis represents the distance along this transect
and the y-axis is time at a fortnightly time step. Figure 6a shows that the NDTI is constantly
high in the Lake, and the increase in NDTI during the winter months (starting in the first
fortnight of May) of the area surrounding the lake is clearly seen. The 4 to 6 week lagged
response of the NDVI to greater moisture availability is seen in Figure 6b. In Figure 6c to e
standard time series plots are shown for individual pixels. The lag between the relative
increase of NDTI in winter and the subsequent increase in NDVI is clearly illustrated in
Figure 6e, where the NDTI is decreasing in September when the maximum NDVI is
observed. The monthly NDTI and NDVI composites are shown in Appendix A.
CSIRO Land and Water Page 11
(a) NDTI ≅ moisture availability
(b) NDVI ≅ vegetation vigour
(c) (d) (e)
Figure 6. Fortnightly time series plot from 2001 for an 80 km transect located across Lake Victoria of (a) NDTI and (b) NDVI. Note these do not have the same colour stretch. The black line in (a) and (b) represent the profile of the first fortnightly compositing period in each case. Per-pixel time series are shown for pixels located at (c) 15 km, (d) 40 km, and (e) 65 km, respectively, with the black line being the NDTI and the grey line the NDVI.
CSIRO Land and Water Page 12
3.2 Coleambally Irrigation Area
The Coleambally Irrigation Area (CIA) is a rice-based irrigation area, with other crops also
grown in summer including corn, soybeans and sorghum (Van Niel and McVicar 2004b; Van
Niel et al. 2003). There are a number of drains west of the irrigation area used to remove
excess water applied to the region. Rice is flood irrigated, with the other row crops having
irrigated water applied to small depressions (called furrows) separating the soil that the rows
of crops are planted along. In winter, cereals (wheat, oats, barely) and grazing pastures are
the primary agricultural activities (Van Niel and McVicar 2003) – the irrigation demand in
winter is lower as landholders attempt to utilise residual soil moisture after successfully
growing summer crops. A review of the application of remote sensing technology to assist
management within an irrigation system is provided by Van Niel and McVicar (2004a); the
following example compliments that work, by highlighting how large area remotely sensed
data (i.e., AVHRR or MODIS) can be used to monitor irrigation system performance noting
that this can be performed for all irrigation systems located in the Murray-Darling Basin, and
here the CIA is used an example.
Figure 7a shows the fortnightly NDTI images from 2001 for the vicinity of the CIA. The extent
of the irrigation area is clearly seen in the first fortnight of January. The increase of moisture
availability associated with the outfall drain to the west of the CIA is seen in the second
fortnight of January and the first fortnight in February, while the dryland agriculture east of
CIA provides strong constant low moisture availability right up to the border of the CIA.
Patchy precipitation is received in the first fortnight of February, and in the last fortnights of
April and November. Widespread heavy precipitation is received in June and July, resulting
in these months having high NDTI values for all parts of the image. The dryland agricultural
land surrounding the CIA dries during spring (September and October), with the winter crops
actively growing in this period. Flood irrigation of rice paddocks commences in mid October
with most paddocks being filled by the end of November. The vegetation vigour, shown by
the NDVI (Figure 7b), lags the application of irrigation water within the CIA, and the
senescence and harvesting of the summer crops in March to June is seen (Figure 7b).
Following this, the gradual greening due to the winter crops is seen, with these being
harvested in late November (Figure 7b). West of the irrigation area the green-up occurring in
February and March (Figure 7b) clearly follows the increase in moisture availability that starts
in the last fortnight of January (Figure 7a). To the north and east of the irrigation area the
increase in vegetation vigour of the dryland cropping and grazing during September and
October (Figure 7b), following the wide spread winter rains is also clearly seen.
CSIRO Land and Water Page 13
0 . 0 0 . 5 1 . 0 0 . 0 0 . 5 1 . 0 (a) NDTI ≅ moisture availability (b) NDVI ≅ vegetation vigour
Figure 7. Fortnightly NDTI images (a) and fortnightly NDVI images (b) around the CIA transect for 2001. NDTI measures moisture availability (high = wet) and NDVI reflects vegetation vigour (high = lush).
The 80 km long transect (see Figure 3) extracted through the centre of the context images
(also running through the centre of the irrigation area) for NDTI and NDVI, see Figure 7a and
b, respectively further highlights the patterns of the three distinctive regions in both the NDTI
(Figure 8a) and NDVI (Figure 8b) images. The three regions are: (1) west of CIA where the
impact of the CIA drain is seen (0 km to 30 km along the transect); (2) the CIA proper (30 km
to 58 km); and (3) dryland agriculture east of the CIA (58 km to 80 km). Figure 8a and b
represents the time series for 2001; the x-axis represents the distance along this transect
and the y-axis is time at a fortnightly time step. While the same gross patterns are revealed
in the context images (which will not be discussed to avoid repetition) some finer scale
CSIRO Land and Water Page 14
features are worthy of discussion. For example, west of the CIA the NDTI increases in the
second fortnight of January, yet it takes another fortnight for the NDVI to respond. During
this period direct evaporation from the soil surface would have been high before the
vegetation responded. Additionally, in the last fortnight of November there was a large
supply of water due to landholder management of flood irrigation of the rice paddocks, while
at the same time the NDVI (as expected from a water surface) was low. In this early part of
the growing season there would have been considerable evaporation from the water surface
of the rice paddocks, which would not be detected in reflective based estimates of ETa.
Figure 8c to e are standard time series plots for individual pixels. The accurate estimation of
ETa from intensive irrigation areas (such as CIA) is an important feature of accurate water
accounting systems that are currently being developed for Australia. The lag between the
relative increase of NDTI and subsequent increases of NDVI are clearly seen. Monthly NDTI
and NDVI composites are shown in Appendix A.
CSIRO Land and Water Page 15
(a) NDTI ≅ moisture availability
(b) NDVI ≅ vegetation vigour
(c) (d) (e)
Figure 8. Fortnightly time series plot from 2001 for an 80 km transect located across Coleambally Irrigation Area of (a) NDTI and (b) NDVI. The black line in (a) and (b) represent the profile of the first fortnightly compositing period in each case. Per-pixel time series are shown for pixels located at (c) 15 km, (d) 40 km, and (e) 65 km, respectively, with the black line being the NDTI and the grey line the NDVI.
CSIRO Land and Water Page 16
3.3 Forest – dryland agriculture interface at Tumut To illustrate the utility of the NDTI/NDVI time series imagery for monitoring changes due to
land-use, not directly related to a hydrological land-use (as the two previous examples of a
lake and an irrigation area show), a region with a forest - dryland agriculture interface was
selected in the vicinity of Tumut, south-eastern NSW (Figure 3). Both the time series of
context images and transect will simply be named ‘Tumut’ to represent this forest - dryland
agriculture land-use interface in the following.
The Tumut NDTI fortnightly composite context images (Figure 9a) illustrate that the moisture
availability is relative lower in the agricultural area in the north-west of the image in summer
(January and December) than the adjacent forested areas located in the south-east of the
context image. This spatial pattern is confirmed in the NDVI Tumut context images (Figure
9b). Throughout winter there was widespread precipitation resulting in subsequent increases
of both the NDTI and NDVI (Figure 9a and b, respectively). As expected, the NDTI of the
agricultural lands decreases more rapidly than that of the adjacent forests in spring
(September and October) and this decrease in moisture availability is associated with
increasing moisture utilisation illustrated by the NDVI of the dryland agriculture being slightly
higher than the NDVI of the adjacent forests in late September through to the end of October,
see Figure 9a and b, respectively.
CSIRO Land and Water Page 17
0 . 0 0 . 5 1 . 0 0 .0 0 0 .4 3 0 .8 5
(a) NDTI ≅ moisture availability (b) NDVI ≅ vegetation vigour Figure 9. Fortnightly NDTI images (a) and fortnightly NDVI images (b) around the forest - dryland agriculture land-uses transect near Tumut for 2001. NDTI measures moisture availability (high = wet) and NDVI reflects vegetation vigour (high = lush).
The 80 km long transect (see Figure 3) running through the centre of the context images
(see Figure 9a and b for NDTI and NDVI, respectively) further highlights the patterns of the
two distinctive land-uses and is illustrated in both the NDTI (Figure 10a) and NDVI (Figure
10b). The two land-uses are: (1) dryland agriculture in the western portion of the image (0
km to 38 km along the transect); and (2) forests (from 38 km to 80 km). The two land-uses
are clearly distinguishable. In the west the dryland agriculture shows a relative lower NDTI in
summer, with moderate values recorded during autumn, with the relative maxima occurring
in spring. The NDVI follows a similar cycle, with about a 6 week to 8 week lag. Figure 10c
CSIRO Land and Water Page 18
shows this lag, with the local maxima of the NDTI occurring in July followed by the maxima of
the NDVI occurring in September. The time series plots of both forested pixels (Figure 10d
and e) show that the NDTI is constantly above 0.6, with more variability and slightly lower
values being observed close to the forest-agriculture interface (Figure 10d) compared to the
forested pixel that is surrounded by forest on all sides (Figure 10e). This finding is in
agreement with the well known feature of enhanced ETa (and hence lower moisture
availability) at the edges of large stands of trees – usually associated with plantations or
fragments of forests following clearing – the so-called ‘edge effect’ (Camargo and Kapos
1995; Giambelluca et al. 2003; Kapos 1989). In both plots the NDVI values were relatively
constant. Monthly NDTI and NDVI composites for this transect are shown in Appendix A.
CSIRO Land and Water Page 19
(a) NDTI ≅ moisture availability
(b) NDVI ≅ vegetation vigour
(c) (d) (e)
Figure 10. Fortnightly time series plot from 2001 for an 80 km transect located across a forest – dryland agriculture land-uses in the vicinity of Tumut (a) NDTI and (b) NDVI. The black line in (a) and (b) represent the profile of the first fortnightly compositing period in each case. Per-pixel time series are shown for pixels located at (c) 15 km, (d) 40 km, and (e) 65 km, respectively, with the black line being the NDTI and the grey line the NDVI.
CSIRO Land and Water Page 20
4 Developing a Prototype Operational NDTI Generation System
Operationally, there are 14 main steps involved in generating spatial NDTI surfaces, they
are:
1. Data Collection 2. Quality assess SILO Point Patch Database Annually 3. Quality assess BoM Wind Data and convert from wind run to wind speed 4. Convert fPAR data to Percent Vegetation Cover 5. Generate annual .DAT files and .SS files 6. Run PRERAD to Generate .PSF files 7. Generate and Interpolate Daily Atmospheric Transmittance 8. Calculate Rs_in at the date-time of image acquisition 9. Use .ALL files in RADWAT to create .COV files 10. Interpolate Ta and ea at the date-time of image acquisition 11. Generate Rn and Ts-Ta at the date-time of image acquisition 12. Run RADWAT to create files containing overpass information 13. Generate Overpass NDTI Surfaces 14. Generate Fortnightly and Monthly NDTI Composites
Each of these processing steps is described in detail below, and a flow chart is included
wherever applicable for easy and quick assimilation of the processing steps. Definition of
terms and acronyms for the terminology used primarily in this section, and in other parts of
the text, is provided in Table 1.
Table 1. Definition of terms used in this and other sections of the report.
Term Definition
SILO PPD SILO Patched Point Dataset (where missing gaps in meteorological
stations have been in-filled by using output from spatial interpolation at
the date-geographic intersection).
fPAR fraction of Photosynthetically Active Radiation absorbed by vegetation
.DAT This type of file contains the quality assessed SILO PPD daily data and
Bureau of Meteorology average daily wind-speed data (if recorded) for
each isolated meteorological station. It is used as input to PRERAD.
.SS This type of file contains information about the study site, here the
locations are never different from the .DAT file. It is used as input to
PRERAD.
CSIRO Land and Water Page 21
.PSF This type of file contains daily extreme and integral meteorological data.
It primarily differs from the .DAT file by having incoming solar radiation
(Rs_in) calculated as a total and fraction diffuse component using methods
outlined in McVicar and Jupp (1999). It is output from PREAD and input
to RADWAT.
.OVP This type of file contains data specific to one geographic location (BoM
station or flux tower) for the time series of overpasses at that location. It
includes date(s) and time(s), and in this study also includes Ts and %
Veg Cover obtained from AVHRR remotely sensed images. If data from
a flux tower were being used in the REBM then Ta, relative humidity,
Rs_in, wind speed, and vegetation height could also be provided. Multiple
times can be provided each day if such data exist (as would be the case
for a flux tower) with a separate .OVP file needed for each geographic
location.
.ALL This type of file only contains all the date-time data specific to each
overpass regardless of cloud cover or satellite coverage; this means that
the temporal interpolation of Ta and ea is conducted for all meteorological
stations for all dates and is independent of remotely sensed data. It is
output from RADWAT.
.COV This type of file only contains geographic information (longitude, latitude
and elevation) and specific time-of-day meteorological data (temporally
interpolated in RADWAT) suitable for spatial interpolation by ANUSPLIN.
It is the output of reformatting the .ALL file (one station, many times) to
produce the .COV file (all stations, one time).
PRERAD This is a FORTRAN executable that is run for each meteorological
station. It performs the required daily time-step calculations (primarily
modelling the radiation environment) before running the REBM.
RADWAT This is a FORTRAN executable that is run for each meteorological
station. It performs the required REBM calculations at the specific dates-
times that optical (thermal and reflective) remotely sensed imagery are
available to force the model.
.NIN This is the Ndti INput file. It contains all the data to drive ANUSPLIN
(specifically SPLINA) to fit a quint-variate partial thin plate spline model,
incorporating a bi-variate thin plate spline function of longitude and
latitude with constant linear dependencies on Ts - Ta, VegCov and Rn.
.NHW This is the Ndti Half Way (or intermediate) file, which contains overpass
CSIRO Land and Water Page 22
output from RADWAT. It is combined with data from overpass surfaces
to create .NIN files.
4.1 Data collection NDTI can be generated from any remotely sensed data where: (1) both thermal and
reflective channels are acquired; and (2) the date and specific time-of-day of data acquisition
are known. The first point means that sensors such as AVHRR, MODIS, TM, ETM+, AASTR
and ASTER are suitable remotely sensed data sources. The second point is valid for single-
overpass imagery. Some compositing algorithms, such as the MODIS land surface
temperature product, is the average surface temperature of all cloud-free observations in a
given compositing period. The averaging of several days means it is unwise to utilise the
resultant MODIS product as input to a process-based REBM. Of course, single afternoon
MODIS images, where the date and specific time-of-day of data acquisition are known are
eminently suitable as input to a process-based REBM.
The current prototype operational system uses AVHRR data managed by CSIRO Marine and
Atmospheric Research (King 1998; King 2000; King 2003). Single afternoon overpasses are
used in the NDTI processing, where the date and time-of-day that the image are acquired are
known. Table 2 shows the number of AVHRR images available for use between 1992 and
2006. We only process AVHRR overpasses if valid (i.e., not cloud affected or outside the
swath) AVHRR data intersects with at least 25 valid BoM stations.
Table 2. Number of AVHRR images from 1992 through 2006 and number of SILO PPD meteorological stations that pass our assessment of > 90% recorded data.
Year Number of images Number of valid meteorological stations
1992 492 234 1993 653 237 1994 472 228 1995 581 234 1996 630 233 1997 645 223 1998 647 226 1999 654 213 2000 646 204 2001 650 215 2002 654 215 2003 653 203 2004 359 200 2005 586 193 2006 567 193 Total 8889
CSIRO Land and Water Page 23
The files from CMAR have the specific naming convention which underpins the scripting of
the entire system. An example is, ah16_20010101_035315_23 – where the field ah
represents AVHRR, 16 is the number of the NOAA series of satellites, next the GMT date is
provided as YYYYMMDD. In our example this is the 1st Jan 2001, then the average GMT
time of non-water and non-cloud affected pixels (i.e., the valid land pixels) is calculated and
provided as HHmmSS, and finally the percentage of the land surface with valid pixels (i.e.,
non-cloud affected and not past the edge of the acquisition swath) are calculated. In the text
above YYYY means year, MM month, DD day, HH hour, mm minute and SS second. The
files are stored in sub-folders arranged by year and month of image acquisition.
Other datasets used in this processing include:
• SILO Point Patch Database (PPD) data: daily, from 1980 through 2006; • Bureau of Meteorology (BoM) windrun data: daily, from 1980 through 2006; • Vegetation coverage percentage images: monthly, from 1992 through 2006
generated from AVHRR data; and • Australian 9 arc-second DEM.
4.2 Quality assess SILO Point Patch Database annually SILO PPD data are infilled from interpolated data wherever actual BoM meteorological
measurements are missing. As we put a higher certainty on measured data than
interpolated data, we performed an analysis to determine how many meteorological stations
had substantially measured data as opposed to substantially interpolated data. This quality
assessment of the SILO PPD data was performed on an annual basis, which allowed for a
user defined threshold for choosing which stations were acceptable for each year. For
example, with a threshold of at least 90% measured data (i.e., < 10% interpolated) for the
first three variables (maximum temperature (Tx), minimum temperature (Tn), and precipitation
(P)), the overall number of stations that would be used if we consider that the station has to
pass the threshold over the entire time between 1992 and 2006 is 144 (magenta line, Figure
11). However, as many more stations pass the threshold for specific years, but not for the
entire time extent, when considered on a yearly basis, many more stations can be used (blue
line, Figure 11).
CSIRO Land and Water Page 24
Figure 11. Number of stations that pass the 90% rule applied to each year in turn (black line) and if 90% rule was applied to all years would result in only 144 stations being used (grey line).
The procedure for SILO PPD quality control is depicted in
Figure 12 below. The SILO PPD text files for each station are input into the IDL program
called checkMetYearly.pro (see Appendix 10.1), which runs the IDL program
stationQualityYearly.pro (see Appendix 10.1 for details). The quality control text report is
output that allows the user to test various threshold values. For our research we used a 90%
threshold on all of Tx, Tn, and P.
SILO PPD data(*.TXT)
QC Report(*.TXT)
checkMetYearly.pro
stationQualityYearly.pro
Figure 12. Flowchart showing the major steps involved in assessing the quality of the SILO PPD annually.
CSIRO Land and Water Page 25
4.3 Quality assess BoM wind data and convert from wind run to wind speed
SILO PPD does not come with any wind speed data. Instead of simply assuming a wind
speed of 2 ms-1, another option is to obtain the daily wind run data (with units of km per day)
measured below 10 feet (or 3 m) above the land-surface from the BoM and convert it to wind
speed using the general rule that 1/3 of the wind occurs at night and 2/3 occurs during the
day (Smith et al. 1991). Therefore, we need to multiply 2/3 by wind run and multiply by 1000
to place the units in m, and divide by daylength (N) in seconds (i.e., multiply N*60*60). The
BoM data were shifted by a day forward to match the SILO PPD data (who already do this
shift for users). That is, BoM wind run data are recorded at 0900 local time. We assumed
that most of this wind run would be attributed to the first 15 hours associated with the
afternoon of the day prior to the recording.
The key IDL function called in the process is calcDayLength, which calculated the daylength
based on the FAO-56 (Allen et al. 1998) calculation from the latitude and day of year (see
Figure 13 and Appendix 10.2). The wind run data were nulled (-1) if the BoM wind run
equalled zero because this only happens when the original wind run data were blank and
were made to be zero because it was read into a floating point variable. Wind run was also
nulled (-1) if ((qualityFlag ne 'W') AND (qualityFlag ne 'S') AND (qualityFlag ne 'I')) AND
((accDays eq 1) OR (accDays eq '')).
WindRun_to_WindSpeed.pro
readBoMWindAscii.pro
calcdaylength.pro
BoM Wind Speed(*.TXT)
BoM Wind Run(*.TXT)
Figure 13. Flowchart showing conversion of BoM daily wind run data (units km per day) into average daily day-time wind speed (units m s-1).
CSIRO Land and Water Page 26
4.4 Convert fPAR data to percent vegetation cover Fractional cover (FC) describes the proportion of ground area covered by foliage and is
generally equivalent to Percentage Foliage Cover. We used measures of fPAR (the fraction
of Photosynthetically Active Radiation absorbed by vegetation) to make estimates of cover.
The fPAR data were derived from 0.01° resolution monthly AVHRR red and near-infrared
reflectance data corrected for atmospheric and sensor calibration effects, following the
method of Donohue et al. (2007). This dataset covers the entire Australian land surface.
fPAR is linearly related to FC (Asrar et al. 1984; Carlson and Ripley 1997; Lu et al. 2003)
and, by making the assumption that when fPAR is 0, FC = 0.0 and that when fPAR is 0.95
(the maximum possible value), FC = 1.00, FC can be estimated as FC = fPAR / 0.95. This
was performed using the using IDL code fapr2Cover.pro (see Figure 14 and Appendix 10.3).
We then subset each monthly vegetation coverage image between 1992 through 2006 to our
south eastern Australia study site. This was done using IDL code subset_f_Coverage.pro
(see Figure 14 and Appendix 10.3). Null data were also changed from -999 to -9999 in the
resultant subset images to comply with our convention. When FC images were produced,
the land mask also excluded lakes from land, we filled these nulls with the mean monthly
data according to a new land mask which masked out ocean only.
CSIRO Land and Water Page 27
SE AustraliaVegetation
Coverage Images
Vegetation Coverage Images
fPAR2cover.pro
fPAR Images
subset_f_coverage.pro
Null fillled SE Australia
Vegetation Coverage Images
fill_nulls_veg_cov_DOIT.pro fill_nulls.pro
Figure 14. Flowchart showing general steps to produce monthly FC images for south east Australia.
4.5 Generate annual .DAT files and .SS files Files with the suffix .DAT and .SS are primary inputs into PRERAD, the .DAT file contains
basic site header information and daily meteorological data that is extreme air temperatures
(Tx, Tn), integral precipitation (P) and average wind speed. The daily meteorological data
must be consecutive with each and every day being present and both the date YYYYMMDD
and day of year fields are required, as these are used in internal checking by PRERAD to
ensure that the user has not inadvertently made an error with the date / day of year
calculations. The .SS file contains information that may be different from the location of the
CSIRO Land and Water Page 28
.DAT file – this logic was to allow for spatial interpolation internal to the MTCLIM package
(Hungerford et al. 1989) – however in our processing the location of the .DAT file and .SS file
were always exactly the same as all spatial interpolation conducted here was performed
using ANUSPLIN (Hutchinson 2004). We generated DAT and SS files using the procedure
described below in Figure 15 (also see Appendix 10.4 for the relevant IDL code). The main
program called make_dat_ss.pro takes in a dictionary file (station file), silo PPD data, and
BoM wind speed data (if it exists) and gleans relevant information from the station file to
generate the appropriate *.DAT files. In this step the *.SS files for each year are also
generated. It puts the data in separate directories by year and also checks for BoM wind
speed data for each station and merges the wind speed with the SILO PPD data.
SILO PPD data(*.TXT)
*.DAT(Base Met)
make_dat_ss.pro
readBoMWindAscii.pro
readSiloMet.pro
Met Station Dictionary
(.TXT)
*.SS(Study Site)
BoM Wind Speed(*.TXT)
Figure 15. Flowchart capturing primary steps to generate the *.DAT and *.SS files each year from the SILO PPD, dictionary of meteorological stations and BoM wind speed database.
4.6 Run PRERAD to generate .PSF files PRERAD is a FORTRAN executable program that performs daily time-step calculations,
primarily of the radiation environment, as documented in McVicar and Jupp (1999). These
calculations produce total daily solar radiation, fraction of diffuse radiation, potential
evapotranspiration and daily atmospheric transmittance (τ ). For each run, PRERAD takes
one *.DAT file and one relevant *.SS file as input data, and then outputs one *.PSF file,
Figure 16. The IDL code Parameter4PRERAD.pro was used to produce yearly parameter
files for PRERAD (Appendix 10.5). We then ran PRERAD for each year at the DOS
command line using each PRERAD parameter file. For example, for year 2001, we would
type “PRERAD_b < 2001_prerad.par” at the DOS prompt. All the years .PSF files were
generated like this via DOS batch files.
CSIRO Land and Water Page 29
Parameter4Prerad.pro
.PSF (Daily
Summary File)
PRERAD.EXE
.DAT (Station
Summary File)
.SS (Station
Summary File)
Figure 16. Flowchart PRERAD processing to produce the .PSF daily summary files.
4.7 Generate and interpolate daily atmospheric transmittance The next step involved bi-variate thin plate spline interpolation of daily atmospheric
transmittance (τ ) using ANUSPLIN Version 4.3 (Hutchinson 2004); an example of this
output is provided in McVicar and Jupp (2002). The daily atmospheric transmittance values
from the .PSF files, generated from PRERAD, are saved as one file per station (i.e., many
days per file (for each geographic location where there is a BoM station)). For input to
ANUSPLIN, we needed to convert these to one file per day (i.e., many stations per file (for
each day in the time series)). This was performed using the IDL code SortPSFByDay.pro
(see Figure 17 and Appendix 10.6). We named the sorted daily atmospheric transmittance
files as “YYYYMMDD.TRN” to help specifically identify them from the .PSF files – these
where YYYY, MM and DD, which are year, month and day, respectively. The .TRN files
were then interpolated using ANUSPLIN (meaning both SPLINA and LAPGRD) to produce
daily atmospheric transmittance ASCII surfaces for future use in the incoming shortwave
radiation (Rs_in) calculation performed with SOL_RAD.
CSIRO Land and Water Page 30
.TRN (Daily
Transmittance File)
SortPsfByDay.PRO
.PSF (Saily
Summary File)
SPLINA
SPLINA Daily .CMD file
.SUR file(+ other
SPLINA files)
LAPGRD
LAPGRD Daily .CMD
file
Daily At_Trans ASCII surface
Create_SPLINA_cmd_Files.pro
Create_SPLINA_Batch_Runs.pro
Create_LAPGRD_cmd_Files.pro
Create_LAPGRD_Batch_Runs.pro
=
=
Figure 17. Flowchart showing the generation of the daily atmospheric transmittance.
4.8 Calculate Rs_in at the date-time of image acquisition Modelling of the incoming shortwave radiation environment was performed in an image
processing package using a program called SOL_RAD; it can be thought of as a specific
time-of-day version of SRAD (Moore et al. 1993; Wilson and Gallant 2000). The main
difference is that SRAD uses the Ångström–Prescott equation (Prescott 1940), which is
based on the ratio of bright sunshine hours (n) to total sunshine hours (N) as its measure of
atmospheric transmittance, whereas in SRAD the daily atmospheric transmittance (τ ) is
derived from south-east Australia optimized (McVicar and Jupp 1999) versions of the Bristow
and Campbell (1984) coefficients. The required topographic information (including grids of
elevation above sea level, slope and aspect generated from an ANUDEM generated 9 arc
second DEM) were appended to the daily atmospheric transmittance ASCII surfaces created
in the previous step (Section 4.7) to successfully model incoming shortwave radiation. This
was performed by executing a series of DOS batch files. RS_IN.BAT was used as the slave
CSIRO Land and Water Page 31
or engine file and CALL_RS_IN_FOR_COV.BAT was the master or driver file that was run
for every day of each year and called RS_IN.BAT many times passing the few parameters
needed. The date and time of the specific overpass are some of the few parameters needed
and these were obtained from the AVHRR file names. Figure 18 shows the flowchart of Rs_in
modelling and Appendix 10.7 shows the IDL code used for this processing step.
Rs_in.bat
BRINGIN
.IMG
IMAGEMAN
.IMG
Bringin .PAR
Imageman.PAR
SOL_RAD SOL_RAD.PAR
(AVHRR image names are to specify the time, dd, mm and yy for Sol_Rad)
Rs_in_Batchrun_WRON.pro
= AVHRR Names
3 Channel image(elevation, slope and aspect)
Rs_in.IMG
Figure 18. Flowchart of processing to modelling incoming shortwave radiation.
4.9 Use .ALL files in RADWAT to create .COV files To assist with modelling other radiation components, specifically longwave incoming, we
used RADWAT to produce specific time-of-day meteorological interpolation input for use in
SPLINA. In this way we used all valid met stations that passed the 90% quality threshold
discussed in section 4.2 to ultimately interpolate vapor pressure (ea) and air temperature (Ta)
at the time of the image acquisition. The input files for use in this current processing step are
referred to as .ALL files as they are used for processing all stations for all times. In section
4.10 we describe a very similar process using remote sensing-derived variables (Rn and Ts -
Ta) in which interpolation cannot be run over all stations for all times because some areas are
either cloud affected or outside the AVHRR swath. By separating these two steps, we
ensure that ea and Ta are interpolated using as many input BoM stations as possible.
Otherwise, if the steps were combined into one, then these variables would be interpolated
with fewer input points than were truly available.
CSIRO Land and Water Page 32
The IDL program called Int_Creation.pro (see Appendix 10.8) reads in the AVHRR image
names and creates input files (what we call .INT files) for RADWAT. The .INT files are in the
format required by RADWAT, which as stated above, outputs an .All file for each station (see
Figure 19 below). The program SortAllByImageAcq.pro (see Appendix 10.8) does the
conversion from Relative Humidity (%) to vapor pressure (kPa) and vapor pressure deficit
(kPa) and also converts the .ALL files from having many image acquisition records for each
station in a separate text file to having many station records for each image acquisition in
separate text files. That is, in stead of having individual .ALL files sorted by station, we
converted them to being sorted by image acquisition, and thus they were directly useable for
spatial interpolation of each separate image acquisition.
We used the Tetens (1930) approximation of saturated vapor pressure outlined in Allen et al.
(1998) to convert from specific time-of-day relative humidity to specific time-of-day vapor
pressure (ea). This was implemented as:
Rel. humidity . ( )100a s ae e T=
(2)
where ea is the atmospheric water vapor pressure (kPa); Rel. humidity is the specific time-of-day relative humidity (%); and es(Ta) is the saturation vapor pressure (kPa) at the specific time-of-day air temperature (°C).
The saturation vapor pressure (es) at air temperature (Ta) is calculated as:
( ) 17.27 TT 0.6108 expT 237.3se ×⎡ ⎤= ⎢ ⎥+⎣ ⎦
(3)
where es(T) is the saturation vapor pressure (kPa) at air temperature T (°C); and T is the air temperature (°C).
As we are not working with daily or monthly extremes there was no need to calculate the
saturation vapor pressure at each of these extremes and average the result to account for
non-linear underlying air temperature (°C) / saturation vapor pressure (kPa) relationship
(Murray 1967). Vapor pressure deficient (VPD) is simply calculated as es – ea.
To maximize subsequent processing flexibility the 8 variables recorded in the resulting .COV
files are:
1. BoM Station number 2. Longitude (decimal degrees) 3. Latitude (decimal degrees) 4. DEM elevation (m) 5. Ta (°C) 6. Relative Himidity (%) 7. ea (kPa)
CSIRO Land and Water Page 33
8. es - ea (or VPD with units kPa)
.INT (All Stations)
RADWAT.PSF (Daily
Summary)
.ALL (All Stations,
All Times)
SortAllByImageAcq.PRO
.COV (Tair, ea, VPD)
=
AVHRR Images
Create_Parameter_For_Radwat_All.Pro
Int_Creation_WRON.pro
U:\AWAP\Data\RADCALC\ALL\run_radwat.bat
Figure 19. Flowchart of processing to create .COV files for all time at all stations (as input to spatially interpolate the meteorological covariates required to generate the NDTI surfaces).
4.10 Interpolate Ta and ea at the date-time of image acquisition Once the .COV files were generated, the ea, Ta and VPD data contained in each .COV file
were then interpolated for all of south eastern Australia using ANUSPLIN. This is depicted
in Figure 20, below, where the output surfaces are labeled with a .GRD extension (see also
Appendix 10.9 for the relevant IDL code). Following McVicar et al. (2007) a tri-variate partial
thin plate spline (TPS) incorporating a bi-variate TPS function of longitude and latitude and a
constant linear dependence on elevation was used to interpolate these meteorological
surfaces. In all cases the order of the spline was 2 (i.e., m = 2); for more details see McVicar
et al. (2005).
CSIRO Land and Water Page 34
SPLINA
Tair.GRD ea.GRD
.SUR,.LOG
& others
LAPGRD
VPD.GRD
.COV (Tair, ea, VPD)
SPLINA Daily .CMD file
Create_SPLINA_cmd_Files_Ta_Ea_VPD.pro
Create_SPLINA_Batch_Runs_Ta_Ea_VPD.pro
LAPGRD CMD file
Create_LAPGRD_cmd_Files_Ta_Ea_VPD.pro
Create_LAPGRD_Batch_Runs_Ta_Ea_VPD.pro
=
=
Figure 20. Flowchart showing the processing to develop specific time-of-day meteorological surfaces that are used as to develop covariates for NDTI spatial interpolation.
4.11 Generate Rn and Ts-Ta at the date-time of image acquisition Net radiation (Rn) is the balance of the incoming and outgoing shortwave radiation
components defined by:
= − + −n s_in s_out l_in l_outR R R R R (4)
where: Rn (W m–2) is allwave net radiation at the surface; Rs_in (W m–2) is the downward shortwave radiation (0.15–4 μm) from the sun and atmosphere;
Rs_out (W m–2) is the upward shortwave radiation reflected by the surface; Rl_in (W m–2) is the longwave radiation (>4 μm) emitted from the atmosphere toward the
surface; and Rl_out (W m–2) is the longwave radiation emitted from the surface into the atmosphere.
Rs_in (W m–2) is derived from SOL_RAD (see Section 4.8); with Rs_out (W m–2) being derived
from Rs_in times the surface albedo (α ). Saunders (1990) estimated α as 0.5 R1 + 0.5 R2
where R1 and R2 are the per-pixel fractional reflectance of AVHRR channel 1 and channel 2,
respectively. As AVHRR reflective channels are used to determine α , it is important that
calibrations are applied to the AVHRR reflective channels. For the AVHRR data we used,
sensor degradation (i.e., responsivity and space count corrections) were applied to NOAA11
and NOAA14 as specified in CALWATCH (Mitchell 1999). To cast this in CALWATCH terms
this means that responsivity (S1 and S2) and space count (C01 and C02) corrections have
been applied to both channels 1 and 2 for NOAA11 and NOAA14 as per CALWATCH
CSIRO Land and Water Page 35
(Mitchell 1999). For NOAA16 and NOAA18 NESDIS pre-launch calibrations were used. To
cast this in CALWATCH terms this means that responsivity (S1 and S2) and space count
(C01 and C02) corrections were applied to both channels 1 and 2 for NOAA16 and NOAA18
as per NESDIS pre-launch calibrations. Since this processing has been performed it has
been shown that the time dependent changes in NOAA16 responsivity are stable with time
(Vermote and Saleous 2006), and NOAA18 has not been acquired for a long enough period
for its degradation to be characterised. Rl_in (W m–2) is derived from:
4_l in a aR Tε σ= (5)
where: εa is the effective atmospheric emissivity; σ is the Stefan-Boltzmann constant (5.67 x 10-8 W m-2 K-4); and Ta is air temperature at reference height (K). Prata’s (1996) model was used to determine εa as :
1/ 21 (1 )exp{ (1.2 3.0 ) }aε ξ ξ⎡ ⎤= − + − +⎣ ⎦ (6)
Where a
46.5T
aeξ⎛ ⎞
= ⎜ ⎟⎝ ⎠
.
where ea is the vapour pressure (hPa or, equivalently, millibars; note that kPa must be multiplied by 10 to convert them to hPa); and Ta is air temperature (K) at reference height.
Rl_out (W m–2) was derived from:
4_l out s sR Tε σ= (7)
where εs is the surface emissivity; and Ts is the surface temperature (K) derived from the radiance measured by thermal AVHRR channels 4
and 5 that are converted to brightness temperatures, denoted T4 and T5 respectively, using Plank’s law. These are then used in a split window algorithm Ts = T5 + 3 (T4 - T5) + 0.5. The split window coefficients were determined by simulating absorption for AVHRR channels 4 and 5 bandwidths using Australian mid-latitude atmospheric data (Maher and Lee 1977) and the LOWTRAN6 package (Kneizys et al. 1983). The same split window algorithm is used to generate the .OVP file.
With εs calculated as
ε ε εs v v v gf f= + −( )1 (8)
where: εv is the vegetation emissivity (0.98); εg is the ground (or soil) emissivity (0.96); and fv is the fraction of vegetation. The constant values for εv and εg were derived from (Wan and Dozier 1996), with fv being
derived from reflective remotely sensed data, using the dataset produced by Donohue et al.
(2007). The relatively small component of outgoing longwave radiation by reflected sky
radiation was ignored (Jackson et al. 1985, p. 156).
CSIRO Land and Water Page 36
IDL code Calc_Rn_Surface_WRON.pro (see Figure 21 and Appendix 10.10) outputs AVHRR
image specific (i.e., specific for the date and time of image acquisition) surfaces of Rn and Ts
– Ta by reading relevant AVHRR images, Vegetation coverage, ea, Ta, and Rs_in surfaces.
The Rs_out, Rl_in, Rl_out, α and Ts are calculated ‘on-the-fly’, reducing the requirement to store
intermediate files on disk – therefore greatly reducing the need for hard disk space.
Calc_Rl_in
Tair.GRDea.GRD Albedo.GRDTs.GRD frac_Veg.GRD Rs_in.GRD
AVHRR Chns 4 and 5.GRD
AVHRR Chns 1 and 2.GRD
Calc_Rl_out Calc_Rs_out(alb*Rs_in)
AVHRR Chns 1 and 2.GRD
Rl_in.GRD Rl_out.GRD Rs_out.GRD
Calc_Rn
Rn.GRD
Split Window Algorithm
Saunders (1990)
Algorithm
At_Tr, DEM (z,slp,asp), ovp time
SOL_RAD
Calc_Rn_Surface_WRON.pro
Ts-Ta.GRD
Ts - Ta
Figure 21. Flowchart showing how Rn and Ts-Ta are generated.
4.12 Run RADWAT to create files containing overpass information RADWAT is a FORTRAN executable program that performs specific time-of-day resistance
energy balance modelling (Jupp 1990; Jupp et al. 1998). Here remotely sensed based Ts
data (derived from AVHRR; any sensor with thermal capacity could be utilised) was used as
input with the REBM being run in a dual-source mode. Using dual-source mode, means that
temperatures of both the ground and vegetation are considered (being modeled as a fraction
of vegetation cover), and to provide the extra constraint needed to solve the dual-source
REBM here a ‘minimum power’ constraint (i.e., where the sum of the square of the latent and
CSIRO Land and Water Page 37
sensible heat fluxes from the ground and vegetation surfaces is minimized thereby
numerically simulating a conservative biophysical system) was solved. RADWAT allows
users to explore many aspects of the energy balance and options are available to run the
model in single-source mode (using both a Penman-Monteith and full case model) or by
providing the extras constraint in the dual-source that the aerodynamic temperature is
equivalent to the vegetation temperature. This constraint is not suitable to the vast areas of
low vegetation coverage in Australia. RADWAT also allows replacing forcing data from Ts to
either flux tower measurements of the sensible heat flux (H) or measurements of moisture
availability (the ratio of ETa / ETp). These options are primarily accessed where such
detailed flux tower data exist to support detailed processing understanding. As is common
for large area operational remote sensing-based systems (like that developed here)
meteorological data is not usually available for the specific time-of-day that the remotely
sensed data was acquired. To calculate specific time-of-day meteorological data RADWAT
implements the algorithms for temporal interpolation for air temperature, vapor pressure,
incoming shortwave radiation and wind speed reported in McVicar and Jupp (1999). From the AVHRR data the .OVP (or OVerPass) files are created for input into RADWAT
(along with the daily summary processed site files .PSF generated in Section 4.6; see also
Appendix 10.5). The specific time-of-day thermal remotely sensed driven output of RADWAT
are .RAD files, are saved as one file per station (i.e., many days per file (for each geographic
location where there is a BoM station)); see Figure 22. For input to ANUSPLIN, we needed
to convert these to one file per day (i.e., many stations per file (for each day in the time
series)). This is achieved by the IDL program called SortRadByImgAcq.PRO (Appendix
10.11) which produces the intermediate output file .NHW (or NDTI “Half Way” files).
Following McVicar and Jupp (2002), to generate specific time-of-day NDTI surfaces
ANUSPLIN was run using a quint-variate partial thin plate spline model, incorporating a bi-
variate thin plate spline function of longitude and latitude with constant linear dependencies
on Ts - Ta, VegCov and Rn. These data were extracted from the specific time-of-day surfaces
of Ts - Ta and Rn (generated in Section 4.11) and monthly surfaces of FC (generated in
Section 4.4) using the IDL program NHW2NIN.PRO (Appendix 10.11). In a previous
analysis of the sensitivity of the NDTI to its various forcing factors it was concluded that the
NDTI was relatively insensitive to perturbations in FC (McVicar and Jupp 2002), and as FC
changes more slowly than moisture availability, use of monthly time-step for FC data was
warranted.
The .NHW stands for NDTI “Half Way” file and .NIN stands for NDTI INput file (to SPLINA).
The data contained in each file are listed below. There are 5 columns (in order) in the .NHW
files being: 1:BoM station number; 2:longitude (dd); 3:latitude (dd); 4:DEM (m); and 5:NDTI
CSIRO Land and Water Page 38
(ratio). In the .NIN files 8 variables are stored. They are (in order): 1: BoM station number;
2:longitude (dd); 3:latitude (dd); 4:DEM (m); 5: Rn (W m-2); 6: Ts - Ta (°C) 7: FC (%); and 8:
NDTI (ratio).
.OVP (cloud free)
RAD_WAT.PSF(Daily
Summary)
.RAD (Cloud Free,
Specific Times)
SortRadByImageAcq.PRO
.NIN (NDTI)
Rn.GRD
.NHW (NDTI)
NHW2NIN.PROTs-Ta.GRD
Veg_Cov.GRD
Create_Parameter_For_Radwat_RAD.pro
Check_Cols.pro
(for NDTI interpolation)
*_Checked.txt
(from PRERAD)
AVHRR Images
AVHRR_Cloud_Check.pro
Met Station Dictionary
OVP_Creation_WRON.pro)
Figure 22. Flowchart showing how the .OVP, .RAD and .NIN files are created.
CSIRO Land and Water Page 39
4.13 Generate overpass NDTI surfaces NDTI surfaces for each relevant AVHRR image overpass time are interpolated using
ANUSPLIN Version 4.3 (Hutchinson 2004), specifically the SPLINA and LAPGRD programs
that are part of the ANUSPLIN package. Following McVicar and Jupp (2002), ANUSPLIN
was run with a quint-variate partial thin plate spline model, incorporating a bi-variate thin
plate spline function of longitude and latitude with constant linear dependencies on Ts - Ta,
VegCov and Rn. The .NIN files (Section 4.12) contain all variables needed by SPLINA
including geographic position, NDTI value (from RADWAT), as well as the specific time-of-
day values of Ts - Ta and Rn (generated in Section 4.11), and monthly values of FC
(generated in Section 4.4) generated at the BoM stations where the REBM was run. These
data are required as input into LAPGRD in order to generate the overpass NDTI surfaces.
Figure 23 schematically shows the steps needed to produce the NDTI surfaces, while
Appendix 10.12 shows the IDL code used.
SPLINA
Rn.GRD
NDTI.GRD
.SUR,.LOG
& others
LAPGRDTs-Ta.GRD
.NIN
Veg_Cov.GRD
SPLINA Daily .CMD file
Create_SPLINA_cmd_Files_NDTI.pro
LAPGRD CMD file
Create_LAPGRD_cmd_Files_NDTI.pro
Create_hdrs.PRO
Create_SPLINA_Batch_Runs_NDTI.pro
=
Create_LAPGRD_Batch_Runs_NDTI.pro
=
Figure 23. Flowchart of running ANUSPLIN to generate the overpass NDTI surfaces.
4.14 Generate fortnightly and monthly NDTI composites The individual specific time-of-day NDTI surfaces generated in section 4.13 above were
combined into both monthly (Section 5) and fortnightly (Section 9) composite images. The
IDL code used to perform the initial compositing is called Compostie_images.pro and is
shown in Appendix 10.13. To generate a single composite image, every specific time-of-day
CSIRO Land and Water Page 40
NDTI image within a specified compositing period was read in to memory and stacked into a
single large multi-band image. The mean value through the stack was then determined for
each x, y location. Instances of missing data due to cloud cover or out-of-range values were
identified and not used in determining the mean value for a given x, y location. NDTI values
were considered out-of-range if they were < 0 or > 1. The single-band mean composite of
the multi-band stack was saved to disk. This process was repeated for every compositing
period between April 1992, when our data record starts, and December 2006, when our data
record ends. To help understanding of the compositing procedure, an example from a
hypothetical 4-day compositing period is given in Figure 24 below.
Day 1 Day 2 Day 3 Day 4 Result
0.55 0.50 0.60 0.55 0.65 0.60 0.60 -9999 0.60 0.55
0.90 0.20 1.00 0.10 1.10 -0.15 1.05 0.15 0.95 0.15 Figure 24. Schematic of NDTI compositing procedure for a theoretical 4-day composite. The numbers in the first four 2 x 2 arrays (left to right) represent single overpass NDTI values at the same locations for four days. Null values (depicted by a -9999, here) and out-of-range values (<0 or >1) are coloured gray and are excluded from the resultant mean composite calculation. The result of the mean NDTI compositing procedure for the 4 day observation period is shown in the final 2 x 2 array: the top-left pixel of the result is the mean of the 4 input top-left pixels, the top-right pixel is the mean of the 3 valid input top-right pixels (Day 4 is excluded due to a null value), the bottom-left pixel is the mean of the two valid bottom-left pixels (Day 3 and Day 4 are excluded due to high out-of-range values), and the bottom-right pixel is the mean of the three valid bottom-right pixels (Day 3 is excluded due to a low out-of-range value). Please note, the compositing periods actually used for this report were fortnightly (nominally 15 days) and monthly (determined by number of days in each month). The minimum compositing period for our work was a fortnight, which was strictly defined as
two composites per month. The first fortnight always consisted of the first 15 days of the
month and the second fortnight consisted of the rest of the days in that month. This means
that the first fortnightly composite was always 15 days, while the second composite was 13
days for non leap year Februaries, 14 days for leap year Februaries, 15 days for 30-day
months, or 16 days for 31-day months. We also generated monthly composites, which were
defined by the number of days in the month, so these ranged between 28 and 31 days. In
most cases, the compositing algorithm accessed multiple specific time-of-day NDTI images
for each day. For example, when generating the 24 fortnightly composites for 2006, 518
specific time-of-day NDTI images were used, which was an average of 21.58 images per
compositing period or 1.42 images per day for 2006.
Any gaps in these initial monthly or fortnightly composites due to clouds or out-of-range
values were then filled based on the overall mean value for the specific month (Figure 26) or
CSIRO Land and Water Page 41
the overall mean value for the specific fortnight (Figure 50). The gap-filled pixles were
identified and saved in a separate image for each composite period. This means that the
original non gap-filled composite images can be recovered so other (more sophisticated)
methods can be assessed in the future. The gap-filled composites are displayed in Section 5
for monthly timesteps and in Section 9 for fortnightly timesteps. The IDL code used to
perform the gap filling of the initial composites is called
Fill_nulls_ndti_composites_hierarchy_2tier_only.pro and is shown in Appendix 10.14.
CSIRO Land and Water Page 42
5 Monthly Gap-filled NDTI Composite Images from 1992 to 2006
The following figures are the gap-filled monthly NDTI composites from 1992 to 2006,
inclusive. The layout of the 12 months is schematically shown below (Figure 25). Each
month is given by the first 3 letters of that month, with the gap-filling following the method
outlined in Section 4.14 used. Gap-filling means that if a cell were blank (this means no
AVHRR data were available, or there are currently no AVHRR data available to produce a
within-range NDTI image) it was filled by the monthly average of entire time series (shown in
Figure 26). For example, in 1994 Table 2 shows there were less images available compared
to most other years, this was due to no AVHRR being acquired late in 1994 (Figure 2).
Additionally, the AVHRR data that were acquired in the winter of 1994 were observed very
late in the day (when Rn is positive and low, or negative, as the sun is close to setting)
resulting in very small Ts – Ta differences, and while RADWAT is numerically stable enough
to solve such challenging input data, physically unreasonable NDTI values occurred during
that period. First we show the monthly average NDTI over the entire time series (Figure 26),
and following this monthly gap-filled composites from 1992 to 2006, see Figure 27 to Figure
41 respectively, are provided.
Jan Feb Mar
Apr May Jun
Jul Aug Sep
Oct Nov Dec
Figure 25. Schematic showing the layout of the progression of monthly composite images, where each month is given by the first 3 letters of that month.
CSIRO Land and Water Page 43
0 . 0 0 . 5 1 . 0 Figure 26. Overall average monthly NDTI composite images from 1992 to 2006, inclusive. When NDTI = 0.0 there is no moisture available for actual evapotranspiration (i.e., ETa = 0 mm day-1 or the environment is dry) and when NDTI = 1.0 moisture is freely available so actual evapotranspiration is occurring at its potential (i.e., ETa = ETp or the environment is wet). Black is the ocean.
CSIRO Land and Water Page 44
0 . 0 0 . 5 1 . 0 Figure 27. Monthly NDTI composite images for 1992 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 45
0 . 0 0 . 5 1 . 0
Figure 28. Monthly NDTI composite images for 1993 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 46
0 . 0 0 . 5 1 . 0
Figure 29. Monthly NDTI composite images for 1994 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 47
0 . 0 0 . 5 1 . 0
Figure 30. Monthly NDTI composite images for 1995 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 48
0 . 0 0 . 5 1 . 0
Figure 31. Monthly NDTI composite images for 1996 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 49
0 . 0 0 . 5 1 . 0
Figure 32. Monthly NDTI composite images for 1997 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 50
0 . 0 0 . 5 1 . 0
Figure 33. Monthly NDTI composite images for 1998 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 51
0 . 0 0 . 5 1 . 0
Figure 34. Monthly NDTI composite images for 1999 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 52
0 . 0 0 . 5 1 . 0
Figure 35. Monthly NDTI composite images for 2000 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 53
0 . 0 0 . 5 1 . 0
Figure 36. Monthly NDTI composite images for 2001 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 54
0 . 0 0 . 5 1 . 0
Figure 37. Monthly NDTI composite images for 2002 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 55
0 . 0 0 . 5 1 . 0
Figure 38. Monthly NDTI composite images for 2003 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 56
0 . 0 0 . 5 1 . 0
Figure 39. Monthly NDTI composite images for 2004 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 57
0 . 0 0 . 5 1 . 0
Figure 40. Monthly NDTI composite images for 2005 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 58
0 . 0 0 . 5 1 . 0
Figure 41. Monthly NDTI composite images for 2006 (see caption of Figure 26 for more detail).
CSIRO Land and Water Page 59
6 Conclusions The NDTI - a direct measurement of moisture availability (ma; the ratio of ETa to ETp) - has
been developed for almost 9,000 afternoon single-overpass AVHRR images from April 1992
to 2006. The NDTI is calculated by inverting a 2-source resistance energy balance model
which is run at the specific date-time of each overpass. Fortnightly and monthly gap-filled
composite NDTI images have been developed from these single-overpasses. The resultant
NDTI images clearly illustrate on a temporal and spatial basis a signal which is driven by
climatic variability.
Ecohydrological assessment of interactions between the NDTI and the commonly used NDVI
for three hydrologically relevant study sites (Lake Victoria, Coleambally Irrigation Area, and a
dryland agriculture-forest interface near Tumut) show, as expected, that the NDTI responds
more quickly to changes in water availability than the NDVI does. This has implications for
adequately modelling soil evaporation that NDVI-based models of ETa can not adequately
account for. This observation (i.e., a more rapid response of the NDTI than the NDVI to a
key component of the water balance) is made at both fortnightly and monthly time-steps.
NDTI surfaces are derived by linking meteorological point data and optical (both thermal and
reflective) remotely sensed data in a way that optimally utilises process understanding of ma
and the inherent strengths of these different data types. NDTI is generated using a
‘calculate-then-interpolate’ approach to best use the strengths of the two primary datasets -
meteorological data are temporally dense (and spatially sparse), whereas remotely sensed
data are spatially dense (and temporally sparse). The spatial interpolation is performed
using covariates that are heavily governed by the spatially dense remotely sensed data,
which allow for better modelling of ma in-between meteorological stations (e.g., over water
bodies, which would otherwise not be well represented). Importantly, the NDTI is readily calculated across large regions; it does not require interpolated gridded precipitation or irrigation data as inputs, and therefore provides a means to detect ‘spurious and unaccounted’ water application thus enabling compliance to be monitored. Given that
ma ≅ NDTI, and that ma = ETa / ETp, then multiplying the resultant NDTI surfaces by daily
ETp surfaces provides a means to measure daily ETa with units of mm day-1.
To rapidly process the almost 9,000 AVHRR images, and to routinely process current and
future remotely sensed data from any number of possible sensors, a prototype operational
system has been developed. The design of the information system to generate the NDTI is
modular, requires little operator input, and automatic error checking methodologies have
CSIRO Land and Water Page 60
been implemented. The completed prototype operational system could be extended to all
Australia. Further research quantifying the spatio-temporal variability and trends between
moisture availability (NDTI) and moisture utilization (NDVI) are warranted to assess risks to:
(1) landscape ecohydrological stability and functioning; and (2) agricultural production.
Resulting products and analysis would be suitable for distribution via on-line web-services as
they could be routinely produced and would be widely relevant to a number of water resource
and environmental managers and policy-makers. An example of one of the style of products
available from such time series analysis are the anomaly images (McVicar and Van Niel
2005) created by placing current conditions into historical context (McVicar et al. 2003).
While the remotely sensed data currently used to generate the NDTI are AVHRR, any sensor
with both thermal and reflective capacity – such as MODIS (MODerate Resolution Imaging
Spectroradiometer), ETM+ (Enhanced Thematic Mapper Plus), AATSR (Advanced Along-
Track Scanning Radiometer) and ASTER (Advanced Spaceborne Thermal Emission and
Reflection Radiometer) – could be used. This means that larger regional-scale studies can
be performed using 1 km resolution data (from the AVHRR, MODIS and AATSR
instruments), with smaller farm-to-paddock scale monitoring undertaken using 60 m
resolution ETM+ data and 90 m resolution ASTER data.
CSIRO Land and Water Page 61
7 References
Allen RG, Pereira LS, Raes D, Smith M (1998) 'Crop evapotranspiration - Guidelines for computing crop water requirements.' FAO Irrigation and Drainage Paper 56, Rome, Italy http://www.fao.org/docrep/X0490E/X0490E00.htm.
Asrar G, Fuchs M, Kanemasu ET, Hatfield JL (1984) Estimating absorbed photosynthetic radiation and leaf-area index from spectral reflectance in wheat. Agronomy Journal 76, 300-306.
Bristow KL, Campbell GS (1984) On the Relationship Between Incoming Solar Radiation and Daily Maximum and Minimum Temperature. Agricultural and Forest Meteorology 31, 159-166.
Camargo JLC, Kapos V (1995) Complex edge effects on soil moisture and microclimate in central Amazonian forest Journal of Tropical Ecology 11, 205-221.
Carlson TN, Ripley DA (1997) On the relation between NDVI, fractional vegetation cover, and leaf area index. Remote Sensing of Environment (China) 62, 241-252.
Donohue RJ, Roderick ML, McVicar TR (2007) 'Correcting long-term AVHRR reflectance data using the vegetation cover triangle.' CSIRO Land and Water, Canberra http://www.clw.csiro.au/publications/science/2007/sr26-07.pdf.
Giambelluca TW, Ziegler AD, Nullet MA, Truong DM, Tran LT (2003) Transpiration in a small tropical forest patch. Agricultural and Forest Meteorology 117, 1-22.
Hungerford RD, Nemani RR, Running SW, Coughlin JC (1989) 'MTCLIM: A mountain microclimate simulation model.' USDA Forest Service, INT-414
Hutchinson MF (2004) ANUSPLIN Version 4.3 User Guide, http://cres.anu.edu.au/outputs/software.php. In. (The Australian National University, Centre for Resource and Environmental Studies: Canberra)
Jackson RD, Pinter Jr. PJ, Reginato RJ (1985) Net Radiation Calculated from Remote Multispectral and Ground Station Meteorological Data. Agricultural and Forest Meteorology 35, 153-164.
Jupp DLB (1990) Constrained two-layer models for estimating evapotranspiration. In 'Proceedings of the 11th Asian Conference on Remote Sensing'. Guangzhou, China, November, 1990 pp. B.4.1-B.4.6
CSIRO Land and Water Page 62
Jupp DLB, Tian G, McVicar TR, Qin Y, Fuqin L (1998) 'Soil Moisture and Drought Monitoring Using Remote Sensing I: Theoretical Background and Methods.' CSIRO Earth Observation Centre, Canberra http://www.eoc.csiro.au/pubrep/scirpt/jstc1.pdf.
Kapos V (1989) Effects of isolation on the water status of forest patches in the Brazilian Amazon Journal of Tropical Ecology 5, 173-185.
King EA (1998) AVHRR Activities at the EOC in Canberra. In 'Proceedings of the Land AVHRR Workshop, 9th Australasian Remote Sensing and Photogrammetry Conference'. Sydney. (Ed. TR McVicar) pp. 59-65. (CSIRO Land and Water)
King EA (2000) Stitching the Australian 1 km archive. In 'Proceedings of the Land EnvSat Workshop, 10th Australasian Remote Sensing and Photogrammetry Conference'. (Ed. TR McVicar) pp. 41-49. (CSIRO Land and Water: Adelaide) http://www.clw.csiro.au/publications/technical2000/tr36-00.pdf.
King EA (2003) 'The Australian AVHRR Data Set at CSIRO/EOC: Origins, Processes, Holdings and Prospects.' CSIRO Earth Observation Centre Report 2003/04 (and CSIRO Atmospheric Research Report 2003/32), CSIRO Australia http://www.eoc.csiro.au/tech_reps/2003/tr2003_04.pdf.
Kneizys FX, Shettle EP, Abreu LW, Anderson GP, Chetwynd JH, Gallery WO, Selby JE, Clough SA (1983) 'Atmospheric transmittance/radiance: computer code LOWTRAN 6. Technical Report AFGL-TR-83-0187.' Optical Physics Division, U.S. Air Force Geophysics Laboratory, Hanscom Air Force Base, Massachusetts
Kustas WP, Humes KS (1997) Spatially distributed sensible heat flux over a semiarid watershed. Part II: Use of a variable resistance approach with radiometric surface temperatures. Journal of Applied Meteorology 36, 293-301.
Kustas WP, Jackson TJ (1999) The impact on area-averaged heat fluxes using remotely sensed data at different resolutions: A case study using Washita ‘92 Data. Water Resources Research 35, 1539-1550.
Kustas WP, Norman JM (1996) Use of remote sensing for evapotranspiration monitoring over land surfaces. Hydrological Sciences Journal des Sciences Hydrologiques 41, 495-516.
Lu H, Raupach MR, McVicar TR, Barrett DJ (2003) Decomposition of Vegetation Cover into Woody and Herbaceous Components Using AVHRR NDVI Time Series. Remote Sensing of Environment 86, 1-18.
Maher JV, Lee DM (1977) 'Upper Air Statistics Australia. Surface to 5 mb 1957 to 1975.' (Department of Science, Bureau of Meteorology: Canberra)
CSIRO Land and Water Page 63
McVicar TR, Briggs PR, King EA, Raupach MR (2003) 'A Review of Predictive Modelling from a Natural Resource Management Perspective: The Role of Remote Sensing of the Terrestrial Environment.' CSIRO Land and Water Client Report to the Bureau of Rural Sciences (also available as CSIRO Earth Observation Centre Report 2003/03 and CSIRO Atmospheric Research Report 2003/31), Canberra, Australia http://www.clw.csiro.au/publications/consultancy/2003/Review_Of_Remote_Sensing.pdf.
McVicar TR, Jupp DLB (1998) The current and potential operational uses of remote sensing to aid decisions on Drought Exceptional Circumstances in Australia: A Review. Agricultural Systems 57, 399- 468.
McVicar TR, Jupp DLB (1999) Estimating one-time-of-day meteorological data from standard daily data as inputs to thermal remote sensing based energy balance models. Agricultural and Forest Meteorology 96, 219-238.
McVicar TR, Jupp DLB (2002) Using covariates to spatially interpolate moisture availability in the Murray-Darling Basin: a novel use of remotely sensed data. Remote Sensing of Environment 79, 199-212.
McVicar TR, Li LT, Van Niel TG, Hutchinson MF, Mu XM, Liu ZH (2005) 'Spatially Distributing 21 Years of Monthly Hydrometeorological Data in China: Spatio-Temporal Analysis of FAO-56 Crop Reference Evapotranspiration and Pan Evaporation in the Context of Climate Change.' CSIRO Land and Water Technical Report, 8/05, Canberra, Australia http://www.clw.csiro.au/publications/technical2005/tr8-05.pdf.
McVicar TR, Van Niel TG (2005) 'Deriving moisture availability from time series remote sensing for drought assessment. Report to the Reference Group of the Australian Water Availability Project (AWAP).' CSIRO Land and Water Client Report, Canberra, Australia http://www.clw.csiro.au/publications/consultancy/2005/deriving_moisture_availability_AWAP.pdf.
McVicar TR, Van Niel TG, Li L, Hutchinson MF, Mu X, Liu Z (2007) Spatially distributing monthly reference evapotranspiration and pan evaporation considering topographic influences. Journal of Hydrology 338, 196-220.
Mitchell RM (1999) 'Calibration status of the NOAA AVHRR solar reflectance channels: CalWatch Revision 1.' CSIRO Atmospheric Research Technical Paper No. 42, Melbourne http://www.eoc.csiro.au/cats/calwatch/.
Moore ID, Norton TW, Williams JE (1993) Modelling Environmental Heterogeneity in Forested Landscapes. Journal of Hydrology 150, 717-747.
Murray-Darling Basin Commission (2006) Lake Victoria. In. (http://www.mdbc.gov.au/rmw/river_murray_system/lake_victoria)
CSIRO Land and Water Page 64
Murray FW (1967) On the computation of saturation vapour pressure. Journal of Applied Meteorology 6, 203-204.
Prata AJ (1996) A new long-wave formula for estimating downward clear-sky radiation at the surface. Quarterly Journal of the Royal Meteorological Society 122, 1127-1151.
Prescott JA (1940) Evaporation from water surface in relation to solar radiation. Transactions of the Royal Society of South Australia 64, 114-125.
Price JC (1991) Timing of NOAA afternoon passes. International Journal of Remote Sensing 12, 193-198.
Saunders RW (1990) The determination of broad band surface albedo from AVHRR visible and near-infrared radiances. International Journal of Remote Sensing 11, 49-67.
Smith M, Allen RG, Monteith JL, Perrier A, Pereira LS, Segeren A (1991) 'Expert consultation on procedures for revision of FAO guidelines for prediction of crop water requirements.' FAO Land and Water Development Division, Rome, Italy
Tetens O (1930) Über einige meteorologische Begriffe. Zeitschrift Geophysic 6, 297-309.
Van Niel TG, McVicar TR (2003) A simple method to improve field-level rice identification: Toward operational monitoring with satellite remote sensing. Australian Journal of Experimental Agriculture 43, 379-387.
Van Niel TG, McVicar TR (2004a) Current and potential uses of optical remote sensing in rice-based irrigation systems: a review. Australian Journal of Agricultural Research 55, 155-185.
Van Niel TG, McVicar TR (2004b) Determining temporal windows of crop discrimination with remote sensing: a case study in south-eastern Australia. Computers and Electronics in Agriculture 45, 91-108.
Van Niel TG, McVicar TR, Fang HL, Liang S (2003) Calculating environmental moisture for per-field discrimination of rice crops. International Journal of Remote Sensing 24, 885-890.
Vermote EF, Saleous NZ (2006) Calibration of NOAA16 AVHRR over a desert site using MODIS data. Remote Sensing of Environment 105, 214-220.
Wan Z, Dozier J (1996) A Generalized Split-Window Algorithm for Retrieving Land-Surface Temperature from Space. IEEE Transactions on Geoscience and Remote Sensing 34, 892-905.
CSIRO Land and Water Page 65
Wilson JP, Gallant JC (2000) Secondary Topographic Attributes. In 'Terrain Analysis'. (Eds JP Wilson and JC Gallant) pp. 87-131. (John Wiley & Sons, Inc.: New York)
CSIRO Land and Water Page 66
8 Appendix A: Monthly NDTI / NDVI Characterisation
For completeness, this appendix shows the same analysis for the same three areas used in
Section 3, expect data are provided at a monthly composite. That is NDTI and NDVI gep-
filled composite images are shown, the time-series from the transect running through the
centre of the composite image are shown, and pixel level time-series plots are shown. The
progression of the monthly images is shown in Figure 42. Results from this section highlight
the temporal detail provided at the fortnightly level (see Section 3) compared to the monthly
level provided here; where more generalised patterns (that may be more suited to some
management applications) are provided.
Jan Feb Mar
Apr May Jun
Jul Aug Sep
Oct Nov Dec
Figure 42. Schematic showing the layout of the progression of monthly composite images, where each month is given by the first 3 letters of that month.
CSIRO Land and Water Page 67
8.1 Lake Victoria
0 . 0 0 . 5 1 . 0 (a) NDTI ≅ moisture availability
- 0 .0 5 0 .4 7 1 .0 0
(b) NDVI ≅ vegetation vigour Figure 43. Monthly NDTI images (a) and monthly NDVI images (b) around the Lake Victoria transect for 2001.
CSIRO Land and Water Page 68
(a) NDTI ≅ moisture availability
(b) NDVI ≅ vegetation vigour
(c) (d) (e)
Figure 44. Monthly time series plot from 2001 for an 80 km transect located across Lake Victoria of (a) NDTI and (b) NDVI. The black line in (a) and (b) represent the profile of the first fortnightly compositing period in each case. Per-pixel time series are shown for pixels located at (c) 15 km, (d) 40 km, and (e) 65 km, respectively, with the black line being the NDTI and the grey line the NDVI.
CSIRO Land and Water Page 69
8.2 Coleambally Irrigation Area
0 . 0 0 . 5 1 . 0 (a) NDTI ≅ moisture availability
0 . 0 0 . 5 1 . 0 (b) NDVI ≅ vegetation vigour
Figure 45. Monthly NDTI images (a) and monthly NDVI images (b) around the CIA transect for 2001.
CSIRO Land and Water Page 70
(a) NDTI ≅ moisture availability
(b) NDVI ≅ vegetation vigour
(c) (d) (e)
Figure 46. Monthly time series plot from 2001 for an 80 km transect located across Coleambally Irrigation Area of (a) NDTI and (b) NDVI. The black line in (a) and (b) represent the profile of the first fortnightly compositing period in each case. Per-pixel time series are shown for pixels located at (c) 15 km, (d) 40 km, and (e) 65 km, respectively, with the black line being the NDTI and the grey line the NDVI.
CSIRO Land and Water Page 71
8.3 Forest - dryland agriculture interface at Tumut
0 . 0 0 . 5 1 . 0 (a) NDTI ≅ moisture availability
0 .0 0 0 .4 3 0 .8 5
(b) NDVI ≅ vegetation vigour Figure 47. Monthly NDTI images (a) and monthly NDVI images (b) around the forest-dryland agriculture transect near Tumut for 2001.
CSIRO Land and Water Page 72
(a) NDTI ≅ moisture availability
(b) NDVI ≅ vegetation vigour
(c) (d) (e)
Figure 48. Monthly time series plot from 2001 for an 80 km transect located across a forest-dryland agriculture interface near Tumut of (a) NDTI and (b) NDVI. The black line in (a) and (b) represent the profile of the first fortnightly compositing period in each case. Per-pixel time series are shown for pixels located at (c) 15 km, (d) 40 km, and (e) 65 km, respectively, with the black line being the NDTI and the grey line the NDVI.
CSIRO Land and Water Page 73
9 Appendix B: Fortnightly Gap-filled NDTI Composite Images from 1992 to 2006
The following figures are the fortnightly NDTI gap-filled composites from 1992 to 2006,
inclusive. The layout of the 24 fortnights is schematically shown below (Figure 49). Each
month is given by the first 3 letters of that month, and 1 and 2 mean the first fortnight (days 1
through 15) and second fortnight (days 16 through to the end of the month) respectively.
First we show the fortnightly average NDTI over the entire time series (Figure 50), following
this the fortnightly composites from 1992 to 2006, see Figure 51 to Figure 65 respectively,
are shown.
Jan_1 Jan_2 Feb_1 Feb_2
Mar_1 Mar_2 Apr_1 Apr_2
May_1 May_2 Jun_1 Jun_2
Jul_1 Jul_2 Aug_1 Aug_2
Sep_1 Sep_2 Oct_1 Oct_2
Nov_1 Nov_2 Dec_1 Dec_2
Figure 49. Schematic showing the layout of the progression of fortnightly composite images, where each month is given by the first 3 letters of that month, and 1 and 2 mean the first fortnight (days 1 through 15) and second fortnight (days 16 through to the end of the month) respectively.
CSIRO Land and Water Page 74
0 .0 0 .5 1 .0 Figure 50. Overall average fortnightly NDTI composite images for 1992 to 2006, inclusive. When NDTI = 0.0 there is no moisture available for actual evapotranspiration (i.e., ETa = 0 mm day-1 or the environment is dry) and when NDTI = 1.0 moisture is freely available so actual evapotranspiration is occurring at its potential (i.e., ETa = ETp or the environment is wet). Black is the ocean.
CSIRO Land and Water Page 75
0 . 0 0 . 5 1 . 0 Figure 51. Fortnightly NDTI composite images for 1992 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 76
0 . 0 0 . 5 1 . 0
Figure 52. Fortnightly NDTI composite images for 1993 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 77
0 . 0 0 . 5 1 . 0
Figure 53. Fortnightly NDTI composite images for 1994 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 78
0 . 0 0 . 5 1 . 0
Figure 54. Fortnightly NDTI composite images for 1995 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 79
0 . 0 0 . 5 1 . 0
Figure 55. Fortnightly NDTI composite images for 1996 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 80
0 . 0 0 . 5 1 . 0
Figure 56. Fortnightly NDTI composite images for 1997 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 81
0 . 0 0 . 5 1 . 0
Figure 57. Fortnightly NDTI composite images for 1998 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 82
0 . 0 0 . 5 1 . 0
Figure 58. Fortnightly NDTI composite images for 1999 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 83
0 . 0 0 . 5 1 . 0
Figure 59. Fortnightly NDTI composite images for 2000 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 84
0 . 0 0 . 5 1 . 0
Figure 60. Fortnightly NDTI composite images for 2001 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 85
0 . 0 0 . 5 1 . 0
Figure 61. Fortnightly NDTI composite images for 2002 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 86
0 . 0 0 . 5 1 . 0
Figure 62. Fortnightly NDTI composite images for 2003 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 87
0 . 0 0 . 5 1 . 0
Figure 63. Fortnightly NDTI composite images for 2004 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 88
0 . 0 0 . 5 1 . 0
Figure 64. Fortnightly NDTI composite images for 2005 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 89
0 . 0 0 . 5 1 . 0
Figure 65. Fortnightly NDTI composite images for 2006 (see caption of Figure 50 for more detail).
CSIRO Land and Water Page 90
10 Appendix C: IDL code This appendix contains all the IDL code referred to in Section 4 and represents the bulk of
the important programming that was performed for this research. The sub-heading titles
below relate to the relevant sub-sections in Section 4 in which specific IDL programs were
referred to.
10.1 IDL code to quality assess SILO Point Patch Database annually Pro checkMetYearly ;Tom Van Niel ;13-Apr-2006 ;generates report of data quality for all SILO PPD text files in the ;SiloPPDdirPath SiloPPDdirPath='W:\Data\MET\SILO_PPD\' SiloPPDTemplate='U:\idl_code\AWAP\templates\siloPPDTemplate.sav' outTextFile='W:\metCheckReportYearly.txt' yrs=27 CD, SiloPPDdirPath staList = FILE_SEARCH('*.txt') outArray=fltarr((yrs)+1,n_elements(staList)) For i=0,n_elements(staList)-1 Do Begin sta=staList(i) data=readSiloMet(SiloPPDTemplate, sta) sta=STRSPLIT( sta, '.', /EXTRACT) outArray[0,i]=float(sta(0)) For j=0, yrs-1 Do Begin lowVal=19800000+(j*ulong(10000)) highVal=lowVal+10000 yearIndex=where((data.(0) ge lowVal) and (data.(0) lt highVal),numDays) ;convert structure to an array before passing to staion qual function yearData=fltarr(8,numDays) For k=0,7 Do Begin yearData[k,*]=data.(k)[yearIndex] EndFor staQual=stationQualityYearly(yearData) outArray[j+1,i]=staQual EndFor print, sta EndFor OPENW, 1, outTextFile PRINTF, 1, outArray, format="(I8, "+strtrim(string(yrs),2)+"F8.2)" CLOSE, 1 End
CSIRO Land and Water Page 91
Function stationQualityYearly, data ;Tom Van Niel ;13-Apr-2006 ;performs data quality check on point patch SILO data ;returns 7 element fltarray of % orig met sta. data (non interp, etc) ;1st element is AND between Tmx,Tmn,P, the remaining 6 elements ; are Tmx,Tmn,P,Evap,Rdn,and VP ;e.g., ; staQual=stationQuality(data) flagList=[3,5,7] totNum=float(n_elements(data[0,*])) ;make tmp array to perform bitwise and b=bytarr(totNum) For m=0,2 Do Begin ;find location of non-interp met stations for the given var index=where(data[flagList(m),*] eq 0,count) ;put these in an array to perform bitwise and a=bytarr(totNum) if (count ne 0) then a[index]=1 dataQual=(count/totNum)*100 ;for the first 3 vars (Tx,Tn,P) do an "and" to find their intersection CASE m OF 0: b=a 1: b and= a 2: b and= a ENDCASE EndFor ;insert the AND of the first three vars into the first element of outarray tnNum=total(b) dataQual=(tnNum/totNum)*100. return, dataQual End
CSIRO Land and Water Page 92
10.2 IDL code to quality assess BoM wind data and convert from wind run to wind speed
Pro WindRun_to_WindSpeed ;Tom Van Niel ;01-Mar-2007 ;converts BOM wind run data to windspeed that will eventually be appended ;to SILO PPD data and saved to *.dat format for use in NDTI calculations ; ;this program outputs a single textfile for each station of BOM windrun ;data. The output file contains: Year, Month, Day, DOY, Windspeed ; ;These separate windspeed text files need to be appended to a copy of the ;SILO PPD data BEFORE conversion to *.dat format ; ;This program also outputs any years that have nDays <> 365 or 366 so ;they can be checked to make sure the data are all consecutive (no gaps) ; ;the following is output to the screen: ;STA, startyear, endyear, yearofinterest, nDaysinYearofInterest, ;startDayinYearofInterest, endDayinYearofInterest ; ;From this info, we can check that the yearofinterest (i.e., the year ;that might not have consecutive data)is in fact either the start or end ;year of record and that the nDays in theyearofinterest equals the ;endDay - startDay + 1 ; ; If the yearofinterest is either the start or end ;year and the nDays match endDay - startDay + 1, then everything is OK. ; ;set null data value ND=-2147483648 ;set ouput directory outname='W:\Data\MET\Wind_Data_BOM\Calculated_Windspeed_from_WindRun\' ;read in station file ENVI_READ_COLS, 'W:\spreadsheets_and_texts\stations_nostrings.txt', sta_array BIGstalist=sta_array(0,*) latlist=sta_array(3,*) ;read in and manipulate BOM wind run data BOMwindDirPath='W:\DATA\MET\wind_data_BOM\' BOMwindAsciiTemplateFN='U:\idl_code\AWAP\BOMwindAsciiTemplate.sav' ;outTextFile='U:\AWAP\spreadsheets_and_texts\BOM_wind_daylength.txt' CD, BOMwindDirPath staList = FILE_SEARCH('*DATA*.txt') summaryArray=lonarr(8,n_elements(staList)) For i=0,n_elements(staList)-1 Do Begin sta=staList(i) ;read in BOM wind run ascii file ;data = READ_ASCII(sta, DATA_START=1, DELIMITER=',') data=readBOMWindAscii(BOMwindAsciiTemplateFN, sta) windrun=data.(15) ;calc DOY for first record year=data.(2) month=data.(3)
CSIRO Land and Water Page 93
day=data.(4) qualityFlag=data.(16) accDays=data.(17) ;make sure to remove last record, which is a bogus line nrec=n_elements(data.(0))-1 year=year[0:nrec-1] month=month[0:nrec-1] day=day[0:nrec-1] qualityFlag=qualityFlag[0:nrec-1] accDays=accDays[0:nrec-1] DOY=day ;put doy of first day of each month in a list to use in DOY calculation doylist=[0,31,59,90,120,151,181,212,243,273,304,334] For j=1,12 do begin index=where(month eq j, count) if (count ne 0) then doy[index]=doy[index]+doylist[j-1] Endfor ;adjust doy if it is a leap year ; leap year if it is divisible by 4 but not by 100 ; If a year is divisible by 4 and by 100, it is not a leap ; year unless it is also divisible by 400 leapindex=where(((year mod 4 eq 0) AND (year mod 100 eq 0) AND (year mod 400 eq 0)) OR $ ((year mod 4 eq 0) AND (year mod 100 ne 0)) AND (month ge 3),leapcount) if (leapcount ne 0) then doy[leapindex]=doy[leapindex]+1 sta=STRSPLIT( sta, '_', /EXTRACT) stanum=long(sta(2)) sta=STRTRIM(STRING(stanum),2) ;now find the station number in the overall list and get its latitude staindex=where(bigstalist eq stanum, stacount) lat=latlist[staindex] ;we have the lat and the doylist, so call the daylength function N=calcDayLength(lat(0), doy) ;as the BOM wind run is predominantly measured on the previous day from ;which it was recorded, need to shift the wind run data to be associated ;with doy-1 e.g., windrun data is measured from 9am Monday to 9am Tuesday ;and then recorded at 9am Tuesday. This means that Tuesday's reading is ;mostly associated with Monday ; ;to fix this, truncate first record of windrun data and last record of ;doy and daylength ; ;when adding back to textfile, start at record 1, not 0 windrun=windrun[1:nrec-2];truncate first record for day shift and last record as it is just a blank line doy=doy[0:nrec-3];truncate last two lines - the last blank line and the 2nd last to match the day shift day=day[0:nrec-3] month=month[0:nrec-3] year=year[0:nrec-3] qualityFlag=qualityFlag[0:nrec-3] accDays=accDays[0:nrec-3] N=N[0:nrec-3]
CSIRO Land and Water Page 94
;1/3 windrun occurs at night, 2/3 occurs during day, ; so mult 2/3 by windrun and mult by 1000 to place units in m ; and divide by daylength in seconds (have to mult N*60*60) windspeed=(((2./3)*windrun)*1000.)/(N*60.*60) ;define first mask for windspeed - if windrun = 0, then windspeed = -1 ;(null) index=where((windspeed eq 0) OR (windrun eq ND), zerocount) ;null windspeed based on the first mask if (zerocount ne 0) then windspeed[index]=-1 ;define 2nd mask for windspeeds based on the qualityFlag and num days ;used to accum. the windrun ; and use the mask to null out any innappropriate values index=where((qualityFlag ne 'W') AND (qualityFlag ne 'S') AND (qualityFlag ne 'I') AND ((accDays eq 1) OR (accDays eq '')),passed, complement=nulls, ncomplement=failed) if (failed ne 0) then windspeed[nulls]=-1 ;now determine how many of the valid records pass and fail the quality ;and accumulation criteria qualFail=0 & accFail=0 if (Passed ne 0) then begin qualindex=where((qualityFlag eq 'W') OR (qualityFlag eq 'S') OR (qualityFlag eq 'I'), qualFail) accindex=where((accDays ne 1) AND (accDays ne ''), accFail) nindex=where(qualityflag eq 'N', ncount) endif summaryArray[*,i]=[stanum,nrec-2, passed, failed, qualFail,accfail,ncount,zerocount] ;split up the windspeed into yearly chunks and check whether there are ;any years with data that does not eq 365 or 366 ; if there are, then print them to the screen firstyear=year[0] lastyear=year[n_elements(year)-1] numyears=lastyear-firstyear+1 yearArray=fltarr(numyears,366) ;print, stanum, firstyear, lastyear For j=0,numyears-1 Do begin index=where(year eq firstyear+j,count) firstindex=index[0] lastindex=index[count-1] firstday=doy[firstindex] lastday=doy[lastindex] if ((count ne 366) AND (count ne 365)) then print, stanum, firstyear, lastyear, firstyear+j, count, firstday, lastday Endfor ;output station file to output directory outArray=[[year],[month],[day],[doy],[windspeed]] outArray=Transpose(outArray) OPENW, 1, outname+sta+'.txt' PRINTF, 1, outArray, format="(4I6, F12.4)" CLOSE, 1 EndFor ;now print out summary of missing and masked values print, 'Station',' nRecs',' validWindData',' Passed',' Failed',' FailedQual',' FailedAccum' print, summaryarray End function calcDayLength, lat, doy ;calculate daylength based on FAO-56 crop ET pub:
CSIRO Land and Water Page 95
; FAO book (ISBN 92-5-104219-5) Crop evapotranspiration - ; Guidelines for computing crop water requirements - ; FAO Irrigation and drainage paper 56 (http://www.fao.org/docrep/X0490E/x0490e00.htm). ;input must be a single latitude value and an array of day of year ;returns same size array of daylength in decimal hours ;e.g., ;ENVI> lat=37.5 ;ENVI> doy=indgen(365)+1 ;ENVI> N=calcDayLength(lat, doy) ;ENVI> print, N[0:5] ; 9.46857 9.47912 9.49052 9.50276 9.51582 9.52971 ;-------------------------------------------------------------------------- ; USING EQ's 24, 25, & 34: ; Daylight hours, N = (24/pi)*Ws ; Ws = sunset hour angle in radians = arccos [-tan (latitude) tan (sol_dec_angle ) ; sol_dec_angle = 0.409*sin(2pi/365*DOY-1.39) ;initialise variables radians=!pi/180. ;calculate solar declination angle solar_dec=0.409*sin(2.*!pi/365*doy-1.39) ;calculate sunset hour angle in radians Ws=acos(-1.*tan(radians*lat)*tan(solar_dec)) ;calculate daylength N=(24/!pi)*Ws ;return daylength array return, N End function readBOMWindAscii, templateFN, dataFN ;reads in met ascii files (e.g., BOM wind run) ; into pre-defined structure ;e.g., for BOM wind run: ; data=readBOMWindAscii('U:\AWAP\idl_code\BOMwindAsciiTemplate.sav', 'U:\AWAP\Data\MET\Wind_Data_BOM\DC02D_Data_090103_999999991387493.txt') ; ;restore the pre-defined ascii template for the met data ; after it is restored, it is a var called by this name restore, templateFN ;read in the data using the structure data = READ_ASCII(dataFN, TEMPLATE=BOMwindAsciiTemplate) return, data End
CSIRO Land and Water Page 96
10.3 IDL code to convert fPAR data to percent vegetation cover PRO fpar2cover, mth, res, dset ; Purpose: convert fPAR to coverage ; mth variable specifying the month - integer value ; dset array containing the returned dataset ; the Equation from Randal is c = 100 * F / 0.95 nd = -999. ; month 135 - 287 for mth = 288, 311 do begin openr,unit,string('W:\DATA\RS\corrected_fpar\fpar01_',mth,'.rst',format='(A,I0,A)'),/get_lun temp = assoc(unit,bytarr(4501,3501),0) dset = temp[0] & free_lun,unit nulCell = where (dset eq 0) ;null is 0 outC = 100 * (dset/100.0 - 1) /0.95 ;fPAR was scaled to 100 - 195. outC[nulCell] = nd ;put back null value OPENW, cIMG, 'W:\DATA\RS\corrected_fpar\cvrg01_'+strtrim(string(mth),2)+'.flt', /get_lun WRITEU, cIMG, outC close, cIMG free_lun, cIMG endfor END pro subset_f_coverage, imageName ;20 April 2007 ;Purpose: subset SEA part of the f_coverage surfaces from whole Australia ; surfaces ;Input: forest coverage surfaces which are converted from Randall's ; fpar surfaces ;Output: forest coverage surfaces just for south east australia shortFile = file_basename(imageName) randall_mth = fix(strmid(shortFile, 6,3, /REVERSE_OFFSET)) year = 1981 + randall_mth/12 month = randall_mth mod 12 + 1 year = strtrim(string(year),2);convert to string if month lt 10 then month = '0' + strtrim(string(month),2) else month = strtrim(string(month),2);convert to string ND = -9999. ;get the right vegetation coverage image and subset for SEA outPath='W:\Data\RS\corrected_fpar\converted_coverage\SE_Australia\ outName = year + month + '_' + strtrim(string(randall_mth),2) + '.flt' samples = 1601 lines = 1701 f_image = fltarr(samples,lines) if (file_test(imageName) eq 0) then print, 'f image not fount' else begin ;open coverage file f_samples = 4501 f_lines = 3501 in_cvrg = fltarr(f_samples,f_lines) subset_start = [2800, 1300];see above 2 lines OPENR, surface_in, imageName, /GET_LUN readu, surface_in, in_cvrg free_lun, surface_in ;get the right subset for South East Aus f_image = in_cvrg[subset_start[0]:subset_start[0]+samples -1, subset_start[1]: subset_start[1]+lines - 1] endelse ;change null from -999. to -9999. w=where (f_image eq -999., count)
CSIRO Land and Water Page 97
if (count gt 0) then begin f_image(w) = -9999. end ;Write out Rn image OPENW, surface_out, outPath + outName, /get_lun WRITEU, surface_out, f_image CLOSE, surface_out free_lun, surface_out End f_path = 'W:\Data\RS\corrected_fpar\converted_coverage\' f_List = file_search(f_path +'*.flt', count=FileNumber) If (FileNumber eq 0) then print, 'no f_image found' else begin ;For j=0, FileNumber-1 Do Begin For j=FileNumber-25, FileNumber-1 Do Begin subset_f_coverage, f_List[j] Endfor endelse end function fill_nulls, inImage, fillImage, ND ;TVN/29-Apr-2007/ ;finds nulls in an input image and fills it with values from another image ; ;assumes inImage and fillImage are same size and type and that ;the fillimage has no nulls outImage=inImage valid=where(inImage ne ND, vCount, complement=nulls, ncomplement=ncount) if (ncount ne 0) then outImage[nulls]=fillImage[nulls] return, outImage END pro fill_nulls_veg_cov_DOIT ;TVN/29-Apr-2007/ ;fills all nulls in each of the monthly veg cover data with monthly mean ;data ;initialise variables samples=1601 lines=1701 basepath='W:\Data\RS\corrected_fpar\converted_coverage\SE_Australia\' outpath=basepath+'\filled_nulls\' fillFileName=basepath+'monthly_mean.flt' ext='flt' ND=-9999. month_num_array=['01','02','03','04','05','06','07','08','09','10','11','12'] z=n_elements(month_num_array) outarray=fltarr(samples,lines) ;Read in fill data fImage = fltarr(samples,lines,z) ;Open input files OPENR, unit, fillFileName, /GET_LUN readu, unit, fImage free_lun, unit For month=0, z-1 DO BEGIN monthstring = month_num_array[month]
CSIRO Land and Water Page 98
fileList=FILE_SEARCH(basepath,'*'+monthstring+'_'+'*.'+ext) numImages=n_elements(fileList) mnthFillImage=fImage[*,*,month] For i=0, numImages-1 Do Begin ;Now read image data into a variable inVAR = fltarr(samples,lines) ;Open input files inVARName = fileList[i] OPENR, unit, inVARName, /GET_LUN readu, unit, inVAR free_lun, unit ;call the function that fills the in image outArray=fill_nulls(inVar, mnthFillImage, ND) ;write out stacked image OPENW, unit, outpath+file_basename(fileList[i]) WRITEU, unit, outarray CLOSE, unit free_lun, unit print, outpath+file_basename(fileList[i]) Endfor EndFor END
10.4 IDL code to generate annual .DAT files and .SS files pro make_dat_ss ;TVN/02-Mar-2007 ;Replaces fortran code called "dic2hdr" ;Generates silo *.dat file and SS file with correct headers ; takes in a dictionary file (station file) and silo PPD data ; gleans relevant info from station file to generate the correct ; header needed for processing, and merges with SILO PPD data to ; make the appropriate *.dat file. Also generates SS files. ; Puts data in separate directories by year ;initialise variables threshold=90 startyear=2006 path='W:\' SiloPPDdirPath=path+'Data\MET\SILO_PPD\' datbasepath=Path+'Data\RADCALC\DATS\' ssbasepath=Path+'Data\RADCALC\SS\' siloTemplateFN='U:\idl_code\AWAP\templates\siloPPDTemplate.sav' staTemplateFN='U:\idl_code\AWAP\templates\staTemplate.sav' staDataFN=path+'spreadsheets_and_texts\stations.txt' ;metCheckyearlyFN=path+'spreadsheets_and_texts\metCheckReportYearly.txt' metCheckyearlyFN=path+'metCheckReportYearly_2006only.txt' BOMwindAsciiTemplateFN='U:\idl_code\AWAP\BOMwindAsciiTemplate.sav' BOMwinddirPath=Path+'Data\MET\Wind_Data_BOM\Calculated_Windspeed_from_WindRun\' ;get sta list from BOM Wind directory CD, BOMwindDirPath BOMstaList = FILE_SEARCH('*.txt') If (n_elements(BOMstalist) eq 0) then print, kdjfkdjf
CSIRO Land and Water Page 99
;read in text files ;read in station text file using structure restore, staTemplateFN staData = READ_ASCII(staDataFN, TEMPLATE=staAsciiTemplate) ;read in metcheckyearly text file, which is a summary of quality of each station for a given year envi_read_cols, metCheckYearlyFN, metCheckYearly totStaList=metCheckYearly[0,*] ;For each year in metCheckYearly, define the list of valid stations based on the input threshold ;start at 1 to skip station identifier column For j=1, n_elements(metCheckYearly[*,0])-1 Do Begin ;get index of valid stations staIndex=where(metCheckYearly[j,*] GE threshold, nValid) staList=totStaList[staIndex] ;go through each valid station in stalist and extract relevant info For i=0, n_elements(staList)-1 Do Begin sta=STRTRIM(STRING(long(staList(i))),2)+'.txt' datpath=datbasepath+STRTRIM(STRING(startyear+j-1),2)+'\' sspath=ssbasepath+STRTRIM(STRING(startyear+j-1),2)+'\' data=readSiloMet(SiloTemplateFN, SiloPPDdirPath+sta) sta=STRSPLIT( sta, '.', /EXTRACT) ;define index for the year of interest yrIndex=where((data.(0) gt ((startyear+j-2)*(10000.)+1220)) AND $
(data.(0) lt ((startyear+j)*(10000.)+0111)),nDays) first=STRTRIM(STRING(data.(0)[yrIndex[0]]),2) first=STRMID(first, 0, 4)+' '+STRMID(first, 4, 2)+' '+STRMID(first, 6,$
2) ;find the selected station in the dictionary and get relevant info staIndex=where(staData.(0) eq long(sta(0))) staName = staData.(2)[staIndex] lat = staData.(5)[staIndex] latdeg = fix(lat) latmin = round((abs(lat) - abs(latdeg))*60.) lon = staData.(6)[staIndex] londeg = fix(lon) lonmin = round((abs(lon) - abs(londeg))*60.) elev = round(staData.(9)[staIndex]) CASE staData.(8)[staIndex] OF 'ACT': timeZone = 10 'NSW': timeZone = 10 'QLD': timeZone = 10 'TAS': timeZone = 10 'VIC': timeZone = 10 'NT': timeZone = 9.5 'SA': timeZone = 9.5 'WA': timeZone = 8 ENDCASE ; outarray=fltarr(5,nDays) outarray[0,*]=data.(1)[yrIndex] outarray[1,*]=data.(2)[yrIndex] outarray[2,*]=data.(4)[yrIndex] outarray[3,*]=data.(6)[yrIndex] outarray[4,*]=-1 ;now check to see if there is windspeed data avail for this station from ;BOM ; if so, then replace relevant part of windspeed staIndex=where(BOMstalist eq sta[0]+'.'+sta[1], staIndexCount)
CSIRO Land and Water Page 100
if (staIndexCount ne 0) then begin ;read in BOM wind SPEED ascii file for the given station ENVI_READ_COLS, BOMwindDirPath+sta[0]+'.'+sta[1], BOMdata ymd=BOMdata(0,*)*10000.+BOMdata(1,*)*100.+BOMdata(2,*) ;define index for the year of interest BOMyrIndex=where((ymd ge ((startyear+j-2)*(10000.)+1222)) AND (ymd le$
((startyear+j)*(10000.)+0110)),BOMnDays) if (nDays ne 0) then Begin ;get the wind data BOMwind=BOMdata[4,*] BOMwind=BOMwind[BOMyrIndex] ;find the starting point referenced through the doy doyStart=BOMdata[3,BOMyrIndex[0]] doyEnd=BOMdata[3,BOMyrIndex[BOMnDays-1]] oafirst=where(outarray[0,*] eq doyStart, oafnum) oalast=where(outarray[0,*] eq doyEnd, oalnum) ;overwrite the -1's in the wind data with the BOM values if (oafnum gt 1) then oafirst=min(oafirst) if (oalnum gt 1) then oalast=max(oalast) outarray[4,oafirst:oalast]=BOMwind endif endif ;output *.DAT file outfile = datpath+STRTRIM(STRING(sta(0)),2)+'.dat' OPENW, 1, outfile PRINTF, 1, STRTRIM(STRING(nDays),2), $ format="(A"+STRTRIM(fix(alog10(nDays)+1),2)+")" PRINTF, 1, staName, 'BOM ID',sta(0), $ format="(A"+STRTRIM(n_elements(aName),2)+", A7, A6)" PRINTF, 1, 'Date of first data (Year Month Day) PRINTF, 1, first, format="(A10)" PRINTF, 1, 'Latdeg Latmin Longdeg Longmin Time Zone PRINTF, 1, latdeg, latmin, londeg, lonmin, timeZone, format="(2I3,I4,I3,$
F5.1)" PRINTF, 1, 'Altitude (m) Slope (deg) Aspect (deg) NE & NW Horizons (deg) PRINTF, 1, elev,0.0,0.0,0.0,0.0, $ format="(I"+STRTRIM(fix(alog10(elev)+1),2)+".1, 4f4.1) PRINTF, 1, 'Base Station Z reference (Ht of screen (m))' PRINTF, 1, 2, format="(I1)" PRINTF, 1, 'Bristow & Campbell Atc(0), B & C Coefficients' PRINTF, 1, '0.807 0.175 0.849' PRINTF, 1, 'Daily Met Data (DOY Tmax (deg C) Tmin (deg C) Precip (mm)$
Wind (ms-1))' PRINTF, 1, outarray, format="(I3,4f8.2)" CLOSE, 1 ;Output the *.SS file outfile = sspath+STRTRIM(STRING(sta(0)),2)+'.ss' OPENW, 1, outfile PRINTF, 1, staName, 'Actual Pnt BOM ID',sta(0), $ format="(A"+STRTRIM(n_elements(aName),2)+", A18, A6)" PRINTF, 1, latdeg, latmin, londeg, lonmin, timeZone, format="(2I3,I4,I3,$
F5.1)" PRINTF, 1, elev,0.0,0.0,0.0,0.0, $ format="(I"+STRTRIM(fix(alog10(elev)+1),2)+".1, 4f4.1) PRINTF, 1, '1.0 0.15'
CSIRO Land and Water Page 101
CLOSE, 1 EndFor EndFor End function readSiloMet, templateFN, dataFN ;reads in silo Met station text file into pre-defined structure ;returns data SILO met data array ;e.g., ; data=readSiloMet('G:\AWAP\idl_code\siloPPDTemplate.sav',$ ; 'G:\AWAP\SILO_PPD\11001.txt') ; ;restore the pre-defined ascii template for the SILO met data ; this template was named siloAsciiTemplate and when restored ; is a var called by this name restore, templateFN ;read in the data using the structure data = READ_ASCII(dataFN, TEMPLATE=siloAsciiTemplate) return, data End function readBOMWindAscii, templateFN, dataFN ;reads in met ascii files (e.g., BOM wind run) ; into pre-defined structure ;e.g., for BOM wind run: ; data=readBOMWindAscii('U:\AWAP\idl_code\BOMwindAsciiTemplate.sav',$ ; 'U:\AWAP\Data\MET\Wind_Data_BOM\DC02D_Data_090103_999999991387493.txt') ; ;restore the pre-defined ascii template for the met data ; after it is restored, it is a var called by this name restore, templateFN ;read in the data using the structure data = READ_ASCII(dataFN, TEMPLATE=BOMwindAsciiTemplate) return, data End
10.5 IDL code to run PRERAD to generate .PSF files ;This pro create parameter files for prerad_b.exe, with one parameter file per year ;Started: 2007.03.09 ;By: Li Lingtao Pro parameter4Prerad rootDir = 'W:\DATA\RADCALC\' for year = 2006, 2006 do begin outFileDir = rootDir + 'psf\' + strtrim(string(year),2) ;check if the destination directory exists, create one if not outDirOK = file_search(outFileDir,COUNT=N_FOUND) IF N_FOUND EQ 0 THEN SPAWN, 'mkdir ' + outFileDir outFile = outFileDir + '\'+ strtrim(string(year),2)+'_prerad.par' openw, fOut, outFile, /get_lun; datFiles=file_search(rootDir+'dats\'+strtrim(string(year),2)+'\',$
'*.dat', count=datFileNum) datFileNameOnly = strarr(datFileNum) for i=0, datFileNum-1 do begin datFileNameOnly[i]=(strsplit(file_basename(datFiles[i]),'.',$
/extract))[0]
CSIRO Land and Water Page 102
endfor ssFiles=file_search(rootDir+'ss\'+strtrim(string(year),2)+'\',$ '*.ss', count=ssFileNum) ssFileNameOnly = strarr(ssFileNum) for i=0, ssFileNum-1 do begin ssFileNameOnly[i]=(strsplit(file_basename(ssFiles[i]),'.',$
/extract))[0] endfor for i=0, datFileNum-1 do begin ssFile = where (ssFileNameOnly eq datFileNameOnly[i], count) if count GT 0 then begin ;check if the output .psf exists, delete it if yes,
;otherwise this par file won't work outFileName = rootDir + $ 'psf\'+strtrim(string(year),2)+'\'+ $ datFileNameOnly[i]+'.psf' outFileOK = file_search(outFileName,COUNT=N_FOUND) IF N_FOUND EQ 1 THEN FILE_DELETE, outFileName;SPAWN, $
'del ' + outFileName printf, fOut,"b" printf, fOut, datFiles[i] printf, fOut, "s" printf, fOut, ssFiles[ssFile] printf, fOut, "d" printf, fOut, outFileName endif endfor printf, fOut,"q" close, fOut free_lun, fOut endfor end
10.6 IDL code to generate and interpolate daily atmospheric transmittance
pro SortPSFByDay ;TVN/02-Apr-2007 ;Takes in psfs and sorts all valid stations into separate ;files for each image acquisition ; ;initialise variables startyear=2006 endyear=2006 path='W:\Data\RadCalc\psf\' outpath='W:\Data\RADCALC\Spatial_Int_Daily\transmittance\' staFile='W:\spreadsheets_and_texts\stations_nostrings.txt' ;read in station file to get lat longs ;read in station file ENVI_READ_COLS, staFile, sta_array BIGstalist=sta_array(0,*) BIGlatlist=sta_array(3,*) BIGlonlist=sta_array(4,*)
CSIRO Land and Water Page 103
For i=startyear, endyear Do Begin ;get sta list from BOM Wind directory CD, path+strtrim(string(i),2) psfList = FILE_SEARCH('*.psf') If (n_elements(psflist) eq 0) then print, kdjfkdjf ;Read in each psf file and concat them together For j=0, n_elements(psflist)-1 Do Begin ;read in psf data into a variable called matrix data = READ_ASCII(path+strtrim(string(i),2)+'\'+psflist[j],$
DATA_START=14, DELIMITER=',') matrix=data.(0) dateArray=lonarr(n_elements(matrix[0,*])) ;However, since read_ascii is a hunk of garbage, it messes up the date ;field. Therefore, use this code to manually read overwrite matrix. ;Still use read_ascii above in order to automatically define the size ;of the matrix aline = '' startrec=14 counter=0 OPENR, unit, path+strtrim(string(i),2)+'\'+psflist[j], /GET_LUN while not eof (unit) do begin readf, unit, aline aRec=strSplit(aline, ',', /Extract) if (counter ge startRec) then datearray[counter-startrec]=aRec[1] counter++ endwhile free_lun, unit ;set up necesarry lists stalist=lonarr(n_elements(matrix[0,*])) lonList=fltarr(n_elements(matrix[0,*])) latList=fltarr(n_elements(matrix[0,*])) ;determine the station, lat, and lon for this file and populate the ;relevant lists ;station list a=strsplit(psflist[j],'.',/EXTRACT) a=long(a[0]) stalist[*]=a ;lon and lat lists aindex=where(BIGstaList eq a) lonlist[*]=BIGlonlist[aindex] latlist[*]=BIGlatList[aindex] ;insert station, lon, and lat lists into the BIG rad array matrix=[[matrix],transpose(stalist), transpose(lonlist),$ transpose(latlist)] ;concat each separate array together into a big array if (j eq 0) then Begin bigArray=matrix ImgID=transpose(datearray) Endif else Begin bigArray=[[bigArray],[matrix]] ImgID=[[ImgID],[transpose(dateArray)]] endelse Endfor ;Sort the big array based on image acquisition
CSIRO Land and Water Page 104
; get list of image acquisitions from the big array ;make a unique identifier for each overpass by using date (*100) plus ;time (to one decimal place) ;imgID needs to be rearranged first imgAcqList=imgID[UNIQ(imgID, SORT(imgID))] ;Loop through each unique image acquistion and get any valid data For k=0, n_elements(imgAcqList)-1 Do Begin index=where(imgID eq imgAcqList[k], count) ;write out all data for this day ;insert filter to the line below if needed (e.g., zen angle < 45 ;degrees) outarray=fltarr(4,count) outarray[0,*]=bigarray[11,index]; station number outarray[1,*]=bigarray[12,index]; longitude outarray[2,*]=bigarray[13,index]; latitude outarray[3,*]=round(bigarray[10,index]*100.); Trdir (trans % * 100) ;name outarray by image acq date and include all stations OPENW, 1, outpath+strtrim(string(i),2)+$
'\'+strtrim(string(long(imgAcqList[k])),2)+'.trn' ;PRINTF, 1, 'Station ', 'Lon ', 'Lat ', 'Trdir_(%*100)' PRINTF, 1, outArray, format="(I10, 2F10.4, I10)" CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+$
strtrim(string(long(imgAcqList[k])),2)+'.trn' Endfor Endfor End
pro Create_SPLINA_cmd_Files ;TVN/LLT Apr-2007 ;LLT Modified on 30 April 2007 ;create cmd files for each surface to be splined ; assumes a yearly directory structure ; requires a template *.cmd file as input at the base directory specified below ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Daily\transmittance\' outpath='W:\DATA\RADCALC\Spatial_Int_Daily\transmittance\SPLIN_cmd_files\' surfacepath='W:\DATA\RADCALC_SURFACES\At_Trans_0m' ext='trn' magicRecord=[19,20,21,22,23] template=outpath+'template_splna.cmd' ;make sure template exists Result = FILE_TEST(template) if (result ne 1) then print, dkfjkdj For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext) If (n_elements(List) eq 0) then print, kdjfkdjf For j=0, n_elements(List)-1 Do Begin filename=strsplit(list[j],'.', /Extract)
CSIRO Land and Water Page 105
mmdd=strmid(filename[0],4,4) ;read in the template file and modify the "magic" record where the ;filename is kept counter=0 aline='' sarray=strarr(1) OPENR, unit, template, /GET_LUN while not eof (unit) do begin readf, unit, aline if counter eq 14 then begin strput, aline, strtrim(string(i),2)+'\'+filename[0],48 endif else begin tmp = where (magicRecord eq counter+1, count) if count gt 0 then strput, aline, $ strtrim(string(i),2)+'\'+filename[0],64 endelse if (counter eq 0) then begin strput, aline, filename[0],0 $
sarray[0]=aline endif else sarray=[[sarray],[aline]] counter++ endwhile free_lun, unit ;output the new modified cmd file using input basename OPENW, 1, outpath+strtrim(string(i),2)+'\'+filename[0]+'_SPLINA.cmd' PRINTF, 1, sarray CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+filename[0]+'_SPLINA.cmd' Endfor Endfor End pro Create_Splina_Batch_Runs ;TVN/05-Apr-2007 ;create batch file to run all splina commands ;assumes a yearly directory structure ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Daily\transmittance\SPLIN_cmd_files\' outpath='W:\DATA\RADCALC\Spatial_Int_Daily\transmittance\SPLIN_cmd_files\' ext='cmd' aline1='splina' aline2='_SPLINA.bat' For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext) nRecs=n_elements(list) If (n_elements(List) eq 0) then print, kdjfkdjf sarray=strarr(nRecs) For j=0, nRecs-1 Do Begin filename=strsplit(list[j],'.', /Extract) sarray[j]=aline1+' < '+list[j]+' > '+filename[0]+'.log' Endfor
CSIRO Land and Water Page 106
;output the new modified cmd file using input basename OPENW, 1, outpath+strtrim(string(i),2)+'\'$
+strtrim(string(i),2)+aline2 PRINTF, 1, sarray CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 Endfor End
10.7 IDL code to calculate Rs_in at the date-time of image acquisition pro Rs_in_BatchRun_WRON ;based on Tom's createsplinbatruns.pro ;create batch file to run rs_in.bat (bringin, combine and sol_rad) for all .cov file times. ; startyear=2006 endyear=2006 ;rootPath='W:\Data\RS\AVHRR_Daily\' rootPath='W:\Data\RS\AVHRR_Daily\' outPath = 'W:\At_Trans\' ext='.hdr' TimeZone = 10 ;monthStr = ['01\','02\','03\','04\','05\','06\','07\','08\','09\','10\','11\','12\'] For i=startyear, endyear Do Begin avhrrPath = rootPath + strtrim(string(i),2);+ '\' + monthStr[m] fileList = file_search(avhrrPath, '*.hdr', count=FileNumber); If (FileNumber eq 0) then print, 'no avhrr image files' else begin cmdArray=strarr(FileNumber) For j=0, FileNumber-1 Do Begin filename=strsplit(fileList[j],'.', /Extract) theFile = file_basename(fileList[j]) dateTime = strmid(theFile, [5, 9, 11, 14], [4,2,2,4]) time_local = fix(dateTime[3])+TimeZone * 100 if time_local lt 1000 then time_local = strmid ('000' +$ strtrim(string(time_local),2), 3,4, /REVERSE_OFFSET ) else$ time_local = strtrim(string(time_local),2) cmdArray[j]='call rs_in '+ time_local+' '+dateTime[2]+'$ '+dateTime[1]+' '+dateTime[0] Endfor ;output the batch file OPENW, 1, outPath+strtrim(string(i),2)+ '\'$ +'call_rs_in_for_cov_'+strtrim(string(i),2)+'_new.bat' PRINTF, 1, cmdArray, format = '(a0)' CLOSE, 1 endelse Endfor End
10.8 IDL code to use .ALL files in RADWAT to create .COV files ;Purpose: Create INT file ;Input: 1 ;Output: Many txt files (one station one file, containing tempral AVHRR dada, see lines below for the ;format) ;Phoenix Overpass Data ;17 2 1 2 1 1.013 ;D 67 19830308 800 283.5 89.5 0.40 1.20 108.0 283.9 88.3 ;D 67 19830308 830 285.6 89.5 0.40 2.00 247.1 285.3 83.7
CSIRO Land and Water Page 107
;Steps: ;1. get all checked AVHRR data first and sort by date - outside this function ;2. read in one checked AVHRR file and write out many (in the INT format shown above) ;3. read the second checked AVHRR file and write out many (append to files created in above step) ;History: 13 March 2007 add function to extract coverage data ;Modified to WRON server a)trunk the seconds in file names (previous version uses rounding) ; b)change file locations ; c)read f imgae from SEA dir drectly Function get_maxDay, year, month m = year mod 4 if (m eq 0 && month eq 2) then begin k = 29 endif else begin case month of '02': k = 28 '04': k = 30 '06': k = 30 '09': k = 30 '11': k = 30 else: k = 31 endcase endelse return, K end Pro WriteStationFile, outputDir, avhrrFileName, staCode ;read in one record - avhrrValue - from Cloud check data, ;use avhrr image name as a key to write out one station file ;avhrrValue array columns timeZone_h = 10 timeZone_m = 0 imgNameStr = strsplit(file_basename (avhrrFileName), '_', /extract) theYear=fix(strmid(imgNameStr[1], 0,4)) theMonth=fix(strmid(imgNameStr[1], 4,2)) theDay=fix(strmid(imgNameStr[1], 6,2)) hh=fix(strmid(imgNameStr[2], 0,2));hour mm=fix(strmid(imgNameStr[2], 2,2));minute hh = hh + timeZone_h ;GMT time to local time mm = mm + timeZone_m if mm GE 60 then begin mm = mm - 60 hh = hh + 1 endif if hh GE 24 then begin maxDay = get_maxDay(theYear, theMonth) hh = hh - 24 theDay = theDay + 1 if theDay GT maxDay then begin theDay = theDay - maxDay theMonth = theMonth + 1 endif if theMonth GT 12 then begin theMonth = theMonth - 12 theYear = theYear + 1 endif
CSIRO Land and Water Page 108
endif DoY = JULDAY(theMonth, theDay,theYear) - JULDAY(1,0,theYear);day of the year ;CONVERT DATE/TIME TO STRING theYear = strtrim(string(theYear),2) if theMonth lt 10 then theMonth = '0' + strtrim(string(theMonth),2) else $
theMonth = strtrim(string(theMonth),2) if theDay lt 10 then theDay = '0' + strtrim(string(theDay),2) else$
theDay = strtrim(string(theDay),2) if hh lt 10 then hh = '0' + strtrim(string(hh),2) else hh = strtrim(string(hh),2) if mm lt 10 then mm = '0' + strtrim(string(mm),2) else mm = strtrim(string(mm),2) outFile = outputDir + strtrim(string(staCode, format = '(i)'),2)+'.INT' if (file_test(outFile) eq 1) then begin ; if the file already exists, append the record openw, /append, fout, outFile, /get_lun endif else begin openw, fout, outFile, /get_lun; otherwise, create one. printf, fout, string(staCode, format = '(i)') +' Overpass Data' ;format= '()';need to modify printf, fout, '17 2 1 2 0 0.411554';format= '()';need to modify endelse ;calculate data needed in INT file SurfT = '-1' Cover = '-1' Height = '-1';use -1 for RadCalc if data missed Wind = '-1' SolarIr = '-1' AirT = '-1' Humidity = '-1' ;need to modify AVHRR value printf, fout, 'D ', DoY, theYear+theMonth+theDay,hh+mm,SurfT, Cover,$ Height,Wind,SolarIr,AirT,Humidity, format = '(a1,1x,i3,1x,a8,1x,a4,a7,1x,a6,5a3)' close, fout free_lun, fout end Pro ManyPts2ManyDates, rootDir_INT, avhrrName, year ;this procedure reads in station info file and find out the valid ones for a specific year theYearCol = year - 1980 ;AVHRR Year = 2004 threshold_staCheck = 90; threshold of valid stations checkedMetSta = 'W:\spreadsheets_and_texts\metCheckReportYearly.txt';checked stations nCols = 0 ; the number of columns in the input file nline1 = 0 ; number of lines in checked met station file aline = ' ' openr, f_MetSta1, checkedMetSta, /get_lun readf, f_MetSta1, aline nCols = n_elements(strsplit(aline, STRING(9B) )) while not eof(f_MetSta1) do begin nline1++ readf, f_MetSta1, aline endwhile point_lun, f_MetSta1, 0; put pointer back arrMetSta1 = lonarr(nCols, nline1);one more column for the station code ;readf, f_MetSta1, arrMetSta1; this is data
CSIRO Land and Water Page 109
envi_read_cols, checkedMetSta, arrMetSta1 tmp = where (arrMetSta1[theYearCol+1,*] GE threshold_staCheck, nValid);if the valus is 1, then the station (line) is OK for the year(column), arrValidSta =fltarr(nValid) arrValidSta = (arrMetSta1[0,*])[tmp] ;first col is station code close, f_MetSta1 free_lun, f_MetSta1 nameStr = strsplit(file_basename (avhrrName), '_', /extract);ah16 20010102 052360 37 Checked.txt for i = 0, n_elements(arrValidSta)-1 do begin WriteStationFile, rootDir_INT, avhrrName, arrValidSta[i] endfor end Pro Modify_INT, inFile ;this procedure modifies the INT file created above (pro WriteStationFile) ;it reads in an INT file, counts the number of records, then wtites out a modified INT with correct line 2. aline = '' nLines = 0 openr, tmpFile, inFile, /get_lun while not eof (tmpFile) do begin readf, tmpFile, aline nLines++ ;number of lines of the INT file endwhile POINT_LUN, tmpFile, 0 tmpArray = strarr(1,nLines) readf, tmpFile, tmpArray ;read in as a one dimensional string array close, tmpFile free_lun, tmpFile ;change first few charactors of line2 pos = strpos(tmpArray[0,1], ' ');end of first number in the line part2= strmid(tmpArray[0,1],pos);get the rest of the line newNum = strtrim(string(nlines - 2),2);number of station records = total lines - two header lines tmpArray[0,1] = newNum + part2 ; modify line 2 openw, tmpFile, inFile, /get_lun printf, tmpFile, tmpArray; write out the new INT file close, tmpFile free_lun, tmpFile end ;Starting point for the function. Get all outputs from AVHRR_Cloud_Check.pro ;rootDir = 'G:\AWAP\RADCALC_test_for_LLT_only\' for year = 2006,2006 do begin ; rootDir_INT = 'W:\DATA\RADCALC\INT\2001\' ;rootDir = 'W:\DATA\RS\AVHRR_Daily\2001\' rootDir_INT = 'W:\DATA\RADCALC\INT\'+strtrim(string(year), 2)+'\' rootDir = 'W:\Data\RS\AVHRR_Daily\'+strtrim(string(year), 2)+'\' files=file_search(rootDir, '*.hdr', count=FileNumber) if FileNumber GT 0 then begin for i=0, FileNumber-1 do begin ManyPts2ManyDates, rootDir_INT, files[i], year
CSIRO Land and Water Page 110
endfor endif ;if the output INT files need to be modified (say for number of Records), do it here. files2=file_search(rootDir_INT, '*.INT', count=INTFileNumber) if FileNumber GT 0 then begin for i=0, INTFileNumber-1 do begin Modify_INT, files2[i] endfor endif endfor end ;This pro create parameter files for radwat.exe, with one parameter file per year ;Started: 2007.03.09 ;By: Li Lingtao ;The difference between this code and Create_Parameter_For_Radwat.pro is: ;this one needs .int and .psf to generate .all file Pro Create_Parameter_For_Radwat_All rootDir = 'W:\DATA\RADCALC\' for year = 2006, 2006 do begin outFileDir = rootDir + 'all\' + strtrim(string(year),2) ;check if the destination directory exists, create one if not outDirOK = file_search(outFileDir,COUNT=N_FOUND) IF N_FOUND EQ 0 THEN SPAWN, 'mkdir ' + outFileDir outFile = outFileDir + '\'+ strtrim(string(year),2)+'_radwat.par' openw, fOut, outFile, /get_lun; psfFiles=file_search(rootDir+'psf\'+strtrim(string(year),2)+'\', '*.psf', count=psfFileNum) psfFileNameOnly = strarr(psfFileNum) for i=0, psfFileNum-1 do begin psfFileNameOnly[i]=(strsplit(file_basename(psfFiles[i]),'.', /extract))[0] endfor intFiles=file_search(rootDir+'int\'+strtrim(string(year),2)+'\', '*.int', count=intFileNum) intFileNameOnly = strarr(intFileNum) for i=0, intFileNum-1 do begin intFileNameOnly[i]=(strsplit(file_basename(intFiles[i]),'.', /extract))[0] endfor for i=0, intFileNum-1 do begin psfFile = where (psfFileNameOnly eq intFileNameOnly[i], count) if count GT 0 then begin ;check if the output .all exists, delete it if yes, otherwise this par file won't work outFileName = rootDir + 'all\'+strtrim(string(year),2)+'\'+ $ intFileNameOnly[i]+'.all' outFileOK = file_search(outFileName,COUNT=N_FOUND) IF N_FOUND EQ 1 THEN FILE_DELETE, outFileName;SPAWN, 'del ' +$
outFileName printf, fOut,"s" printf, fOut, psfFiles[psfFile] printf, fOut, "r" printf, fOut, intFiles[i] printf, fOut, outFileName endif
CSIRO Land and Water Page 111
endfor printf, fOut,"q" close, fOut free_lun, fOut endfor end pro SortAllByImageAcq ;TVN/17-Apr-2007 ;Takes in output from RadCalc and sorts all stations into separate ;files for each image acquisition. ; ;Outputs sorted data by image acquisition date for input to spline (spline in, ;called "cov") ; ;initialise variables startyear=2006 endyear=2006 path='W:\Data\RADCALC\ALL\' outpath='W:\Data\RADCALC\Spatial_Int_Overpass\' staFile='W:\spreadsheets_and_texts\stations_nostrings_DEM.txt' ND = -9999. ;read in station file to get lat longs ;read in station file ENVI_READ_COLS, staFile, sta_array BIGstalist=sta_array(0,*) BIGlatlist=sta_array(3,*) BIGlonlist=sta_array(4,*) BIGelevlist=sta_array(5,*) BIGDEMlist=sta_array(10,*) For i=startyear, endyear Do Begin ;get sta list from BOM Wind directory CD, path+strtrim(string(i),2) radList = FILE_SEARCH('*.all') If (n_elements(radlist) eq 0) then print, kdjfkdjf ;Read in each rad file (i.e., one station at a time) and concat them together For j=0, n_elements(radlist)-1 Do Begin envi_read_cols, path+strtrim(string(i),2)+'\'+radlist[j], matrix ;set up necesarry lists stalist=lonarr(n_elements(matrix[0,*])) lonList=fltarr(n_elements(matrix[0,*])) latList=fltarr(n_elements(matrix[0,*])) demList=fltarr(n_elements(matrix[0,*])) elevList=fltarr(n_elements(matrix[0,*])) ;determine the station, elev, lat, and lon for this file and populate the relevant lists ;station list a=strsplit(radlist[j],'.',/EXTRACT) a=long(a[0]) stalist[*]=a ;lon and lat lists aindex=where(BIGstaList eq a) lonlist[*]=BIGlonlist[aindex] latlist[*]=BIGlatList[aindex] demlist[*]=BIGDEMList[aindex]
CSIRO Land and Water Page 112
elevlist[*]=BIGelevList[aindex] ;check if DEM eq -999 (null), if so, replace with station elev from dictionary nullDEM=where(demlist eq -999, nullDEMcount) if (nullDEMcount ne 0) then Begin demlist[nullDEM]=elevlist[nullDEM] nullELEV=where(elevlist[nullDEM] eq -1, nullelevcount) ;if elev from dictionary is null (-1) then stop program - this needs ;to be fixed. Bad idea to make station elev nulls eq -1 as this is a valid ;elevation - need to get original elevs again, then we can do something ;different here. if (nullelevcount ne 0) then print, dkfjdkj endif ;insert station, lon, and lat and DEM lists into the BIG rad array matrix=[[matrix],transpose(stalist), transpose(lonlist), transpose(latlist), transpose(demlist)] ;concat each separate array together into a big array if (j eq 0) then bigArray=matrix else bigArray=[[bigArray],[matrix]] Endfor ;Add Ea and VPD columns to the big array based on LLT's code ;which uses Tair and Rh ; E0 =0.6108 * exp (17.27 * bigarray[8,*] / (bigarray[8,*] + 237.3)) Ea = bigarray[9,*]/100.0 * E0 VPD = E0 - Ea nd_Tair = where (bigarray[8,*] eq ND, count_Ta) nd_Rh = where (bigarray[9,*] eq ND, count_Rh) if count_Ta NE 0 then begin Ea [nd_Tair] = ND VPD [nd_Tair] = ND endif if count_Rh NE 0 then begin Ea [nd_Rh] = ND VPD [nd_Rh] = ND endif ;insert Ea and VPD into the BIG rad array bigarray=[[bigarray],Ea,VPD] ;Sort the big "ALL" array based on image acquisition ; get list of image acquisitions from the big array ;make a unique identifier for each overpass by using date (*10000) plus time (hhmm) imgID=(bigArray[2,*]*10000.)+bigArray[5,*] imgAcqList=imgID[UNIQ(imgID, SORT(imgID))] ;Loop through each unique image acquistion and get associated data For k=0, n_elements(imgAcqList)-1 Do Begin ;split up the uniq image acq ids into date and time str=STRTRIM(STRING(ulong64(imgAcqlist[k])),2) imgDate=STRMID(str,0,8) imgTime=STRMID(str,8,4) ;define the items in the big array that have this date and time index=where((bigArray[2,*] eq long(imgDate)) and (bigArray[5,*] eq fix(imgTime)), count)
CSIRO Land and Water Page 113
;write out specified data for this image acquisition date outarray=fltarr(8,count) outarray[0,*]=bigarray[33,index]; station number outarray[1,*]=bigarray[34,index]; longitude outarray[2,*]=bigarray[35,index]; latitude outarray[3,*]=bigarray[36,index]; DEM outarray[4,*]=bigarray[8,index]; Ta (Deg C) outarray[5,*]=bigarray[9,index]; Rh (%) outarray[6,*]=bigarray[37,index]; Ea (kPa) outarray[7,*]=bigarray[38,index]; VPD (kPa) OPENW, 1, outpath+strtrim(string(i),2)+'\'+strtrim(string(long(imgDate)),2)+'_'+imgTime+'.cov' PRINTF, 1, outArray, format="(I10, 7F10.4)" CLOSE, 1 Endfor Endfor End
10.9 IDL code to interpolate Ta and ea at the date-time of image acquisition
pro Create_SPLINA_cmd_Files_Ta_Ea_VPD ;TVN/LLT Apr-2007 ;create cmd files for each surface to be splined for overpass variables (Ta, Ea and VPD) ;assumes a yearly directory structure ;requires a template *.cmd file as input at the base directory specified below ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Overpass\' outpath='W:\DATA\RADCALC\Spatial_Int_Overpass\SPLIN_cmd_files\' ext='cov'; magicRecord=[20,21,22,23,24] template=outpath+'template_Ta_Ea_VPD_splna.cmd' ;make sure template exists Result = FILE_TEST(template) if (result ne 1) then print, dkfjkdj For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext, count=n) If (n eq 0) then print, kdjfkdjf For j=0, n_elements(List)-1 Do Begin filename=strsplit(list[j],'.', /Extract) ;read in the template file and modify the "magic" record where the filename is kept counter=0 aline='' sarray=strarr(1) OPENR, unit, template, /GET_LUN while not eof (unit) do begin readf, unit, aline if counter eq 15 then begin strput, aline, strtrim(string(i),2)+'\'+filename[0],37
CSIRO Land and Water Page 114
endif else begin tmp = where (magicRecord eq counter+1, count) if count gt 0 then strput, aline, strtrim(string(i),2)+'\'+filename[0],53 endelse if (counter eq 0) then begin strput, aline, filename[0],0 sarray[0]=aline endif else sarray=[[sarray],[aline]] counter++ endwhile free_lun, unit ;output the new modified cmd file using input basename OPENW, 1, outpath+strtrim(string(i),2)+'\'+filename[0]+'_Ta_Ea_VPD_SPLINA.cmd' PRINTF, 1, sarray CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+filename[0]+'_Ta_Ea_VPD_SPLINA.cmd' Endfor Endfor End pro Create_SPLINA_Batch_Runs_Ta_Ea_VPD ;TVN/05-Apr-2007 ;create batch file to run all splina commands for overpass variables (Ta, Ea and VPD) ;assumes a yearly directory structure ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Overpass\SPLIN_cmd_files\' ext='cmd' aline1='splina' aline2='_SPLINA.bat' For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext) nRecs=n_elements(list) If (n_elements(List) eq 0) then print, kdjfkdjf sarray=strarr(nRecs) For j=0, nRecs-1 Do Begin filename=strsplit(list[j],'.', /Extract) sarray[j]=aline1+' < '+list[j]+' > '+filename[0]+'.log' Endfor ;output the batch file OPENW, 1, path+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 PRINTF, 1, sarray CLOSE, 1 print, path+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 Endfor End pro Create_Lapgrd_cmd_Files_Ta_Ea_VPD ;TVN/LLT Apr-2007
CSIRO Land and Water Page 115
;LLT modified 30 April 2007 ;create cmd files for each surface to be lapgrd-ed for overpass variables (Ta, Ea and VPD) ;assumes a yearly directory structure ;requires a template *.cmd file as input at the base directory specified below ;initialise variables startyear=2006 endyear=2006 path = 'W:\DATA\RADCALC\Spatial_Int_Overpass\' outpath='W:\DATA\RADCALC\Spatial_Int_Overpass\LAPGRD_cmd_files\' ext='cov' template=outpath+'template_Ta_Ea_VPD_lapgrd.cmd' ;make sure template exists Result = FILE_TEST(template) if (result ne 1) then print, dkfjkdj For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext, count = n) If (n eq 0) then print, kdjfkdjf For j=0, n_elements(List)-1 Do Begin filename=strsplit(list[j],'.', /Extract) mmdd=strmid(filename[0],4,4) ;read in the template file and modify the "magic" record where the filename is kept counter=0 aline='' sarray=strarr(1) OPENR, unit, template, /GET_LUN while not eof (unit) do begin readf, unit, aline if counter eq 14 then begin strput, aline, strtrim(string(i),2)+'\Ta\'+filename[0],28 endif else if counter eq 15 then begin strput, aline, strtrim(string(i),2)+'\Ea\'+filename[0],28 endif else if counter eq 16 then begin strput, aline, strtrim(string(i),2)+'\VPD\'+filename[0],28 endif else begin if counter eq 0 then strput, aline, strtrim(string(i),2)+'\'+filename[0],53 endelse if (counter eq 0) then sarray[0]=aline else sarray=[[sarray],[aline]] counter++ endwhile free_lun, unit ;output the new modified cmd file using input basename OPENW, 1, outpath+strtrim(string(i),2)+'\'+filename[0]+'_Ta_Ea_VPD_LAPGRD.cmd' PRINTF, 1, sarray CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+filename[0]+'_Ta_Ea_VPD_LAPGRD.cmd' Endfor Endfor End pro Create_LAPGRD_Batch_Runs_Ta_Ea_VPD ;TVN/05-Apr-2007
CSIRO Land and Water Page 116
;Modified LLT 30 April 2007 ;create batch file to run all LAPGRD commands for Overpass variables (Ta, Ea and VPD) ;assumes a yearly directory structure ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Overpass\LAPGRD_cmd_files\' ext='cmd' aline1='LAPGRD' aline2='_LAPGRD.bat' For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext) nRecs=n_elements(list) If (n_elements(List) eq 0) then print, kdjfkdjf sarray=strarr(nRecs) For j=0, nRecs-1 Do Begin filename=strsplit(list[j],'.', /Extract) sarray[j]=aline1+' < '+list[j]+' > '+filename[0]+'.log' Endfor ;output the batch file OPENW, 1,path+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 PRINTF, 1, sarray CLOSE, 1 Endfor End
10.10 IDL code to generate Rn and Ts-Ta at the date-time of image acquisition
pro Calc_Rn_Surface_WRON, yearStr, imageName ;19 April 2007 ;Purpose: Calculate Rn and Tdiff (Tdif = Ts - Ta) ;Input: Rs_in surfaces, and Rs_out, Rl_in, Rl_out on-fly ;Output: Rn surface, and Tdif surface ;Equation: Rn = Rs_in - Rs_out + Rl_in - Rl_out ;27 April 2007 Modified to read subset f images ;4 may 2007, edit for WRON timeZone = 10 dateTime = strmid(imageName, [0, 2, 4], [2,2,4]);mm, dd, time: 12231530.IMG utm_Time = fix(dateTime[2]) - timeZone * 100 if utm_Time lt 1000 then utm_Time = strmid ('000' + strtrim(string(utm_Time),2), 3,4,$
/REVERSE_OFFSET ) else utm_Time = strtrim(string(utm_Time),2) ; put here if utm_Time is minus. as for 2001 there are all positives ;initialise variables ND = -9999. avhrr_Path='W:\Data\RS\AVHRR_Daily\' + yearStr + '\' + dateTime[0] + $ '\';ah16_20010101_035300_23
CSIRO Land and Water Page 117
avhrr_filename_start = yearStr+ dateTime[0]+dateTime[1]+'_'+utm_Time avhrr_file = file_search(avhrr_Path,'*'+avhrr_filename_start+'*', count = n ) if (n GE 1) then avhrr_filename = avhrr_file[0] else begin ;in case time is different to avhrr image time as we rounded it in Rs_in creation rem = utm_Time mod 100 if (rem GT 0) then utm_Time = utm_Time -1 else utm_Time = utm_Time - 41 if utm_Time lt 1000 then utm_Time = strmid ('000' + strtrim(string(utm_Time),2), 3,4,$
/REVERSE_OFFSET ) else utm_Time = strtrim(string(utm_Time),2) avhrr_filename_start = yearStr+ dateTime[0]+dateTime[1]+'_'+utm_Time avhrr_file = file_search('*'+avhrr_Path,avhrr_filename_start+'*', count = n ) avhrr_filename = avhrr_file[0] endelse Ea_Path='W:\Data\RCALC_SURF\Overpass\' + yearStr + '\Ea\' Ea_filename = yearStr+ $ dateTime[0]+dateTime[1]+'_'+dateTime[2]+'_Ea_lapgrd.grd';20010101_1353_Ea_lapgrd.grd Ea_filename = Ea_Path + Ea_filename Ta_Path='W:\Data\RCALC_SURF\Overpass\' + yearStr + '\Ta\' Ta_filename = yearStr+ $ dateTime[0]+dateTime[1]+'_'+dateTime[2]+'_Ta_lapgrd.grd';20010101_1353_Ta_lapgrd.grd Ta_filename = Ta_Path + Ta_filename outPathRn='W:\Data\RCALC_SURF\Overpass\' + yearStr + '\Rnet\' outPathTdiff='W:\Data\RCALC_SURF\Overpass\' + yearStr + '\Tdiff\' samples = 1601 lines = 1701 Rs_in_image = fltarr(samples,lines) Ea_image = fltarr(samples,lines) Ta_image = fltarr(samples,lines) f_image = fltarr(samples,lines) ;get Rs_in from the surfaces created by microBRIAN Rn_in_path = 'W:\Rs_in\' Rs_in_image = read_binary(Rn_in_path+yearStr+'\'+imageName,DATA_START = 512, $
DATA_TYPE = 2,DATA_DIMS = [1601, 1701]); Rs_in_image OPENR, surface_in, Ea_filename, /GET_LUN readu, surface_in, Ea_image free_lun, surface_in Ea_image = Ea_image * 10 ;convert to hPa from kPa OPENR, surface_in, Ta_filename, /GET_LUN readu, surface_in, Ta_image free_lun, surface_in Ta_image = Ta_image + 273.15 ; convert to K from C ;use SEA f image directly ;get the right vegetation coverage image and subset for SEA f_path = 'W:\DATA\RS\corrected_fpar\converted_coverage\SE_Australia\filled_nulls\' month_in = strtrim((fix(yearStr) - 1981) * 12 + fix(dateTime[0]) -1, 2) ;get Randall's month f_filename = yearStr+ dateTime[0]+'_'+month_in+'.flt' f_filename = f_path + f_filename OPENR, surface_in, f_filename, /GET_LUN readu, surface_in, f_image free_lun, surface_in ;calculate Rs_out. avhrr_image = fltarr(samples,lines,8)
CSIRO Land and Water Page 118
OPENR, surface_in, avhrr_filename, /GET_LUN readu, surface_in, avhrr_image free_lun, surface_in ; Note that the Chn 1 and 2 data are provided as a reflectance % from EAK ; To properly invoke the scaling for the albedo, this must be a fraction, hence divided by 100 albedo = ((avhrr_image[*,*,0] + avhrr_image[*,*,1] ) * 0.5)/100 Rs_out_image = albedo * Rs_in_image ;calculate Rl_in from Ea and Ta surfaces created by ANUSPLIN ;Eq come from Tim's note ;Rl_in = EPSILONa * SIGMA * Ta ^ 4 ;EPSILONa = [1-(1+XI)exp{-(1.2+3.0*XI)^0.5} ] ;XI = 46.5 * (ea / Ta) XI =46.5 * (Ea_image / Ta_image);Ea in hPa, Ta in K EPSILONa = (1- (1+XI) * exp(-(1.2+3.0*XI)^0.5)) SIGMA = 5.67 * 10 ^ (-8.0) Rl_in_image = EPSILONa * SIGMA * (Ta_image) ^ 4 ;calculate Rl_out from Ts and vegetation coverage ;Eq come from Tim's note ;Rl_out = EPSILONs * SIGMA * Ts ^ 4 ;EPSILONs = fv*EPSILONv + (1-fv) * EPSILONg EPSILONv = 0.98 EPSILONg = 0.96 ; Note AVHRR Chn 4 is channel 3 in the data EAK processed; when counting from 0 this = 2nd array Ts_image = avhrr_image[*,*,3] + 3*(avhrr_image[*,*,2] - avhrr_image[*,*,3]) + 0.5 ;t5 + 3*(t4-t5) + 0.5 t_dif = Ts_image - Ta_image ; Note that Randall's veg cover image is provided as a %, it needs to be a fraction hence is ; divided by 100 in the following EPSILONs = f_image/100. * EPSILONv + (1 - f_image/100.) * EPSILONg Rl_out_image = EPSILONs * SIGMA * Ts_image ^ 4 Rn_image = Rs_in_image - Rs_out_image + Rl_in_image - Rl_out_image w=where (Ea_image eq ND, count) if (count gt 0) then begin Rn_image(w) = ND t_dif(w) = ND end w=where (f_image eq ND, count) if (count gt 0) then begin Rn_image(w) = ND t_dif(w) = ND end w=where (Rs_in_image eq -32767, count) if (count gt 0) then begin Rn_image(w) = ND t_dif(w) = ND end
CSIRO Land and Water Page 119
w=where (avhrr_image[*,*,1] eq -999, count) if count gt 0 then begin Rn_image(w) = ND t_dif(w) = ND end ;Write out Rn image outRn_filename = yearStr + dateTime[0]+dateTime[1]+'_'+dateTime[2]+'_Rn.flt' OPENW, surface_out, outPathRn + outRn_filename, /get_lun WRITEU, surface_out, Rn_image CLOSE, surface_out free_lun, surface_out t_dif_filename = yearStr + dateTime[0]+dateTime[1]+'_'+dateTime[2]+'_tdif.flt' OPENW, surface_out, outPathTdiff + t_dif_filename, /get_lun WRITEU, surface_out, t_dif CLOSE, surface_out free_lun, surface_out End Rs_in_path = 'W:\Rs_in\' startYear = 2006 EndYear = 2006 for i = startYear, endYear do begin yearString = strtrim(string(i), 2) Rs_in_List = file_search(Rs_in_path+yearString+'\'+'*.IMG', count=FileNumber) If (FileNumber eq 0) then print, 'no Rs_in images found' else begin For j=0, FileNumber-1 Do Begin theFile = file_basename(Rs_in_List[j]) Calc_Rn_Surface_WRON, yearString, theFile Endfor endelse endfor end
10.11 IDL code to run RADWAT to create files containing overpass information
;IDL 6 function ;Purpose: Cloud checking: read point values of met stations from AVHRR surfaces ;input files: AVHRR surfaces + met station location file 'U:\AWAP\spreadsheets_and_texts\stations.txt' ; and 'U:\AWAP\spreadsheets_and_texts\metCheckReportYearly.txt' ;output file: a text file showing the AVHRR channel values and cloud condition of the stations ;Version History: 8 Dec 2006, based on Extract_pt_from_Surface.pro for ACIAR CSHC data Function AVHRR_Cloud_Check, rootDir, Surface_fileName ;read in two met station files: one contains lat/long of all stations(stations.csv), one contains met data ;quality for a specific year(metCheckReportYearly.csv) ;open checked met station file to get valid station codes theYearCol = 21 ;AVHRR Year = 2001 threshold_staCheck = 90; threshold of valid stations threshold_Zenith = 50; threshold of valid stations checkedMetSta = 'U:\AWAP\spreadsheets_and_texts\metCheckReportYearly.txt';checked stations nCols = 0 ; the number of columns in the input file nline1 = 0 ; number of lines in checked met station file nline2 = 0; number of lines in big met station file
CSIRO Land and Water Page 120
aline = ' ' openr, f_MetSta1, checkedMetSta, /get_lun readf, f_MetSta1, aline nCols = n_elements(strsplit(aline, STRING(9B) )) while not eof(f_MetSta1) do begin nline1++ readf, f_MetSta1, aline endwhile point_lun, f_MetSta1, 0; put pointer back arrMetSta1 = lonarr(nCols, nline1);one more column for the station code readf, f_MetSta1, arrMetSta1; this is data ;if the valus is 1, then the station (line) is OK for the year(column), tmp = where (arrMetSta1[theYearCol+1,*] GE threshold_staCheck, nValid) n_OutFileCols = 11 ;staCode,TimeZone, Long, Lat,X, Y, c1, c2, c3, c4, cloudFlag staCode =0 timeZone=1 Longi =2 Lat=3 X1=4 Y1=5 c1=6 c2=7 c3=8 c4=9 cloudFlag=10 arrValidSta =fltarr(n_OutFileCols,nValid) arrValidSta[staCode,*] = (arrMetSta1[0,*])[tmp] ;first col is station code close, f_MetSta1 free_lun, f_MetSta1 ;open the met station file with lat/long to get lat/longs of the valid stations xyMetSta = 'U:\AWAP\spreadsheets_and_texts\stations.txt' openr, f_MetSta2, xyMetSta, /get_lun readf, f_MetSta2, aline; this is header line nCols = n_elements(strsplit(aline, STRING(9B) )) while not eof(f_MetSta2) do begin readf, f_MetSta2, aline nline2++ endwhile point_lun, f_MetSta2, 0; put pointer back arrMetSta2 = strarr(nCols, nline2); to put all info of lat/long station file readf, f_MetSta2, aline; this is header line for i = 0, nline2-1 do begin;this is data readf, f_MetSta2, aline arrMetSta2[*,i] = strsplit(aline, STRING(9B), /extract) endfor ;thisState = 'ACT' tZone = 10 for i = 0, nValid-1 do begin thisSta = where (float(strtrim((arrMetSta2[0,*]),2)) eq arrValidSta[staCode,i]) thisState = strtrim(arrMetSta2[8,thisSta],2);state
CSIRO Land and Water Page 121
case strupcase(thisState) of 'NT': tZone = 9.5 'SA': tZone = 9.5 'WA': tZone = 8 else: tZone = 10 endcase arrValidSta[timeZone,i] = tZone arrValidSta[Longi,i] = float(strtrim(arrMetSta2[6,thisSta],2));longi arrValidSta[Lat,i] = float(strtrim(arrMetSta2[5,thisSta],2));lat endfor close, f_MetSta2 free_lun, f_MetSta2 ;up to now, we have an array contains valid met stations and their lat/longs ;get image info from image file name. this may be used late. fname_split = strsplit(Surface_fileName, '_', /extract);ah16_20010129_054602_16 imgDate = fname_split[1] imgTime = fname_split[2] imgCloudflag = fname_split[3] ;get info from ENVI header headerFile = rootDir+Surface_fileName+'.hdr' OPENR, fHeader, headerFile, /GET_LUN paraArray = fltarr(6) while not eof (fHeader) do begin Readf, fHeader, aline for i=0, 6 do begin if (STRMID(aline, 0, 5) eq 'sampl') then begin tmp = strsplit(aline, /extract, ' ') paraArray[0] = float(strtrim(tmp[2],2)) endif else if(STRMID(aline, 0, 5) eq 'lines') then begin tmp = strsplit(aline, /extract, ' ') paraArray[1] = float(strtrim(tmp[2],2)) endif else if(STRMID(aline, 0, 5) eq 'bands') then begin tmp = strsplit(aline, /extract, ' ') paraArray[2] = float(strtrim(tmp[2],2)) endif else if(STRMID(aline, 0, 5) eq 'map i') then begin tmp = strsplit(aline, /extract, ' ') paraArray[3] = float(strtrim(tmp[7],2)) ;longi at starting point paraArray[4] = float(strtrim(tmp[8],2)) ;lat at starting point paraArray[5] = float(strtrim(tmp[9],2)) ;pixel size endif endfor endwhile close, fHeader free_lun, fHeader ;header files from EAK untrustful, so just use fixed numbers samples = 1601 lines = 1701 bands = 8 topLeftLong = 137.99500 topLeftLat = -22.99500 pixelsize = 0.01 bottomRightLong = topLeftLong + pixelsize * samples
CSIRO Land and Water Page 122
bottomRightLat = topLeftLat - pixelsize * lines ;The AVHRR image has 8 channels according to EAK, their names are ;normalisedReflectance_avhrr_1 ;normalisedReflectance_avhrr_2 ;brightnessTemperature_avhrr_4 ;brightnessTemperature_avhrr_5 ;satelliteAzimuthGrid ;satelliteZenithGrid ;sunAzimuthGrid ;sunZenithGrid ;read in surface file in_Surface = fltarr(samples,lines,bands) OPENR, f_Surface, rootDir+Surface_fileName, /GET_LUN readu, f_Surface, in_Surface for i = 141 ,nValid -1 do begin dn_avhrr_1 = -999 x=float(arrValidSta[Longi,i]) ;longi y=float(arrValidSta[Lat,i]) ;lat x_img = floor((x-topLeftLong)/pixelsize)+1;check this equation with Tom y_img = floor((topLeftLat-y)/pixelsize)+1;check this equation with Tom
if ( x GE (topLeftLong) && x LE bottomRightLong && (y LE topLeftLat) && $ (y GE bottomRightLat)) then begin
dn_avhrr_1 = in_Surface[x_img-1,y_img-1, 0]; channel 1. dn_avhrr_2 = in_Surface[x_img-1,y_img-1, 1]; channel 2. dn_avhrr_4 = in_Surface[x_img-1,y_img-1, 2]; channel 3. dn_avhrr_5 = in_Surface[x_img-1,y_img-1, 3]; channel 4. dn_satelliteZenith = in_Surface[x_img-1,y_img-1, 5]; channel 6. arrValidSta[X1,i] = x_img ;x coor on image, not lat/long arrValidSta[Y1,i] = y_img ;y coor on image, not lat/long arrValidSta[c1,i] = dn_avhrr_1 arrValidSta[c2,i] = dn_avhrr_2 arrValidSta[c3,i] = dn_avhrr_4 arrValidSta[c4,i] = dn_avhrr_5 endif ;if dn_avhrr_1 EQ -999 then arrValidSta[cloudFlag,i] = 0 else arrValidSta[cloudFlag,i]= 1;cloud is 0 ;if take into account Zenith angle, use following code instead th eline above if dn_avhrr_1 EQ -999. then arrValidSta[cloudFlag,i] = 0 else begin
if (dn_satelliteZenith GT threshold_Zenith) || (dn_satelliteZenith eq -999.) then $ arrValidSta[cloudFlag,i]= 1 else begin
arrValidSta[cloudFlag,i]= 2 endelse endelse endfor ;the array of valid met is now ready, this is for one pixel. close, f_Surface free_lun, f_Surface OPENW, f_out, rootDir+Surface_fileName+'_Checked.txt', /GET_LUN printf, f_out, 'StaCode','Zone','Long','Lat','X_IMG','Y_IMG','avhrr_1','avhrr_2',$
'avhrr_4','avhrr_5','Cloud_Zenith', format='(a7,a5,2a12,2a6,4a12,a14)' printf, f_out, arrValidSta, format = '(i7,f5.1,2f12.6, 2i6,4f12.6,i10)' close, f_out free_lun, f_out cfc = where (arrValidSta[cloudFlag,*] GE 1, count) ;cloud free cells aCloudRatio = fltarr(5)
CSIRO Land and Water Page 123
aCloudRatio[0] = nValid ; number of valid station aCloudRatio[1] = count ; number of cloud free station aCloudRatio[2] = count*100.00/nValid ; percentage of cloud free station ;cloud free cells with Zenith angle smaller than threshold zenith_OK = where (arrValidSta[cloudFlag,*] eq 2, count2) aCloudRatio[3] = count2 ; number of cloud free station aCloudRatio[4] = count2*100.00/nValid ; percentage of cloud free station return, aCloudRatio; this ratio is for the whole AVHRR image end monthStr = ['01\','02\','03\','04\','05\','06\','07\','08\','09\','10\','11\','12\'] for j = 0, 0 do begin rootDir = 'U:\AWAP\Data\RS\AVHRR_Daily\2001\' rootDir = rootDir + monthStr[j] files = file_search(rootDir, '*.hdr', count=FileNumber); if (FileNumber GT 0) then begin outRationArray = strarr(6, FileNumber); surppose each imag has a hdr file for i=0, FileNumber-1 do begin ;fName=file_basename(files[i]) fName=strsplit(files[i], /extract, '.') fName_base = file_basename(fName) ;thisIMGRation = AVHRR_Cloud_Check(rootDir, fName_base[0]) thisIMGRation = AVHRR_Cloud_Check(rootDir, 'ah16_20010102_052334_37') outRationArray[0, i] = fName_base[0] outRationArray[1, i] = string(fix(thisIMGRation[0])) outRationArray[2, i] = string(fix(thisIMGRation[1])) outRationArray[3, i] = string(thisIMGRation[2], format = '(f6.2)') outRationArray[4, i] = string(fix(thisIMGRation[3])) outRationArray[5, i] = string(thisIMGRation[4], format = '(f6.2)') endfor
OPENW, f_out, rootDir +'AVHRR_img_summary.txt', /GET_LUN printf, f_out, 'Image name', 'Valid_Sta', 'CloudFree_Sta', 'Cloudfree(%)','Zen<=50', $
'Zen_Cloud_OK(%)', format='(a23, a10, a14, a13, a17, a16)' printf, f_out, outRationArray, format='(a23, a10, a14, a13, a17, a16)' close, f_out free_lun, f_out
endif endfor end ;Purpose: Create OVP file ;Input: outputs from AVHRR_Cloud_Check.pro ;(many txt files each contains many points' AVHRR image data, see lines below for the format). ;StaCode Long Lat X_IMG Y_IMG avhrr_1 avhrr_2 avhrr_4 avhrr_5 1=Cloudfree ; 15594 134.686493 -23.463499 0 0 0.000000 0.000000 0.000000 0.000000 0 ; 17031 138.063705 -29.648100 7 665 29.307428 28.258816 314.406067 311.286957 1 ; ;Output: Many txt files (one station one file, containing tempral AVHRR dada, ;see lines below for the format) ;Phoenix Overpass Data ;17 2 1 2 1 1.013 ;D 67 19830308 800 283.5 89.5 0.40 1.20 108.0 283.9 88.3 ;D 67 19830308 830 285.6 89.5 0.40 2.00 247.1 285.3 83.7
CSIRO Land and Water Page 124
;Steps: ;1. get all checked AVHRR data first and sort by date - outside this function ;2. read in one checked AVHRR file and write out many (in the OVP format shown above) ;3. read the second checked AVHRR file and write out many (append to files created in above step) ; ;History: 13 March 2007 add function to extract coverage data ;Modified to WRON server a)trunk the seconds in file names (previous version uses rounding) ; b)change file locations ; c)read f imgae from SEA dir drectly Function Get_Coverage_Value,year,month,x_array, y_array ;from Randall's month to year_month: year = 1981 + a/12, month = a mod 12 + 1 ;from year_month to Randall's month year_int=fix(year) month_int=fix(month) out_cvrg_array = fltarr(n_elements(x_array)) out_cvrg_array[*] = -9999. samples = 1601 lines = 1701 f_image = fltarr(samples,lines) f_path = 'W:\DATA\RS\corrected_fpar\converted_coverage\SE_Australia\filled_nulls\' month_in = strtrim(string((year_int - 1981) * 12 + month_int -1), 2) ;Randall's month f_filename = year + month + '_'+month_in+'.flt' f_filename = f_path + f_filename OPENR, surface_in, f_filename, /GET_LUN readu, surface_in, f_image free_lun, surface_in if (file_test(f_filename) eq 0) then out_cvrg_array[*] = -9999. else begin ;open coverage file for i = 0 ,n_elements(x_array) -1 do begin if (x_array[i] GT 0 and y_array[i] GT 0) then $
out_cvrg_array[i] = f_image[x_array[i]-1,y_array[i]-1] else out_cvrg_array[i] = -9999. endfor endelse return, out_cvrg_array end Function get_maxDay, year, month m = year mod 4 if (m eq 0 && month eq 2) then begin k = 29 endif else begin case month of '02': k = 28 '04': k = 30 '06': k = 30 '09': k = 30 '11': k = 30 else: k = 31 endcase endelse return, K end Pro WriteStationFile, outputDir, avhrrFileName, avhrrValue, coverage ;read in one record - avhrrValue - from Cloud check data, ;use avhrr image name as a key to write out one station file
CSIRO Land and Water Page 125
;avhrrValue array columns staCode = 0 tZone = 1 Longi = 2 Lat = 3 X_IMG = 4 Y_IMG = 5 avhrr_1 = 6 avhrr_2 = 7 avhrr_4 = 8 avhrr_5 = 9 Cloudfree = 10 timeZone_h = 10 timeZone_m = 0 if (avhrrValue[Cloudfree] GT 0) then begin;the station is not covered by cloud ;ah16 20010102 052360 37 Checked.txt imgNameStr = strsplit(file_basename (avhrrFileName), '_', /extract) theYear=fix(strmid(imgNameStr[1], 0,4)) theMonth=fix(strmid(imgNameStr[1], 4,2)) theDay=fix(strmid(imgNameStr[1], 6,2)) hh=fix(strmid(imgNameStr[2], 0,2));hour mm=fix(strmid(imgNameStr[2], 2,2));minute hh = hh + timeZone_h ;GMT time to local time mm = mm + timeZone_m if mm GE 60 then begin mm = mm - 60 hh = hh + 1 endif if hh GE 24 then begin maxDay = get_maxDay(theYear, theMonth) hh = hh - 24 theDay = theDay + 1 if theDay GT maxDay then begin theDay = theDay - maxDay theMonth = theMonth + 1 endif if theMonth GT 12 then begin theMonth = theMonth - 12 theYear = theYear + 1 endif endif DoY = JULDAY(theMonth, theDay,theYear) - JULDAY(1,0,theYear);day of the year ;CONVERT DATE/TIME TO STRING theYear = strtrim(string(theYear),2) if theMonth lt 10 then theMonth = '0' + strtrim(string(theMonth),2) else $
theMonth = strtrim(string(theMonth),2) if theDay lt 10 then theDay = '0' + strtrim(string(theDay),2) else $
theDay = strtrim(string(theDay),2) if hh lt 10 then hh = '0' + strtrim(string(hh),2) else hh = strtrim(string(hh),2) if mm lt 10 then mm = '0' + strtrim(string(mm),2) else mm = strtrim(string(mm),2)
CSIRO Land and Water Page 126
outFile = outputDir + strtrim(string(avhrrValue[staCode], format = '(i)'),2)+'.ovp' if (file_test(outFile) eq 1) then begin ; if the file already exists, append the record openw, /append, fout, outFile, /get_lun endif else begin openw, fout, outFile, /get_lun; otherwise, create one.
;format= '()';need to modify printf, fout, string(avhrrValue[staCode], format = '(i)') +' Overpass Data' printf, fout, '17 2 1 2 0 0.411554';format= '()';need to modify endelse ;calculate data needed in OVP file SurfT = avhrrValue[avhrr_5] + 3*(avhrrValue[avhrr_4] - avhrrValue[avhrr_5]) + 0.5; if coverage eq -999. then Cover = strtrim(string(-1),2) else $
Cover = string(coverage, format = '(f6.2)'); if coverage eq -9999. then Cover = strtrim(string(-1),2) else begin if coverage eq 0. then coverage = 0.01 Cover = string(coverage, format = '(f6.2)') endelse Height = -1;use -1 for RadCalc if data missed Wind = -1 SolarIr = -1 AirT = -1 Humidity = -1 printf, fout, 'D ', DoY, theYear+theMonth+theDay,hh+mm,SurfT, Cover,$ Height,Wind,SolarIr,AirT,Humidity, $
format = '(a1,1x,i3,1x,a8,1x,a4,f7.2,1x,a6,5i3)';need to modify AVHRR value close, fout
free_lun, fout endif end Pro ManyPts2ManyDates, rootDir_ovp, avhrrName ;this procedure reads in a text file (one image, many stations) ;then send one record - a station - to Pro WriteStationFile to write out if file_test(avhrrName) eq 0 then begin print, 'file ' + avhrrName+ ' not exist, task cancelled' endif header='' nLines=0 openr, f_In_IMG, avhrrName, /get_lun readf, f_In_IMG, header POINT_LUN, -f_In_IMG, pos nCols = n_elements(strsplit(header, ' '));Need to check if nCols is stll correct later while not eof (f_In_IMG) do begin readf, f_In_IMG, header nLines++ ;number of lines of the OVP file endwhile POINT_LUN, f_In_IMG, pos aIMG = fltarr(nCols,nLines) readf, f_In_IMG, aIMG close, f_In_IMG free_lun, f_In_IMG
CSIRO Land and Water Page 127
nameStr = strsplit(file_basename (avhrrName), '_', /extract);ah16 20010102 052360 37 Checked.txt theYear=strmid(nameStr[1], 0,4) theMonth=strmid(nameStr[1], 4,2) col_x = 4;envi col number of the image col_y = 5;envi line number of the image cvrg_arr = Get_Coverage_Value(theYear,theMonth,aIMG[col_x,*], aIMG[col_y,*] ) ;get % coverage here can save running time than station by station for i = 0, nLines-1 do begin
;line by line (i.e., station by station) creating output by calling WriteStationFile WriteStationFile, rootDir_ovp, avhrrName, aIMG[*,i], cvrg_arr[i] endfor end Pro Modify_OVP, inFile ;this procedure modifies the OVP file created above (pro WriteStationFile) ;it reads in an OVP file, counts the number of records, then wtites out a modified ;OVP with correct line 2. aline = '' nLines = 0 openr, tmpFile, inFile, /get_lun while not eof (tmpFile) do begin readf, tmpFile, aline nLines++ ;number of lines of the OVP file endwhile POINT_LUN, tmpFile, 0 tmpArray = strarr(1,nLines) readf, tmpFile, tmpArray ;read in as a one dimensional string array close, tmpFile free_lun, tmpFile ;change first few charactors of line2 pos = strpos(tmpArray[0,1], ' ');end of first number in the line part2= strmid(tmpArray[0,1],pos);get the rest of the line newNum = strtrim(string(nlines - 2),2);number of station records = total lines - two header lines tmpArray[0,1] = newNum + part2 ; modify line 2 openw, tmpFile, inFile, /get_lun printf, tmpFile, tmpArray; write out the new OVP file close, tmpFile free_lun, tmpFile end ;Starting point for the function. Get all outputs from AVHRR_Cloud_Check.pro for year = 2006, 2006 do begin
rootDir_OVP = 'W:\DATA\RADCALC\OVP\'+strtrim(string(year), 2)+'\' rootDir = 'W:\Data\RS\AVHRR_Daily\'+strtrim(string(year), 2)+'\'
files=file_search(rootDir, '*_Checked.txt', count=FileNumber) if FileNumber GT 0 then begin for i=0, FileNumber-1 do begin ManyPts2ManyDates, rootDir_ovp, files[i] endfor endif
;if the output ovp files need to be modified (say for number of Records), do it here. files2=file_search(rootDir_OVP, '*.ovp', count=OVPFileNumber)
CSIRO Land and Water Page 128
if FileNumber GT 0 then begin for i=0, OVPFileNumber-1 do begin Modify_OVP, files2[i] endfor endif endfor end
;This pro create parameter files for radwat.exe, with one parameter file per year ;Started: 2007.03.09 ;By: Li Lingtao ;The difference between this code and Create_Parameter_For_Radwat_All.pro is: ;this one needs .ovp and .psf to generate .rad file, the other needs .int/psf to generate .all file Pro Create_Parameter_For_Radwat_RAD rootDir = 'W:\DATA\RADCALC\' for year = 2006, 2006 do begin outFileDir = rootDir + 'rad\' + strtrim(string(year),2) ;check if the destination directory exists, create one if not outDirOK = file_search(outFileDir,COUNT=N_FOUND) IF N_FOUND EQ 0 THEN SPAWN, 'mkdir ' + outFileDir outFile = outFileDir + '\'+ strtrim(string(year),2)+'_radwat_rad.par' openw, fOut, outFile, /get_lun; psfFiles=file_search(rootDir+'psf\'+strtrim(string(year),2)+'\', '*.psf', count=psfFileNum) psfFileNameOnly = strarr(psfFileNum) for i=0, psfFileNum-1 do begin psfFileNameOnly[i]=(strsplit(file_basename(psfFiles[i]),'.', /extract))[0] endfor ovpFiles=file_search(rootDir+'ovp\'+strtrim(string(year),2)+'\', '*.ovp', count=ovpFileNum) ovpFileNameOnly = strarr(ovpFileNum) for i=0, ovpFileNum-1 do begin ovpFileNameOnly[i]=(strsplit(file_basename(ovpFiles[i]),'.', /extract))[0] endfor for i=0, ovpFileNum-1 do begin psfFile = where (psfFileNameOnly eq ovpFileNameOnly[i], count) if count GT 0 then begin ;check if the output .rad exists, delete it if yes, otherwise this par file won't work outFileName = rootDir + 'rad\'+strtrim(string(year),2)+'\'+ $ ovpFileNameOnly[i]+'.rad' outFileOK = file_search(outFileName,COUNT=N_FOUND) IF N_FOUND EQ 1 THEN FILE_DELETE, outFileName;SPAWN, 'del ' +$
outFileName printf, fOut,"s" printf, fOut, psfFiles[psfFile] printf, fOut, "r" printf, fOut, ovpFiles[i] printf, fOut, outFileName endif endfor printf, fOut,"q"
CSIRO Land and Water Page 129
close, fOut free_lun, fOut endfor end
pro Check_Cols,startyear,endyear,path,ext,expectedCols,n_headerlines ;/LLT/TVN ;this is to check if each line in a file contains right number of records ;18 April 2007 ;initialise variables Count=0 For i=startyear, endyear Do Begin ;put all files into a list List = FILE_SEARCH(path+strtrim(string(i),2),'*.'+ext) If (n_elements(List) eq 0) then print, kdjfkdjf For j=0, n_elements(List)-1 Do Begin fileName=List[j]
cols = 0 openr, fin, fileName, /get_lun aline='' lineNum = n_HeaderLines for k = 0, n_HeaderLines -1 do begin ;skip headers readf, fin, aline endfor; k while not eof(fin) do begin lineNum++ readf, fin, aline cols=n_elements(strsplit(aline, /extract, ',')) if (cols ne expectedCols) then Begin print, ' '+fileName + ' has ' + STRTRIM(string(cols),2) +' columns in line '+ $ STRTRIM(string(lineNum),2)+', expected$
'+STRTRIM(STRING(expectedCols),2) Count++ endif endwhile close, fin free_lun, fin EndFor; j Endfor; i Print, ' Total number of '+STRTRIM(STRING(COUNT),2)+' files had the wrong number of columns' End pro SortRadByImageAcq ;TVN/02-Apr-2007 ;Takes in output from RadCalc (or RADWAT?) and sorts all valid stations into separate ;files for each image acquisition. ;Outputs sorted data by image acquisition date for input to spline (spline in, ;called "sin") ; ;TVN/17-Apr-2007/ adjust original code so it does not output Tair nor Rh, uses the ;25 station filter ; ;TVN/20-Apr-2007/ added a filter to select records that have unreasonably high or ;low ET or NDTI values. The original *.sin files are the same and are still output ;but now we also output two additional files per image acquisition. These are
CSIRO Land and Water Page 130
;*.ehw and *.nhw (i.e., et half way, and ndti half way) where these new file ;have the unrealistic ET or NDTI records removed. The initial rules for removing ;records are: ; For NDTI: if (NDTI LT -0.25) OR (NDTI GT 1.25) ; For ET: if (ET LT 0) OR (NDTI GT 1.25) ; ;initialise variables startyear=2006 endyear=2006 path='W:\Data\RADCALC\RAD\' outpath='W:\Data\RADCALC\Spatial_Int_Overpass\' staFile='W:\spreadsheets_and_texts\stations_nostrings_DEM.txt' staThresh=25 ;read in station file to get lat longs ;read in station file ENVI_READ_COLS, staFile, sta_array BIGstalist=sta_array(0,*) BIGlatlist=sta_array(3,*) BIGlonlist=sta_array(4,*) BIGelevlist=sta_array(5,*) BIGDEMlist=sta_array(10,*) For i=startyear, endyear Do Begin ;get sta list from BOM Wind directory CD, path+strtrim(string(i),2) radList = FILE_SEARCH('*.rad') If (n_elements(radlist) eq 0) then print, kdjfkdjf ;Read in each rad file (i.e., one station at a time) and concat them together For j=0, n_elements(radlist)-1 Do Begin envi_read_cols, path+strtrim(string(i),2)+'\'+radlist[j], matrix ;set up necesarry lists stalist=lonarr(n_elements(matrix[0,*])) lonList=fltarr(n_elements(matrix[0,*])) latList=fltarr(n_elements(matrix[0,*])) demList=fltarr(n_elements(matrix[0,*])) elevList=fltarr(n_elements(matrix[0,*])) ;determine the station, elev, lat, and lon for this file and populate the relevant lists ;station list a=strsplit(radlist[j],'.',/EXTRACT) a=long(a[0]) stalist[*]=a ;lon and lat lists aindex=where(BIGstaList eq a) lonlist[*]=BIGlonlist[aindex] latlist[*]=BIGlatList[aindex] demlist[*]=BIGDEMList[aindex] elevlist[*]=BIGelevList[aindex] ;check if DEM eq -999 (null), if so, replace with station elev from dictionary nullDEM=where(demlist eq -999, nullDEMcount) if (nullDEMcount ne 0) then Begin demlist[nullDEM]=elevlist[nullDEM] nullELEV=where(elevlist[nullDEM] eq -1, nullelevcount) ;if elev from dictionary is null (-1) then stop program - this needs ;to be fixed. Bad idea to make station elev nulls eq -1 as this is a valid ;elevation - need to get original elevs again, then we can do something
CSIRO Land and Water Page 131
;different here. if (nullelevcount ne 0) then print, dkfjdkj endif ;insert station, lon, and lat and DEM lists into the BIG rad array matrix=[[matrix],transpose(stalist), transpose(lonlist), transpose(latlist), transpose(demlist)] ;concat each separate array together into a big array if (j eq 0) then bigArray=matrix else bigArray=[[bigArray],[matrix]] Endfor ;j ;Sort the big "ALL" array based on image acquisition ; get list of image acquisitions from the big array ;make a unique identifier for each overpass by using date (*10000) plus time (hhmm) imgID=(bigArray[2,*]*10000.)+bigArray[5,*] imgAcqList=imgID[UNIQ(imgID, SORT(imgID))] ;Loop through each unique image acquistion and get any valid data For k=0, n_elements(imgAcqList)-1 Do Begin ;split up the uniq image acq ids into date and time str=STRTRIM(STRING(ulong64(imgAcqlist[k])),2) imgDate=STRMID(str,0,8) imgTime=STRMID(str,8,4) ;define the items in the big array that have this date and time index=where((bigArray[2,*] eq long(imgDate)) and (bigArray[5,*] eq fix(imgTime)), count) ;write out specified data for this image acquisition date only if it passes ;the filter for number of stations. this filter is the intersection between ;met stations and valid image data (e.g., cloud free) ; If (count GE staThresh) then Begin; 25 records (stations) are needed to interpolate outarray=fltarr(6,count) outarray[0,*]=bigarray[33,index]; station number outarray[1,*]=bigarray[34,index]; longitude outarray[2,*]=bigarray[35,index]; latitude outarray[3,*]=bigarray[36,index]; DEM outarray[4,*]=bigarray[14,index]; ET Day (mm/day) outarray[5,*]=bigarray[20,index]; NDTI (ratio) ;output the ehw file eIndex=where((outarray[4,*] GE 0) AND (outarray[5,*] LE 1.25), eCount) ;only print out the ehw file if there are more than 25 stations (or whatever we change
;staThresh to be) AND ET Day is GE 0 AND NDTI is LE 1.25 ; ;NOTE, this will result in some ehw files that contain less than 25 points ;if (eCount ge staThresh) then begin if (eCount ge 1) then begin OPENW, 1, outpath+strtrim(string(i),2)+'\'+imgDate+'_'+imgTime+'.ehw' PRINTF, 1, [[outarray[0,eindex]],outarray[1,eindex],$
outarray[2,eindex],outarray[3,eindex],outarray[4,eindex]],$ format="(I10, 4F12.6)"
CLOSE, 1 endif ;only print out the nhw file if there are more than 25 stations (or whatever we
;change staThresh to be) AND NDTI is between -0.25 and 1.25 nIndex=where((outarray[5,*] GE -0.25) AND (outarray[5,*] LE 1.25), nCount) ;if there are any records, then print out the nhw file ;if (nCount ge staThresh) then begin
CSIRO Land and Water Page 132
if (nCount ge 1) then begin OPENW, 1, outpath+strtrim(string(i),2)+'\'+imgDate+'_'+imgTime+'.nhw' PRINTF, 1, [[outarray[0,nindex]],outarray[1,nindex],outarray[2,nindex],$
outarray[3,nindex],outarray[5,nindex]], format="(I10, 4F12.6)" CLOSE, 1 endif ;put summary values of nhw and ehw into an array if (k eq 0) then enhwMatrix=[imgDate,imgTime,eCount,nCount,Count] else $ enhwMatrix=[[enhwMatrix],[imgDate,imgTime,eCount,nCount,Count]] Endfor ;k ;output summary text file of enh and nhw counts OPENW, 1, outpath+strtrim(string(i-1),2)+'ehw_nhw_sinCounts.txt' PRINTF, 1, ' YYYMMDD',' HHMM',' ehw',' nhw',' sin' PRINTF, 1, enhwMatrix, format="(I10, 4I6)" CLOSE, 1 Endfor ;i End ;IDL 6 function ;Purpose:create ,nin file for interpolation of NDTI ;Method: Extract point Rn, Ta and f data for stations and add to .sin ;input file: Rn, Ta and f surfaces, .sin files (point location) ;output file: one txt file containing .nhw + Rn, Tdiff and f value ;Version History: 20 April 2007 ;based on Extract_pts_from surfaces Pro nhw2nin_WRON, nhwFile fName = file_basename(nhwFile);20011231_1404.nhw fName_Tokens = strmid(fName, [0,4,6,9], [4,2,2,4]);yy,mm,dd,hhmm samples = 1601 lines = 1701 pixelsize = 0.01 leftLongi = 137.995 topLat = -22.995 rightLongi = 154.005 bottomLat = -40.005 ;Read in nhw file ;the 6 columns in nhwArray are: ;0:station number, ;1:longitude (dd) ;2:latitude (dd) ;read in .nhw file envi_read_cols, nhwFile, nhwArray n_Cols = n_elements(nhwArray[*,0]) n_Lines = n_elements(nhwArray[0,*]) ;read in Rn surfaces Rn_Dir = 'W:\Data\RCALC_SURF\Overpass\' + fName_Tokens[0] + '\Rnet\' Rn_fName = $ fName_Tokens[0]+fName_Tokens[1]+fName_Tokens[2]+'_'+fName_Tokens[3]+'_Rn.flt Rn_fName = Rn_Dir + Rn_fName ;read in Tdiff surfaces Tdiff_Dir = 'W:\Data\RCALC_SURF\Overpass\' + fName_Tokens[0] + '\Tdiff\' Tdiff_fName = $ fName_Tokens[0]+fName_Tokens[1]+fName_Tokens[2]+'_'+fName_Tokens[3]+'_tdif.flt Tdiff_fName = Tdiff_Dir + Tdiff_fName
CSIRO Land and Water Page 133
;read in f surfaces f_Dir = 'W:\DATA\RS\corrected_fpar\converted_coverage\SE_Australia\filled_nulls' f_fName = file_Search(f_Dir, fName_Tokens[0]+fName_Tokens[1]+'*.flt', count=n) if n GT 1 then f_fName = f_fName[0] ;grab point data from surfaces Rn_surface = fltarr(samples,lines) Tdiff_surface = fltarr(samples,lines) f_surface = fltarr(samples,lines) thisRn = fltarr(n_Lines) thisTdiff = fltarr(n_Lines) thisf = fltarr(n_Lines) OPENR, fSur, Rn_fName, /GET_LUN readu, fSur, Rn_surface free_lun, fSur OPENR, fSur, Tdiff_fName, /GET_LUN readu, fSur, Tdiff_surface free_lun, fSur OPENR, fSur, f_fName, /GET_LUN readu, fSur, f_surface free_lun, fSur for i = 0, n_Lines-1 do begin if ( nhwArray[1,i] GE leftLongi && nhwArray[1,i] LE rightLongi && $ (nhwArray[2,i] GE bottomLat) && (nhwArray[2,i] LE topLat)) then begin thisRn[i] = Rn_surface[floor((nhwArray[1,i]-leftLongi)/pixelsize), $
floor((topLat-nhwArray[2,i])/pixelsize)] thisTdiff[i] = Tdiff_surface[floor((nhwArray[1,i]-leftLongi)/pixelsize), $ floor((topLat-nhwArray[2,i])/pixelsize)] thisf[i] = f_surface[floor((nhwArray[1,i]-leftLongi)/pixelsize), $
floor((topLat-nhwArray[2,i])/pixelsize)] endif endfor ;add 3 more columns to .nhw file - note that the NDTI (or independent var for ANUSPLIN MUST be last column in the date file) ninArray =fltarr(n_Cols+3, n_Lines) ninArray[0:n_Cols-2,*] = nhwArray[0:n_Cols-2, *] ninArray[n_Cols-1,*] = thisRn ninArray[n_Cols,*] = thisTdiff ninArray[n_Cols+1,*] = thisf ninArray[n_Cols+2,*] = nhwArray[n_Cols-1,*] ;if Rn, Tdiff or f is null, delete the line ND = -9999. invalid_Rn = where (ninArray[4,*] eq ND) invalid_Tdiff = where (ninArray[5,*] eq ND) invalid_f = where (ninArray[6,*] eq ND) ;if Rn and Tdiff out of SPLINA ranges, get rid of them as well. ttt = [-1, invalid_Rn,invalid_Tdiff,invalid_f] invalid_all = ttt [uniq(ttt, sort(ttt))] if n_elements(invalid_all) LE 1 then begin ;output .nin file openw, fout, (strsplit(nhwFile, '.', /extract))[0] + ".nin", /get_lun
CSIRO Land and Water Page 134
printf, fout, ninArray, format= '(i10, 7F11.4)' free_lun, fout endif else begin ; if ((n_Lines - n_elements(invalid_all)+1) ge staValid) then begin ninArray_out = fltarr(n_Cols + 3, n_Lines - n_elements(invalid_all) + 1) q = 0 for p = 0, n_Lines - 1 do begin ;big array with null if (where (invalid_all eq p) eq -1) then begin ninArray_out[*,q] = ninArray[*,p] q ++ endif endfor ;output .nin file openw, fout, (strsplit(nhwFile, '.', /extract))[0] + ".nin", /get_lun printf, fout, ninArray_out, format= '(i10, 7F11.4)' free_lun, fout ; endif endelse end for i = 2006,2006 do begin nhwFileDir = 'W:\DATA\RADCALC\Spatial_Int_Overpass\' + strtrim(string(i),2)+'\' nhw_List = file_search(nhwFileDir +'*.nhw', count=FileNumber) If (FileNumber eq 0) then print, 'no .nhw file found' else begin For j=0, FileNumber-1 Do Begin nhw2nin_WRON, nhw_List[j] Endfor endelse endfor end
10.12 IDL code to generate overpass NDTI surfaces pro Create_Splina_cmd_Files_NDTI ;TVN/LLT Apr-2007 ;create cmd files for each surface to be splined ; assumes a yearly directory structure ; requires a template *.cmd file as input at the base directory specified below ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Overpass\' outpath='W:\DATA\RADCALC\Spatial_Int_Overpass\SPLIN_cmd_files\' ;utpath='W:\Data\RADCALC\Spatial_Int_Overpass\SPLIN_cmd_files\' ext='nin' template=outpath+'template_NDTI_splna.cmd' ;magicRecord=15 magicRecord=[22,23,24,25,26] ;make sure template exists Result = FILE_TEST(template) if (result ne 1) then print, dkfjkdj For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*.'+ext, count=n)
CSIRO Land and Water Page 135
If (n eq 0) then print, kdjfkdjf For j=0, n_elements(List)-1 Do Begin filename=strsplit(list[j],'.', /Extract) ;read in the template file and modify the "magic" record where the filename is kept counter=0 aline='' sarray=strarr(1) OPENR, unit, template, /GET_LUN while not eof (unit) do begin readf, unit, aline if counter eq 17 then begin strput, aline, strtrim(string(i),2)+'\'+filename[0],37 endif else begin tmp = where (magicRecord eq counter+1, count) if count gt 0 then strput, aline, strtrim(string(i),2)+'\'+filename[0],53 endelse if (counter eq 0) then begin strput, aline, filename[0],0 sarray[0]=aline endif else sarray=[[sarray],[aline]] counter++ endwhile free_lun, unit ;output the new modified cmd file using input basename OPENW, 1, outpath+strtrim(string(i),2)+'\'+filename[0]+'_NDTI_SPLINA.cmd' PRINTF, 1, sarray CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+filename[0]+'_NDTI_SPLINA.cmd' Endfor Endfor End pro Create_SPLINA_Batch_Runs_NDTI ;TVN/05-Apr-2007 ;create batch file to run all splina commands ; assumes a yearly directory structure ;initialise variables startyear=2006 endyear=2006 path='W:\DATA\RADCALC\Spatial_Int_Overpass\SPLIN_cmd_files\' ext='NDTI_SPLINA.cmd' aline1='SPLINA' aline2='_NDTI_SPLINA.bat' For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*'+ext) nRecs=n_elements(list) If (n_elements(List) eq 0) then print, kdjfkdjf sarray=strarr(nRecs)
CSIRO Land and Water Page 136
For j=0, nRecs-1 Do Begin filename=strsplit(list[j],'.', /Extract) sarray[j]=aline1+' < '+list[j]+' > '+filename[0]+'.log' Endfor ;output the batch file OPENW, 1, path+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 PRINTF, 1, sarray CLOSE, 1 print, path+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 Endfor End pro Create_Lapgrd_cmd_Files_NDTI ;TVN/LLT Apr-2007 ;Modified by LLT ;create cmd files for each surface to be splined ; assumes a yearly directory structure ; requires a template *.cmd file as input at the base directory specified below ;initialise variables startyear=2001 endyear=2001 ;path = 'W:\DATA\RADCALC\Spatial_Int_Overpass\' path = 'W:\DATA\RADCALC\Spatial_Int_Overpass\SPLIN_cmd_files\' outpath='W:\DATA\RADCALC\Spatial_Int_Overpass\LAPGRD_cmd_files\' ext='_NDTI_SPLINA.sur' template=outpath+'template_NDTI_lapgrd.cmd' ;make sure template exists Result = FILE_TEST(template) if (result ne 1) then print, 'Trouble finding the template' For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*'+ext, count = n) If (n eq 0) then print, kdjfkdjf For j=0, n_elements(List)-1 Do Begin filename=strsplit(list[j],'.', /Extract) ;get right f_image yyyymm = strmid(filename[0], 0,6) yyyymmdd = strmid(filename[0], 0,8) time = strmid(filename[0], 9,4) f_image = $ FILE_SEARCH('W:\DATA\RS\corrected_fpar\converted_coverage\SE_Australia\filled_nulls\',$
yyyymm+'*.flt') ;read in the template file and modify the "magic" record where the filename is kept counter=0 aline='' sarray=strarr(1) OPENR, unit, template, /GET_LUN while not eof (unit) do begin readf, unit, aline
CSIRO Land and Water Page 137
if counter eq 11 then begin strput, aline, strtrim(string(i),2)+'\Rnet\'+yyyymmdd+'_'+time,28 endif else if counter eq 13 then begin strput, aline, strtrim(string(i),2)+'\Tdiff\'+yyyymmdd+'_'+time,28 endif else if counter eq 18 then begin strput, aline, strtrim(string(i),2)+'\NDTI\'+yyyymmdd+'_'+time,28 endif else if counter eq 15 then begin aline = f_image endif else begin if counter eq 0 then strput, aline, strtrim(string(i),2)+'\'+filename[0],58 endelse if (counter eq 0) then sarray[0]=aline else sarray=[[sarray],[aline]] counter++ endwhile free_lun, unit ;output the new modified cmd file using input basename OPENW, 1, outpath+strtrim(string(i),2)+'\'+strmid(filename[0],0,13)+'_NDTI_LAPGRD.cmd' PRINTF, 1, sarray CLOSE, 1 print, outpath+strtrim(string(i),2)+'\'+filename[0]+'_NDTI_LAPGRD.cmd' Endfor Endfor End pro Create_LAPGRD_Batch_Runs_NDTI ;TVN/05-Apr-2007 ;Modified LLT 30 April 2007 ;create batch file to run all LAPGRD commands for Overpass variables (Ta, Ea and VPD) ;assumes a yearly directory structure ;initialise variables startyear=2001 endyear=2001 path='W:\DATA\RADCALC\Spatial_Int_Overpass\LAPGRD_cmd_files\' ext='NDTI_LAPGRD_Tdiff_Interp.cmd' aline1='W:\Programs\Anuspl43\bin\LAPGRD' aline2='_NDTI_LAPGRD_Tdiff_Interp.bat' For i=startyear, endyear Do Begin ;put all files into a list CD, path+strtrim(string(i),2) List = FILE_SEARCH('*'+ext) nRecs=n_elements(list) If (n_elements(List) eq 0) then print, kdjfkdjf sarray=strarr(nRecs) For j=0, nRecs-1 Do Begin filename=strsplit(list[j],'.', /Extract) sarray[j]=aline1+' < '+list[j]+' > '+filename[0]+'.log' Endfor ;output the batch file OPENW, 1,path+strtrim(string(i),2)+'\'+strtrim(string(i),2)+aline2 PRINTF, 1, sarray CLOSE, 1 Endfor End
CSIRO Land and Water Page 138
pro create_HDRs, imgName, hdrTemplate nline = 0 aline = ' ' openr, f_hdr, hdrTemplate, /get_lun while not eof(f_hdr) do begin nline++ readf, f_hdr, aline endwhile hdrArray=strarr(1,nline) point_lun, f_hdr, 0; put pointer back readf, f_hdr, hdrArray close, f_hdr free_lun, f_hdr outFile = (strsplit(imgName, '.', /extract))[0] + '.hdr' openw, tmpFile, outFile, /get_lun printf, tmpFile, hdrArray; write out file close, tmpFile free_lun, tmpFile end img_Path = 'W:\Data\RCALC_SURF\Overpass\' startYear = 2006 EndYear = 2006 arcHeader = 'W:\Data\RCALC_SURF\ArcHeader_Template.hdr' enviHeader ='W:\Data\RCALC_SURF\EnviHeader_Template.hdr' for i = startYear, endYear do begin yearString = strtrim(string(i), 2) img_List = file_search(img_Path+yearString+'\Tdiff\','*.flt', count=FileNumber) If (FileNumber eq 0) then print, 'no images found' else begin For j=0, FileNumber-1 Do Begin theFile = file_basename(img_List[j]) create_HDRs, img_List[j], arcHeader Endfor endelse endfor end
10.13 IDL code to generate initial fortnightly and monthly NDTI composites
Pro Composite_images ;TVN/18-Apr-2007/ ; ;initialise variables samples=1601 lines=1701 ND=-9999.;-999.;-9999. startyear=2001 endyear=2001 origPath='W:\Data\RCALC_SURF\Overpass\' outPath='W:\Data\RCALC_SURF\Overpass\composites\FN_Ta_2001\' yearFlag=1 ext='grd'
CSIRO Land and Water Page 139
count = 0 minVal=0;-0.25;-1.;0. maxVal=1;1.25;1. compositeType='mean';'max';'mean' productType='NDTI';’NDVI’ monthStrArray=['01','02','03','04','05','06','07','08','09','10','11','12'] ;numDays=31; numDays = compositing period, e.g., 10 daily, 14 daily, etc...will be specified by user ;set up day values to start the search on for 10 daily compositing; ;this array will change for other compositing periods, and will eventually be ;provided by the user when the program is called... dayStrArray=['01','16'] ;fortnightly numDaysArray=[15,16]; fortnightly ;dayStrArray=['01']; monthly ;numDaysArray=[31]; monthly For year=startyear, endyear Do Begin yearStr=STRTRIM(STRING(year),2) path=origPath+yearStr+'\'+productType+'\' For month=0, n_elements(monthStrArray)-1 Do Begin monthStr=monthStrArray[month] For interval=0,n_elements(dayStrArray)-1 Do Begin dayStr=dayStrArray[interval] numDays=numDaysArray[interval] ;define list of all files within the specific compositing period List=returnNDTIfileList(path, yearStr, monthStr, dayStr, numDays, ext) ;make sure the file search found something if (list[0] ne '-1') then begin ;put all the images in the list into a stack stack=returnFloatStack(List, samples, lines, 0) ;Get mean, min, or max of the stack as the first go of compositing outArray=fltarr(samples,lines) For s=0, samples-1 Do Begin For l=0, lines-1 Do Begin stackIndex=where((stack[s,l,*] NE ND) AND (stack[s,l,*] GE minVal) AND (stack[s,l,*] LE maxVal), stackCount) if (stackCount NE 0) then begin if (compositeType eq 'mean') then outarray[s,l]=mean(stack[s,l,stackindex]) if (compositeType eq 'max') then outarray[s,l]=max(stack[s,l,stackindex]) if (compositeType eq 'min') then outarray[s,l]=min(stack[s,l,stackindex]) endif else outarray[s,l]=ND Endfor;l Endfor;s ;write out composite images ;if the composit period is less than monthly, then print out one way: if (n_elements(numDaysArray) GT 1) then Begin if (yearFlag eq 1) then $ outName=outpath+yearStr+'\'+yearStr+MonthStr+'_'+STRTRIM(STRING(INTERVAL),2)+$
'_'+compositeType+'COMPOSITE.flt' else $ $ outName=outpath+yearStr+MonthStr+'_'+STRTRIM(STRING(INTERVAL),2)+'_'+$ compositeType+'COMPOSITE.flt'
OPENW, unit, outName, /GET_LUN WRITEU, unit, outarray CLOSE, unit free_lun, unit print, outName
CSIRO Land and Water Page 140
endif else Begin if (yearFlag eq 1) then $ outName=outpath+yearStr+'\'+yearStr+MonthStr+'_'+compositeType+'COMPOSITE.flt' else $ outName=outpath+yearStr+MonthStr+'_'+compositeType+'COMPOSITE.flt' OPENW, unit, outName, /GET_LUN WRITEU, unit, outarray CLOSE, unit free_lun, unit print, outName endelse endif EndFor;interval EndFor;month EndFor;year END function returnFloatStack, List, samples, lines, band ;TVN/24-Apr-2007/ ;returns a floating point stack of the specified band for the list of image names ; ;where band=the idl index of the band number that should be stacked -assumes bsq ; ;example: return a stack of all the NDTI image in the second half of Jan 2001 ; List=returnNDTIfileList('U:\AWAP\Data\RCALC_SURF\Overpass\', '2001', '01', '16', 16, 'grd') ; stack=returnFloatStack(List, 1601, 1701, 0) ;Stack each image in the list numImages=n_elements(List) print, numImages stack = fltarr(samples,lines,numImages) For m=0, numImages-1 Do Begin ;read image data into a variable ;Open input files if (band ne 0) then invar=fltarr(samples,lines,band+1) else invar=fltarr(samples,lines) inVARName = List[m] OPENR, unit, inVARName, /GET_LUN readu, unit, invar close, unit free_lun, unit if (band ne 0) then stack[*,*,m]=invar[*,*,band] else stack[*,*,m]=invar Endfor;m return, stack End function returnNDTIfileList, p, yearStr, monthStr, dayStr, numDays, ext ;TVN/24-Apr-2007/ ;returns a list of all available NDTI images in a given path ;for a given year, month, start day, and number of days ; ;example: return a list of all the NDTI images for the second half of Jan 2001 ; ;List=returnNDTIfileList('U:\AWAP\Data\RCALC_SURF\Overpass\NDTI\', '2001', '01', '16', 16, 'grd') ; ;ENVI> print, list ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001011614403_NDTI_lapgrd.grd
CSIRO Land and Water Page 141
; U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001011714303_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001011814193_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001011815593_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001011914093_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001011915493_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012013593_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012113483_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012213383_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012313273_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012413173_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012414583_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012514483_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012614383_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012714273_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012814173_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012815573_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012914063_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001012915473_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001013013563_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001013015373_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001013113463_NDTI_lapgrd.grd ;U:\AWAP\Data\RCALC_SURF\Overpass\2001\NDTI\2001013115263_NDTI_lapgrd.grd ;initialise variables tmpCount=0 fnStart = '*'+yearStr+monthStr numArray=indgen(numDays)+fix(dayStr) searchStrings=strarr(numDays) ;insert '0's in front of the number strings if necessary For i=0,numDays-1 Do begin str=STRTRIM(STRING(numarray[i]),2) numDigits=STRLEN(str) ;if numDigits eq 1, then insert a zero in front of it if (numDigits eq 1) then searchStrings[i]=fnStart+'0'+str+'*.'+ext else searchStrings[i]=fnStart+str+'*.'+ext End;i ;search based on the possible filenames for the specified compositing period ;put all files into a list For j=0, n_elements(searchStrings)-1 Do begin ;for recursive searching use two search variables tmp=FILE_SEARCH(p,searchStrings[j]) ;for non-recursive searching, cd into the path and use one search variable ;CD, p ;tmp=FILE_SEARCH(searchStrings[j]) If (tmp[0] ne '') then begin if (tmpCount eq 0) then List=tmp else List=[LIST,tmp] tmpCount++ Endif EndFor;j ;if no files were found in the search, set the list to -1 if (n_elements(list) eq 0) then list=['-1'] return, list End
CSIRO Land and Water Page 142
10.14 IDL code to gap-fill fortnightly and monthly NDTI composites pro fill_nulls_ndti_composites_hierarchy_2tier_only ;TVN/12-July-2007/ ;fills all nulls in each of the Monthly or FN'ly composites ; ;initialise variables stYear=1990 enYear=2006 samples=1601 lines=1701 ext='.flt' interval='MTH';'FN' basePath='W:\Data\RCALC_SURF\Overpass\composites\'+interval+'\';0 to 1 mean composites GMname=basePath+'grandMean_'+interval+ext;Grand mean composites outpath='W:\Data\RCALC_SURF\Overpass\composites\'+interval+'_2tier_filled_nulls_z1\' ND=-9999. month_num_array=['01','02','03','04','05','06','07','08','09','10','11','12'] if (interval eq 'FN') then Begin numIntervals=2 z=24 endif else Begin numIntervals=1 z=12 endelse array=fltarr(samples,lines); zero to one FN mean composite GMstack=fltarr(samples,lines,z) ;Read in grand mean data just once. OPENR, unit, GMName, /GET_LUN readu, unit, GMstack free_lun, unit ;define the background nulls backgrndNulls=where(GMstack[*,*,0] eq ND, BNcount) ;go through each (FN for each) Month for each Year For year=styear, enYear Do Begin yearStr=STRTRIM(STRING(year),2) Count=0 For month=0, 11 DO BEGIN monthStr=month_num_array[month] For i=0,numIntervals-1 Do BEGIN intervalString=STRTRIM(STRING(i),2) p=basePath+yearStr+'\' if (interval eq 'FN') then searchStr=yearStr+monthStr+'_'+intervalString+'_*'+ext else searchStr=yearStr+monthStr+'*'+ext FileList=FILE_SEARCH(p,searchStr,count=FileCount) if (FileCount gt 1)then Begin print, 'File count for '+searchStr+', should have been 1 or less. Skipping this one..' break endif
CSIRO Land and Water Page 143
;Now read in the zero-to-one FN image and the all-data-truncated image into variables if (FileCount ne 0) then Begin fileName = FileList[0] OPENR, unit, fileName, /GET_LUN readu, unit, array free_lun, unit endif else array[*,*]=ND; if there was no file for this FN (or MTH) then use a completely nulled array ;get the correct band of the GM stack GMarray=GMstack[*,*,count] outarray=Array msk=bytarr(samples,lines) ;Fill nulls in all data composite with GM composite valid=where(array ne ND, vCount, complement=nulls, ncomplement=ncount) if (ncount ne 0) then begin outArray[nulls]=GMarray[nulls] msk[nulls]=3 endif if (vcount ne 0) then msk[valid]=2 ;make sure to set the ocean nulls to 0 in the mskarray msk[backgrndNulls]=0 ;write out version 1 of the filled image - using 2 tiers and it's associated mask if (fileCount eq 0) then begin if (interval eq 'FN') then outName=yearStr+monthStr+'_'+intervalString+'_'+'meanCOMPOSITE_filled'+ext else outName=yearStr+monthStr+'_'+'meanCOMPOSITE_filled'+ext endif else begin tmp=strsplit(FileList[0],'.',/extract) outName=file_basename(tmp[0])+'_filled.flt' endelse OPENW, unit, outpath+yearStr+'\'+outName, /GET_LUN WRITEU, unit, outarray CLOSE, unit free_lun, unit print, outpath+yearStr+'\'+outName if (fileCount eq 0) then begin if (interval eq 'FN') then outName=yearStr+monthStr+'_'+intervalString+'_'+'meanCOMPOSITE_filled_LUT.byt' else outName=yearStr+monthStr+'_'+'meanCOMPOSITE_filled_LUT.byt' endif else begin tmp=strsplit(FileList[0],'.',/extract) outName=file_basename(tmp[0])+'_filled_LUT.byt' endelse OPENW, unit, outpath+yearStr+'\'+outName, /GET_LUN WRITEU, unit, msk CLOSE, unit free_lun, unit print, outpath+yearStr+'\'+outName
CSIRO Land and Water Page 144
Count++ EndFor EndFor EndFor END
Tim R. McVicar, Tom G. Van Niel, Lingtao Li, Edward A. King and Randall J. Donohue
Deriving moisture availability from time series remote sensing for ecohydrological applications: Development of a prototype near real-time operational system.
CSIRO Land and Water Science Report 37/07
For further information:
CSIRO Land and WaterTim McVicarPhone: (02) 6246 5741Email: [email protected]: www.clw.csiro.au
Deriving m
oisture availability from tim
e series remote sensing C
SIRO Land and W
ater Science Report 37/07