restoration of missing data due to clouds on optical...
Post on 23-Jan-2021
1 Views
Preview:
TRANSCRIPT
Restoration of Missing Data due to Clouds on Optical Satellite Imagery Using Neural
Restoration of Missing Data due to Clouds on Optical Satellite Imagery Using Neural
Sergii Skakun1, Nataliia Kussul1, Ruslan Basarab2 1 Space Research Institute NAS and SSA Ukraine 2 National University of Life and Environmental Sciences of Ukraine Sentinel-2 for Science Workshop May 22, 2014, ESA/ESRIN, Frascati, Italy
S2 for
Science
2014 2
Content
• Objective of the study
• Methodology – Reconstruction of missing data using self-organising
Kohonen maps (SOM)
• Data used – Landsat-8
– Sich-2
• Results – Reconstruction of missing data in Landsat-8 and
Sich-2
– Large scale crop mapping using reconstructed imagery (Landsat-8)
• Discussion & conclusions
S2 for
Science
2014 3
Objective of the study
• Clouds and shadows – limiting factor in exploitation of optical
satellite imagery
– cause missing data
• Existing approaches (filling missing data) – inpainting-based (Lorenzi et al. 2011)
– multispectral-based • using MODIS (Roy et al. 2008)
– multitemporal-based • using SOM for MODIS time-series (Latif
& Mercier 2010)
• Objectives – to qualitatively assess the use of SOMs
for restoring missing data on time-series of high and medium resolution satellite images
– to provide crop mapping using restored images
Landsat-8 images over
JECAM Ukraine
02.05.2013
18.05.2013
02.05.2013
18.05.2013
02.05.2013
18.05.2013
19.06.2013
05.07.2013
S2 for
Science
2014 4
Methodology: SOM
• Self-organizing Kohonen maps (SOMs) – type of artificial neural network
– unsupervised learning
– produces a 2D, discretized representation of the input space
SOM architecture
SOM training process
lLl
i wxx ,1
minarg)(
Training sample Neuron
winner Updating weights
Llnnhnnn jilll ,1)),()(()()()1( )(, wxww x
S2 for
Science
2014 5
Methodology: Restoration
• Performed for each spectral band separately
• Training phase: – Training samples selected
automatically: on a regular grid
– Only pixels with all valid (i.e. non-missing) values considered
Restoration of missing values
Landsat-8 time-
series (Band 4)
X1 X2 X3 Nan X5 Nan
Input
SOM: selection of neuron winner
X1 X2 X3 Nan X5 Nan
wl1 wl2 wl3 wl4 wl5 wl6
Missing
Only valid components are considered for
finding a neuron winner
wi1 wi2 wi3 wi4 wi5 wi6
Missing components
are taken from
neuron winner
S2 for
Science
2014 6
Data used & experiment setup
• Landsat-8 – Resolution: 30 m
– Pre-processing:
• DN -> TOA -> SR
• Simplified Model for Atmospheric Correction (SMAC) (Rahman & Dedieu 1994)
– Dates:
• 16 April; 2 and 18 May; 19 June 2013
• Sich-2 – Resolution: 8 m
– DN used
– Dates:
• 3 June; 4, 14 and 19 September 2013
• Qualitative assessment – 2 ROIs selected that
were artificially assigned Nan values
– Metrics: RMSE and Relative RMSE (RRMSE)
ROI1
ROI2
Landsat-8
(true colour)
Sich-2 (false colour)
S2 for
Science
2014 7
Results (1)
Average RMSE error of reconstructing missing values for
ROI1 and ROI2 on Landsat-8 images. RMSE values are
shown depending on the number of missing values in the
time-series (M=1, 2, 3) and fraction of pixels taken for
training. X-axis is shown in a logarithmic scale.
S2 for
Science
2014 8
Results (2)
Average RRMSE error of reconstructing missing values for
ROI1 and ROI2 on Landsat-8 images. RRMSE values are
shown depending on the number of missing values in the
time-series (M=1, 2, 3) and fraction of pixels taken for
training. X-axis is shown in a logarithmic scale.
S2 for
Science
2014 9
Results (3)
Average RMSE and RRMSE error of reconstructing missing
values for ROI1 and ROI2 on Sich-2 images. RMSE and
RRMSE values are shown depending on the number of missing
values in the time-series (M=1, 2, 3) and fraction of pixels taken
for training. X-axis is shown in a logarithmic scale.
S2 for
Science
2014 10
Results (4)
Restoration of ROI1 for Landsat-8
S2 for
Science
2014 11
Crop mapping (1)
• Landsat-8 time-series: – Dates (6 images):
• 16 April, 02 May, 18 May, 19 June, 05 July, 06 August 2013
– Path/row:
• 181/24, 181/25, 181/26
– Pre-processing:
• TOA->DN->SR – SR: Simplified Model for
Atmospheric Correction (SMAC) (Rahman & Dedieu 1994)
– Aerosol optical depth: Aeronet station (Kyiv)
• Clouds & shadows detection – Fmask (Zhu & Woodcock
2012)
• Filling missing values – SOMs
TOA (top) and SR (bottom) for Landsat-8
acquired on 08 August 2013. A true color
composition of Landsat-8 bands 4-3-2. TOA and
SR reflectance are scaled from 0 to 0.15.
S2 for
Science
2014 12
Crop mapping (2)
Example of restoration for
Landsat-8 image
acquired on 05 July 2013
(true color composite)
S2 for
Science
2014 13
Crop mapping (3)
• Ground observations – Along the roads
• ~ 390 polygons
• Classes: No. LUCAS Description
1 Axx Artificial
2 B11 Winter wheat (and barley)
3 B32 Winter rapeseed
4 B12, B14 Spring crops (wheat, barley)
5 B16 Maize
6 B22 Sugar beet
7 B31 Sunflower
8 B33 Soybeans
9 B19, B39, B40
Other cereals, other annual crops, temporary grass
10 Cxx, B60 Forest, fruit trees
11 Exx Grassland
12 Fxx Bare land
13 Gxx Water
Location of along the
roads surveys within
Kyiv oblasts
Distribution of training (50%) and test (50%) data
Train
Test
S2 for
Science
2014 14
Crop mapping (4)
• Methodology – Inputs:
• Restored SR values (bands 2-7) for 6 multi-temporal images
• Total inputs: 36
– Ensemble of 6 neural nets:
• 30, 40, 50, 60, 70 and 80 hidden units
– Ensemble: • Max average a-
posteriori probability
Best single net
Ensemble
OA, % 83.90 85.07
Kappa 0.807 0.820
PA, % UA, %
1 Artificial 100.0 98.4
2 Winter wheat 95.4 91.6
3 Winter rapeseed 93.2 99.3
4 Spring crops 28.3 27.2
5 Maize 91.1 86.1
6 Sugar beet 94.7 90.2
7 Sunflower 84.2 86.4
8 Soybeans 68.8 77.5
9 Other cereals 71.3 75.7
10 Forest 97.0 92.7
11 Grassland 90.6 88.9
12 Bare land 86.9 98.8
13 Water 100.0 98.6
S2 for
Science
2014 15
Crop mapping (5)
Landsat-8 image of 05
July 2013 (true color
composite)
Restored Landsat-8
image from 05 July
2013 using SOMs
Crop map
S2 for
Science
2014 16
Conclusions
• SOMs for high & med resolution satellite images – encodes samples with non-missing values during
a training phase, and then reconstructs missing values from SOM weights
• Accuracy of reconstruction – Landsat-8: most accurate for NIR bands (11-15%)
comparing to visible bands (16-19%)
– Sich-2: relative error green (4.3%), red (5.8%), and NIR (8%)
• Efficient for large scale crop mapping
S2 for
Science
2014 17
Thank you!
top related