spatial and spectral evaluation of image fusion methods
DESCRIPTION
Spatial and Spectral Evaluation of Image Fusion Methods. Sascha Klonus Manfred Ehlers Institute for Geoinformatics and Remote Sensing University of Osnabrück. Content. Introduction Image Fusion Test Site Fusion Results Color Distortions Evaluation Methods and Results Ehlers Fusion - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/1.jpg)
Spatial and Spectral Evaluation of Image Fusion Methods
Sascha KlonusManfred Ehlers
Institute for Geoinformatics and Remote SensingUniversity of Osnabrück
![Page 2: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/2.jpg)
Content
Introduction Image Fusion
Test Site
Fusion Results
Color Distortions
Evaluation Methods and Results
Ehlers Fusion
Conclusions and Future Work
![Page 3: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/3.jpg)
Remote sensors have different spatial resolution for panchromatic and multispectral imagery
The ratios vary between 1:2 and 1:5
For multisensor fusion the ratios can exceed 1:30(e.g. Ikonos/Landsat)
Data Fusion: Why is it Necessary?
![Page 4: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/4.jpg)
Objectives of Image Fusion
Sharpen images Improve geometric corrections Provide stereo-viewing capabilities Enhance certain features Complement data sets Detect changes Substitute missing information Replace defective data
Pohl & van Genderen (1998)
![Page 5: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/5.jpg)
Meaning of Pan-Sharpening
Spatial Spectral +
panchromatic &high geometric resolution
multi-/hyperspectral image &low geometric resolution
multi-/hyperspectral &high geometric resolution
![Page 6: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/6.jpg)
Fusion Methods
Color Transformations Modified IHS Transformation
Statistical Methods Principal Component Merge
Numerical Methods Brovey CN Spectral Sharpening Gram-Schmidt Spectral Sharpening Wavelet based Fusion
Combined Methods Ehlers Fusion
![Page 7: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/7.jpg)
Test Site
![Page 8: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/8.jpg)
Original Data
Quickbird Multispectral image (2004-09-04)
Quickbird Panchromatic image (2004-09-04)
Formosat Multispectral image (2004-01-30)
Ikonos Multispectral image (2005-08-03)
![Page 9: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/9.jpg)
Single Sensor Fusion: Quickbird
Quickbird Multispectral imageFused with BroveyFused with CN Spectral SharpeningFused with EhlersFused with WaveletFused with Gram-Schmidt Fused with PCFused with modified IHS
![Page 10: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/10.jpg)
Multisensor Fusion: Ikonos
Ikonos Multispectral imageFused with BroveyFused with CN Spectral SharpeningFused with EhlersFused with modified IHSFused with PCFused with Gram-Schmidt Fused with Wavelet
![Page 11: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/11.jpg)
Multisensor Fusion: Formosat
Formosat Multispectral imageFused with BroveyFused with CN Spectral SharpeningFused with EhlersFused with modified IHSFused with PCFused with Gram-Schmidt Fused with Wavelet
![Page 12: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/12.jpg)
Panchromatic band has a different spectral sensitivity
Multisensoral differences (e.g. Ikonos and SPOT merge)
Multitemporal (seasonal) changes between pan and ms image data
Fusion Problem: Color Distortion
Inconsistent panchromatic information is fused into the multispectral bands
![Page 13: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/13.jpg)
Spectral Comparison Methods (1)
fusedorg
fusedorg
xxbias
ss
biasRMSE
22
s = standard deviation
org = Original image
fused = Fused image
x = Mean
RMSE
Correlation coefficients
)()(
),(),(
yVarxVar
yxCovyxKor
Visual (Structure and Colour Preservation)
![Page 14: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/14.jpg)
Results RMSE
Quickbird Ikonos Formosat
Mod. IHS 7.1822 11.0714 1.7792
PC 42.5058 23.8979 23.3095
Brovey 222.1732 143.8303 14.3147
CN-Sharpening 46.7389 123.5038 70.4803
Gram-Schmidt 5.6883 5.1425 0.4078
Wavelet 3.1915 2.5379 0.0290
Ehlers 1.0028 4.3645 0.4201
![Page 15: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/15.jpg)
Results Correlation Coefficients
Quickbird Ikonos Formosat
Mod. IHS 0.8737 0.5731 0.5892
PC 0.8811 0.6512 0.6199
Brovey 0.8611 0.6020 0.5677
CN-Sharpening 0.8611 0.6020 -0.0546
Gram-Schmidt 0.8690 0.6774 0.6110
Wavelet 0.9698 0.9719 0.9720
Ehlers 0.9510 0.8094 0.9324
![Page 16: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/16.jpg)
Spectral Comparison Methods (2)
Per Pixel Deviation
Degrade
Degraded to ground resolution of original image(Formosat = 8m)
Original multispectral image (Formosat 8m) Band 1 2.56
Band 2 2.92
Band 3 3.49
Band 4 3.35
Result: Vector containing the deviation per pixel
Fused image (Formosat 2m)
![Page 17: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/17.jpg)
Mean Per Pixel Deviation
Quickbird Ikonos Formosat
IHS 27.28 42.95 11.93
PC 48.80 53.27 22.52
Brovey 209.15 136.62 19.48
CN-Sharpening 48.73 117.61 70.71
Gram-Schmidt 30.35 45.29 11.99
Wavelet 7.20 13.47 3.04
Ehlers 17.28 25.86 4.29
![Page 18: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/18.jpg)
Spatial Comparison Methods (1)
Edge Detection
-
--
Band 1 91.16 %
Band 2 92.10 %
Band 3 92.64 %
Mean 91.96 %
![Page 19: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/19.jpg)
Results Edge Detection
Quickbird Ikonos Formosat
Mod. IHS 92.71 % 92.44 % 95.54 %
PC 95.10 % 93.28 % 93.44 %
Brovey 94.69 % 95.16 % 97.87 %
CN-Sharpening 94.69 % 95.16 % 90.69 %
Gram-Schmidt 95.02 % 95.53 % 97.82 %
Wavelet 85.00 % 83.82 % 84.81 %
Ehlers 91.85 % 90.35 % 94.40 %
![Page 20: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/20.jpg)
Spatial Comparison Methods (2)
Highpass Filtering
Correlation
Band 1 0.8012
Band 2 0.7820
Band 3 0.7912
Mean 0.7918
![Page 21: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/21.jpg)
Highpass Correlation Results
Quickbird Ikonos Formosat
Mod. IHS 0.9336 0.9149 0.9420
PC 0.9900 0.0021 0.8073
Brovey 0.9715 0.9765 0.9895
CN-Sharpening 0.9714 0.9764 -0.0170
Gram-Schmidt 0.9857 0.9879 0.9652
Wavelet 0.3976 0.3627 0.3799
Ehlers 0.8997 0.8689 0.9349
![Page 22: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/22.jpg)
FFT Filter Based Data Fusion (Ehlers Fusion)
Panchromatic Image
Multispectral Image
R
G
B
Basis: IHS Transform and Filtering in the Fourier Domain
FFT FourierSpectrum
FFT FourierSpectrum
HPFPanHP
LPFILPI
H
S
R‘
G‘
B‘
IHS-1
ILP+PanHP
H
S
FFT-1
![Page 23: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/23.jpg)
Panchromatic image and its spectrum
Original panchromatic image
Panchromatic Spectrum
![Page 24: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/24.jpg)
Filtersetting effects
Frequency
Intensity
Cut-off Frequencyfn
Filtered Panchromatic Spectrum
![Page 25: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/25.jpg)
Effects in the spatial domain
Filtered panchromatic image Fused image
![Page 26: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/26.jpg)
Filtersetting effects
Frequency
Intensity
Cut-off Frequencyfn
Filtered Panchromatic Spectrum
![Page 27: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/27.jpg)
Effects in the spatial domain
Filtered panchromatic image Fused image
![Page 28: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/28.jpg)
Filtersetting effects
Filtered Panchromatic Spectrum
Frequency
Intensity
Cut-off Frequencyfn
![Page 29: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/29.jpg)
Effects in the spatial domain
Filtered panchromatic image Fused image
![Page 30: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/30.jpg)
Results
Ehlers Fusion shows the best overall results in all images
It works also if the panchromatic Information does not match the spectral sensitivity of the merged bands (multitemporal and multisensoral fusion)
Its performance is superior to standard fusion techniques (IHS, Brovey Transform, PC Merge)
Wavelet preserves the spectral characteristics at the cost of spatial improvement
Ehlers Fusion is integrated in a commercial image processing system (Erdas Imagine 9.1)
![Page 31: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/31.jpg)
Future Work
Fusion of radar- and optical Data
Development of one method to evaluate the spatial and spectral quality of an fused image
Comparison with the algorithm of Zhang (PCI Geomatica)
Research on automation for filter design
![Page 32: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/32.jpg)
Thanks for your Attention
Questions???
![Page 33: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/33.jpg)
Ehlers Fusion Program
![Page 34: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/34.jpg)
Ehlers Fusion Program
![Page 35: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/35.jpg)
Ehlers Fusion Program
![Page 36: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/36.jpg)
Ehlers Fusion Program
![Page 37: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/37.jpg)
Ehlers Fusion Program
![Page 38: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/38.jpg)
Multispectral image and its spectrum
Original multispectral intensity
Multispectral intensity spectrum
![Page 39: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/39.jpg)
Filtersetting effects
Filtered multispectral spectrum
Frequency
Intensity
Cut-off Frequencyfn
![Page 40: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/40.jpg)
Filtersetting effects
Filtered multispectral spectrum
Frequency
Intensity
Cut-off Frequencyfn
![Page 41: Spatial and Spectral Evaluation of Image Fusion Methods](https://reader036.vdocuments.us/reader036/viewer/2022062305/56815736550346895dc4d84c/html5/thumbnails/41.jpg)
Filtersetting effects
Filtered multispectral spectrum
Frequency
Intensity
Cut-off Frequencyfn