Evaluation of Pan-Sharpening Techniques Using Lagrange Optimization
DOI:
https://doi.org/10.46604/aiti.2020.4288Keywords:
image fusion, pan-sharpening, optimization, spectral consistencyAbstract
Earth’s observation satellites, such as IKONOS, provide simultaneously multispectral and panchromatic images. A multispectral image comes with a lower spatial and higher spectral resolution in contrast to a panchromatic image which usually has a high spatial and a low spectral resolution. Pan-sharpening represents a fusion of these two complementary images to provide an output image that has both spatial and spectral high resolutions. The objective of this paper is to propose a new method of pan-sharpening based on pixel-level image manipulation and to compare it with several state-of-art pansharpening methods using different evaluation criteria. The paper presents an image fusion method based on pixel-level optimization using the Lagrange multiplier. Two cases are discussed: (a) the maximization of spectral consistency and (b) the minimization of the variance difference between the original data and the computed data. The paper compares the results of the proposed method with several state-of-the-art pan-sharpening methods. The performance of the pan-sharpening methods is evaluated qualitatively and quantitatively using evaluation criteria, such as the Chi-square test, RMSE, SNR, SD, ERGAS, and RASE. Overall, the proposed method is shown to outperform all the existing methods.
References
L. Wald, “Data fusion: a conceptual approach for an efficient exploitation of remote sensing images,” 2nd International Conference, France, pp. 17-24, 1998.
C. Pohl and J. L. V. Genderen, “Review article multisensor image fusion in remote sensing: concepts, methods and applications,” International Journal of Remote Sensing, vol. 19, no. 5, pp. 823-854, 1998.
T. M. Lillesan. W. J. Carper, and R. W. Kiefer “The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogrammetric Engineering and Remote Sensing, vol. 56, no. 4, pp. 459-467, April 1990.
J. P. Chavez and P. S. Chavez “Comparison of the spectral information content of landsat thematic mapper and SPOT for three different sites in the Phoenix, Arizona region,” Photogrammetric Engineering & Remote Sensing, vol. 54, no. 12, pp. 1699-1708, January 1988.
K. Edwards and P. A. Davis, “The use of intensity-hue-saturation transformation for producing color shaded-relief images,” Photogrammetric Engineering & Remote Sensing, vol. 60, pp. 1369-1374, November 1994.
T. M. Tu, P. S. Huang, C. H. Huang, and C. P. Chang, “A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery,” IEEE Geoscience and Remote Sensing Letters, vol. 1, no. 4, pp. 309-312, October 2004.
A. R. Gillespie, A. B. Kahle, and R. E. Walker, “Color enhancement of highly correlated images. II. channel ratio and “chromaticity” transformation techniques,” Remote Sensing of Environment, vol. 22, no. 3, pp. 343-365, August 1987.
D. B. Yuan, X. Hong, S. Yu, L. Li, and Y. Zhao, “Analysis of four remote image fusion algorithms for Landsat7 ETM+ PAN and Multi-spectral imagery,” International Journal of Online Engineering, vol. 10, no. 3, p. 49, 2014.
J. Qu, Y. Li, and W. Dong, “Guided filter and principal component analysis hybrid method for hyperspectral pansharpening,” Journal of Applied Remote Sensing, vol. 12, no. 1 pp. 1-18, 2018.
M. Ghadjati, A. Moussaoui, and A. Boukharouba, “A novel iterative PCA-based pansharpening method,” Remote Sensing Letters, vol. 10, no. 3, pp. 264-273, 2019.
M. B. Devi and R. Devanathan, “Pansharpening using data-centric optimization approach,” International Journal of Remote Sensing, vol. 40, no. 20, pp. 7784-7804, 2019.
J. P. Chavez, S. C. Sides, and J. A. Anderson, “Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic,” Photogrammetric Engineering and Remote Sensing, vol. 57, no. 3, pp. 265-303, March 1991.
G. Hong, Y. Zhang, and B. Mercer, “A wavelet and IHS integration method to fuse high resolution SAR with moderate resolution multispectral images,” Photogrammetric Engineering & Remote Sensing, vol. 75, no. 10, pp. 1213-1223, October 2009.
Y. Zhang and G. Hong, “An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images,” Information Fusion, vol. 6, no. 3, pp. 225-234, September 2005.
S. Ourabia and Y. Smara, “A new pansharpening approach based on nonsubsampled contourlet transform using enhanced PCA applied to SPOT and ALSAT-2A satellite images,” Journal of the Indian Society of Remote Sensing, vol. 44, no. 5, pp. 665-674, October 2016.
A. Svab Lenarcic and K. Oštir, “High-resolution image fusion: methods to preserve spectral and spatial resolution,” Photogrammetric Engineering and Remote Sensing, vol. 72, no. 5, pp. 565-572, May 2006.
C. Yang, Q. Zhan, H. Liu, and R. Ma, “An IHS-Based pansharpening method for spectral fidelity improvement using ripplet transform and compressed sensing,” Sensors, vol. 18, no. 11, pp. 1-20, November 2018.
T. Wang, F. Fang, F. Li, and G. Zhang, “High-quality Bayesian pansharpening,” IEEE Trans Image Process, vol. 28, no. 1, pp. 227-239, January 2019.
K. Nikolakopoulos, “Comparison of nine fusion techniques for very high resolution data,” Photogrammetric Engineering & Remote Sensing, vol. 74, pp. 647-659, 2008.
H. Li, L. Jing, and Y. Tang, “Assessment of pansharpening methods applied to WorldView-2 imagery fusion,” Sensors, vol. 17, no. 1, pp. 1-30, January 2017.
E. I. Ulzurrun, C. G. Martin, J. M. Ruiz, A. G. Pedrero, and D. R. Esparragon, “Fusion of high resolution multispectral imagery in vulnerable coastal and land ecosystems,” Sensors, vol. 17, no.2, pp. 1-23, February 2017.
H. Li, L. Jing, Y. Tang, and H. Ding, “An improved pansharpening method for misaligned panchromatic and multispectral data,” Sensors, vol. 18, no. 2, pp. 1-16, February 2018.
C. Chen, Y. Li, W. Liu, and J. Huang, “Image fusion with local spectral consistency and dynamic gradient sparsity,” 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2760-2765, 2014.
X. Meng, J. Li, H. Shen, L. Zhang, and H. Zhang, “Pansharpening with a guided filter based on three-layer decomposition,” Sensors, vol. 16, no. 7, pp. 1-15, July 2016.
A. Sekrecka and M. Kedzierski, “Integration of satellite data with high resolution ratio: improvement of spectral quality with preserving spatial details,”, Sensors, vol. 18, no. 12, pp. 1-22, December 2018.
C. Ballester, V. Caselles, J. Verdera, and B. Rougé, “A variational model for P+XS image fusion,” International Journal of Computer Vision, vol. 69, pp. 43-58, 2006.
F. Fang, F. Li, C. Shen, and G. Zhang, “A variational approach for pansharpening,” IEEE Transactions on Image Processing, vol. 22, no. 7, pp. 2822-2834, 2013.
M. Möller, T. Wittman, A. L. Bertozzi, and M. Burger, “A variational approach for sharpening high dimensional images,” SIAM Journal on Imaging Sciences, vol. 5, no. 1, pp. 150-178, 2012.
L. J. Deng, G. Vivone, W. Guo, M. Dalla Mura, and J. Chanussot, “A variational pansharpening approach based on reproducible Kernel Hilbert space and Heaviside function,” IEEE Trans Image Process, vol. 27, no. 9, pp. 4330-4344, September 2018.
C. Kwan, J. H. Choi, S. H. Chan, J. Zhou, and B. Budavari, “A super-resolution and fusion approach to enhancing hyperspectral images,” Remote Sensing, vol. 10, no. 9, pp. 1-28, September 2018.
Y. Qu, H. Qi, and C. Kwan, “Unsupervised and unregistered hyperspectral image super-resolution with Mutual Dirichlet-Net,” Computer Vision and Pattern recognition, 2019.
R. Borsoi, T. Imbiriba, and J. Carlos Moreira Bermudez, “Super-resolution for hyperspectral and multispectral image fusion accounting for seasonal spectral variability,” IEEE Transactions on Image Processing, vol. 29, pp. 116-127, July 2019.
H. AanÆs, J. R. Sveinsson, A. A. Nielsen, T. Bovith, and J. A. Benediktsson, “Model-based satellite image fusion,” IEEE Transactions on Geoscience and Remote Sensing,” vol. 46, no. 5, pp. 1336-1346, 2008.
H. Aanaes, A. A. Nielsen, T. Bovith, J. R. Sveinsson, and J. A. Benediktsson, “Spectrally consistent satellite image fusion with improved image priors,” Proc. of the 7th Nordic Signal Processing Symposium-NORSIG 2006, IEEE Press, Janury 2006, pp. 14-17.
G. Cliche, F. Bonn, and P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogrammetric Engineering & Remote Sensing, vol. 51, pp. 311-316, 1985.
X. Meng, H. Shen, H. Li, L. Zhang, and R. Fu, “Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: practical discussion and challenges,” Information Fusion, vol. 46, pp. 102-113, 2018.
P. Ghamisi, B. Rasti, N. Yokoya, Q. Wang, B. Hofle, L. Bruzzone, and P. M. Atkinson, “Multisource and multitemporal data fusion in remote sensing: A comprehensive review of the state of the art,” IEEE Geoscience and Remote Sensing Magazine, vol. 7, no. 1, pp. 6-39, March 2019.
G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. A. Licciardi, and L. Wald, “A critical comparison among pansharpening algorithms,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2565-2586, December 2014.
S. Kahraman and A. Erturk, “Review and performance comparison of pansharpening algorithms for RASAT images,” Journal of Electrical & Electronics Engineering, vol. 18, no. 1, pp. 109-120, 2018.
R. L. Plackett, “International Statistical Review”, vol. 51, pp. 59-72, 1983.
M. Fallah and A. Azizi, “Quality assessment of image fusion techniques for multisensor high resolution satellite images (Case study: IRS-P5 AND IRS-P6 satellite images),” In: Wagner W., Székely, B. (eds.): ISPRS TC VII Symposium – 100 Years ISPRS, Vienna, Austria, vol. 38, January 2010.
Yuhendra, I. Alimuddin, J. T. S. Sumantyo, and H. Kuze, “Assessment of pan-sharpening methods applied to image fusion of remotely sensed multi-band data,” International Journal of Applied Earth Observation and Geoinformation, vol. 18, pp. 165-175, 2012.
T. Ranchin, L. Wald, and M. Mangolini, “The ARSIS method: a general solution for improving spatial resolution of images by the means of sensor fusion,” Fusion of Earth Data, Cannes, France, pp. 53-58, February 1996.
C. Myungjin, “A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 6, pp. 1672-1682, 2006.
A. Vesteinsson, J. R. Sveinsson, J. A. Benediktsson, and H. Aanaes, “Spectral consistent satellite image fusion: using a high resolution panchromatic and low resolution multi-spectral images,” Proc. 2005 IEEE International Geoscience and Remote Sensing Symposium, vol. 4, pp. 2834-2837, July 2005.
Published
How to Cite
Issue
Section
License
Submission of a manuscript implies: that the work described has not been published before that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication. Authors can retain copyright in their articles with no restrictions. is accepted for publication. Authors can retain copyright of their article with no restrictions.
Since Jan. 01, 2019, AITI will publish new articles with Creative Commons Attribution Non-Commercial License, under The Creative Commons Attribution Non-Commercial 4.0 International (CC BY-NC 4.0) License.
The Creative Commons Attribution Non-Commercial (CC-BY-NC) License permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
