Abstract
Cross-domain few-shot hyperspectral image classification focuses on learning prior knowledge from a large number of labeled samples from source domains and then transferring the knowledge to the tasks which contain few labeled samples in target domains. Following the metric-based manner, many current methods first extract the features of the query and support samples, and then directly predict the classes of query samples according to their distance to the support samples or prototypes. The relations between samples have not been fully explored and utilized. Different from current works, this paper proposes to learn sample relations on different levels and take them into the model learning process, to improve the cross-domain few-shot hyperspectral image classification. Building on current method of "Deep Cross-Domain Few-Shot Learning for Hyperspectral Image Classification" which adopts a domain discriminator to deal with domain-level distribution difference, the proposed method applies contrastive learning to learn the class-level sample relations to obtain more discriminable sample features. In addition, it adopts a transformer based cross-attention learning module to learn the set-level sample relations and acquire the attention from query samples to support samples. Our experimental results have demonstrated the contribution of the multi-level relation learning mechanism for few-shot hyperspectral image classification when compared with the state of the art methods. All the codes are available at github https://github.com/HENULWY/STBDIP.
Similar content being viewed by others
Availability of data and materials
The datasets generated and analysed during the current study are available at https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes and http://naotoyokoya.com/Download.html.
Code availability
The codes are available at https://github.com/HENULWY/STBDIP.
References
Li S, Song W, Fang L, Chen Y, Benediktsson JA (2019) Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing PP (99) 1–20
Jia S, Jiang S, Lin Z, Li N, Xu M, Yu S (2021) A survey: Deep learning for hyperspectral image classification with few labeled samples. Neurocomputing 448:179–204
Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing 42(8):1778–1790
Li J, Bioucas-Dias JM, Plaza A (2012) Semisupervised hyperspectral image classification using soft sparse multinomial logistic regression. IEEE Geoscience and Remote Sensing Letters 10(2):318–322
Licciardi G, Marpu PR, Chanussot J, Benediktsson JA (2011) Linear versus nonlinear pca for the classification of hyperspectral data based on the extended morphological profiles. IEEE Geoscience and Remote Sensing Letters 9(3):447–451
Zhang C, Zheng Y (2014) Hyperspectral remote sensing image classification based on combined svm and lda. Multispectral, Hyperspectral, and Ultraspectral Remote Sensing Technology, Techniques and Applications V 9263:462–468
Qian Y, Ye M, Zhou J (2012) Hyperspectral image classification based on structured sparse logistic regression and threedimensional wavelet texture features. IEEE Transactions on Geoscience and Remote Sensing 51(4):2276–2291
Falco N, Benediktsson JA, Bruzzone L (2015) Spectral and spatial classification of hyperspectral images based on ica and reduced morphological attribute profiles. IEEE Transactions on Geoscience and Remote Sensing 53(11):6223–6240
Yu S, Jia S, Xu C (2017) Convolutional neural networks for hyperspectral image classification. Neurocomputing 219:88–98
Paoletti ME, Haut JM, Plaza J, Plaza A (2019) Deep learning classifiers for hyperspectral imaging: A review. ISPRS Journal of Photogrammetry and Remote Sensing 158(Dec.) 279–317
Chen X, Li M, Yang X (2015) Stacked denoise autoencoder based feature extraction and classification for hyperspectral images. Journal of Sensors 2016(2015-11-30), 1–10
Mei S, Ji J, Geng Y, Zhang Z, Li X, Du Q (2019) Unsupervised spatial-spectral feature learning by 3d convolutional autoencoder for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing 57(9):6808–6820
Lee H, Kwon H (2017) Going deeper with contextual cnn for hyperspectral image classification. IEEE Trans Image Process 26(10):4843–4855
Zhang H, Li Y, Zhang Y, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network. Remote Sensing Letters 8(4–6):438–447
Zhong Z, Li J, Luo Z, Chapman M (2017) Spectral-spatial residual network for hyperspectral image classification: A 3-d deep learning framework. IEEE Transactions on Geoscience and Remote Sensing 56(2):847–858
Liu B, Yu X, Zhang P, Tan X, Wang R, Zhi L (2018) Spectral-spatial classification of hyperspectral image using threedimensional convolution network. Journal of Applied Remote Sensing 12(1):016005–016005
Mou L, Ghamisi P, Zhu XX (2017) Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 55(7):3639–3655
Liu B, Yu X, Yu A, Zhang P, Wan G (2018) Spectral-spatial classification of hyperspectral imagery based on recurrent neural networks. Remote sensing letters 9(10–12):1118–1127
Zhou F, Hang R, Liu Q, Yuan X (2019) Hyperspectral image classification using spectralspatial lstms - sciencedirect. Neurocomputing 328:39–47
Qin A, Shang Z, Tian J, Wang Y, Zhang T, Tang YY (2018) Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters 16(2):241–245
Wan S, Gong C, Zhong P, Pan S, Li G, Yang J (2020) Hyperspectral image classification with context-aware dynamic graph convolutional network. IEEE Transactions on Geoscience and Remote Sensing 59(1):597–612
Hong D, Gao L, Yao J, Zhang B, Plaza A, Chanussot J (2020) Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 59(7):5966–5978
Tang H, Li Y, Han X, Huang Q, **e W (2019) A spatial-spectral prototypical network for hyperspectral remote sensing image. IEEE Geoscience and Remote Sensing Letters 17(1):167–171
Ma X, Ji S, Wang J, Geng J, Wang H (2019) Hyperspectral image classification based on two-phase relation learning network. IEEE Transactions on Geoscience and Remote Sensing 57(12):10398–10409
Li Z, Liu M, Chen Y, Xu Y, Du Q (2021) Deep cross-domain few-shot learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing PP(99), 1–18
Bai J, Huang S, **ao Z, Li X, Zhu Y, Regan AC, Jiao L (2022) Few-shot hyperspectral image classification based on adaptive subspaces and feature transformation. IEEE Transactions on Geoscience and Remote Sensing 60:1–17
Zhang Y, Li W, Zhang M, Wang S, Tao R, Du Q (2022) Graph information aggregation cross-domain few-shot learning for hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems
Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D (2020) Supervised contrastive learning. Advances in neural information processing systems 33:18661–18673
Koch G, Zemel R, Salakhutdinov R et al (2015) Siamese neural networks for one-shotimage recognition. ICML deep learning workshop 2(1)
Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. Advances in neural information processing systems 29
Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. Advances in neural information processing systems 30
Sung F, Yang Y, Zhang L, **ang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. Proceedings of the IEEE conference on computer vision and pattern recognition, 1199–1208
Zhang B, Li X, Ye Y, Huang Z, Zhang L (2021) Prototype completion with primitive knowledge for few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3754–3762
Xue W, Wang W (2020) One-shot image classification by learning to restore prototypes. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):6558–6565
Xu W, Xu Y, Wang H, Tu Z (2021) Attentional constellation nets for few-shot learning
Finn C, Abbeel P, Levine S (2017) Modelagnostic meta-learning for fast adaptation of deep networks. International conference on machine learning, 1126–1135
Jamal MA, Qi G-J (2019) Task agnostic metalearning for few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 11719–11727
Garcia V, Bruna J (2018) Few-shot learning with graph neural networks. 6th International Conference on Learning Representations, ICLR 2018
Kim J, Kim T, Kim S, Yoo CD (2019) Edgelabeling graph neural network for few-shot learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11–20
Chen C, Yang X, Xu C, Huang X, Ma Z (2021) Eckpn: Explicit class knowledge propagation network for transductive few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6596–6605
Zhang C, Lyu X, Tang Z (2019) Tgg: Transferable graph generation for zero-shot and few-shot learning. Proceedings of the 27th ACM International Conference on Multimedia, 1641–1649
Ma Y, Bai S, An S, Liu W, Liu A, Zhen X, Liu X (2020) Transductive relationpropagation network for few-shot learning. IJCAI 20:804–810
Wang W, Dou S, Jiang Z, Sun L (2018) A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote sensing 10(7):1068
Mou L, Lu X, Li X, Zhu XX (2020) Nonlocal graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 58(12):8246–8257
Liu B, Yu X, Zhang P, Yu A, Fu Q, Wei X (2017) Supervised deep feature extraction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 56(4):1909–1921
Huang L, Chen Y (2020) Dual-path siamese cnn for hyperspectral image classification with limited training samples. IEEE Geoscience and Remote Sensing Letters 18(3):518–522
Rao M, Tang P, Zhang Z (2020) A developed siamese cnn with 3d adaptive spatialspectral pyramid pooling for hyperspectral image classification. Remote Sensing 12(12):1964
Sun J, Shen X, Sun Q (2022) Hyperspectral image few-shot classification network based on the earth mover’s distance. IEEE Transactions on Geoscience and Remote Sensing 60:1–14
Gao K, Liu B, Yu X, Qin J, Zhang P, Tan X (2020) Deep relation network for hyperspectral image few-shot classification. Remote Sensing 12(6):923
Rao M, Tang P, Zhang Z (2019) Spatial-spectral relation network for hyperspectral image classification with limited training samples. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(12):5086–5100
Zuo X, Yu X, Liu B, Zhang P, Tan X (2022) Fsl-egnn: Edge-labeling graph neural network for hyperspectral image few-shot classification. IEEE Transactions on Geoscience and Remote Sensing 60:1–18
** B, Li J, Li Y, Song R, Hong D, Chanussot J (2022) Few-shot learning with classcovariance metric for hyperspectral image classification. IEEE Transactions on Image Processing 31:5079–5092
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30
Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. Advances in neural information processing systems 31
Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery with 3d convolutional neural network. Remote Sensing 9(1):67
Yu C, Wang J, Chen Y, Huang M (2019) Transfer learning with dynamic adversarial adaptation network. IEEE Transactions on Geoscience and Remote Sensing pp. 778–786
Zhu Y, Zhuang F, Wang J, Ke G, Chen J, Bian J, **ong H, He Q (2020) Deep subdomain adaptation network for image classification. IEEE transactions on neural networks and learning systems 32(4):1713–1722
Liu B, Yu X, Yu A, Zhang P, Wan G, Wang R (2018) Deep few-shot learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 57(4):2290–2304
Acknowledgements
We would like to thank our anonymous reviewers for their valuable comments and suggestions. This work was supported by the Henan Province Science and Technology Research Project under Grant 232102110276, and National Natural Science Foundation of China under Grant 42371433.
Author information
Authors and Affiliations
Contributions
Chun Liu drafted the manuscript. Longwei Yang designed expriments. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Ethics approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, C., Yang, L., Li, Z. et al. Multi-level relation learning for cross-domain few-shot hyperspectral image classification. Appl Intell 54, 4392–4410 (2024). https://doi.org/10.1007/s10489-024-05384-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-024-05384-3