SpringerLink
Log in

Transfer Learning: Survey and Classification

  • Conference paper
  • First Online:
Smart Innovations in Communication and Computational Sciences

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1168))

Abstract

A key notion in numerous data mining and machine learning (ML) algorithms says that the training data and testing data are essentially in the similar feature space and also have the alike probability distribution function (PDF). Though, in several real-life applications, this theory might not retain true. There are issues where training data is costly or tough to gather. Thus, there is a necessity to build high-performance classifiers, trained using more commonly found data from distinct domains. This methodology is stated as transfer learning (TL). TL is usually beneficial when enough data is not available in the target domain but the large dataset is available in source domain. This survey paper explains transfer learning along with its categorization and provides examples and perspective related to transfer learning. Negative transfer learning is also discussed in detail along with its effects on the accomplishment of learning in target domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now
Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 103.50
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 129.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Weiss, K., Khoshgoftaar, T.M., Wang, D.D.: A survey of transfer learning. J. Big Data 3(1), 9 (2016)

    Article  Google Scholar 

  2. Mureşan, H., Oltean, M.: Fruit recognition from images using deep learning. Acta Univ. Sapientiae Inform. 10(1), 26–42 (2018)

    Article  Google Scholar 

  3. Asefisaray, B., Haznedaroğlu, A., Erden, M., Arslan, L.M.: Transfer learning for automatic speech recognition systems. In: 2018 26th Signal Processing and Communications Applications Conference (SIU), May 2018, pp. 1–4. IEEE

    Google Scholar 

  4. Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recogn. Lett. 119, 3–11 (2019)

    Article  Google Scholar 

  5. Chowdhary, M., Kumar, A., Singh, G., Meer, K.R., Kar, I.N., Bahl, R.: STMicroelectronics International NV and STMicroelectronics lnc.: Method and apparatus for determining probabilistic context awareness of a mobile device user using a single sensor and/or multi-sensor data fusion. U.S. Patent 9,870,535 (2018)

    Google Scholar 

  6. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  7. Taylor, M.E., Stone, P.: Cross-domain transfer for reinforcement learning. In: Proceedings of the 24th International Conference on Machine learning, pp. 879–886. ACM (2007)

    Google Scholar 

  8. Simões, R.S., Maltarollo, V.G., Oliveira, P.R., Honorio, K.M.: Transfer and multi-task learning in QSAR modeling: advances and challenges. Front. Pharmacol. 9, 74 (2018)

    Article  Google Scholar 

  9. Javaid, A., Niyaz, Q., Sun, W., Alam, M.: A deep learning approach for network intrusion detection system. In: Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS), May 2016, pp. 21–26

    Google Scholar 

  10. Alothman, B.: Similarity based instance transfer learning for botnet detection. Int. J. Intell. Comput. Res. (IJICR) 9, 880–889 (2018)

    Article  Google Scholar 

  11. Wang, W., Wang, H., Zhang, C., Xu, F.: Transfer feature representation via multiple kernel learning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence, Feb 2015

    Google Scholar 

  12. Kumagai, W.: Learning bound for parameter transfer learning. In: Advances in Neural Information Processing Systems, pp. 2721–2729 (2016)

    Google Scholar 

  13. Wang, D., Li, Y., Lin, Y., Zhuang, Y.: Relational knowledge transfer for zero-shot learning. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Google Scholar 

  14. Rajesh, M., Gnanasekar, J.M.: Annoyed realm outlook taxonomy using twin transfer learning. Int. J. Pure Appl. Math. 116(21), 549–558 (2017)

    Google Scholar 

  15. ** transductive transfer learning for recognition of epileptic electroencephalogram signals. IEEE Trans. Cybern. 49(6), 2200–2214 (2018)

    Article  Google Scholar 

  16. Yao, Q., Yang, H., Yu, A., Zhang, J.: Transductive transfer learning-based spectrum optimization for resource reservation in seven-core elastic optical networks. J. Lightw. Technol. (2019)

    Google Scholar 

  17. Siddhant, A., Goyal, A., Metallinou, A.: Unsupervised transfer learning for spoken language understanding in intelligent agents. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4959–4966, July, 2019

    Google Scholar 

  18. Lv, J., Chen, W., Li, Q., Yang, C.: Unsupervised cross-dataset person re-identification by transfer learning of spatial-temporal patterns. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7948–7956 (2018)

    Google Scholar 

  19. George, D., Shen, H., Huerta, E.A.: Classification and unsupervised clustering of LIGO data with Deep Transfer Learning. Phys. Rev. D 97(10), 101501 (2018)

    Article  Google Scholar 

  20. Bianco, S., Celona, L., Napoletano, P., Schettini, R.: On the use of deep learning for blind image quality assessment. SIViP 12(2), 355–362 (2018)

    Article  Google Scholar 

  21. Geng, M., Wang, Y., **ang, T., Tian, Y.: Deep transfer learning for person re-identification (2016). ar**v preprint ar**v:1611.05244

  22. Dai, W., Yang, Q., Xue, G.-R., Yu, Y.: Self-taught clustering. In: Proceedings of the 25th International Conference on Machine Learning, pp. 200–207. ACM (2008)

    Google Scholar 

  23. Wang, Z., Dai, Z., Póczos, B., Carbonell, J.: Characterizing and avoiding negative transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11293–11302 (2019)

    Google Scholar 

  24. Ulmer, A., Kapach, Z., Merrick, D., Maiya, K., Sasidharan, A., Alikhan, A., Dang, D.: Actively Preventing Negative Transfer (2019)

    Google Scholar 

  25. Paul, A., Vogt, K., Rottensteiner, F., Ostermann, J., Heipke, C.: A comparison of two strategies for avoiding negative transfer in domain adaptation based on logistic regression. Int. Arch. Photogram. Remote Sens. Spat. Inf. Sci. ISPRS Arch. 42(2), 845–852 (2018)

    Google Scholar 

  26. Mi, X., Chen, W., Zhang, Y.: A study on negative transfer of Non-English majors’ writing under the error-analysis theory. In: 2018 8th International Conference on Management, Education and Information (MEICI 2018). Atlantis Press, Dec (2018)

    Google Scholar 

  27. Gui, L., et al.: Negative transfer detection in transductive transfer learning. Int. J. Mach. Learn. Cybern. 9(2), 185–197 (2018)

    Article  Google Scholar 

  28. Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., Dean, J.: A guide to deep learning in healthcare. Nat. Med. 25(1), 24 (2019)

    Article  Google Scholar 

  29. **a, L., Goldberg, D.M., Hong, S., Garvey, P.: Transfer Learning in Knowledge-Intensive Tasks: A Test in Healthcare Text Analytics (2019)

    Google Scholar 

  30. Khan, S., Islam, N., Jan, Z., Din, I.U., Rodrigues, J.J.C.: A novel deep learning based framework for the detection and classification of breast cancer using transfer learning. Pattern Recogn. Lett. 125, 1–6 (2019)

    Article  Google Scholar 

  31. Raghu, M., Zhang, C., Kleinberg, J., Bengio, S.: Transfusion: Understanding transfer learning with applications to medical imaging (2019). ar**v preprint ar**v:1902.07208

  32. He, K., Girshick, R., Dollár, P.: Rethinking imagenet pre-training (2018). ar**v preprint ar**v:1811.08883

  33. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015). ar**v preprint ar**v:1502.03167

  34. Kute, R.S., Vyas, V., Anuse, A.: Component-based face recognition under transfer learning for forensic applications. Inf. Sci. 476, 176–191 (2019)

    Article  Google Scholar 

  35. Salaken, S.M., Khosravi, A., Nguyen, T., Nahavandi, S.: Seeded transfer learning for regression problems with deep learning. Expert Syst. Appl. 115, 565–577 (2019)

    Article  Google Scholar 

  36. Chakraborty, S., Gilyén, A., Jeffery, S.: The power of block-encoded matrix powers: improved regression techniques via faster Hamiltonian simulation (2018). ar**v preprint ar**v:1804.01973

  37. Sid-Lakhdar, W.M., Aznaveh, M.M., Li, X.S., Demmel, J.W.: Multitask and Transfer Learning for Autotuning Exascale Applications (2019). ar**v preprint ar**v:1908.05792

  38. Nguyen, L.D., Lin, D., Lin, Z., Cao, J.: Deep CNNs for microscopic image classification by exploiting transfer learning and feature concatenation. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5 (2018)

    Google Scholar 

  39. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  40. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence, Feb 2017

    Google Scholar 

  41. Zanini, P., Congedo, M., Jutten, C., Said, S., Berthoumieu, Y.: Transfer learning: Riemannian geometry framework with applications to brain–computer interfaces. IEEE Trans. Biomed. Eng. 65(5), 1107–1116 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Shri Ramswaroop Memorial University, Barabanki, India

    Nidhi Agarwal

  2. Department of Electrical and Electronics Engineering, G D Goenka University, Gurugram, India

    Akanksha Sondhi & Khyati Chopra

  3. Department of Electronics Engineering, National Institute of Technology Uttarakhand, Srinagar, Uttarakhand, India

    Ghanapriya Singh

Authors
  1. Nidhi Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Akanksha Sondhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khyati Chopra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ghanapriya Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nidhi Agarwal .

Editor information

Editors and Affiliations

  1. Computer Science Engineering Department, ABES Engineering College, Ghaziabad, Uttar Pradesh, India

    Shailesh Tiwari

  2. Department of Computer Science and Engineering, National Institute of Technology Agartala, Agartala, Tripura, India

    Munesh C. Trivedi

  3. Computer Science Engineering Department, Motilal Nehru National Institute of Technology Allahabad, Allahabad, Uttar Pradesh, India

    Krishn Kumar Mishra

  4. Computer Science Engineering Department, Motilal Nehru National Institute of Technology Allahabad, Allahabad, Uttar Pradesh, India

    A.K. Misra

  5. Department of Digital Technologies, University of Mauritius, Reduit, Moka, Mauritius

    Khedo Kavi Kumar

  6. Information Systems Department, Faculty of Information and Communication Technology, Institut Teknologi Sepuluh Nopember (ITS), Surabaya, Indonesia

    Erma Suryani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Agarwal, N., Sondhi, A., Chopra, K., Singh, G. (2021). Transfer Learning: Survey and Classification. In: Tiwari, S., Trivedi, M., Mishra, K., Misra, A., Kumar, K., Suryani, E. (eds) Smart Innovations in Communication and Computational Sciences. Advances in Intelligent Systems and Computing, vol 1168. Springer, Singapore. https://doi.org/10.1007/978-981-15-5345-5_13

Download citation

Publish with us

Policies and ethics

Search

Navigation