SpringerLink
Log in

UnseenSignalTFG: a signal-level expansion method for unseen acoustic data based on transfer learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This study introduces a transfer learning-based approach for signal-level expansion of unseen acoustic signal data, aiming to address the scarcity of acoustic signal data in a specific domain. By establishing connections and sharing knowledge between the source and target domains, the method successfully mitigates cross-domain disparities, overcoming challenges posed by unavailable data in the target domain, thereby elevating the quality and precision of data expansion. Diverging from conventional methods that predominantly emphasize feature-level expansion, the proposed approach accentuates the preservation of data signal integrity and effectively achieves the expansion of unseen class samples within the target domain.The effectiveness of this method has been validated across four different types of signal datasets. In the bearing dataset, the expansion of unseen data achieved accuracies of 99% at the signal level and 95% at the spectral level. These experimental results not only demonstrate the method’s advantages in augmenting both seen and unseen data but also highlight its effectiveness and application potential in achieving comprehensive expansion of target signals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data Availability

We have made dataset available at https://github.com/dongxidong/UnseenSignalTFG.

References

  1. Cui Y, Liu F, **g X, Mu J (2021) Integrating sensing and communications for ubiquitous iot: Applications, trends, and challenges. IEEE Netw 35(5):158–167

    Article  Google Scholar 

  2. Gangsar P, Tiwari R (2020) Signal based condition monitoring techniques for fault detection and diagnosis of induction motors: A state-of-the-art review. Mechl Syst Signal Process 144:106908

    Article  Google Scholar 

  3. Lv Y, Zhao W, Zhao Z, Li W, Ng KK (2022) Vibration signal-based early fault prognosis: Status quo and applications. Adv Eng Inf 52:101609

    Article  Google Scholar 

  4. Murat F, Yildirim O, Talo M, Baloglu UB, Demir Y, Acharya UR (2020) Application of deep learning techniques for heartbeats detection using ecg signals-analysis and review. Comput Biol Med 120:103726

    Article  Google Scholar 

  5. Peimankar A, Puthusserypady S (2021) Dens-ecg: A deep learning approach for ecg signal delineation. Expert Syst Appl 165:113911

    Article  Google Scholar 

  6. Elhaki O, Shojaei K, Mehrmohammadi P (2022) Reinforcement learning-based saturated adaptive robust neural-network control of underactuated autonomous underwater vehicles. Expert Syst Appl 197:116714

    Article  Google Scholar 

  7. Nowicki Jessee MA, Hamburger M, Ferrara M, McLean A, FitzGerald C (2020) A global dataset and model of earthquake-induced landslide fatalities. Landslides 17(6):1363–1376

    Article  Google Scholar 

  8. Liu S, Jiang H, Wu Z, Li X (2022) Data synthesis using deep feature enhanced generative adversarial networks for rolling bearing imbalanced fault diagnosis. Mech Syst Signal Process 163:108139

    Article  Google Scholar 

  9. Wang R, Zhang S, Chen Z, Li W (2021) Enhanced generative adversarial network for extremely imbalanced fault diagnosis of rotating machine. Measurement 180:109467

    Article  Google Scholar 

  10. Ding F, Cui Y (2023) Museflow: music accompaniment generation based on flow. Appl Intell 53(20):23029–23038

    Article  Google Scholar 

  11. Dibaj A, Ettefagh MM, Hassannejad R, Ehghaghi MB (2021) A hybrid fine-tuned vmd and cnn scheme for untrained compound fault diagnosis of rotating machinery with unequal-severity faults. Expert Syst Appl 167:114094

    Article  Google Scholar 

  12. Chen J, Hu W, Cao D, Zhang Z, Chen Z, Blaabjerg F (2022) A meta-learning method for electric machine bearing fault diagnosis under varying working conditions with limited data. IEEE Trans Ind Inf 19(3):2552–2564

    Article  Google Scholar 

  13. Zhong H, Lv Y, Yuan R, Yang D (2022) Bearing fault diagnosis using transfer learning and self-attention ensemble lightweight convolutional neural network. Neurocomputing 501:765–777

  14. Wang Q, Michau G, Fink O (2020) Missing-class-robust domain adaptation by unilateral alignment. IEEE Trans Ind Electron 68(1):663–671

    Article  Google Scholar 

  15. Zhang W, Li X, Ma H, Luo Z, Li X (2021) Open-set domain adaptation in machinery fault diagnostics using instance-level weighted adversarial learning. IEEE Trans Ind Inf 17(11):7445–7455

    Article  Google Scholar 

  16. Wan Z, Yang R, Huang M, Zeng N, Liu X (2021) A review on transfer learning in eeg signal analysis. Neurocomputing 421:1–14

    Article  Google Scholar 

  17. Li W, Huang R, Li J, Liao Y, Chen Z, He G, Yan R, Gryllias K (2022) A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios: Theories, applications and challenges. Mech Syst Signal Process 167:108487

    Article  Google Scholar 

  18. Zhuang F, Qi Z, Duan K, ** D, Zhu Y, Zhu H, **ong H, He Q (2020) A comprehensive survey on transfer learning. Proc IEEE 109(1):43–76

    Article  Google Scholar 

  19. Zhao B, Zhang X, Zhan Z, Wu Q (2021) Deep multi-scale adversarial network with attention: A novel domain adaptation method for intelligent fault diagnosis. J Manuf Syst 59:565–576

  20. Chen L, Li Q, Shen C, Zhu J, Wang D, **a M (2021) Adversarial domain-invariant generalization: A generic domain-regressive framework for bearing fault diagnosis under unseen conditions. IEEE Trans Ind Inf 18(3):1790–1800

    Article  Google Scholar 

  21. Li T, Zhao Z, Sun C, Yan R, Chen X (2021) Domain adversarial graph convolutional network for fault diagnosis under variable working conditions. IEEE Trans Instrum Meas 70:1–10

    Google Scholar 

  22. Tian Y, Wang Y, Peng X, Zhang W (2023) A fault diagnosis method for few-shot industrial processes based on semantic segmentation and hybrid domain transfer learning. Appl Intell 53(23):28268–28290

    Article  Google Scholar 

  23. Tian J, Han D, Li M, Shi P (2022) A multi-source information transfer learning method with subdomain adaptation for cross-domain fault diagnosis. Knowl-Based Syst 243:108466

    Article  Google Scholar 

  24. Chen Y-C, Lin Y-Y, Yang M-H, Huang J-B (2019) Crdoco: Pixel-level domain transfer with cross-domain consistency. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1791–1800

  25. Wang Q, Du J, Wu H-X, Pan J, Ma F, Lee C-H (2023) A four-stage data augmentation approach to resnet-conformer based acoustic modeling for sound event localization and detection. IEEE/ACM Trans Audio Speech Lang Process 31:1251–1264

    Article  Google Scholar 

  26. Rashid KM, Louis J (2019) Window-war**: a time series data augmentation of imu data for construction equipment activity identification. In: ISARC. Proceedings of the international symposium on automation and robotics in construction, vol 36, pp 651–657. IAARC Publications

  27. Steven Eyobu O, Han DS (2018) Feature representation and data augmentation for human activity classification based on wearable imu sensor data using a deep lstm neural network. Sensors 18(9):2892

    Article  Google Scholar 

  28. Xu K, Kong X, Wang Q, Yang S, Huang N, Wang J (2022) A bearing fault diagnosis method without fault data in new working condition combined dynamic model with deep learning. Adv Eng Inf 54:101795

    Article  Google Scholar 

  29. Lou H, Qi Z, Li J (2018) One-dimensional data augmentation using a wasserstein generative adversarial network with supervised signal. In: 2018 Chinese control and decision conference (CCDC), pp 1896–1901. IEEE

  30. Chen G, Zhu Y, Hong Z, Yang Z (2019) Emotionalgan: Generating ecg to enhance emotion state classification. In: Proceedings of the 2019 International conference on artificial intelligence and computer science, pp 309–313

  31. Haradal S, Hayashi H, Uchida S (2018) Biosignal data augmentation based on generative adversarial networks. In: 2018 40th Annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 368–371. IEEE

  32. Zang T, Zhu Y, Liu H, Zhang R, Yu J (2022) A survey on cross-domain recommendation: taxonomies, methods, and future directions. ACM Trans Inf Syst 41(2):1–39

    Article  Google Scholar 

  33. Himeur Y, Al-Maadeed S, Kheddar H, Al-Maadeed N, Abualsaud K, Mohamed A, Khattab T (2023) Video surveillance using deep transfer learning and deep domain adaptation: Towards better generalization. Eng Appl Artif Intell 119:105698

    Article  Google Scholar 

  34. Li Q, Chen L, Kong L, Wang D, **a M, Shen C (2023) Cross-domain augmentation diagnosis: An adversarial domain-augmented generalization method for fault diagnosis under unseen working conditions. Reliab Eng Syst Saf 234:109171

    Article  Google Scholar 

  35. Zhao C, Shen W (2023) Mutual-assistance semisupervised domain generalization network for intelligent fault diagnosis under unseen working conditions. Mech Syst Signal Process 189:110074

    Article  Google Scholar 

  36. Dai K, Li X, Huang X, Ye Y (2022) Sentatn: learning sentence transferable embeddings for cross-domain sentiment classification. Appl Intell 52(15):18101–18114

    Article  Google Scholar 

  37. Pan T, Chen J, **e J, Zhou Z, He S (2020) Deep feature generating network: A new method for intelligent fault detection of mechanical systems under class imbalance. IEEE Trans Ind Inf 17(9):6282–6293

    Article  Google Scholar 

  38. Rombach K, Michau G, Fink O (2023) Controlled generation of unseen faults for partial and open-partial domain adaptation. Reliab Eng Syst Saf 230:108857

    Article  Google Scholar 

  39. Hung H-T, Wang C-Y, Yang Y-H, Wang H-M (2019) Improving automatic jazz melody generation by transfer learning techniques. In: 2019 Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC), pp 339–346. IEEE

  40. Han X, Yin J, Du P, Zhang X (2014) Experimental demonstration of underwater acoustic communication using bionic signals. Appl Acoust 78:7–10

    Article  Google Scholar 

  41. Stojanovic M (1996) Recent advances in high-speed underwater acoustic communications. IEEE J Ocean Eng 21(2):125–136

    Article  Google Scholar 

  42. Mousavi SM, Sheng Y, Zhu W, Beroza GC (2019) Stanford earthquake dataset (stead): A global data set of seismic signals for ai. IEEE Access 7:179464–179476

    Article  Google Scholar 

  43. Li J, Chen X, He Z (2013) Adaptive stochastic resonance method for impact signal detection based on sliding window. Mech Syst Signal Process 36(2):240–255

    Article  Google Scholar 

  44. Zhang S, Wei H-L, Ding J (2023) An effective zero-shot learning approach for intelligent fault detection using 1d cnn. Appl Intell 53(12):16041–16058

    Article  Google Scholar 

  45. Pang L, Men S, Yan L, **ao J (2020) Rapid vitality estimation and prediction of corn seeds based on spectra and images using deep learning and hyperspectral imaging techniques. IEEE Access 8:123026–123036

    Article  Google Scholar 

  46. Zhang J, Zhou K, Wang Z (2023) A novel water pollution detection method based on acoustic signals and long short-term neural network. Appl Intell 53(10):12355–12371

    Article  Google Scholar 

  47. Zhang J, Zhou K (2023) Identification of solid and liquid materials using acoustic signals and frequency-graph features. Entropy 25(8):1170

    Article  Google Scholar 

  48. Zhao Z, Li T, Wu J, Sun C, Wang S, Yan R, Chen X (2020) Deep learning algorithms for rotating machinery intelligent diagnosis: An open source benchmark study. ISA Trans 107:224–255

    Article  Google Scholar 

  49. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  50. Liang C, Chen C, Liu Y, Jia X (2021) A novel intelligent fault diagnosis method for rolling bearings based on compressed sensing and stacked multi-granularity convolution denoising auto-encoder. IEEE Access 9:154777–154787

    Article  Google Scholar 

  51. Sakoe H, Chiba S (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans Acoust Speech Signal Process 26(1):43–49

    Article  Google Scholar 

  52. Kim J, Kang H, Kang P (2023) Time-series anomaly detection with stacked transformer representations and 1d convolutional network. Eng Appl Artif Intell 120:105964

    Article  Google Scholar 

  53. Zhu H, Cheng J, Zhang C, Wu J, Shao X (2020) Stacked pruning sparse denoising autoencoder based intelligent fault diagnosis of rolling bearings. Appl Soft Comput 88:106060

    Article  Google Scholar 

  54. Liu K, Peng Q, Sun H, Fei M, Ma H, Hu T (2022) A transferred recurrent neural network for battery calendar health prognostics of energy-transportation systems. IEEE Trans Ind Inf 18(11):8172–8181

    Article  Google Scholar 

  55. Zhao P, Cao D, Wang Y, Chen Z, Hu W (2023) Gaussian process-aided transfer learning for probabilistic load forecasting against anomalous events. IEEE transactions on power systems

  56. Wang Y, Ding H, Sun X (2022) Residual life prediction of bearings based on senet-tcn and transfer learning. IEEE Access 10:123007–123019

  57. Han B, Jiang X, Wang J, Zhang Z (2023) A novel domain adaptive fault diagnosis method for bearings based on unbalance data generation. IEEE Trans Instrum Meas

  58. Zhou K, Sisman B, Liu R, Li H (2021) Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 920–924. IEEE

  59. Anicet Zanini R, Luna Colombini E (2020) Parkinson’s disease emg data augmentation and simulation with dcgans and style transfer. Sensors 20(9):2605

    Article  Google Scholar 

  60. Lee P, Jeon S, Hwang S, Shin M, Byun H (2023) Source-free subject adaptation for eeg-based visual recognition. In: 2023 11th International winter conference on brain-computer interface (BCI), pp 1–6. IEEE

Download references

Author information

Authors and Affiliations

  1. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, **’an, 710121, Shaanxi, China

    **aoying Pan, Jia Sun, MingZhu Lei, YiFan Wang & Jie Zhang

  2. School of Computer Science and Technology, **’an University of Post and Telecommunications, **’an, 710121, Shaanxi, China

    **aoying Pan, Jia Sun, MingZhu Lei, YiFan Wang & Jie Zhang

  3. **’an Key Laboratory of Big Data and Intelligent Computing, **’an, 710121, Shaanxi, China

    Jie Zhang

Authors
  1. **aoying Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. MingZhu Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. YiFan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

**aoying Pan: Conceptualization, Formal analysis, Investigation. Jia Sun: Methodology, Data curation, Software, Validation, Visualization, Writing - original draft. Mingzhu Lei: Conceptualization, Methodology, Visualization. YiFan Wang: Conceptualization, Methodology, Visualization. Jie Zhang: Conceptualization, Methodology, Supervision, Validation.

Corresponding author

Correspondence to Jie Zhang.

Ethics declarations

Competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A   Appendix

A   Appendix

Table 29 Comparison results of feature extraction analysis
Full size table
Table 30 Comparative results of transfer signal generation analysis
Full size table
Table 31 Comparison results of feature extraction with different layers of analysis
Full size table
Table 32 Comparison results of the analysis of different layers of the transfer module
Full size table
Table 33 Comparative results of cross-domain consistency loss analysis
Full size table
Table 34 Comparison results of the expansion of dataset II
Full size table
Table 35 Comparison results of the expansion of Dataset III
Full size table
Table 36 Classification results for Dataset I
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, X., Sun, J., Lei, M. et al. UnseenSignalTFG: a signal-level expansion method for unseen acoustic data based on transfer learning. Appl Intell 54, 7317–7351 (2024). https://doi.org/10.1007/s10489-024-05568-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05568-x

Keywords

Search

Navigation